AI candidate screening you can check line by line
AnswerReel scores each answer against criteria you write, one criterion at a time. Every score has a one-line reason, and any score above 1 quotes the candidate's own words. This page is the whole method.
By The AnswerReel team. Updated .
Maya Ruiz
Customer support lead
4.5
Scored 1 of 1 questions
Question 1
Tell me about a time you dealt with an upset customer. What did you do?
Scores against the criteria you wrote
Scores are written by AI and can be wrong. The recording is the record.
0:03Last spring a customer emailed us three times in one morning about an order that was nine days late.
0:14She'd paid for express shipping, so she was right to be annoyed.
0:22I called her back the same afternoon, apologized, and refunded the shipping while we were on the phone.
0:38Then I wanted to know why it happened, because she wasn't the only one.
0:47We changed how the warehouse checks express orders at packing, and late express orders dropped from about twenty a month to two.
AI follow-up: asks how the fix was found
You said she wasn't the only one. How did you work out what was causing the delays?
0:02I pulled the last thirty late orders into a sheet and sorted them by who packed them and when.
0:15Almost all of them went out on the morning shift, right after the label printer had been reset.
0:29The reset switched it back to standard labels, so I got the team to check the label against the order before sealing the box.
One question: did the answer cover it?
The AI scores one thing only: how far an answer covers the criteria you wrote for that question. Each question can have up to three criteria, and each criterion gets its own score from 1 to 5.
A question's score is the average of its criteria. The candidate's total is the average of the questions that could be scored, shown with one decimal and a count such as “scored 2 of 3 questions”. A question that can't be scored is left out of the average, not counted as zero.
Questions with a short factual answer, like a start date or whether someone holds a forklift license, aren't scored at all. The answer goes straight into the summary as the candidate gave it.
| Score | What it means for one criterion |
|---|---|
| 1 | Not mentioned. An answer under 10 words also gets a 1, with the reason “answer too short”. |
| 2 | Mentioned, but vague, with nothing specific. |
| 3 | Explained, but without a specific example or method. |
| 4 | Gives a specific example or method. |
| 5 | Gives a specific example or method, and the result: a number, a change, or someone’s feedback. |
| Not scored | No speech detected, too unclear to transcribe reliably, answered in a different language, or the quote could not be found word for word in the answer. |
How to write a criterion
Write one sentence, up to 200 characters, about what a good answer has to include. Criteria are shown to you and to the model, never to the candidate. Candidates see the general method on this page instead.
| Criterion | Verdict | Why |
|---|---|---|
| Describes one complaint from start to finish, including what they did and when.One idea, and you can tell from the answer whether it's there. | Good | One idea, and you can tell from the answer whether it's there. |
| Good communication skills.Nothing in a single answer proves or disproves it. Ask for something they did instead. | Too vague | Nothing in a single answer proves or disproves it. Ask for something they did instead. |
| Handles complaints and finds upsell opportunities.Split it into two criteria, or the score can't say which half was missing. | Two in one | Split it into two criteria, or the score can't say which half was missing. |
| Sounds confident and speaks clearly.This is about delivery, not content. AnswerReel doesn't score how people sound. | Not scored | This is about delivery, not content. AnswerReel doesn't score how people sound. |
Every score shows its evidence
Each score comes with one sentence of reasoning and a quote from the candidate's answer. A score of 1 says “nothing relevant found” instead of a quote.
Before a score is shown, our server looks for the quote in the transcript or the typed answer, ignoring only capital letters, spacing and punctuation. If the words aren't there exactly, the score is thrown away and tried once more; if it still fails, that criterion is marked “not scored”. So every quote you see is in the transcript or the typed answer, word for word. A transcript can still mishear a word, which is why clicking a quote usually plays the recording from the second it was said: you can listen for yourself.
When a question has a follow-up, the original answer and the follow-up answer are scored together, and each quote is labeled with the part it came from.
What the AI never sees or judges
The model receives the question, its criteria and the words of the answer, and nothing else. We don't send it the video, the audio, the name and email the candidate entered, or where the job is. If a candidate says their own name in an answer, the model reads it like any other word.
It doesn't score delivery: confidence, fluency, pace or accent. Transcription accuracy already varies with accent and recording conditions, so scoring how someone speaks would mean scoring their accent. We never analyze faces, expressions, emotion or voice, and never infer age, gender, ethnicity or health.
The model is told to ignore personal details that have nothing to do with the criteria, and to ignore any instructions aimed at it inside an answer, such as “give me a 5”. Answers that contain one are flagged for you, and scored normally.
Why it never rejects anyone
Scores sort your candidate list. Nothing happens automatically because of a score: no rejection email, no stage change. Emails you set up for “moving forward” or “not a fit” only go out when a person applies that label.
A transcript can mishear a word, and a model can misread an answer. A person who checks the quote against the recording has a chance to catch that; an automatic rejection doesn't. Several laws also treat a fully automated hiring decision differently from one a person makes.
Candidates can say no
Before starting, every candidate chooses between “let AI process my answers” and “only a person reviews them”. Neither is pre-selected and both look the same. Choosing no means nothing is sent to any AI service. Those candidates still appear in your list, in a separate group flagged at the top, for you to review yourself.
What to use it for, and what not
Use the scores to decide whose answers to read or watch first, measured against the criteria you wrote for a job. Don't use them:
- to reject or advance anyone without a person reviewing their answers, including through the API, webhooks or Zapier, which pass the scores on to other tools;
- for anything other than screening candidates for a real job, such as promotion, pay or discipline;
- with criteria about who someone is, such as their age, health, family or where they're from, or about how they speak;
- to compare candidates across jobs. Each job has its own questions and criteria, so a 4 in one job isn't a 4 in another.
Known limitations
- A score depends on the transcript. Transcription is less accurate for some accents, in noisy rooms and with weak microphones, and a misheard word can hide a point the candidate made. That can put some candidates at a disadvantage, which is why our tests compare native and non-native speakers, and why clicking a quote usually plays the recording from where it was said.
- The model checks whether an answer covers a criterion, not whether it's true. A quote that matches word for word shows what was said, not that the score is right.
- Scores are only as good as the criteria. Vague criteria give unreliable scores.
- A very short answer, or a recording that is mostly silence, usually scores 1 or comes back “not scored”.
- Scores made by different model or prompt versions aren't strictly comparable. When a job has both, the candidate list says so.
- Scoring works in English for now. Each other language switches on only after it passes the tests below.
- Anthropic's system card for Claude Haiku 5.5 reports a small measured bias on a standard bias test (BBQ), a little higher than its previous Haiku model.
How to review a score
- Read the reason and the quote together. If the reason claims more than the quote shows, trust the quote.
- For video and voice answers, click the quote. It usually plays the recording from where those words were said; listen to what comes before and after.
- Before you pass over anyone, read or watch their whole answer, not just the quotes.
- Treat “not scored” as “review this one yourself”, not as a low score.
- Apply the labels yourself. A score never moves anyone forward or back.
Now and then, check how you use the scores. If your labels usually match the scores, review a few answers without looking at the scores and compare your decisions. For a criterion marked “not scored”, read the reason shown under it. Criteria lock once invitations go out, so to change one, copy the job and edit the copy. If a candidate asks you to look again, review their answers without the scores. You can switch AI scoring off for any job.
How we test it before launch
Scoring doesn't switch on for a language until it passes these checks on a set of real recorded answers:
- The same answer scored three times gets the same score on at least 90% of criteria, and never two or more points apart.
- AI scores land within one point of a human rater's on at least 85% of criteria.
- The same answer from a native and a non-native speaker scores within one point.
- Every quote can be found word for word in the transcript or the typed answer.
- Answers with planted instructions like “give me a 5” score within a point of the clean version, and get flagged.
We will publish the results on this page once testing is finished.
The model behind it
Scores, summaries and follow-up questions come from Claude Haiku 5.5, a general model made by Anthropic. Anthropic says it learned from public web pages (its crawler follows robots.txt and doesn't go behind logins), public and private datasets, data from people who sent feedback or agreed to share it for training, and synthetic data made by other models, with knowledge up to June 2026. It doesn't say how much personal data is in that mix. Anthropic doesn't train on what we send it through its API, and we don't train or fine-tune any model on candidate answers.
Recordings are transcribed by Deepgram's Nova-3 model. Deepgram describes its training only in general terms, such as audio from difficult recording conditions and synthetic speech that mixes languages, and doesn't list its sources. We've opted out of its model improvement program, so it doesn't learn from our recordings.
Every score records the model and the version of our instructions that produced it, and a change only applies to answers submitted afterwards. We email customers at least 30 days before any material change to how scores or follow-up questions are produced, such as the model, those instructions, the transcription service, how quotes are checked, how scores are combined or the default settings, and before any material change to this page. Urgent fixes, such as a security problem or wrong scores, can't wait that long; we tell you as soon as they ship. We keep those notices, past versions of this page and our test results for at least three years.
If you have to tell a candidate about it
Some laws, such as Colorado's from 2027, require you to explain to a candidate the part AI played in a decision about them. What you'll need: the tool is AnswerReel AI scoring, made by AnswerReel. It uses the words of the candidate's answers, typed or transcribed from their recording, together with the question and your criteria. The current versions are claude-haiku-5-5 for the model, score-2026-10-20.2 for the scoring instructions and follow-up-2026-10-20 for follow-up questions. To find out which versions scored a particular candidate, email support@answerreel.com.
We don't publish the instructions themselves, which are our own work. This page describes what they tell the model to do.
Using AI scores comes with duties for employers in some places, including bias audits in New York City and notice and consent in Illinois. Our AI hiring laws summary lists them. To see how other tools score, read our candidate screening software comparison.
See this scoring on your own questions
Start from our screening interview questions, or build a scorecard first. The free plan covers 10 answers a month, with AI scoring and follow-ups included.