AI Fluency Interview Questions: A Round You Can Score
By Pooja Goenka · Published 2026-10-14
To test AI fluency in a developer interview, give the candidate a confident AI-written answer that contains exactly one planted mistake, and score whether they find it, explain why it is wrong, and leave the correct lines alone. Then ask how they would verify that code before it ships, because on the job most of AI fluency is checking what the AI hands you.
A full round adds three more question types and a rubric written down in advance, so the score doesn't depend on which interviewer the candidate got.
The round in brief:
- Define AI fluency as judgement about AI output: knowing when it is wrong and how to prove it.
- Ask four kinds of question: find the planted mistake, plan the verification, decide what to delegate, and diagnose a bad prompt.
- Score each on a 0 to 3 scale with written anchors, agreed before the interview.
- Count a confident criticism of a correct line as a miss.
- Run it in about 30 minutes with a shared document.
What "AI fluency" means for an engineer
Vendors use the term differently. The most useful definition I've found comes from Anthropic's AI Fluency course, built with Professors Rick Dakan and Joseph Feller. It splits the skill into four parts, which they call the 4D framework: Delegation (deciding whether and how to use AI for a task), Description (telling the AI clearly what you want), Discernment (judging whether what comes back is any good) and Diligence (taking responsibility for what you do with it).
For hiring engineers, Discernment and Diligence carry the most weight. Writing a clearer prompt is quick to learn. Noticing that a fluent, well-formatted answer is wrong, and knowing how to prove it, takes real understanding of the material.
The platforms that assess this say similar things. HackerRank's description of Chakra (5 August 2026) lists scenarios such as reviewing AI-suggested code and catching a subtle bug a model introduced, and scores candidates on CS fundamentals, AI fluency and judgement. CoderPad's AI fluency docs tell reviewers to check whether the candidate gave the AI enough context, verified its output and improved weak responses. The docs don't include a rubric and recommend writing your own in advance, which is what the one below is for.
Four AI fluency interview questions, with examples
Each type maps to one of the four Ds. Use all four for a dedicated round, or pick one or two for a block inside a longer interview.
1. Find the planted mistake (Discernment)
Give a question from the role and a five to seven-line AI-style answer with exactly one wrong line (worked example below). Say: "An AI wrote this. One line contains a technical mistake. Review it as you would a colleague's pull request, out loud."
2. Plan the verification (Diligence)
Show a short AI-generated function that is correct, and say: "An assistant wrote this and the tests pass. What would you check before you merge it?"
You're listening for specific checks tied to the code: edge cases they would add tests for, inputs the tests don't cover, a library behaviour they would look up in the docs, what they'd watch in logs after release.
3. Decide what to delegate (Delegation)
Describe a real task from your team, such as adding rate limiting to an API endpoint, and ask: "Which parts would you hand to an AI assistant, which would you write yourself, and why?"
Strong answers separate boilerplate from decisions. They might hand off the test scaffolding and the config wiring and keep the choice of limits, the behaviour under load and anything touching authentication. There isn't one right split, and what you're scoring is whether the reasoning holds up.
4. Diagnose a bad prompt (Description)
Show a prompt and the poor output it produced, for example a request for "a function to parse dates" that returned code assuming US month-first format. Ask: "What was missing from the prompt, and how would you rewrite it?"
Good candidates name the missing context, which here is the input formats, the locale, what to do with invalid input and the expected return type.
A worked planted-mistake example
The question: a teammate's JavaScript sorts a list of prices and the order comes out wrong. Here is the AI's explanation.
Array.prototype.sort()sorts the array in place and returns the same array.- With no comparator, it converts elements to strings and compares them, so
[10, 9, 1].sort()gives[1, 10, 9].- To sort numbers, pass a comparator function.
- The comparator only needs to say whether the first value is bigger, so
(a, b) => a > bsorts ascending.- The comparator
(a, b) => a - balso works, and returns a negative, zero or positive number.
Line 4 is the mistake. The comparator must return a negative number, zero or a positive number. a > b returns true or false, which converts to 1 or 0 and never goes negative, so the engine can't tell when a should come first. I ran [5, 1, 4, 2, 3, 10, 9, 8, 7, 6, 12, 11].sort((a, b) => a > b) in Chrome 152 and the array came back in its original order, unsorted. Every small array I tried came back in its original order too, so the code only looks right when the input was already sorted, as hand-written test arrays often are.
Line 2 is the decoy. It sounds wrong to people who haven't hit it, and a candidate who confidently "corrects" it has flagged a true line. That's a different result from missing line 4, and you should record it separately.
A strong reason sounds like: "The comparator has to return a negative number for 'a first', and a boolean never does, so the order is undefined." A weak reason is "booleans are bad practice". The candidate found the right line but may not know why it breaks.
A scoring rubric you can copy
Agree this table before the interview and score each row privately before you discuss the candidate with anyone.
| Dimension | 0 | 1 | 2 | 3 |
|---|---|---|---|---|
| Finds the planted mistake | Approves everything, or flags only correct lines | Finds it after two or more probes | Finds it after one probe | Finds it unprompted |
| Explains why | No reason, or a wrong one | Vague ("looks off") | Right idea, imprecise | Names the mechanism precisely |
| Avoids false flags | Confidently attacks a correct line and holds the view | Attacks a correct line, then withdraws when asked how to check it | Raises a doubt on a correct line as a question | Leaves correct lines alone, or flags a doubt with a check |
| Plans verification | No concrete check | Generic ("test it more") | One specific check | Specific tests or doc lookups tied to the risky lines |
| Delegation judgement | Hands off everything or nothing, no reason | Splits the task with weak reasons | Sensible split, partly justified | Clear split, keeps the risky decisions, explains why |
| Prompt diagnosis | Can't say what was missing | Names one gap | Names most gaps | Names the gaps and still plans to check the output |
Two scoring rules matter more than the table. A confident wrong critique scores 0 on the first row however well it is argued, because in real work it sends a team chasing a bug that isn't there. And a candidate who says "I'm not sure about line 4, here is the test I'd run to find out" deserves credit on the verification row even if they didn't commit to an answer.
The table works as a row in any structured scorecard. My technical interview scorecard template has a technical-judgement row it slots into.
Common scoring errors
The usual ones are rewarding the longest critique, leaking the answer through your reactions, planting two mistakes by accident, and reusing one example until it spreads among candidates. The costliest is discussing before scoring, because the first interviewer to speak sets the tone. How to run a hiring debrief covers private-first scoring.
How this fits with AI-assisted live coding
Some companies now let candidates use an AI assistant during the coding round itself. 404 Media reported in July 2025 that Meta planned to let some coding candidates use one, saying in an internal post that it was more representative of the environment its engineers would work in. Platforms such as CoderPad and HackerRank give you a coding environment with an assistant built in and a replay of how the candidate used it.
That format shows how someone works with AI over a longer task. The planted-mistake round is narrower and cheaper, and gives a clean signal on whether the candidate checks what a model tells them. I'd use both if you have the time. For a fuller ten-minute version with probes and a second example, see how to test whether candidates can spot mistakes in AI answers.
Where LogicWiz fits
LogicWiz Interviews runs an inverted round built on this idea. The AI writes a confident, well-structured answer with one subtle technical error and asks the candidate to review it. If they miss it, the AI pushes back gently without giving it away, and a confident critique of the wrong thing scores as a miss. Interviewers submit scorecards privately before they see each other's, and the debrief view highlights where they disagree.
Candidates can practise the same skill free. LogicWiz Interviews for candidates has a spot-the-mistake mode across 16 tracks, with 3 free AI sessions a day on a free account, and the LogicWiz Arena game Spot Aria's Mistake is five short rounds of finding the wrong line.
Common questions
Should candidates be allowed to use AI in interviews?
It depends on what you want to measure. Allowing an assistant shows how someone works with it on a longer task, as Meta reportedly began trialling in 2025. A planted-mistake round measures judgement directly and works either way.
How long should an AI-judgement round take?
About ten minutes for the planted-mistake question alone, and around 30 minutes for all four question types. It fits as a block inside a longer technical interview.
What does a strong answer look like?
The candidate names the wrong line, explains the mechanism behind the error, leaves the correct lines alone and says how they would confirm it, such as a specific test or a doc lookup.
Can this replace a coding round?
No. It shows how someone reviews and verifies, and it says little about whether they can build something from scratch. Pair it with a build or debug task.