Five AI interview questions that reveal judgment

A marketing candidate can write a polished AI prompt and still miss an unsupported product claim in the output. Your interview needs to show how they inspect the work, explain a decision, and handle a task that doesn’t go as planned.

Choose a task from the role before choosing the questions. For a B2B SaaS marketer, that might be turning verified product facts into a page brief or checking a weekly report. The questions below help you examine that work without turning the interview into a tool-name quiz.

How should you set up the interview?

Give candidates the same task context, questions, and evaluation criteria. OPM’s structured-interview guidance describes using predetermined questions and consistent standards to assess job-related competencies.

Tell candidates what they’ll demonstrate and which tools you provide. Let them use a cleared example or the shared exercise; don’t require confidential work from a former employer. Ask about adjustments they need to participate, and agree the scope before assigning work outside the interview.

Assess a skill the role needs on entry. OPM’s work-sample guidance distinguishes those requirements from skills you’ll teach after hiring. A candidate shouldn’t fail a test for not knowing your internal workflow yet.

1. Which part of this task would you give AI?

Look for a decision about the work and a way to inspect the result. Ask the candidate to separate gathering inputs, drafting, checking, and publication for your selected task.

For a product page, they might use AI to organize approved facts into an outline while retaining responsibility for the buyer’s question and final claims. Ask what they’d keep manual and why. If their first answer is broad, let them narrow it using your brief.

Anthropic’s guidance on simple workflows recommends adding complexity when the task requires it. Ask the candidate why an agent is needed and whether a simpler draft-and-check process would do the job.

2. What would you change in this prompt?

Look for a specific change connected to a specific output problem. Give the candidate the prompt and its draft, then ask them to explain what they’d change and how they’d test it.

A request for an “engaging page” may need a named reader, verified facts, and a destination. A useful revision states those inputs and gives the writer a way to handle missing information. Ask the candidate to compare the revised output with the original brief, rather than judging only whether it sounds smoother.

A previous prompt with permitted revision notes is also useful evidence. Treat an inability to share private material as an access constraint, and offer the common exercise instead.

3. How would you catch an unsupported claim?

Look for a source check that can change the publishing decision. Give the candidate a draft containing a capability claim the product brief doesn’t establish.

Ask which source or person could confirm it. Then ask what they’d do while the claim remains unverified. A candidate can remove it, ask a focused question, or leave the draft unfinished with the gap recorded. The content-review method shows how to compare a draft with its brief.

NIST’s generative AI risk profile covers confidently stated false content. The interview should let you observe whether the candidate notices that problem in a marketing draft. Naming the risk is useful; showing the unsupported sentence and resolving it is stronger evidence.

4. How would you choose a tool for this job?

Look for a comparison based on the task’s requirements. Ask what they’d test using the same inputs, and what would make them choose or reject an option.

Relevant criteria might include output accuracy, source handling, revision effort, data access, and cost. Give them access to the tools needed for your exercise, or ask them to describe the comparison they’d run. Don’t make ownership of several paid subscriptions a requirement for demonstrating judgment.

Their preferred product can differ from yours. Follow the reasoning through to an acceptance check, and record where they still need evidence. Use the marketing engineer role overview to define the job before choosing the exercise. The marketing engineer skills guide connects these choices to the broader role you’re hiring for.

5. What happens when the workflow fails?

Look for a visible handoff to someone who can resolve the problem. Ask what happens if a source file is missing, a connection fails, or the system saves a partial draft.

The candidate should explain how they detect the failure, preserve useful work, and communicate what’s unfinished. Ask which actions they’d stop and who owns the recovery. A claim that the workflow “just retries” needs a stopping condition and a way to avoid duplicate work.

This is also a useful follow-up to assigning bounded jobs to agents. You need to know who checks the output after the automation runs.

What can you use as the shared exercise?

Use a small, labeled teaching brief that represents the role’s work. This illustrative exercise contains no client data or candidate quote:

• The reader is comparing software for sales-to-delivery handoffs.
• The supplied product facts support assigning a receiving owner and a next action.
• Integration support is unknown until documentation confirms it.
• The requested output is a page outline, a fact-check note, and one unresolved question.

Ask every candidate to work from those inputs. Record evidence under five headings, covering task choice, prompt revision, source checking, tool selection, and failure handling. Use consistent descriptions such as demonstrated, needs a follow-up, or not demonstrated in this exercise, with the passage or action that supports each judgment.

Before the next interview, choose the task and write those criteria. If the task exposes a workflow you want built for your business, Custom AI Builds can scope its inputs and completion checks. Keep that implementation decision separate from your hiring decision.