Hiring Your First AI Engineer

The most common failure in this hire is not choosing badly between candidates. It is running a process before deciding which of three quite different jobs is being filled, and then being surprised when the person who arrives is excellent at a job you did not have.

Decide which job it is first

Write down, in one sentence, what will be different in six months if this hire goes well. Then check which of these it describes, because the three want different people and the titles will not tell you apart:

• Ship a feature that uses models. You need a product engineer who is comfortable with uncertainty. The binding skill is engineering judgement, and the AI-specific knowledge is learnable in weeks by the right person.
• Make an existing AI feature reliable and affordable. Different person. You need evaluation discipline and production instincts, and the ability to say no to changes that cannot be measured.
• Train or tune models on your data. This is ML engineering and it is a genuinely different hire with a different interview loop. If you are not sure you need it, you probably do not yet — the question of whether to fine-tune at all is usually answered no on the first pass.

If the honest answer is “all three”, you are hiring a founding engineer for a function, not a specialist, and you should weight breadth and self-direction far above depth in any one area. Say that in the advert; it changes who applies.

What to screen for

Five signals, in descending order of how much they predict. The first two are worth more than the rest combined.

• Evidence of measurement. Has this person ever reported a number about the quality of something they built, with a definition and a sample size? A candidate who says “we got extraction accuracy from 71% to 84% on a frozen set of 400 documents” is in a different category from one who says the new prompt was better. This is the strongest single signal available, because it is the habit that is hardest to fake and the one most teams lack.
• Something shipped to real users, and kept running. Building a demo is a solved problem. Operating something over time — noticing regressions, handling a provider incident, keeping the cost stable — is what the job actually is.
• Debugging under uncertainty. Ask for a story about a failure that took a while to localise. Listen for hypotheses, experiments, and at least one wrong turn. Candidates whose stories have no wrong turns are telling you a story, not a debug.
• Cost awareness. Can they estimate what something will cost before building it? Not precisely — with visible assumptions. This predicts whether unit economics will be somebody’s problem before or after the invoice.
• Security instinct about untrusted text. Not expertise, just awareness that content arriving from a document or a web page can carry instructions. If they raise it unprompted when the design involves external content, that is a strong signal.

What to ignore on a CV

• Framework and vendor logos. The list of libraries somebody has touched predicts very little, and the libraries turn over faster than the hiring process. What you want is whether they can tell you when they chose not to use one.
• “Prompt engineering” as a headline skill. Not because it is worthless — the underlying work is real — but because it is claimed universally and absorbed into other roles. It carries no information.
• Certificates and course completions, which mostly evidence attendance; what a certificate can and cannot prove has its own page.
• Model or paper name-dropping without a system attached. Familiarity with the discourse is not familiarity with the work.
• The absence of AI experience, if the rest is strong. A senior backend engineer with measurement discipline becomes productive here quickly; the transfer is large and specific. Screening this out is the most common way a first AI hire goes wrong, because the pool of people with a long AI track record is small and the pool of people who can do the job is not.

An exercise worth somebody’s Saturday

A take-home is defensible only if it is short, paid or clearly capped, and about judgement rather than throughput. This one takes about three hours and produces something you can talk about for forty minutes.

BRIEF  (timebox: 3 hours, stop when the time is up)

Attached: 60 short documents of a single type, and a target
schema with 6 fields. 12 of the documents are deliberately
awkward.

Build something that produces a validated record from a
document, and answer these in a README:

1. What is your accuracy, how did you define a failure, and
   on how many documents did you measure it?
2. Which documents fail, and is there a pattern?
3. What does one document cost to process, and how did you
   work that out?
4. What would you do with another day, and why that first?
5. What in your solution would you not put in production
   as it stands?

You may use any model or library. Do not spend more than
three hours; we will read what is there.

The brief works because the awkward documents make it impossible to reach 100%, which forces every candidate into the interesting territory: what do you do about the residual? Answers range from ignoring it, through routing it to a review queue, to designing a confidence signal — and the range is exactly what you are trying to observe.

The rubric

Score before discussing with other interviewers, and write one sentence of evidence per line. A rubric everyone fills in after the debrief is a record of the debrief, not of the candidate.

                                   0 = absent
                                   1 = present
                                   2 = strong

[ ] Measured something, and stated the sample size
[ ] Failure definition is written down and checkable
[ ] Noticed the pattern in the failing documents
[ ] Cost estimate exists with visible assumptions
[ ] Handled an invalid response rather than assuming success
[ ] Named a limitation without being asked
[ ] Chose a smaller solution where a bigger one was available
[ ] Code is legible; a colleague could change it

Hire bar: no zeros on the first two lines.
Everything else is a conversation.

The bar is placed there deliberately. A candidate who did not measure anything and cannot define failure will make every subsequent decision on the team by argument rather than by evidence, and that is the expensive failure mode for a first AI hire.

On compensation

This page will not give you a number, and you should be suspicious of pages that do. Compensation for this work varies by country, by city, by company stage and by quarter, and any figure written here would be both unsourced and out of date by the time you read it. What can be said honestly is what determines the number, so you can reason about your own position:

• It is an engineering salary with a modifier, not a separate market. The largest determinant is the seniority band your organisation already runs for engineers of that level, which you already know.
• Scarcity attaches to specifics, not to the field. “Has used models” is not scarce. “Has operated a model-backed system under a quality bar and a budget for two years” is much rarer, and that is what commands a premium — which is also an argument for hiring the strong generalist and growing the specificity.
• Who else is bidding matters more than the title. If your candidate pool overlaps with companies that pay at the top of your market, you are competing with them whatever your internal band says.
• Look it up for your own market, from more than one source. Published salary bands in adverts in your city, your national salary surveys, sector-specific compensation reports, and — most usefully — what your own last two engineering hires accepted. Date everything you find; a figure from eighteen months ago is an estimate, not a fact.