Hiring Product Builders

TL;DR

• Hire against a scorecard derived from the work, not a fashionable profile or list of tools.
• Recent, specific evidence matters. Employer brand, titles, and polished AI language are context, not proof.
• Use the same structured questions and scoring anchors for every candidate. Add a short, paid work sample only when it tests work the interviews cannot.
• Evaluate judgement, learning, and accountability as well as speed. A prolific builder who cannot review or own the result creates expensive risk.

---

AI changes the work of product management, but it does not remove the need for a fair, valid hiring process. The aim is to find someone who can own the product-builder learning loop: frame a consequential problem, choose the right artefact, build enough to learn, evaluate the result, and operate what ships.

The strongest signal is evidence of that loop in action. The hiring process should make the evidence comparable without demanding that every candidate has followed the same career path.

Start with a work-derived scorecard

Before sourcing, identify the outcomes the person must own in their first year. Translate those outcomes into a small set of observable capabilities.

Define what good evidence looks like at the level you are hiring. Do not add a capability merely because it is fashionable. If the role does not need someone to configure an agent or write code, do not turn that into a hidden requirement.

Source for evidence, not pedigree

Useful candidates can appear in many places: established companies, small teams, public projects, internal functions, domain-heavy industries, consultancies, and independent work. Broaden the lanes, then apply the same scorecard.

Employer brand can indicate experience with scale or strong peers. A public prototype can indicate initiative. Neither proves that the person made the decisions or can repeat the performance in your environment.

Specific outreach works because it connects the candidate's evidence to the job: mention a product decision, project, article, or domain insight and explain why it is relevant. Avoid treating public visibility as a requirement. Caregivers, employees under confidentiality, and people from less public cultures may have excellent private evidence.

Audit the funnel by source and stage. A concentrated funnel is a prompt to inspect reach and criteria, not proof that one demographic or company type is the only place talent exists.

Screen for verifiable specificity

A resume or application should give you claims to test. Look for the relationship between a problem, the candidate's decisions, the artefact or system they changed, the measured result, and what they learned.

Good specificity sounds like this:

We changed retrieval and model configuration after evals showed failures on long policy documents. Cost fell within the target range while quality stayed above the agreed threshold. I owned the test set and rollout decision.

The exact model name matters less than whether the person can explain the choice and evidence. Generic language may reflect weak experience, poor resume writing, confidentiality, or unfamiliarity with the hiring convention. Use a short, consistent screen to distinguish those causes rather than guessing.

Do not use recency as a proxy for age. Ask for recent learning and evidence relevant to the work. A candidate with deep experience and current practice may be stronger than someone who happened to use a new tool last week.

A structured interview loop

Use the shortest loop that gathers independent evidence for the scorecard. A practical design has three parts.

1. Evidence screen

Ask every candidate the same core questions:

1. Walk me through a recent product decision where AI changed the available options.
2. What did you build or change to reduce the most important uncertainty?
3. How did you decide whether the output was good enough?
4. What failed, and what changed because of that failure?
5. What did the complete workflow cost, including human review or rework?

Probe for the candidate's contribution, not just the team's outcome. Record evidence against scoring anchors before discussing the candidate with other interviewers.

2. Work walkthrough

Ask the candidate to explain one real piece of work in depth. Give them options for protecting confidential information: redact details, use an older project, or reconstruct the decision with synthetic data.

Useful follow-ups include:

• What was the customer problem, and what evidence changed your view of it?
• Which uncertainty did you address first, and why?
• What alternatives did you reject?
• What did the prototype or eval fail to represent?
• Where did a specialist change the decision?
• What would you do differently now?

The aim is to understand judgement and ownership. A beautiful artefact is not automatically a good product decision.

3. Collaborative working session

Give the candidate a realistic, bounded problem with enough context to begin. Let them choose how to work, including whether to use AI. Observe how they frame the problem, seek missing context, select an artefact, test assumptions, and review generated output.

Do not reward theatrical speed. A candidate who pauses to identify a safety boundary or asks for customer evidence may be demonstrating better judgement than someone who produces a polished prototype immediately.

Use the same brief, time, available tools, accessibility adjustments, and scoring anchors for comparable candidates. Tell them what is being assessed.

When to use a work sample

A work sample is useful only when it adds evidence the interviews cannot. Keep it short, representative, and bounded. Pay for substantial take-home work at an appropriate market rate, and do not use candidate work as free production labour.

Offer an equivalent alternative when a take-home format creates an accessibility, caregiving, or employment-conflict barrier. Avoid tasks that require candidates to expose employer information or pay for specialised tools.

A work sample might ask someone to:

• Draft an eval plan for a supplied workflow.
• Review a prototype and identify the next uncertainty to reduce.
• Compare two approaches using a small supplied dataset.
• Turn a customer-evidence pack into a product bet and test plan.

Score the decision trail, not just the final polish. Generated presentation quality is easy to obtain; coherent judgement under constraints is more informative.

Separate fluency from tool loyalty

Tools and models change. Strong candidates can explain how they choose, test, and replace them. Look for:

• Deliberate context management
• Verification of generated work
• Evals tied to the user outcome
• Awareness of permissions, privacy, latency, and cost
• Ability to recover when a tool fails
• A learning practice that survives tool churn

Avoid trivia about one vendor interface. If a tool is essential on day one, say so in the role requirements and test only the depth the job actually needs.

Make the decision auditable

Each interviewer should submit evidence and a score before the debrief. Discuss conflicting evidence capability by capability. Do not average away a serious risk, and do not let one charismatic interaction replace the complete record.

Check for common sources of noise:

• Prestige being treated as competence
• Familiar communication style being treated as culture fit
• Fast output being treated as good judgement
• Tool vocabulary being treated as hands-on experience
• One interviewer's enthusiasm becoming everyone else's conclusion
• Unequal access to preparation, tools, or task context

Reference checks should test specific claims with the candidate's consent and follow applicable law and policy. Ask about work, ownership, learning, and operating behaviour rather than inviting vague character judgements.

Offer and onboarding

Set compensation from role scope, labour-market evidence, internal equity, and expected impact. Publish the range early where possible and apply consistent criteria.

Onboarding should continue the evidence loop. Give the new hire a bounded, real problem and enough context to ship a small learning artefact early. Review how they frame, build, evaluate, and collaborate. Do not force a production launch merely to satisfy an arbitrary first-month deadline.

Agree on a development plan using the competency model and AI fluency spectrum. The hiring process identifies a starting profile, not a finished person.

Hiring review checklist

• The scorecard comes from real outcomes and names observable evidence.
• Requirements distinguish essential capability from preferred background.
• Sourcing reaches more than one familiar network or company type.
• Every candidate receives comparable questions, context, time, and scoring.
• Candidates can protect confidential work and request reasonable adjustments.
• Any substantial take-home work is paid and not used as free labour.
• Interviewers record evidence independently before the debrief.
• The decision can be explained without relying on pedigree, charisma, or tool-name recognition.
• Compensation and onboarding use consistent, explicit criteria.

The anti-pattern: hiring the performance of AI fluency

The candidate names every current tool, produces a polished prototype quickly, and speaks confidently about agents. The loop never tests whether they framed the right problem, reviewed the output, understood the economics, or owned a failure.

Three months later, the team has more artefacts and less clarity. Speed was real; judgement was assumed.

Hire the learning loop. Tools amplify the person operating it.