Building an AI product portfolio

Why this matters #

Anyone can now build an impressive AI demo in an afternoon, so demos alone no longer tell a hiring manager much. What’s rare, and what an AI product engineering role actually needs, is evidence of judgment: that you found a real problem, made sensible trade-offs, measured quality, shipped carefully and knew what it cost.

A portfolio is where you show that evidence. This article covers what a strong AI product portfolio contains, how to choose projects, which kinds of evidence convince people, and how to present it so a busy reader gets the point in five minutes.

What a strong portfolio shows #

A reviewer looking at an AI product engineering portfolio is asking a handful of questions, whether or not they say them out loud:

• Can you find a real problem? Not “I built a chatbot”, but “support agents spent their mornings on the same twenty questions”.
• Do you make good decisions? Model choice, what the AI does versus what the user does, when not to use AI at all.
• Do you measure quality? An eval set, a threshold, results, and what you did about failures.
• Can you ship? A working product people used, ideally with a staged rollout and metrics.
• Do you understand cost and risk? Cost per success, limits, privacy, guardrails.
• Are you honest about limits? What didn’t work and what you’d do differently.

Every project in your portfolio should answer several of these. A portfolio that only answers “can you build it?” looks like every other one.

Problem

Who had it, how often, and the evidence

Agents answered ~20 repeated questions daily; 40% of tickets tagged how-to

Decisions

The two or three choices that mattered, with reasons

Drafts, not auto-send; no draft without a matching article

Evidence

Eval results, metrics, cost

88% on a 100-case eval; acceptance 46%; $0.03 per accepted draft

Limits

What didn't work and what you'd change

Billing questions still weak; next step was retrieval over invoices

Show text descriptionHide text description

4 parts, in order:

1. Problem: Who had it, how often, and the evidence Example: Agents answered ~20 repeated questions daily; 40% of tickets tagged how-to
2. Decisions: The two or three choices that mattered, with reasons Example: Drafts, not auto-send; no draft without a matching article
3. Evidence: Eval results, metrics, cost Example: 88% on a 100-case eval; acceptance 46%; $0.03 per accepted draft
4. Limits: What didn't work and what you'd change Example: Billing questions still weak; next step was retrieval over invoices

Choosing projects #

Depth beats breadth. Two or three projects taken all the way through the lifecycle are far stronger than ten demos. One project with a spec, evals, a rollout and a case study shows more skill than a gallery of prompts.

Good sources of portfolio projects, roughly in order of strength:

1. Work you did in your job, written up within what your employer allows. Real users and real stakes are the strongest evidence. If details are confidential, describe the problem, decisions and results in general terms, with permission.
2. A product with real users, however small: a tool for a community, a side project, a feature for a nonprofit. A dozen real users beat a thousand imagined ones.
3. The projects in this wiki. The six projects are designed to build on each other and to produce exactly this kind of evidence. Do them with real data and, where possible, real users, and they become strong portfolio pieces.
4. Open-source contributions to AI tooling or evaluation, which show you can work in someone else’s codebase.

Avoid projects that are a thin wrapper around a model with no problem, no measurement and no users. They don’t hurt, but they don’t help either.

Evidence that convinces #

Claims are easy; evidence is persuasive. For each project, collect:

• Eval results: the size and source of your eval set, your criteria, the threshold, and results before and after key changes. A small, honest table beats a vague “high accuracy”.
• Product metrics: acceptance rate, edit rate, task completion, whatever you defined in the spec, with how you measured it.
• Cost: cost per successful task, and what you did to improve it.
• Screenshots and a short video of the feature in use, including an unhappy path such as a refusal or a cut-off answer handled well. Showing that you designed the failure states is unusually convincing.
• Artifacts: the one-page spec, the change log, a link to the code if you can share it.

Be careful with numbers. Every number should have a source and a sample size, and you should be able to explain how you got it in an interview. One inflated claim discovered in an interview undoes the credibility of everything else.

Presenting it #

Reviewers are busy. Design for a five-minute read with the option to go deeper:

• A short portfolio page, with one card per project: a one-line problem, your role, a key result and a link to the case study.
• A case study per project, structured so the summary at the top tells the whole story. Writing a product case study covers how.
• A two-minute video for your strongest project: the problem, a live walkthrough including an unhappy path, and the result.
• Clear ownership. Say what you did and what others did. Team projects are fine; vague credit isn’t.

Keep everything accessible and fast: a simple, readable page beats an elaborate one. A broken demo link is worse than no demo, so check them regularly or use recorded videos.

Common mistakes #

Try it #

Exercise

About 45 min

Outline your portfolio page

1. Write the one sentence a hiring manager should believe after five minutes.
2. Choose two or three projects, from work, a real product or this wiki’s projects.
3. For each, fill in the four parts: problem, decisions, evidence and limits. Mark any evidence you don’t have yet.
4. Plan the first screen: project cards with one-line problem, role and key result.

Show a hint

The gaps you mark in step 3 are your to-do list. Missing eval results or metrics are usually the most valuable thing to add to an existing project.

Show a worked solution

Example outline:

• Sentence: “Ships AI features that real users rely on, with evals and cost under control.”
• Project 1, support reply drafts (work, with permission): problem from ticket data; decisions on drafting versus sending; evidence from a 100-case eval, acceptance rate and cost per accepted draft; limit: billing questions.
• Project 2, catalog search with citations (wiki project 2, real catalog from a friend’s shop): relevance eval against keyword search; result: better on most query types, worse on exact product codes, so a hybrid was added.
• Missing: a video for project 1; cost data for project 2.

Check your understanding #

Key takeaways #