Build, buy or fine-tune
Decide whether to build on a model API, buy an existing AI product, or fine-tune a model, with the costs, risks and evidence each option needs.
On this page
Why this matters
Two of the most expensive AI mistakes are opposites. One is building, over months, something you could have bought in an afternoon. The other is buying a tool for the one capability that’s supposed to make your product different, and discovering you can’t shape it to your users.
A third option, customizing or fine-tuning a model, is often discussed and less often justified. This article lays out the three options, when each fits, the evidence to gather before committing, and a checklist you can run in a meeting.
Three options
| Option | What it means | Example |
|---|---|---|
| Buy | Use an existing AI product or service as-is, or with light configuration | A support platform’s built-in reply suggestions; a transcription service |
| Build on a model API | Build your own feature: prompts, retrieval, tools, UX and evals around a general model | Drafting replies that use your help center and your tone, inside your product |
| Customize the model | Change the model itself with your data, through fine-tuning where a provider offers it, or by training your own | A model tuned to a narrow, high-volume format |
Most AI product engineering is the middle row: building on a general model through an API, and putting the effort into everything around it. That’s where this wiki spends most of its time, for a simple reason: it’s where most teams can create differentiated value at a reasonable cost.
When to buy
Buy when the capability is a commodity for your business: something many products need, that isn’t why customers choose you, and where good options already exist.
Signs buying is right:
- The job is generic: transcription, translation, spam filtering, generic meeting notes.
- Speed matters more than control. You need it this quarter, not after a build.
- A vendor’s product does most of what you need, and the gaps don’t matter to your users.
- You’d rather spend engineering time on what makes your product different.
What to check before buying:
- Data handling: what the vendor stores, for how long, and whether it’s used for training. See data privacy.
- Quality on your inputs: run your own probe set through the product, not their demo.
- Cost at your volume: per-seat or per-use pricing can look cheap at pilot scale and expensive at full scale.
- Exit plan: how hard would it be to switch or build later? Can you export your data?
When to build on a model API
Build when the capability is close to your core value, depends on your data or workflow, or needs a user experience no vendor offers.
Signs building is right:
- The feature needs your context: your documents, your users’ data, your product’s actions.
- How it behaves is part of your product’s identity: tone, judgment, integration into your UX.
- You need to measure and improve it continuously with your own evals and metrics.
- Vendors exist but none fits your workflow without awkward compromises.
Building on an API is less work than it sounds. You’re not training anything: you’re writing prompts, assembling context, designing the experience and measuring it. The heavy lifting of the model itself is the provider’s.
When customizing a model is worth it
Fine-tuning and custom models are tempting because they sound like the “real” AI work. In practice, they’re worth it only in specific circumstances, and only after cheaper options have run out.
Customization is worth considering when all of these are true:
- You’ve exhausted prompting, context and model choice. Your evals show a persistent gap that better prompts, examples, retrieval and a more capable model haven’t closed.
- The task is narrow and stable. A consistent format or domain that won’t change every month.
- You have enough high-quality examples. Typically far more labeled, reviewed examples than a prompt can hold.
- The volume justifies it. Lower per-request cost or latency at your scale outweighs the cost of training, evaluating and maintaining a custom model.
- You can own it. Someone will retrain, re-evaluate and redeploy it as data, requirements and base models change.
That last point is the most underestimated. A custom model is a long-term maintenance commitment. When a new general model arrives that’s better at your task out of the box, a fine-tuned older model can become a liability. Availability also varies: which models can be fine-tuned, and where, depends on the provider and platform, so check current options before planning around it.
Show text descriptionHide text description
A sequence of yes-or-no questions, asked in order. A "yes" answer stops with the outcome shown; a "no" moves to the next question.
- Is this capability a commodity that does not differentiate your product? If yes: Buy, after checking data handling, quality on your inputs and cost at your volume.
- Can a general model with good prompts, context and tools meet your quality bar on your evals? If yes: Build on a model API.
- Is the task narrow and stable, with plenty of reviewed examples and enough volume to pay for it? If yes: Consider customizing a model, with an owner for retraining.
If the answer to every question is no: Rescope the feature, or keep a human in the loop for the part models can't do yet.
A decision checklist
Run these questions with your team before committing. Write the answers down; they become the rationale section of your spec.
- Differentiation: would customers choose us partly because of this capability?
- Data: does it depend on our data, our users’ data or actions in our product?
- Fit: has anyone tested the best vendor option and a quick API prototype on our real inputs?
- Quality gap: if we’re considering customization, what eval result shows prompting and model choice aren’t enough?
- Cost at scale: what does each option cost per successful task at our expected volume, including people’s time?
- Ownership: who maintains it a year from now? What happens when a better model or vendor appears?
- Exit: how hard is it to change our mind later?
Notice that question 3 asks for a test of both buying and building. A day spent prototyping each usually settles the debate faster than a week of discussion.
Common mistakes
- Building commodities. Months spent on a generic capability a vendor already does well is time not spent on what makes you different.
- Buying your differentiator. If the AI capability is why customers choose you, a generic vendor tool limits how good it can get.
- Fine-tuning first. Customizing a model before exhausting prompts, context and model choice is expensive and often unnecessary.
- Judging vendors by their demos. Test every option on your own real inputs and quality bar.
- Forgetting ownership. Custom models and in-house builds need maintenance. Plan who owns them before committing.
Try it
Exercise
About 30 min
Run the build-buy-fine-tune checklist on a feature
- Pick an AI capability your product needs or might need.
- Answer the seven checklist questions in writing.
- Walk it through the decision path.
- Write a two-sentence recommendation, including what evidence would change your mind.
Show a hint
If you’re unsure whether something differentiates you, look at why customers chose you in sales notes or reviews. If the capability never comes up, it’s probably a commodity for you.
Show a worked solution
Example: “meeting transcription and summaries” in a project management product.
- Transcription is a commodity: buy a transcription service, after checking data retention.
- Summaries that link decisions to the product’s tasks and owners depend on product data and workflow: build that part on a model API.
- Customization isn’t justified: no eval gap, no evidence general models fall short.
Recommendation: buy transcription, build the task-linked summaries. Revisit if evals show summaries consistently miss decisions that a tuned model could catch.
Check your understanding
Key takeaways
- Buy commodities, after checking data handling, quality on your inputs, cost at scale and how to exit.
- Build on a model API when the capability depends on your data, workflow or user experience. That’s most AI product work.
- Customize a model only when evals prove general models fall short, the task is narrow and stable, and someone will own it.
- Settle debates with a quick prototype of each option on real inputs.
Progress is saved in this browser and lights up the roadmap.
Related articles
- AI product engineer vs AI, ML, product and full-stack roles How the AI product engineer differs from AI engineers, ML engineers, product managers and full-stack engineers, and the path into the role from each. Level: Beginner
- Unit economics of AI features Work out the unit economics of an AI feature, from cost per successful task and value per user to margins by plan and the levers that improve them. Level: Expert
- What LLMs are good and bad at A product-focused map of LLM strengths and weaknesses, from language tasks to tool use, and where models still fail, so you scope features realistically. Level: Intermediate