What LLMs are good and bad at
A product-focused map of LLM strengths and weaknesses, from language tasks to tool use, and where models still fail, so you scope features realistically.
On this page
Why this matters
Most AI feature failures aren’t bugs in your code. They’re scoping mistakes: a feature that needs something the model doesn’t do reliably, discovered after the launch announcement. Knowing the shape of what models do well, and where they still slip, lets you scope features that work on the first try and design around the weak spots.
This article is a product engineer’s map, not a benchmark report. Capabilities improve quickly, so treat it as a set of questions to ask and then check against your own inputs, which is what the last section shows you how to do.
Strong fits
Language models are strongest when the task is mostly about understanding and producing language, and when an occasional imperfect answer can be caught or corrected. These are reliable starting points for product features:
| Task | Product examples | Why it fits |
|---|---|---|
| Drafting | Replies, descriptions, release notes | A draft only needs to be a good start; the user finishes it |
| Summarizing | Threads, meetings, documents | Compresses language; errors are visible against the source |
| Extracting | Fields from emails, invoices, contracts | Clear target structure; output can be validated in code |
| Classifying and routing | Tickets, feedback, leads | Small output space; easy to measure accuracy |
| Transforming | Tone, format, translation, simplification | The meaning is given; the model only changes the form |
| Answering from provided sources | Help-center Q&A, document Q&A | Grounded in text you supply, so claims can be cited |
| Writing and explaining code | Snippets, refactors, explanations | Verifiable with tests, linters and compilers |
The common thread: the input contains most of what’s needed, and the output can be checked, by the user, by code, or against a source.
Weak fits
Models are weaker, or need extra machinery, when the task depends on something the model doesn’t have or can’t guarantee:
- Facts that aren’t in the input. A model’s built-in knowledge has a cutoff date and gaps, and models can state wrong facts fluently. This is often called hallucination. For anything factual that matters, provide the source in the context or retrieve it.
- Exact results every time. Models can get precise tasks right most of the time, which isn’t the same as always. Totals, counts, dates and IDs need code to compute or verify them.
- Your private, live state. The model doesn’t know a user’s account, today’s inventory or what happened five minutes ago unless you send it. Tools and retrieval close that gap.
- Long chains of steps without checks. Each step can be slightly wrong, and errors compound. Break work into steps you can verify, or give the model tools to check itself.
- Consistency across runs. The same input can produce different outputs. Fine for drafts; a problem when users expect the same answer twice. Structured outputs, clear instructions and storing results you’ve already generated help.
- Knowing what it doesn’t know. Models don’t reliably signal when they’re unsure. Design for that in the UX, as covered in communicating uncertainty.
None of these makes a feature impossible. Each tells you what to add: a source, a tool, a validation step, a human check.
Show text descriptionHide text description
A two-by-two grid. The vertical axis is language and judgment in the task, increasing upwards. The horizontal axis is tolerance for imperfect output, increasing to the right.
- top left (high language and judgment in the task, low tolerance for imperfect output): Assist a human. Language-heavy but errors are costly: reviewing contracts, medical notes. The model drafts or flags; a person decides.
- top right (high language and judgment in the task, high tolerance for imperfect output): Strong fit. Drafting, summarizing, classifying, transforming. Start here. This is the recommended quadrant.
- bottom left (low language and judgment in the task, low tolerance for imperfect output): Use code. Exact, structured and unforgiving: billing, permissions. Models can help parse input at most.
- bottom right (low language and judgment in the task, high tolerance for imperfect output): Code first, model optional. Structured but forgiving: suggestions, tagging. Rules often suffice; a model handles the long tail.
Tools change the map
The weak fits above shrink considerably when you give the model help:
- Retrieval supplies facts the model doesn’t have: your documents, your catalog, today’s data. Answers become grounded and citable. Context, retrieval and memory covers the trade-offs.
- Tool use lets the model ask your code to do things it does badly or can’t do: look up an order, run a calculation, query a database. The model decides what to do; your code does it exactly.
- Code execution lets the model write and run code for analysis and arithmetic instead of estimating.
- Structured outputs constrain the response to a schema your code can rely on.
This is why “can a model do X?” is the wrong question. The right one is “can a model, plus the sources and tools I can give it, do X reliably enough for this job?” The answer is yes much more often, at the price of more engineering and more to secure.
Testing a capability yourself
Public benchmarks measure general ability on someone else’s tasks. Your feature succeeds or fails on yours. Before committing to a feature, run a quick capability probe:
- Collect 10 to 20 real inputs, including the messy and awkward ones. Invented inputs flatter the model.
- Write the simplest prompt that describes the task, with any sources the feature would have.
- Run every input with the model you’d use in production.
- Read every output and mark it good, fixable or wrong. Note why each wrong one failed.
- Look at the failures. Missing information? Add a source. Exact values wrong? Add a tool or a check. Inconsistent? Add structure. Simply beyond the model? Rescope.
An hour of probing will tell you more about feasibility than any amount of reading. Prompt-first prototyping turns this into a repeatable harness.
Common mistakes
- Trusting the model’s own knowledge for facts that matter. Provide sources or retrieve them, and show them to users.
- Asking for exact results without checks. Compute totals, dates and IDs in code, or verify what the model produced.
- Testing on invented inputs. Clean, made-up examples overstate quality. Probe with real, messy inputs.
- Judging feasibility by benchmarks. General scores don’t predict performance on your task. Run your own probe.
- Treating limitations as fixed. Many weak fits become strong ones with retrieval, tools or structured outputs. Ask what’s missing before rejecting a feature.
Try it
Exercise
About 45 min
Probe a model on ten realistic inputs for your feature
- Collect ten real inputs for the feature you specified in Product Foundations.
- Write a simple prompt and run each input through Claude Opus 5.5, in the Console workbench (opens in a new tab) or with a short script.
- Mark each output good, fixable or wrong, with a one-line reason.
- For each wrong output, decide what would fix it: a source, a tool, structure, a better prompt, or rescoping.
Show a hint
Include at least three inputs you expect to be hard: unusually long, ambiguous, badly formatted or in an unexpected language. Those reveal the limits fastest.
Show a worked solution
Example results for “extract the order number and requested action from support emails”:
- 7 good, 2 fixable, 1 wrong.
- Fixable: two emails mentioned several order numbers. Fix: structured output with a list of order numbers, plus a “which one is the subject” field.
- Wrong: the model invented an order number for an email that had none. Fix: allow a null value explicitly in the schema, and validate every extracted number against the orders database with a tool.
- Conclusion: a strong fit once nulls are allowed and numbers are verified in code.
Check your understanding
Key takeaways
- Models are strongest when the input contains what’s needed and the output can be checked.
- Weak spots are facts not in the input, exact results, private live state, long unchecked chains and knowing when they’re unsure.
- Retrieval, tools, code execution and structured outputs turn many weak fits into strong ones.
- Confirm feasibility with a probe on 10 to 20 real inputs, not benchmarks.
Progress is saved in this browser and lights up the roadmap.
Related articles
- Choosing a model: a decision framework Choose the right model for each AI feature with a repeatable framework that weighs quality on your evals, latency, cost per success, limits and effort settings. Level: Intermediate
- Prompt-first prototyping Prototype an AI feature with nothing but prompts and real inputs before writing app code, and learn whether the core idea works in an afternoon. Level: Intermediate
- Estimating cost per task Estimate what an AI feature costs per task and per successful task, from token counts and model prices to retries and caching, with a reusable cost calculator. Level: Intermediate