Glossary
Plain-language definitions of the terms used across the AI Product Engineer Wiki, from acceptance rate to zero-shot prompting, with links to each topic.
On this page
How to use this glossary
Short, plain-language definitions of the terms used across the wiki, each linking to the article that covers it in depth. Terms are grouped by first letter.
A
Abstention. When an AI feature declines to answer, or answers only part of a question, because it lacks what it needs. Done well, it says what’s missing and what to do next. See communicating uncertainty.
Acceptance rate. The share of AI outputs that users actually use (send, insert, keep), out of those shown. A core behavior metric. See metric definitions.
Adaptive thinking. A mode where the model decides how much to reason before answering. Thinking tokens are billed as output. See estimating cost per task.
A/B test. An experiment that randomly splits users between variants to measure which performs better on real behavior. See A/B testing AI features.
Approval flow. A step where a person confirms an AI-proposed action before it runs. See human-in-the-loop and approval flows.
B
Baseline. The performance of the current or non-AI solution, which an AI feature has to beat. Also the stored eval scores a CI run compares against. See when not to use AI.
Batch processing. Sending non-urgent requests to be processed asynchronously at a discount. See estimating cost per task.
C
Calibration. Checking that a grader, such as an LLM judge, agrees with human grades, and adjusting it until it does. See LLM-as-judge.
Citation. A reference from a claim in an answer to the passage in a source that supports it. See citations and showing sources.
Cohen’s kappa. A measure of agreement between two graders that corrects for agreement by chance. See rubrics and human grading.
Context window. The maximum amount of text, in tokens, a model can take into account in one request, including its response. See context windows, retrieval and memory.
Cost per success. Total model cost divided by the number of successful outcomes. The unit that matters for AI economics. See unit economics.
E
Edit ratio. How much of an AI output a user changed before accepting it, from 0 (kept as is) to 1 (rewritten). See editable outputs and undo.
Effort. A setting that controls how much a model reasons before answering, trading depth against cost and latency. See choosing a model.
Eval (evaluation). A set of cases, expectations and graders that scores how often an AI feature is good enough. See why evals are product metrics.
F
Feature flag. A setting that turns a feature, prompt version or model on or off for a share of users without a deploy. See feature flags and staged rollouts.
Fine-tuning. Further training of a model on your own examples. Worth it only in specific cases. See build, buy or fine-tune.
G
Golden dataset. A curated set of real inputs, each with what a good output must do, used for evals. See building golden datasets.
Grounding. Basing a model’s answer on sources provided in the request, rather than on its built-in knowledge. See citations and showing sources.
Guardrail. A deterministic check around a model that makes specific failures impossible, at input, output or action. See guardrails.
Guardrail metric. A metric that must not get worse while you try to improve the primary one. See defining success metrics.
H
Hallucination. A fluent, confident statement that is false or unsupported. Reduced by grounding and caught by citations and evals. See what LLMs are good and bad at.
Held-out set. Part of an eval set used only for reporting, never for tuning prompts, to detect overfitting. See linking eval scores to business KPIs.
J
Job to be done. The progress a person is trying to make in a particular situation. See jobs-to-be-done for AI products.
K
Kill switch. A flag that turns an AI feature off for everyone in seconds, falling back to the non-AI path. See feature flags and staged rollouts.
L
Latency budget. The time an interaction can take before the experience suffers. See latency budgets and perceived speed.
Lethal trifecta. The combination of private data, untrusted content and a way to send data out, which makes data exfiltration through prompt injection possible. See prompt injection.
LLM-as-judge. Using a model to grade AI outputs against a rubric. See LLM-as-judge.
M
max_tokens. The maximum number of output tokens for a response. Hitting it cuts the answer off, with stop_reason: "max_tokens". See error, refusal and empty states.
Must-never-fail case. An eval case where a wrong answer causes real harm, held to a 100% threshold. See regression eval suites in CI.
P
Prompt caching. Reusing the processed beginning of a prompt across requests, at reduced cost and latency. See context windows, retrieval and memory.
Prompt injection. Instructions planted in text a model reads, trying to override yours. See prompt injection and tool-use risks.
Prompt version. An identifier for a specific prompt, recorded with every request so outcomes can be attributed. See the iteration loop.
R
Rate limit. A cap on how many requests, or how much spend, a user can make in a period. See abuse, misuse and rate limiting.
Refusal. A request declined by the model or its safety measures, reported as stop_reason: "refusal". See error, refusal and empty states.
Retrieval (RAG). Finding the passages relevant to a request and adding them to the model’s context. See context windows, retrieval and memory.
Rubric. A list of specific, observable criteria used to grade outputs consistently. See rubrics and human grading.
S
Staged rollout. Exposing a feature to a growing share of users, with exit criteria at each stage. See feature flags and staged rollouts.
Stop reason. The field on a response that says how it ended: finished, cut off, tool call, refusal and others. See the model selection cheat sheet.
Streaming. Showing a response as it’s generated, token by token. See streaming and loading states.
Structured output. A response constrained to match a schema, such as JSON with specific fields. See structured outputs and tool calls.
T
Time to first token. How long until the first output appears; what users feel as the wait. See latency budgets.
Token. The unit models read and write, often part of a word. Models are priced per input and output token. See estimating cost per task.
Tool use. Letting a model ask your code to do something, with arguments in a defined shape. See structured outputs and tool calls.
Trace. The record of one AI request: context, outcome, tokens, latency and, where allowed, content. See tracing and observability.
U
Unit economics. The per-unit view of a feature’s money: cost and value per successful task, and margin per user. See unit economics.
W
Wizard-of-Oz test. A test where a person plays the AI behind a realistic interface, to learn what users need before building. See Wizard-of-Oz tests.
Z
Zero-shot prompting. Asking a model to do a task with instructions only, no worked examples. Adding a few examples is called few-shot prompting. See prompt-first prototyping.