Your AI works in the demo. Production is a different set of conditions: adversarial inputs instead of curated ones, concurrent tenants instead of one friendly user, retrieved documents and tool outputs that carry instructions instead of trusted text. This is the checklist we use to find where a working prototype breaks before real users and enterprise security reviewers do — 28 checks grouped by the surface they defend, each one specific enough to score.
Why AI demos break in production
A demo runs under conditions you control. You pick the inputs, you drive the happy path, the document set is small, and there is exactly one polite user — you. Production removes every one of those guarantees at once, and each removed guarantee maps to a failure class:
- Curated inputs become hostile ones. The demo never sends a prompt-injection payload, a 40,000-token paste, or a request in a language the eval never saw. Production does, on day one.
- The determinism illusion. It worked five times in the demo. The sixth run, with temperature unpinned, drifts into a broken format or a confident wrong answer. If you never measured variance, you never had reliability — you had luck.
- The trust boundary collapses. In the demo, all text is trusted because you wrote it. In production, retrieved documents and tool outputs reach the model with the same authority as the user’s message — the root of indirect prompt injection (OWASP LLM01).
- One user becomes many tenants. Retrieval scoped by the UI, not the data layer, leaks across tenants the moment two customers share an index. The demo never had a second tenant to leak to.
- Tools gain a blast radius. A tool that only reads in the demo can write, pay, or delete in production, and an agent steered off-path now holds real credentials.
The checklist below is organized so each surface maps to the demo guarantee it removes. Treat every item as a claim you must be able to prove with a reproducible artifact — a request trace, an eval run, an access-control test — not a screenshot of a vibe.
The checklist
Twenty-eight checks across seven surfaces. Items marked (G) are gates: they must be a full pass before you ship, regardless of your total. Scoring comes after the tables.
Model behavior and evals
| # | Check | What “done” means |
|---|---|---|
| 1 (G) | Offline eval set exists | An eval set built from real or realistic user inputs runs on every model, prompt, or retrieval change, with a written pass threshold recorded per release. |
| 2 | Variance is measured | Temperature, top-p, and seed are pinned for eval runs; output variance is measured, not assumed away. |
| 3 | Refusal behavior tested both ways | The system is tested for over-refusal (blocking valid requests) and under-refusal (answering ones it should decline). |
| 4 | Known failures have regression cases | Each observed failure mode — hallucinated facts, format drift, truncation on long inputs — has at least one case in the eval set that would catch its return. |
Prompts and injection surface
| # | Check | What “done” means |
|---|---|---|
| 5 (G) | Injection tested at every entry point | Direct and indirect prompt injection are tested against every place untrusted text reaches the model — chat, retrieved docs, tool output, uploaded files — not just the chat box (OWASP LLM01, MITRE ATLAS). |
| 6 | No secrets in the prompt | No API keys, credentials, or private instructions live in the system prompt; the prompt is treated as recoverable and contains nothing you cannot afford to leak. |
| 7 | Untrusted input is enumerated | Every channel through which untrusted text enters the context is listed and handled as hostile by default. |
| 8 | Actionable output is contained | Output that can trigger effects — markdown links, HTML, tool calls — is filtered or sandboxed so an injected instruction cannot exfiltrate data or drive a side effect. |
RAG and data boundaries
| # | Check | What “done” means |
|---|---|---|
| 9 (G) | Retrieval is access-scoped | Retrieval is filtered to the caller’s tenant and permissions at query time by the data layer — the vector store enforces access control, not the UI. |
| 10 | Ingested docs are sanitized | Documents are sanitized on ingest so embedded instructions in source content cannot hijack downstream generations (indirect injection). |
| 11 | Cross-tenant leakage is tested | A cross-tenant probe confirms one customer’s query cannot surface another customer’s data through the retrieval path. |
| 12 | Sensitive fields are classified and redacted | PII and secrets are classified before indexing; prompts and logs redact them (OWASP LLM02 Sensitive Information Disclosure). |
Agent tool-use
| # | Check | What “done” means |
|---|---|---|
| 13 | Every tool has a written contract | Each callable tool documents allowed inputs, side effects, and blast radius if misused. |
| 14 (G) | High-impact tools are least-privilege | Writes, payments, deletes, and external sends require confirmation or run under scoped credentials — never the app’s full permissions (OWASP Agentic Security Initiative). |
| 15 | Loops and cost are bounded | Step limits, budgets, and timeouts prevent an agent from running away into an unbounded loop or runaway spend. |
| 16 | Tool output is validated before reuse | A tool response is validated before it re-enters the context; a compromised or manipulated response cannot escalate into new privileged actions. |
Auth and tenancy
| # | Check | What “done” means |
|---|---|---|
| 17 (G) | Every model-facing endpoint is authorized | Authentication and authorization are enforced server-side on every path to the model and its tools; there is no unauthenticated route in. |
| 18 | Rate limits and quotas per caller | Per-user and per-tenant limits stop one caller from exhausting the model budget or degrading service for others. |
| 19 | Isolation holds across the full path | Tenant isolation is verified across prompt, retrieval, cache, and logs — not just the primary database. |
| 20 | Credentials live in a secret manager | Model and provider keys are stored in a secret manager, rotated, and never shipped to the client or embedded in a prompt. |
Infra and monitoring
| # | Check | What “done” means |
|---|---|---|
| 21 | Requests are traceable end to end | Every request stores input, retrieved context, tool calls, and output under an ID you can replay — evidence, not screenshots. |
| 22 | Cost and latency are alerted | Per-request cost and latency are monitored with anomaly alerts; a spike often signals injection or a loop before a human notices. |
| 23 | Failure degrades safely | Timeouts, retries, and fallbacks are defined for provider outages; a model failure returns a safe response, not a stack trace. |
| 24 | Logs are reproducible and redacted | Logs capture enough to reconstruct an incident while redacting secrets and PII, under a defined retention policy. |
Release gates
| # | Check | What “done” means |
|---|---|---|
| 25 (G) | Evals gate the release | A prompt, model-version, or retrieval change cannot ship without passing the eval set — it is wired into CI, not run by hand. |
| 26 | A red-team pass has run | A red-team pass covering OWASP LLM Top 10 (plus the Agentic Top 10 where tools are used) has run against a production-like build. |
| 27 (G) | Rollback and kill switch exist | A written rollback plan and a kill switch can disable the AI feature or fall back to a non-AI path within minutes. |
| 28 | The feature is owned in production | A named owner, an on-call path, and an incident runbook exist, aligned to the NIST AI RMF functions (Govern, Map, Measure, Manage). |
How to score it
Score each item on three levels so the result reflects evidence, not intent:
- 0 — not in place.
- 1 — partial, or in place but undocumented and unproven.
- 2 — in place with a reproducible artifact you could hand to a reviewer (an eval run, a trace, a passing access-control test).
Then apply two rules in order:
- Gates are pass or fail, not averaged. Any item marked (G) that scores below 2 blocks release on its own. The gates protect the failure classes that turn into incidents: a broken eval loop (1), untested injection (5), unscoped retrieval (9), over-privileged tools (14), an unauthorized endpoint (17), a missing CI eval gate (25), no rollback path (27). A high total with a red gate is one unhandled failure class away from a bad day.
- Read the shape, not just the sum. With all gates green, a list of mostly 2s with a few documented 1s is a production candidate under monitoring. Zeros and ones clustered in one surface tell you where to harden next — an all-red RAG column means retrieval boundaries are your live risk, whatever the rest looks like.
Score honestly. A 1 you rounded up to a 2 is the check you fail in front of the enterprise reviewer, when it costs the deal instead of an afternoon.
When to bring in an independent assessment
Self-scoring has a structural blind spot: the team that built the system shares the assumptions that make its weaknesses invisible. You cannot injection-test a trust boundary you did not know you crossed, and an internal review rarely attacks its own design as hard as an adversary will. That is the gap an independent assessment closes.
Bring in an outside assessment when any of these is true:
- An enterprise customer’s security review is between you and the contract, and you need adversarial evidence mapped to a framework their team recognizes.
- Your system gives an agent real tool access — writes, payments, external sends — where a single failure has a blast radius.
- You are moving an AI-built prototype toward production and want the failure modes found before launch, not by users after it. That is the work in prototype-to-production hardening.
An AI product readiness assessment runs this checklist as an adversary would, and every finding carries reproducible evidence — request traces, not screenshots of a vibe. We reduce risk and prove exactly what we tested. We will not sell you a certificate that your AI is “safe” — no one can. What you get is a scored map of your real surface and the traces to back it, which is what survives a review.
See how we work for the process behind each surface, and go deeper on the two surfaces most teams under-test: LLM red teaming and prompt injection testing.