What to keep after a wrong AI answer

When a customer has already seen a wrong answer, keep enough of that one request that a person who was not in the room can reconstruct what the model was asked, which model answered, what it was allowed to do, and what the customer was shown. If you cannot do that for the complaint in front of you, you do not have an incident. You have a mood, and a mood cannot be fixed twice the same way. The record is not a research programme. It is a short list of fields you decided to store before anyone was angry, and a retention period at least as long as the customer can still ask about that answer.

This is not the question of why a retrieval demo falls over in production, and it is not the question of how to build the golden set that scores the system before release. Those are real, and they are different articles: RAG is a data problem is the first, and the evaluation suite is the second. A suite tells you a rate moved. It does not reconstruct the answer one person already read. Chunk sizes, embedding models, and pass thresholds will not help you on the call you are about to join. The customer is not asking for your architecture. They are asking what happened at 14:07.

NIST's Generative Artificial Intelligence Profile, published in July 2024 as a companion to the AI Risk Management Framework, puts the obligation in one suggested action. Under GOVERN 1.5, action GV-1.5-003 is to "Maintain a document retention policy to keep history for test, evaluation, validation, and verification (TEVV), and digital content transparency" for generative AI (NIST, AI 600-1, 26 July 2024). A neighbouring action, GV-1.6-003, says a system inventory should be able to name underlying foundation models, the versions of those models, and the access modes, alongside data provenance such as source, signatures, and versioning. Read together, they are a government standards body saying: keep the history, and know which model it was. They do not specify your log schema. They make "we don't store that" a governance gap rather than a storage preference.

The field list that engineers actually emit is being written down in the OpenTelemetry semantic conventions for generative AI calls. In the v1.41.0 conventions — still marked Development, so treat them as a shared vocabulary rather than a finished law — `gen_ai.request.model` is conditionally required when it is available, and "if the model is supplied by a vendor, then the value must be the exact name of the model requested." `gen_ai.response.model` is separate, and recommended: the exact name of the model that actually answered, which is not always the name you asked for. A response identifier, finish reasons, and input and output token counts are recommended. The messages themselves, and the system instructions, are marked Opt-In (OpenTelemetry, GenAI spans, semantic conventions v1.41.0).

That opt-in is the trap. A trace that satisfies the convention's default can tell you the model name and the token count and still be useless on the call, because the customer is complaining about words. The words were left off on purpose: prompts contain the customer's data, and spraying them into every engineer's debug view is its own incident. The decision is not whether the convention requires the text. It does not. The decision is that you keep the input that was sent and the output that was shown, in a store you control, with access limited to the people who handle the complaint. Exact model names on the trace. The words beside them, not on the billboard.

Write the minimum down so it can be checked on a boring Wednesday, before you need it. A request identifier you can quote back to the customer. The time. The account or user it belonged to, which is not the same as their name pasted into a log. The exact model requested and the exact model that answered. The input as sent, not a paraphrase written after the complaint. The output as shown, which may differ from the model's output if a person edited it, and that edit is part of the record. If a tool was called, the tool's name and the result that came back — not a novel about the tool, the fact of the call. Token counts, because the cost of this answer is sometimes the reason it was wrong: a truncated context, a retry storm. And a version or a hash of the instructions the model was given, so "we changed the prompt last week" can be confirmed rather than remembered.

How long is a policy, not a vibe. NIST's action is to have one. A workable floor for a product that only answers questions: keep the record at least as long as that customer can still complain about that answer. If your refund window is thirty days, thirty days is the floor for a wrong sentence, and it is a floor, not a suggestion to delete on day thirty-one when the answer was medical, financial, or something the customer acted on. If the system can write — create a ticket, send an email, change a row — the record includes the before and after of the write, and it lives as long as that row can still be disputed. What breaks when AI meets a system of record is the catalogue of those writes. This article only insists that the write and the answer that caused it can be put back next to each other.

Two stores get confused, and the confusion is how teams end up with neither. An evaluation set is a sample you chose, scored on a schedule, to see whether the system is drifting. A complaint record is one request a named person already saw. You want both, and they answer different emails. The evaluation set will not contain the customer's question unless you are luckier than you deserve. The complaint record will not tell you whether this was the tenth failure today unless you also count. Keeping every prompt forever in a spreadsheet named final-final is not a policy. It is how customer data ends up in someone's downloads folder. Name the store, name who can open it, and name the day it is deleted.

The delivery-performance numbers people reach for here are easy to misuse, so the columns are worth separating. In DORA's 2024 report, nearly 3,000 working professionals fell into clusters the researchers did not set in advance. The elite cluster's change fail rate was 5%, and failed-deployment recovery was under an hour. The low cluster's change fail rate was 40%, and recovery sat between one week and one month. The low cluster was 25% of respondents, with an 89% uncertainty interval of 23–26% on that share — the interval is not attached to the 40% (DORA, Accelerate State of DevOps Report 2024). A wrong model answer is not the same event as a deployment that needs immediate remediation. The transferable part is the recovery column. Under an hour is only possible when the record of what just shipped already exists. A week or a month is what it feels like to reconstruct a request from a chat thread and a guess.

Run the reconstruction once, on a request you pick, not on a request a customer picks. Take an answer from yesterday and produce the list without asking the person who was on call. If you cannot, the gap is specific: usually the input was never stored, or the model name was stored as "the latest," which is not a name. Fix the gap before you write a longer policy. A policy that describes fields you do not capture is a brochure.

There is a system for which this list is the wrong purchase. Nobody outside the team has seen an answer. You are still finding out whether the idea works, and a wrong sentence costs an afternoon of your own. Build the evaluation habit first, on examples you chose, and do not stand up a complaint archive for users you do not have. The free production-readiness diagnostic is aimed at the moment you are about to claim the system is ready, not at the moment a customer is already holding a bad answer. If you are in the second moment, the archive is late, and late is still better than absent: start with today's request, and accept that yesterday is gone.

The offer that matches a pilot which already reached a user and cannot explain itself is not another demo. It is the work of making the next wrong answer reconstructable, and of deciding whether the current one should still be in front of people. That is what we mean by an AI pilot failed to reach production even when something is technically "live": live without a record is a demonstration that happens to have customers. The capability underneath the record — a scored set, a gate, and a place the trace goes — is AI evaluation and observability. One without the other is how teams know their pass rate and still cannot answer 14:07.

What the record looks like when it was built in from the start, rather than after the complaint, is visible in the shape of the product and not in a headcount. A multi-assistant platform we built included prompt monitoring with dashboards and alerts, because a platform that runs more than one assistant cannot explain a bad answer from memory of which assistant it was. The monitoring is the point. Figures for how long that took, or how many people, are not, and several of those figures on this site remain estimates. If your product has one assistant, the same shape is smaller: one model name, one input, one output, one identifier, kept on purpose.

When the customer writes in, reply with the identifier first, not with an apology essay. Either you can pull the request, or you say you cannot and you say what you will keep from this hour forward. Inventing a reconstruction is worse than the original wrong answer, because the customer will compare it with what they saw. The fields are ordinary. The failure mode is deleting them, or never having opted in to the words, and then calling the gap a model problem. The model may also be wrong. You cannot know which, at 14:07, unless the request is still there.