When to turn a live AI feature off

Turn a live AI feature off — or, more often, turn off the part that acts — when the next answer can hurt a user and nobody who is awake can stop that answer before it lands. Narrow it when a human can still approve the step that matters: the refund, the send, the write into the system of record. Keep it serving, and spend the day tuning, only when three things are already true. You can reproduce the last bad answer on the same inputs. The blast radius of the next one is a draft a person will read. And someone named is watching the metric you agreed, on a cadence shorter than the harm. If you cannot say which of those you are in, you are not tuning. You are leaving a feature on because turning it off feels like admitting the launch failed.

That decision is not the diagnosis. When a production answer is wrong, five checks decide which module failed, and they have an order: indexed, returned, present in the prompt that was sent, used, and the right shape. What to check when a production RAG answer is wrong is that order, and it says to stop at the first failure. Run it on a captured answer. It does not tell you whether the feature should still be answering the next customer while you run it. Teams lose the afternoon by treating those as one meeting. They are two. The stop rule is allowed to fire before anyone knows which check failed. Waiting for the root cause is how a bad answer becomes a morning's worth of them.

The industries that already do this well write the trigger down before the incident, not during it. Google's site reliability text defines a postmortem as "a written record of an incident, its impact, the actions taken to mitigate or resolve it, the root cause(s), and the follow-up actions to prevent the incident from recurring," and it lists the triggers teams commonly use: user-visible downtime or degradation beyond a threshold, data loss of any kind, on-call intervention such as a release rollback, a resolution time above a threshold, and a monitoring failure, "which usually implies manual incident discovery." Then the sentence that matters for a feature you just shipped: "It is important to define postmortem criteria before an incident occurs so that everyone knows when a postmortem is necessary" (Google, Postmortem Culture: Learning from Failure, Site Reliability Engineering, Beyer, Jones, Petoff and Murphy, eds.). A kill rule is the same kind of object. If the only person who may turn the feature off is the person who shipped it, and they are asleep, you do not have a rule. You have a hope.

The same chapter's standard for the write-up is the reason the rule has to be blameless to be usable. A blameless record "assumes that everyone involved in an incident had good intentions and did the right thing with the information they had." If turning the feature off is treated as an accusation, the person on the hook will tune a prompt at midnight instead, because tuning looks like work and a rollback looks like blame. The review question Google uses is the one to steal: "Was key incident data collected for posterity?" You cannot answer it after someone has edited the prompt that produced the bad answer. The fields that make that one request reconstructable — the model, the input, the output the customer was shown, an identifier — are what to keep after a wrong AI answer. This article is only the decision to stop serving the next one. Change the system after that record exists, not before. The five artefacts the checklist needs are useless if the first edit destroyed them.

NIST's AI Risk Management Framework puts the same duty in the manage function, without giving you the threshold. Manage 4.3: "Incidents and errors are communicated to relevant AI actors, including affected communities. Processes for tracking, responding to, and recovering from incidents and errors are followed and documented." Manage 4.1 puts incident response, recovery, and decommissioning inside the post-deployment monitoring plan, next to appeal and override (NIST, Artificial Intelligence Risk Management Framework (AI RMF 1.0), NIST AI 100-1, January 2023). Decommissioning is on the list on purpose. Not every failure is a prompt tweak. Some failures mean the feature should not be in the path until the record exists. The framework does not prescribe your risk tolerance. It says the process has to exist and has to be written. A Slack thread that starts with "is this bad?" is not that process.

Public catalogues are a poor substitute for the rule, and the numbers show why. The OECD's AI Incidents and Hazards Monitor classifies news-detected events. An AI incident, in the definition it adopted, is an event where development, use, or malfunction of an AI system leads to actual harm: injury, disruption of critical infrastructure, a violation of rights or of a protective legal obligation, or harm to property, communities, or the environment. A hazard is the plausible version of the same list. Since November 2024 the monitor's own method table says that "on average, about 1 000 AI events are analysed daily, resulting in around 10 AI incidents or hazards," and that the monitor "detects on average 30 incidents and hazards per day" (OECD.AI, Overview and methodology of the AI Incidents and Hazards Monitor). Read 10 out of 1,000 as a filter on news, not as your error rate. The OECD says these public reports are a subset, drawn from news articles, classified by models, and not independently verified. Your user's bad answer will not be one of the thirty. It will be in your logs, for as long as your provider keeps them, and then it will be gone. A stop rule that waits for a category of harm large enough to make a newspaper has already declined to protect the person in front of the feature.

Four stop conditions are enough, and they are written so a person who did not build the model can apply them. First: the feature can take an action a human will not see before a customer does, and the last action was wrong. Sending an email, issuing a refund, changing an address, writing a row the ledger will believe. Turn the action off. Leave a draft path on if a person still approves the send. Second: you cannot reproduce the last bad output, because the prompt, the retrieved context, or the model version was not kept, and a customer has already seen it. You are not in a tuning loop. You are in an evidence gap. Stop the unattended path until the next answer is capturable. Third: the metric you agreed — tickets resolved without a reopen, hours, cycle time — has moved the wrong way for a window you defined in advance, not for an afternoon of anecdotes. Fourth: the failure is data loss, or a write the customer cannot undo, which is the SRE trigger "data loss of any kind" translated into a product. Any one of the four is sufficient. You do not need all four, and you do not need a root cause.

Four things that feel like stop conditions and are not. A single user disliked the tone. A founder is embarrassed by a demo. A competitor launched something louder. The model vendor published a newer checkpoint and someone wants to be on it by Friday. None of those is harm, and using them as the kill rule trains the team to ignore the rule when the harm is real, because the rule has already been spent on taste. Park those in a backlog review. The stop rule is for the next answer, not for the roadmap.

The middle path is the one most teams skip, and it is usually the correct one. Full off is right when the feature's only mode is an action, or when you cannot tell a draft from an action in the current design. Narrowing is right when the system can still propose and a person can still dispose. That is a product decision you can make in an hour if the approval step exists, and a rebuild if it does not. An AI support agent we built for a US direct-to-consumer brand is the shape of the narrow path, not a story about an outage: answers grounded in order data, policies, and the catalogue; direct actions such as a lookup, a return, or an address change sitting behind permission guardrails; escalation that carries the conversation to a person when confidence is low. The evaluation suite measures resolution accuracy before and after a release. None of that is a claim that the feature was ever switched off. It is a claim that the off switch was designed as a gate on the action, which is cheaper than discovering you needed one after the first wrong send.

Who is allowed to throw the switch has to be a name, and it should not be only the author. The author is the person most likely to believe the next prompt will fix it, which is sometimes true and is the wrong bias to give a veto. Give the authority to whoever holds the customer relationship that morning and to whoever is on call for the system, either of whom may act alone. Require them to write one sentence: which of the four conditions fired, what they turned off, and when it will be looked at again. Do not require them to convene the people who shipped it. A committee is how a feature stays on until Monday.

What you tell the customer is a different document from the stop itself, and it should not wait for the diagnosis either. The first message says what is affected, what you have confirmed is not affected, what you did, and when the next update is. It does not include the cause, because the first cause is often wrong. What you owe a customer when your product breaks is that message in full, including the difference between a contract credit and a real loss. Turning the feature off is the operational half. The note is the commercial half. Teams that write a beautiful explanation and leave the feature serving have done the half the customer can see and skipped the half that stops the next harm.

There is a point at which the right purchase is not another day of tuning and not a permanent off switch either. The answer is already live, someone has acted on it, and the checks point at a pipeline you cannot see from the chat window: the index, the rank, the prompt that was sent, the claim. An AI pilot failed to reach production is an audit of what is salvageable, in the first one to two weeks, then the unglamorous rest — evaluation, guardrails, permissions, integration — inside an engagement that finishes in eight to twelve weeks. One metric is agreed up front. If the build does not move it, you do not pay for the build. The audit is priced against what is actually there, not against a template. Bring the frozen answer. Without it the first days reconstruct an incident you already lived through, which is the failure the checklist names and this decision is meant to prevent.

Do not buy that engagement for the wrong stall. If no user has ever seen the feature, you do not have an incident. You have a pilot that has not started, and the right instrument is a two-week AI readiness assessment, whose findings are yours whether or not anyone builds the next slice. If the blocker is that nobody can say what a correct answer is, the goal is undefined, and a rescue is the wrong shape for an undefined goal. If you wanted a demo rather than a system that can be turned down on purpose, say so, and do not staff an incident process around a performance. The free production-readiness diagnostic is the cheap way to find out which of those you are looking at, from a description, before you spend the week arguing about a prompt.

Write the four conditions down while the feature is behaving. Put a name next to the authority. Put a date next to the log retention, because the provider's retention is the real deadline on the evidence, and it is a date you can read on their page rather than a number worth inventing here. The first time the rule fires, the person who uses it should be thanked in the same sentence as the fix, or the rule will not be used again. A feature you cannot turn off is not a launch. It is a bet that the next answer will be kinder than the last one, made by someone who will not be the person the answer reaches.