When the user rewrote the rule

The user's message told the model to ignore the instruction you wrote, or you cannot tell whether it did. Stop serving the sentence. Stop any tool that ran after that message. Show the instruction that was supposed to win, show the user text as it was concatenated into the prompt, and show whether a tool was called. If you cannot show the prompt that was sent, write that, and do not ask the model whether it obeyed. Asking the model replaces the prompt you have not opened with a confession you have not checked.

This is the instruction that lost, or might have lost. It is not an error treated as a yes. When the guard fails open is the night the check threw, timed out, or came back empty, and the product allowed the action. Here the check can have returned a decision. The user's text is what changed the instruction, and the guard may have approved the result. An exception is not a rewritten instruction. A rewritten instruction is not an exception. It is not two sentences for one question. When one question gets two answers stops you until one sentence can be shown twice. The instruction can be stable and the user text can still have replaced it, once. It is not where a supporting passage sat. Where the supporting passage sat asks you to point at a passage in the prompt or to say you cannot see the prompt. A passage can sit in the middle and still not be an instruction the user overwrote. Position is not obedience. It is not whether the cited page supports the sentence. Open the page the answer cited is that check. A page can support a sentence the user told the model to prefer over your rule.

Two catalogues use the same words and do not draw the same box. Schulhoff and colleagues, in the paper that reports the HackAPrompt competition, say they use prompt hacking for prompt-based strategies, and that the term encompasses prompt injection and jailbreaking. They define prompt injection as overriding the original instructions in the prompt with special user input. They define jailbreaking as getting a model to do or say unintended things through prompting. They also say those definitions were updated from a previous version of the paper. OWASP's LLM01:2025 uses the words the other way around. A prompt injection vulnerability, on that page, occurs when user prompts alter the model's behavior or output in unintended ways. The page says the two ideas are often used interchangeably, then says jailbreaking is a form of prompt injection, the case where the inputs cause the model to disregard its safety protocols entirely. OWASP's containment is not Schulhoff's containment. Do not cite either page as if it had agreed with the other. Quote the box you mean (Schulhoff et al., HackAPrompt, EMNLP 2023; OWASP, LLM01:2025 Prompt Injection).

The competition is a direct attack, typed by a person into a box, against an instruction the organizers wrote. It is not your tickets. They report over 2,800 participants from more than 50 countries and more than 600,000 adversarial prompts. Over 2,800 is not an integer they printed, and more than 600,000 is not a sum you should compute from the tables below. The datasheet says the competition ran from May 2023 to June 2023 and the dataset was collected in June 2023. There are ten challenges. A footnote says challenge 0 is a demonstration. Within the first few days, competitors had solved 9 of 10 challenges. The tenth was never solved. They leave challenge 10, the one whose user input could only be emoji, out of most analyses, and they say it may not have a solution. 9 of 10 is not 10 of 10. Challenge 0 is not the tenth. The models were gpt-3, which they identify as text-davinci-003, ChatGPT, which they identify as gpt-3.5-turbo, and FlanT5-XXL. A later re-run mentioned gpt-3.5-turbo-0613, which they say was not available during the competition. The 0613 re-run is not the competition rate.

Success is two datasets, and the rates are not neighbors you can average. The submissions dataset, prompts sent to the leaderboard, has 41,596 prompts, 34,641 successful, 83.2%. The playground dataset, anonymous prompts tried on the interface, has 560,161 prompts, 43,295 successful, 7.7%. 83.2 is not 7.7. 41,596 is not 560,161. 34,641 is not 43,295. The paper says the submissions set is smaller by an order of magnitude and denser. An order of magnitude is their description of that pair. It is not a third rate. Most prompts were written by hand. Some teams built tools. The tooling is not the 83.2, and the 83.2 is not a rate at which a random message from your customer rewrites a rule. These are people trying to win a competition, on three models, in 2023.

A third table is captioned as model usage on the submissions dataset, and the rows do not add up to the submissions total. Flan has 227,801 prompts and 19,252 successes, printed as 8%. ChatGPT has 276,506 and 19,930, printed as 7%. GPT-3 has 55,854 and 4,113, printed as 7%. Those three prompt counts add to 560,161, and those three success counts add to 43,295, which are the playground totals in the other table, not 41,596 and not 34,641. The caption names one dataset. The arithmetic matches the other. Do not cite the 8% or either 7% as a share of the 41,596. The two 7% figures are not the same count: 19,930 of 276,506 is not 4,113 of 55,854, even though both are printed as 7. Do not replace either 7 with a percentage you computed. 8 is not 7. 8 is not 83.2. 7 is not 7.7.

The appendix prints still other integers for what sound like the same piles. It says the playground dataset contains 589,331 anonymous entries. It says the submissions dataset contains 7,332 entries, and that those files contain 58,257 prompts, with an average of 7.95 prompts per submission. 589,331 is not 560,161. 7,332 is not 41,596. 58,257 is not 41,596 and is not 34,641. 7.95 is an average of prompts per file. It is not a success rate, and it is not 7.7. The paper demonstrates the submissions set with the 83.2 table and, in the appendix, describes the files. Both can be in the document. Neither authorizes you to pick the integer that makes a sentence shorter.

A few other figures in the same paper are easy to stick onto the 83.2 and do not belong there. ChatGPT received a 2× score multiplier because it was harder in their pre-competition tests. The default multiplier is 1.0. 2 is not 1, and neither is a success rate. Their worked example, a difficulty of 3 and a prompt of 500 tokens scoring 57,000, is an illustration of the formula. It is not a result anyone achieved. They classified 1,000 prompts with a model and report about 75% agreement with the authors' labels. About 75 is agreement on a label. It is not 83.2, and it is not 7.7. They documented 29 techniques in the taxonomy they built. 29 is not 9 out of 10. They say they have already found at least 6,000 prompts that only work some of the time. At least 6,000 is not 41,596, and it is the paper's own warning that a prompt which worked in June 2023 is not a rate for a model you call now. The limitations say the tests covered a few models, most of them behind closed APIs, that the dataset is a snapshot, and that prompt drift means the same text may not work later. Read the 83.2 inside that fence.

OWASP does not print those rates, and it does not say the same thing about a defense that is only a sentence. The page says retrieval and fine-tuning do not fully mitigate prompt injection. "Do not fully mitigate" is not "never works," and it is not the 7.7. Schulhoff's conclusion says a significant takeaway is that prompt-based defenses do not work, including evaluating one model's output with another model. That is their sentence about this competition. It is not OWASP's sentence. OWASP lists, as mitigation item 1, an instruction that tells the model to ignore attempts to modify core instructions, and it also says that given how models work it is unclear if there are fool-proof methods of prevention. Item 1 is a sentence in a system prompt. The competition's takeaway is that a sentence was not the thing that held. Do not record item 1 as the proof that the user's text lost. OWASP separates direct injection, a user's input altering the model, from indirect injection, where the instruction arrives in content the model reads, such as a page or a file. The competition measured the direct case, people typing into a challenge. If the contradicting instruction was in a retrieved page, you still stop, and the text you show is the page, not the chat bubble. Scenario 4 on the OWASP page is that case: a modified document in a repository a retrieval system uses. Scenario 1 is a support chatbot told to ignore previous guidelines and to use tools. Neither scenario is a count.

Do not serve the sentence, and do not let the tool commit, until the three objects are on one page. First, the instruction you wrote: the system text, the tool policy, the line that said what the model was for. If that text is only in a dashboard you cannot open for the request that already happened, write that you cannot see the instruction that was sent. A template you hope was current is not the instruction. Second, the user text, copied from the prompt that was sent, not paraphrased by a later call. If the user text is ordinary and the contradiction is inside a retrieved page, show the page and the span. Third, the tool calls, if any: the name, the arguments, and whether they ran. A sentence that only talked is a different stop from a sentence that issued a refund, a send, or a write. Stop the ones that write. Keep the prompt. The model is not a witness to what it was told.

Three moves feel like the proof and are a second problem. Adding a line that says to ignore instructions to ignore the instructions, and calling the incident closed. That is OWASP's item 1, and it is the class of defense the competition's conclusion does not treat as sufficient. Asking the model which instruction it followed. You will receive a fluent answer about a prompt you still have not opened. And treating 83.2% as the share of your tickets that are attacks. 83.2 is a share of leaderboard submissions in a contest where the goal was to succeed. Your denominator is messages from customers who were trying to ask about an order. Those are not the same people. A passage in the middle of a long prompt, a stale page, and two answers to one question remain the other articles. Do not file this incident on those pages unless that is the failure you actually saw.

The shape of a product that can refuse a tool without refusing every answer is narrower than a contest. The support agent we shipped for a DTC brand answers order status, returns, and product questions from order data, policies, and the catalog, and it can look up an order, start a return, or change an address behind a permission check, with escalation to a person. It is not a prompt-injection study. Nothing in that description is a headcount, a duration, or a result figure. The shape is the split: a lookup can keep answering while a return does not start, if the instruction that allows the return is no longer the instruction you wrote. A design that has one boolean for whether the model may do anything cannot make that split. OWASP's fifth mitigation is a person approving high-risk actions. The escalation is that shape. It is not a substitute for keeping the prompt.

There is a version of this night where hiring a production engagement is the wrong purchase. Nothing is in production, and the text in front of you is a demo prompt you can still edit. Nothing was retrieved, and the user did not send a second instruction. You are looking at two answers to two questions. A customer has already seen another customer's data. The first hour is the tenant boundary. The wrong value was a write the agent already committed, and the work now is to reverse that row. Reversal is a different page from the prompt you should have kept. Or no metric was ever named, so you cannot say what the instruction was for. A tighter system prompt will not invent the metric.

Where the feature is already live and the prompt shows the user's text against your instruction, or shows that you cannot see the prompt, the work is the stop, the record, and the tool. That sequence is what an AI pilot failed to reach production engagement owes you when the pilot's only defense was a sentence: an audit in weeks one and two, then a path that cannot act without a check a person can open, measured against a number you agreed, and rolled out behind a human-review gate. The build is scoped after the audit. If the agreed metric does not move, you do not pay for the build. The findings of a two-week AI readiness assessment are yours whether or not that build happens. AI evaluation and observability is how you learn, afterwards, whether the cases you saved still fail. The competition's percentages will not be that score.

An unsubscribe you can show for one address, and an account that was allowed to merge, are different stops: show the unsubscribe they sent and name who merged their change. Do not record either on this page. A mailing list is not a prompt. A branch rule is not a system instruction.

Before you tell anyone the model resisted, open the prompt that was sent. Show your instruction. Show the text that was not supposed to be an instruction. Show the tool, or show that you cannot see one of those three. Do not ask the model to grade its own obedience, and do not quote a contest rate as your incident rate. The sentence in the system prompt is a hope. The prompt you sent is the page you can defend.

Naming the system text, the messages, and the tools on the request you stored is the neighboring proof: which prompt did that request send. Showing that the user's sentence was not supposed to be an instruction does not, by itself, show the bytes that left.