Which schema did that call require?

A customer is looking at a wrong field, and someone in the room says the schema would not have allowed it. Do not treat the reply as something a schema produced, and do not ask the model which schema it followed, until you can name the schema on that call: the name, the required keys, and whether strict was set. Show a different call whose schema name or required keys are not this one. Show that you stored the request, not a recollection of it. If any of the three is a sentence the model produced after the fact, you have not shown the schema. You have a plausible object and a guess.

This is the schema the request required. It is not the document a citation named. Which document did that citation name? is an index, a title, and a span. A citation can be well formed while the schema name was never stored, and a schema can be strict while the cited page does not say what the field says. It is not the stop the response carried. Name the stop the response carried is a stop reason. A stop of max_tokens can cut a schema off, and the stop is still not the schema name. It is not the prompt. Which prompt did that request send? is the text. A prompt can mention a schema that the request did not require. It is not the model id. Name the model the request hit is which model answered. A model can be right while the schema on the request was a different name.

None of what follows is a rate for this product, and none of it is legal advice. The fields below are one vendor's announcement of structured outputs, including the example it prints. The coverage numbers below that are a 2025 benchmark of other people's schemas, including that vendor as one row. A perfect score on someone else's eval is not the schema on the call you stored tonight. They do tell you which numbers people mix up when they try to sound precise about a schema they have not opened.

OpenAI's announcement says gpt-4o-2024-08-06 with structured outputs scores 100% on their evals of complex JSON schema following, and gpt-4-0613 scores less than 40%. 100% is not less than 40%, and the page does not print the integer 40. The same post says the model reached 93% on the benchmark before a deterministic constraint, and the constraint is what they say brings the figure to 100%. 93 is not 100. response_format with strict true is described for gpt-4o-2024-08-06 and gpt-4o-mini-2024-07-18. Tools with strict true are described for gpt-4-0613, gpt-3.5-turbo-0613, and later. The 100% sentence is not a claim about every model that accepts strict. The math example sets model gpt-4o-2024-08-06, schema name math_response, required keys steps and final_answer, and additionalProperties false. The user string is solve 8x + 31 = 2, 17 characters. The system string on that example is You are a helpful math tutor., 28 characters plus the period, 29 in all. 17 is not 29. math_response is 13 characters. 13 is not 17. The page tells the reader to print final_answer. It does not print the value of x. Solving the equation yourself gives -3.625. -3.625 is not on the page, and it is not a required key. A schema match is not a correct final_answer. The post says structured outputs do not prevent a wrong step inside a value (OpenAI, Michelle Pokrass, Introducing Structured Outputs in the API).

The limits on that page are not the 100% sentence. A new schema's first request is described as typically under 10 seconds, and a complex schema as up to a minute. 10 seconds is not 60 seconds. A refusal sets a refusal boolean to true, and the post says the model can then fail to follow the schema. true on a refusal is not a schema match. Reaching max_tokens can also fail the schema. A stop is not a name. Structured outputs are not compatible with parallel function calls, and the post says to set parallel_tool_calls to false. false is not strict true. The price sentence says switching to gpt-4o-2024-08-06 saves 50% on inputs, at $2.50 per million input tokens, and 33% on outputs, at $10.00 per million output tokens, compared with gpt-4o-2024-05-13. 50 is not 33. $2.50 is not $10.00. $10.00 divided by $2.50 is 4, not 2, so the output price is not double the input price. Five libraries are named as prior art: outlines, jsonformer, instructor, guidance, and lark. 5 is not 100. gpt-4o-2024-05-13 is not gpt-4o-2024-08-06. Do not paste the 100% onto a call whose request did not set strict.

The benchmark people reach for when they want a coverage rate is a different document, and its schema counts are not a name on your request. Saibo Geng, Hudson Cooper, Michał Moskal, Samuel Jenkins, Julian Berman, Nathan Ranchin, Robert West, Eric Horvitz, and Harsha Nori introduce JSONSchemaBench. The abstract says the benchmark comprises 10K real-world JSON schemas. Table 1's counts are GlaiveAI-2K 1,707, Github-Trivial 444, Github-Easy 1,943, Snowplow 403, Github-Medium 1,976, Kubernetes 1,064, Washington Post 125, Github-Hard 1,240, JSONSchemaStore 492, and Github-Ultra 164. Those ten numbers sum to 9,558. 9,558 is not 10,000. The page does not print the difference, so do not invent it. The row is named GlaiveAI-2K and the count is 1,707. 1,707 is not 2,000. Ten datasets is the row count. 10 is not 9,558. They start from over 6,000 GitHub schemas and, after the split, the five GitHub rows sum to 5,767. 5,767 is not 6,000. They exclude Github-Trivial and Github-Ultra from the experiments and keep them in the benchmark. 444 plus 164 is 608. The page does not print how many schemas remain in the experiments, so do not invent that remainder either (Geng, Cooper, Moskal, Jenkins, Berman, Ranchin, West, Horvitz, and Nori, JSONSchemaBench).

The coverage table and the announcement do not share one 100%. On GlaiveAI, OpenAI's declared coverage is 0.89, empirical coverage is 0.89, and the compliance rate is 1.00. 1.00 is not 0.89. On Github-Easy, OpenAI is 0.30 declared, 0.29 empirical, and 0.97 compliance. 0.29 is not 1.00, and it is not the announcement's 100%. On Github-Medium, OpenAI is 0.13, 0.12, and 0.92. 0.12 divided by 0.13 is 0.92 to two decimal places, so that compliance cell matches the division, and it is still not 1.00. On Kubernetes, OpenAI is 0.21, 0.21, and 1.00. 0.21 is not 0.89. Guidance on GlaiveAI is 0.98 declared and 0.96 empirical. 0.96 is not 0.98. The introduction says constrained decoding can speed generation by 50% against unconstrained decoding. On GlaiveAI the unconstrained tokens-per-output-token figure is 15.40 and Guidance is 6.37. Half of 15.40 is 7.70. 6.37 is not 7.70. The reduction is 58.6% of 15.40, not 50%. The introduction says downstream tasks improve by up to 4%. The results section says Guidance improves on the unconstrained model by approximately 3% in every task, and that every framework scores higher than unconstrained. Table 8's GSM8K cells are 80.1% unconstrained and 83.8% with Guidance. The difference is 3.7 points, not 4 and not 3. Last letters moves from 50.7% to 54.0%, a difference of 3.3. Shuffle objects moves from 52.6% to 55.9%, also 3.3. 3.7 is not 3.3. Llamacpp on shuffle objects prints 52.6, the same cell as unconstrained. 52.6 is not higher than 52.6. XGrammar's GSM8K cell is 83.7, which is not Guidance's 83.8.

Table 5 counts test-suite categories, and it has four framework columns, not six. The paper evaluates six frameworks, and this table prints Outlines, Llamacpp, XGrammar, and Guidance. 4 is not 6. OpenAI is not a column there. Guidance's counts are minimal 30, partial 25, moderate 21, high 17, and full 13. 30 is not 13. Outlines' full-coverage count is 0. 0 is not 13. Outlines' minimal count is 20. The text says some supported features take Outlines from 40 seconds to 10 minutes. 40 seconds is not 10 minutes, and neither is the announcement's under-10-seconds first request. The task schema in the benchmark is a JSON object with reasoning and answer. The announcement's math schema requires steps and final_answer. reasoning is not steps. answer is not final_answer. Do not paste one required list over the other. The toy example says a model might emit 89,000 where the correct token string is 89000, and that blocking the comma might yield 890000. Three strings. 89000 is not 89,000, and 890000 is not 89000. A comma is not a schema name.

Do the three proofs on one real call, in the log that actually holds it. First, the binding. Copy the schema name, the required keys, and whether strict was true. If the log does not store the response format, write that. Do not infer math_response because an announcement uses it. If strict is absent, write that it was absent. Do not upgrade a missing flag to true because a blog post reached 100% with the flag on. Second, the other call. A different name, or the same name with a different required list, is not this one. A benchmark coverage is not that second call. If you cannot show a call that is not this schema, you have a counter, not a record. Third, the stored request. The object you defend is the one you wrote down before the model answered, not the model's later description of what it was constrained by. If the only copy is the reply, write that. A request you cannot show is the finding. It is not a reason to tell the customer the field was valid.

Three moves feel like the proof and are a second problem. Editing the stored request so the schema matches the reply, and then describing the missing name as a cleanup. Quoting 100% because it looks precise, while the call you were asked about is one row, and the benchmark's OpenAI row on an easy GitHub slice is 0.29. Asking the model which schema it followed. The model will pick a plausible name. It will not open your log. A citation you have not kept, while the customer is arguing about a source, is which document did that citation name?. Opening that span does not name the schema.

The path is ordinary when an operator can open one assistant's request without opening the next assistant's request as the way to prove it. Inflectiv Helios is a platform of specialized assistants, with prompt monitoring. The shape that matters here is one call you can point at, distinct from the next call. That case is not a schema ledger. Nothing in the description is a headcount, a duration, or a result figure. If the only control on the screen changes every assistant's required keys together, you do not have this proof. You have a shared prompt.

There is a version of this afternoon where a rescue is the wrong purchase. Nothing is in production. No model is being called. The request was not logged. A customer has already seen another customer's data. The wrong value was already a write. Or no metric was ever named. 100% is not your call, and math_response is not your call. A score does not close a leak, and a lawyer is the right hire when the question is whether a policy was broken.

When the reply is already in front of a customer and the log cannot name the schema, the missing object is specific. The name, the required keys, and the strict flag on that request, a different request that is not that one, and a stored request rather than a recollection. An AI pilot failed to reach production engagement is for the system that is already in front of customers and cannot yet show those three: audit in the first one to two weeks, finished in 8 to 12 weeks, scoped after the audit, unpaid if the agreed metric does not move, rollout behind a human-review gate. The schema you wrote tonight is the kind of case the harness has to include, or the next release will match a shape and still be wrong inside the value. The harness is AI evaluation and observability. An AI integration sprint is the shape where the acceptance sentence can require the schema object on every stored call. If you do not yet know whether the pilot can be saved, a two-week AI readiness assessment hands you the findings either way. The free production-readiness diagnostic is the short way to see whether you are looking at a missing name, a required key the reply ignored, or a system that has not reached a customer. Those are the terms. They are not 100%, and they are not 0.29.

The container that code used is a different object: which container did that code use?. Do not record it on this page. A container id is not a schema name. A memory limit is not a required key.

The invoice a charge paid, and the release a tag named, are different objects: which invoice did that charge pay? and which release did that tag name?. Do not record either on this page. An invoice status is not a schema name. A tag is not a required key.

Before you tell them the field was allowed, do the three proofs and write the name. Name the schema, the required keys, and whether strict was set, or write that the request did not store them. Show a different call that is not that schema. Show that the request is in a log you hold. Do not ask the model which schema it was, and do not treat a valid-looking object as a reason to skip storing the format. A benchmark can tell you how often a grammar accepted someone else's schemas. The binding is the schema you can defend.