Skip to content
Insights · AI Agents

Which file did that batch write?

Name the file that batch wrote: the batch id, the output file, and the counts. A different status can share an id. Another log omits it.

Scroll
AI AgentsOct 30, 202612 min readBy Salman Naqvi, Founder & CEO
Which file did that batch write?

A customer says the overnight job finished, or says the file they downloaded is not the one the job wrote. Do not treat a finished answer as the file the batch wrote, and do not ask the model which file it was, until you can name the file. Tonight, three proofs, in a log you stored. Name the file that batch wrote: the batch id, the output file id, and the request counts. If output_file_id is null, write null. Show a different response whose status, or whose output file, is not this one. Show another organization's list that does not hold this batch. If any of the three needs the person who uploaded the input and then left, you have not named the file. You have a paragraph and a guess.

This is one OpenAI batch. It is not the batch a request joined. Which batch did that request join? is a different vendor's batch id and a custom id. A message batch can finish while this output file is still null. It is not the file a store accepted. Which file did that store accept? is a vector-store file id and a byte count. A store can hold a file while this batch has not written an output. It is not the job a fine-tune finished. Which job did that fine-tune finish? is a job id and a model. A fine-tune can finish while this batch's output file is a different id. It is not the tax id a customer stored. Which tax id did that customer store? is a tax id and a value. It is not the secret an alert named. Which secret did that alert name? is an alert number. An output file is none of those.

None of what follows is a score for the product you are running, and none of it is legal advice. The objects below are the examples in the OpenAI OpenAPI description of batches, and the schema on the batches reference. One id is used for a batch that is still validating, a batch that completed, a batch that is cancelling, and a retrieve response whose endpoint is not the endpoint the others print. The figures after that are a 2024 paper about other people's serving capacity. A multiple on someone else's GPU is not the file id in the log you opened.

The create example stores a batch and does not store an output file. The id is batch_abc123. 12 characters. object is batch. 5 characters. endpoint is /v1/chat/completions. 20 characters. errors is null. input_file_id is file-abc123. 11 characters. 12 is not 11. completion_window is 24h. 3 characters. status is validating. 10 characters. output_file_id is null. error_file_id is null. created_at is 1711471533. in_progress_at, expires_at, finalizing_at, completed_at, failed_at, expired_at, cancelling_at, and cancelled_at are null. Eight timestamps, all null, beside one created time. request_counts are total 0, completed 0, and failed 0. 0 plus 0 is 0, which matches total, and the status is still validating. metadata.customer_id is user_123456789. 14 characters. metadata.batch_description is Nightly eval job. 16 characters. The schema says metadata is at most 16 pairs, keys at most 64 characters, and values at most 512 characters. This example has 2 pairs. 2 is not 16. The key batch_description is 17 characters. 17 is not 64. The value is 16 characters. 16 is not 512. The schema also describes a model, and the example it gives is gpt-6-astra. 11 characters, the same length as file-abc123, and not that string. The create example does not print a model at all (OpenAI, Batch examples).

The list example keeps the id and replaces the file, the counts, and the description. status is completed. 9 characters. 9 is not 10. output_file_id is file-cvaTdG. 11 characters, the same length as file-abc123, and not the same string. error_file_id is file-HOWS94. Also 11 characters, and not the output file. errors is still null. A null errors object beside a populated error file is what this example prints. It is not a batch with no errors, and it is not a missing file. created_at is 1711471533 again. in_progress_at is 1711471538. The gap is 5 seconds. expires_at is 1711557933. 1711557933 minus 1711471533 is 86400 seconds, which is 24 hours exactly. The window string is 24h. The integers match that window, and the page does not print 86400. finalizing_at is 1711493133. completed_at is 1711493163. 1711493163 minus 1711493133 is 30 seconds. 1711493163 minus 1711471533 is 21630 seconds: 6 hours and 30 seconds. The page does not print 21630. 21630 is not 86400. The batch completed with 64770 seconds left before expires_at. 64770 is 17 hours, 59 minutes, and 30 seconds. The page does not print 64770. A window of 24h is not the time the batch took. request_counts are total 100, completed 95, and failed 5. 95 plus 5 is 100. That addition matches. failed_at is null. A failed count of 5 is not status failed, and it is not a failed_at time. metadata.batch_description is Nightly job. 11 characters. 11 is not 16. The create example's description was Nightly eval job. Same customer_id, same batch id, two descriptions. The list request asks for limit 2. The response has 1 object. first_id and last_id are both batch_abc123. has_more is false. 1 is not 2. The list parameter's default is 20. 2 is not 20.

The retrieve example is the same id with a shorter endpoint. endpoint is /v1/completions. 15 characters. 20 minus 15 is 5, the length of chat/. The create example and the list example both print /v1/chat/completions. The retrieve example drops chat/. status is completed. The output file, the error file, the timestamps, and the counts match the list example, including description Nightly job. errors is null again, beside error file file-HOWS94 again. Do not paste /v1/chat/completions onto a retrieve response that printed /v1/completions, and do not paste /v1/completions onto the create response. The schema's status enum has 8 names: validating, failed, in_progress, finalizing, completed, expired, cancelling, and cancelled. validating and cancelling are both 10 characters and are not the same word. completed and cancelled are both 9 and are not the same word. failed is a status and also a field inside request_counts. The completed examples store status completed and failed 5. Those are not the same fact (OpenAI, Batches).

The cancel example is the same id again, and it disagrees with the completed one about what the batch had written. status is cancelling. output_file_id is null. error_file_id is null. finalizing_at is null. completed_at is null. cancelling_at is 1711475133. 1711475133 minus 1711471533 is 3600 seconds, one hour. 3600 is not 86400, and 3600 is not 21630. expires_at is still 1711557933, so the window end did not move. request_counts are total 100, completed 23, and failed 1. 23 plus 1 is 24. 24 is not 100. 100 minus 24 is 76. The page does not print 24 or 76. 23 is not 95. 1 is not 5. The completed example's completed_at, 1711493163, is 18030 seconds after this cancelling_at: 5 hours and 30 seconds. Same created_at, same id, and the completed story happens after the cancelling story in the integers. metadata.batch_description is Nightly eval job again, the create text, not the list's Nightly job. endpoint is /v1/chat/completions again, not the retrieve example's /v1/completions. A cancel response that clears the output file does not rewrite a completed response that printed file-cvaTdG. If your log has both, you have two examples, not one history.

The usage object the schema describes is not in any of those examples. The reference says usage is only populated on batches created after 7 September 2025. created_at 1711471533 is 26 March 2024. The examples are earlier than the date the field begins, and they do not print usage. An absent usage is not a token count of 0, and it is not the request_counts object. input_tokens is not total. The schema says usage includes input tokens, output tokens, and a total. request_counts includes completed, failed, and total. Two objects, and the word total is on both. A total of requests is not a total of tokens.

The paper people cite when they want a batch figure did not open this file. Agrawal, Kedia, Panwar, Mohan, Kwatra, and their coauthors, in Sarathi-Serve, report serving capacity under a latency constraint. The abstract says 2.6 times the serving capacity of vLLM for Mistral-7B on one A100, up to 3.7 times for Yi-34B on two A100s, and up to 5.6 times for Falcon-180B with pipeline parallelism. 2.6 is not 3.7. 3.7 is not 5.6. Later, the same paper says that under a strict SLO, on Yi-34B and one trace, the system sustains up to 4.0 times the load of Orca and 3.7 times the load of vLLM. 4.0 is not 3.7. For LLaMA2-70B it says up to 6.3 times Orca and 4.3 times vLLM. 6.3 is not 5.6. 4.3 is not 2.6. The abstract's 5.6 and the body's 6.3 are not the same multiple, and neither is batch id batch_abc123 (Agrawal and colleagues, Sarathi-Serve).

The same paper's latency table does not do what its caption says, and its 1024 is three different settings. Table 4 is 128 requests of Yi-34B on two A100s, with a token budget of 1024. Hybrid-batching-only prints P50 time to first token 0.53 seconds and P99 time between tokens 0.68 seconds on one trace, and 3.78 and 1.38 on the other. Chunked-prefills-only prints 1.04, 0.17, 5.38, and 0.20. The combined row prints 0.76, 0.14, 3.90, and 0.17. The caption says the combination lowers both time to first token and time between tokens. Against hybrid-batching-only, 0.76 is higher than 0.53, and 3.90 is higher than 3.78. 0.76 minus 0.53 is 0.23. 3.90 minus 3.78 is 0.12. The page does not print 0.23 or 0.12. Time between tokens does fall: 0.68 minus 0.14 is 0.54, and 1.38 minus 0.17 is 1.21. 0.68 divided by 0.14 begins 4.857. The page does not print 4.857, and 4.857 is not 2.6 and not 5.6. Against chunked-prefills-only, the combined time to first token does fall, by 0.28 and by 1.48. The caption does not say which row is the baseline. Figure 3 uses prompt length 1024 for a different measurement. Figure 14 uses chunk lengths 512, 1024, and 2048, and says the smallest, 512, has overhead of at most about 25 percent. 1024 divided by 512 is 2. 2048 divided by 1024 is 2. A prompt length, a chunk length, and a token budget can share an integer and still be three settings. 128 requests is not total 100 on the batch example. 128 minus 100 is 28. The page does not print 28.

Do the three proofs on one real batch, in the log that actually holds it. First, the binding. Copy the batch id, the endpoint, the status, the output file id, the error file id, and the three request counts. If output_file_id is null, write null. If errors is null and error_file_id is not, write both. Add the counts yourself. 95 plus 5 matched 100 on one example, and 23 plus 1 did not match 100 on another. If your rows fail, write the failure. If the description in the create response is not the description in the list response, write both, and do not keep the shorter one because it looks finished. Second, the other response. A different status, even with the same batch id in a published example, is not this one. The validating example, the completed example, and the cancelling example are that shape, and they are still examples. A serving multiple is not that second response. Third, the other organization. A list that belongs to someone else does not contain this id. If the only list is one shared key, write that. A tenant you cannot name is the finding.

Three moves feel like the proof and are a second problem. Cancelling the batch because the output was embarrassing, and then describing the missing file as a cleanup. The cancel example sets output_file_id back to null and leaves status cancelling. A null file is not a deleted file, and it does not make file-cvaTdG on the completed example disappear. Quoting 2.6 or 5.6 or 6.3 because they look precise, while the file you were asked about is one object, and the paper that printed them had already used 5.6 in the abstract and 6.3 in the body. Asking a model which file the batch wrote. The model will pick a plausible name. It will not open the batch. A store you have not named is which file did that store accept?. Opening that attachment does not name this output file, and a byte count on a store file is not request_counts.total.

The path is ordinary when an operator can open one customer's batches without opening the next customer's batches as the way to prove it. Inflectiv Helios is a platform of assistants, one conversation history per user. The shape that matters here is one customer you can point at, in an account you administer, distinct from the next customer's jobs. That case is not a batch ledger. Nothing in that sentence is a headcount, a duration, or a result figure. If the only control on the screen lists every customer's output files together, you do not have this proof. You have a shared key.

There is a version of this call where hiring the path is the wrong purchase. Nothing is in production, because no model is being called. The request was not logged. A customer has already seen another customer's file. Naming a batch id does not close a leak. The wrong value was already written back to a system of record. An output file does not undo a write. Or no metric was ever named, so there is nothing for a later eval to move. The criteria for choosing a firm sit on the page written for a top SaaS development company. That page is criteria. It does not rank anyone, including us. 5.6 is not your file, and file-cvaTdG on a sample is not your file.

When the feature is already live and the log does not store the file, the missing object is specific. A record, in an account you administer, of the batch id, the output file id, and the request counts, plus a different response that is not that one, plus another organization's list that does not hold it. An AI pilot failed to reach production engagement is scoped after an audit, not before it. The audit is the first one to two weeks. The engagement runs eight to twelve weeks. The build is unpaid if the agreed metric does not move. Rollout, when it happens, sits behind a human-review gate. If you do not yet know whether the pilot can be saved, a two-week AI readiness assessment hands you the findings either way. AI evaluation and observability is how you learn, afterwards, whether the cases you saved still fail when the output file changes. The paper's multiples will not be that score. If the first slice is a fixed capability, an AI integration sprint is the shape where the acceptance sentence can include the three proofs.

Before you tell anyone which file the batch wrote, do the three proofs and write the id. Name the batch id, the output file, and the counts, or write that the output file was null. Show a different response that is not that one, even if the batch id matches. Show another organization's list that does not contain it. Do not paste file-cvaTdG onto a cancel response that stored null, and do not paste 5.6 onto a sentence that prints 6.3. The binding is the file you can defend.

Questions

You stop treating a finished answer as the file the batch wrote, and you stop asking the model which file it was, until you can name the batch id, the output file id, and the request counts, show a different response whose status or output file is not this one, and show that another organization's list does not hold it. If the output file was null, write that.

No. A capacity multiple on someone else's GPU is not the id on this file. A file you can defend is the batch id, the output file, and the counts, or a written line that the output file was null. 5.6 in one paper is not file-cvaTdG, and 100 on a sample is not your total.

When nothing is in production, when no model is being called, when the request was not logged, when a customer has already seen another customer's file, when the wrong value was already a write, or when no metric was ever named.

Find this useful? Tell Google to show you more of it.

Let's put AI to work in your business.

A 30-minute call. You bring the workflow or the roadmap — we'll tell you what's feasible, what it costs, and what we'd build first.

Book a call→