Before a subcontracted change ships under your name, three things have to be true when you are alone. You opened the diff. You can say what it changes in the product the client was sold, including a prompt or a tool definition, not only a function name. The check that was supposed to fail if that sentence is false has been run, and you saw the result rather than a sentence that it passed. A message that says it looks good, sent by a person who did not need you to open anything, is not a review. Do not ship it.
This is the read you do on the change itself. It is not the hour you spend running the demo before the client sees it, and it is not the brief you sent so the work would look like yours. The demo asks whether the product behaves, on data shaped like the client's, including the wrong answer. The brief asks whether the vocabulary, the code structure, the acceptance sentence, and the model voice were named before the work started. A score you are willing to repeat to the client is a third meeting: a number you produced, on a set frozen before anyone could tune to it. All three can be in order while the change that is about to ship is a link you have not opened. The rest of this article is what you read, and what you refuse to treat as a reading.
A person looks at the code. NIST's Secure Software Development Framework defines the two activities people collapse. Code review is "a person looks directly at the code to find issues." Code analysis is "tools are used to find issues in code, either in a fully automated way or in conjunction with a person." The task that follows, PW.7.2, is the one an agency can perform without a new department: "Perform the code review and/or code analysis based on the organization's secure coding standards, and record and triage all discovered issues and recommended remediations in the development team's workflow or issue tracking system" (NIST, SP 800-218, February 2022). The practice those tasks sit under says the point is to identify vulnerabilities "so that they can be corrected before the software is released." The first example under PW.7.2 is the shape that passes: "Perform peer review of code, and review any existing code review, analysis, or testing results as part of the peer review." A tool result nobody opened is the analysis half, stranded. A comment of approval with no diff behind it is neither half.
The organization in that sentence is yours. The standards are the ones you would apply to a change your own employee wrote, because the client will meet the change under your name. The tracker the issues are recorded in is yours as well. A list that lives in the subcontractor's tool, which you can see only while the contract is friendly, is not a record you hold. PW.7.2 does not say the author of the change is the reviewer. The example says peer review. On a subcontract, the peer who can refuse to ship is the agency. The person who wrote the change can explain it. Explanation is not the review.
What a large review process actually measured, and what it did not. Sadowski, Söderberg, Church, Sipko, and Bacchelli studied code review at Google: 12 interviews, a survey of 44 respondents, and review logs for 9 million reviewed changes. On an average workday, about 20,000 changes are committed that meet the filters they describe. They excluded changes with no reviewers, used a name heuristic to drop robot authors while still keeping a robot change if a human reviewed it, stayed in the main codebase, dropped changes not yet committed, and dropped diffs the tool reported as zero source lines, such as a binary-only change (Sadowski et al., Modern Code Review: A Case Study at Google, ICSE-SEIP 2018). Those filters are why the medians are not "all of Google's commits." They are the reviewed, human-touched, source-line changes in that codebase.
Inside that set, the changes are small and the wait is short, and both facts are easy to misuse. Over 35% of changes modify only a single file, and about 90% modify fewer than 10 files. Over 10% modify only a single line of code. The median number of lines modified is 24. Initial feedback takes a median of under an hour for small changes and about 5 hours for very large changes. The median latency for the entire review process, across all sizes, is under 4 hours. Fewer than 25% of changes have more than one reviewer, over 99% have at most five, and the median reviewer count is 1. A footnote on that sentence is a different count: 30% of changes have comments by more than one commenter, "meaning that about 5% of changes have additional comments from someone that did not act as an approver." Reviewer, in their definition, is someone who approves. A commenter who did not approve is not that person. Do not quote the 25% and the 30% as the same fact. Over 80% of changes involve at most one iteration of resolving comments. The average number of comments peaks at 12.5 per change for changes of about 1,250 lines, and changes larger than that often contain generated code or large deletions, which draw fewer comments. None of these is a quota for your agency. The paper says Google's median is smaller than the medians Rigby and Bird reported for other companies, and that developers there are pushed toward small changes. A 24-line median from a monorepo with a required review tool is not a rule that your Friday release must be 24 lines. It is evidence that a change a person cannot finish reading is a change the reading will not cover.
The study is also explicit about who it did not ask. The authors did not interview research scientists or developers who interact with specialist reviewers, "such as security reviews," and they call the result biased toward general developers. A prompt, a tool definition, or a permission on a system of record is closer to the review they left out than to a one-line fix in a familiar file. Treat the speed figures as a description of ordinary changes at one company in that log window. Do not treat "median reviewer count of 1" as permission for the subcontractor to be the only person who looked. At Google the one reviewer is inside the company that owns the code. On your engagement the equivalent is someone who answers to you.
What the survey adds is that review can be valued without being a bug hunt, and that a small sample said so. All 44 respondents agreed that code review at Google is valuable: the table prints 0, 0, 0, 14, 30 from strongly disagree to strongly agree. The same table, for whether this particular change's review was a good use of time, is 2, 4, 14, 11, 13. Valuable in general is not the same as worthwhile on the change in front of you. Only 2 respondents said the comments had found a bug. The historical reason the paper records for introducing review at Google is that other developers had to be able to understand the code, not that review was the primary way defects were counted. Developers spent an average of 3.2 hours a week reviewing, median 2.6, measured across five weeks that started in October 2016, by grouping tool interactions into sessions with no more than ten minutes between them. A separate internal satisfaction figure, 97% of developers satisfied with the review tool, is not the 44-person table. Quote them apart. The useful translation for an agency is narrow. You are not reading the diff to out-debug the author in an afternoon. You are reading it to see whether you can explain the change, and whether it is the change you sold. A process your client cannot see you perform is a status line.
What you open. The diff, not the summary. The files, and whether a prompt, a tool schema, a migration, or a permission changed alongside the function. The test that names the acceptance sentence, and the output of that test, not a checkbox. A dependency added in the same change, because a new package is a new supplier your client's questionnaire will eventually ask about. A secret, a key, or a customer row pasted into a fixture. If the change is too large to read, the Google size distribution is the argument for sending it back, not the argument for skimming. You do not need their median. You need a change you can finish. The accounts you keep so you can run this without them — the repository, the model organization, the dataset — are what to keep when you subcontract AI. This article starts where that ownership is already yours. If the repository is theirs, you cannot do this read, and the read is not the first problem.
Record what you refused, in your tracker, with the change it belongs to. PW.7.2's verb is triage, not approval. An issue you noticed and mentioned in chat is not triaged. A comment resolved by the author with no note of what changed is the iteration the Google logs are full of, and it is fine when you can see the new snapshot. It is not fine when the new snapshot is a force-push you did not re-open. Re-open it. The second snapshot is a different change. The first approval does not cover it.
There is a version of this hour where hiring a review practice is the wrong purchase. The work is not sold, so there is no client-facing change to stand behind — that is a sales question, not a diff. The client has asked to meet the people writing the code. That is a disclosure decision, and it is when to tell a client you subcontract, not a reason to skip the read or to pretend you wrote the change. The room only has a slide. There is no diff to open, and a review of a picture is a meeting. You cannot say what the change must prove, because the acceptance sentence was never written. That belongs in what a white-label brief must hold, before anyone is reading code. Or the buyer only wants you to walk the fixture while they watch. The hour before the client sees the product is check the demo before the client does. Do not substitute the demo for the diff. A product can behave in a demo and still contain a change you cannot explain.
What a change you can re-open looks like, when the artefact is a prompt rather than a function, is a record outside the author's laptop. Inflectiv Helios is a multi-assistant platform we built under our own name, with prompt monitoring on a dashboard. It is not a white-label engagement and it should not be read as one. Nothing in that description is a headcount, a duration, or a result figure. The shape is the point: a prompt that shipped is a prompt you can look at after the author has closed the editor. A subcontracted prompt that exists only in a chat thread is the object this review refuses, because you cannot open it twice.
Before you tell the client the build moved, do the three checks and write down which one failed. Open the diff. Say, in a sentence you could forward, what it changes in the thing you sold. Look at the test result, including a failure if the change was supposed to be risky. If you cannot do the first one, you are not behind. You are about to put your name on a change you have not seen. A white label software development partner that expects this will put the repository in your organization and the review on your side of the pull request before the first week is billed, because the read takes them nothing they were not already doing on their own work. One that can only offer a status line is telling you who is allowed to ship. Where the next slice is a single capability with the acceptance sentence already named, an AI integration sprint is the shape that can require this read as the definition of done, rather than as a favor at the end. The number you will later be asked to stand behind, if the change touches how the system is scored, is the same discipline as AI evaluation and observability: a result you can re-run. The diff is how you notice the result moved. It is not a substitute for having run it.
Find this useful? Tell Google to show you more of it.