How to tell if your system is production-ready

Production-ready is not a property your system has. It is a claim somebody is about to make about it — and like any claim, it is either backed by evidence or it isn't. That is why the argument in the room is usually unwinnable. One person is describing how carefully the code was written, another is describing what happens on a bad Tuesday, and nobody has named the observation that would settle it. The answerable version is narrower: for each thing you are about to assert — it stays up, it keeps secrets, it survives being popular, it can be fixed at two in the morning by somebody who did not build it — what would you have to watch happen before you believed it? Everything below is that question asked six times, with the drill that answers each one attached. None takes longer than an hour or needs a tool you do not already have.

The best-documented version of this review is not a checklist at all — it is a job. Google's Site Reliability Engineering book describes the function directly: "We created a dedicated team of “Launch Coordination Engineers” to consult with engineering teams on the technical aspects of a successful launch. The team also curated a “launch checklist” of common questions to ask about a launch, and recipes to solve common issues" (Google, Reliable Product Launches at Scale, Site Reliability Engineering). The volume that justifies that investment also explains what a small company can and cannot copy: "Google sometimes performs up to 70 launches per week", and over one stretch "In 3.5 years, one LCE ran 350 launches through the LCE Checklist", on a team that "averaged five engineers during this time period". By 2008, 30% of reviews were classified low-risk and got an almost trivial version of the checklist.

The transferable part is not the list. It is the two rules the list is curated by, and they are far stricter than anything most teams apply to their own process documents. "Every question’s importance must be substantiated, ideally by a previous launch disaster." And: "Every instruction must be concrete, practical, and reasonable for developers to accomplish." The same chapter notes that "at one point, adding new questions to Google’s launch checklist required approval from a vice president" — a deliberately absurd bar, because "there is a near-infinite number of questions to ask about any system, and it is easy for the checklist to grow to an unmanageable size". Two things follow. A readiness checklist downloaded from the internet is a record of somebody else's incidents. And a question you cannot state as a concrete action somebody could finish this week is not a check — it is an anxiety.

The first drill is the one that exposes the most in the least time: draw the request path, out loud, from a user's tap to the database and back. The checklist's version is blunt — "What is your request flow from user to frontend to backend?" and "Do you have any single points of failure in your design?" Do it on a whiteboard with whoever built it, naming every hop: the CDN, the app server, the queue, the database, each third-party API, the model provider. Two failures show up almost immediately in systems built fast — a hop nobody can describe, usually an integration wired up in an afternoon months ago, and a dependency everyone assumed was optional sitting directly in the path of the login flow. If it takes more than fifteen minutes to draw, the honest finding is that nobody currently holds the whole system in their head, which is itself an answer.

The second drill: write down every process that keeps the system running and cannot be done by a computer. The Google chapter puts the reason plainly — "automation is never perfect, and every service has processes that need to be executed by a human: creating a new release, moving the service to a different data center, restoring data from backups, and so on. For reliability reasons, we strive to minimize single points of failure, which include humans." Its checklist question: "Are there any manual processes required to keep the service running?" Write the list, then write a name beside each one. Every process with exactly one name against it is an outage waiting for that person to be on a plane. The book's fix is not automation: "Processes should be documented in such a way that any team member can execute a given process in an emergency." An afternoon of writing buys more resilience here than a month of engineering.

The third drill is a number, not an opinion: what multiple of today's peak have you actually run this at? If the answer is one, you have not tested capacity, you have observed it. The reason to pick a large multiple is in the same chapter: "Public interest is notoriously hard to predict, and some Google products had to accommodate launch spikes up to 15 times higher than initially estimated." It also states the arithmetic people skip about redundancy — "if you need three replicated deployments to serve 100% of your traffic at peak, you need to maintain four or five deployments, one or two of which are redundant, in order to shield users from maintenance and unexpected malfunctions." The finding that matters from a load test is never the number it survived, but which component gave way first — almost never the one the team expected, and knowing it converts a future incident into scheduled work.

The fourth drill applies to anything in your system that makes requests without a human pressing a button — a mobile app syncing in the background, a page that refreshes itself, a cron job, and now, routinely, an agent in a retry loop. The assumption that traffic is bounded by how fast people can click does not hold for any of them, and the checklist's two action items are the whole defence: "Make sure that your client backs off exponentially on failure. Make sure that you jitter automatic requests." Without the second, every client that failed at the same moment comes back at the same moment, and your recovery is a synchronised stampede. It is usually a five-line change, and it is skipped almost universally in systems built quickly, because nothing in development fails often enough to reveal it.

The fifth drill is about data, where the vocabulary itself misleads people. Companies invest in backup strategies, the same book observes, when "the real focus of such backup efforts should be data recovery, which distinguishes real backups from archives. As is sometimes observed: No one really wants to make backups; what people really want are restores" (Google, Data Integrity: What You Read Is What You Wrote, Site Reliability Engineering). So the drill is not "do we have backups". It is: restore last night's backup into a scratch environment, time it, and check the restored data is usable by the application rather than merely present. A backup that has never been restored is a belief. Two things typically surface — it takes far longer than anyone assumed, and one datastore, usually the newest, was never in the backup at all.

The sixth drill is the one that tells you whether you can operate the thing at all: take one real complaint from the last fortnight and reconstruct exactly what the system did. Not roughly, exactly — the request, the inputs, the downstream calls, the timings, and for anything model-driven, the prompt and the response. If that is possible in minutes, you have observability. If it requires guessing, you do not, and every future incident will be debugged by hypothesis. It is also the check that most often changes a plan, because the fix — one structured log line per request carrying an identifier you can search on — is a few days of work.

Then the finding nobody wants: passing all six does not buy a permanent state. The US National Institute of Standards and Technology states the problem in a sentence worth borrowing outside government: "Initial authorization to operate is based on evidence available at one point in time, but systems and environments of operation change." Its answer is a discipline it defines as "maintaining ongoing awareness of information security, vulnerabilities, and threats to support organizational risk management decisions", and it is realistic about how that gets done: "Organization-wide monitoring cannot be efficiently achieved through manual processes alone or through automated processes alone. Where manual processes are used, the processes are repeatable and verifiable to enable consistent implementation" (NIST, SP 800-137, Information Security Continuous Monitoring for Federal Information Systems and Organizations, September 2011). For a product team: a readiness review has an expiry date, its manual half is fine as long as it is written down and repeatable, and the drill you ran once and never scheduled again has stopped being evidence.

If there is a model in the system, one more claim needs evidence of a different kind. Every check above asks whether the system did what it was built to do; a model can do exactly that and still be wrong. What settles it is a fixed set of real inputs with known-good outputs, scored automatically, running where a bad score can block a release — AI evaluation and observability work rather than testing in the ordinary sense. The tell has the same shape as the others: if the answer to how you know last week's prompt change was an improvement is that somebody tried a few examples, there is no evidence behind the claim, however good the examples were.

The hardest part of a readiness review is not any of the drills. It is that you cannot be the only person who runs it on your own system. Every check above depends on noticing an assumption, and the assumptions are invisible precisely to the people who made them — which is why Google's version is a separate team rather than a form the launching engineers fill in. Small companies do not have that team. What they can do is make the review adversarial on purpose: hand the six drills to somebody who did not build the thing, ask them to try to fail each one, and treat every answer of "that's fine, it's just how we do it" as a finding rather than a reassurance. If nobody suitable is available, our free production-readiness diagnostic reads a description of what you built and returns the failure modes most likely to bite first, saying where it is inferring rather than being told — a second reader, if not a substitute for someone who can see your code.

What to do on Monday, in order, and none of it needs budget approval. Draw the request path and mark every single point of failure. List the manual processes and the one name beside each. Run the system at five times peak and record which component gave way first. Restore a backup somewhere harmless and time it. Take one complaint and try to reconstruct the exact request. Whatever you could not do is your list, and it will be shorter than you feared. The failures that arrive in the first quarter of real use are catalogued in the first ninety days after launch, the gaps that show up specifically in software built fast with AI tools are in what breaks when an AI-built prototype meets real users, and how we work through the same ground on a real engagement is written up under methodology. The claim you are trying to earn is not that nothing will go wrong, but that when something does you will find out before your customers do.