Should we stop shipping features and write tests?

The question always arrives in the same shape. We are still shipping, but every release feels heavier than the last, something broke last Thursday that nobody touched, and someone has started saying we should stop adding features and write tests. The short answer is no — not the way the question means it. The feeling behind it is real evidence about your system, and the remedy it proposes would not fix what the feeling is detecting. Four measurements settle the argument, none takes longer than an afternoon, and the one legitimate version of stopping is a policy you write before you need it rather than a decision you make in the week you lose your nerve.

Start with the framing, because that is the part that is wrong. "Ship features" and "be stable" are being treated as two ends of one dial, and the largest ongoing measurement programme in software delivery says they are not. DORA splits software delivery performance into throughput — change lead time, deployment frequency, failed deployment recovery time — and instability: change fail rate and deployment rework rate. Its summary of what the data shows is flat: "DORA's research has repeatedly demonstrated that speed and stability are not tradeoffs. In fact, we see that the metrics are correlated for most teams. Top performers do well across all five metrics, and low performers do poorly" (DORA, DORA's software delivery performance metrics, last updated January 5, 2026). The same guide quotes Dave Farley on what the real trade-off is: "…the real trade-off, over long periods of time, is between better software faster and worse software slower."

So a team that is slow and unstable is not paying for speed. It is not getting speed. That reframes the question from *which do we sacrifice* to *what is making both worse at once* — which has an answer you can go and find.

Before you go looking, accept that nobody on the team can answer it from memory. In 2025, METR ran a randomised controlled trial with 16 experienced open-source developers on their own repositories — 246 real issues, each randomly assigned to allow or disallow AI tools. The result: "When developers are allowed to use AI tools, they take 19% longer to complete issues—a significant slowdown that goes against developer beliefs and expert forecasts. This gap between perception and reality is striking: developers expected AI to speed them up by 24%, and even after experiencing the slowdown, they still believed AI had sped them up by 20%" (METR, Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity, July 10, 2025).

That study is now the most-quoted number in this argument, and the honest thing to say about it is that its authors have superseded it. In February 2026 METR published an update: "Based on conversations with study participants, we believe it is likely that developers are more sped up from AI tools now — in early 2026 — compared to our estimates from early 2025. However, because of the selection effects in our experiment, our data is only very weak evidence for the size of this increase." The selection effect is worth reading twice, because it is the same bias that will distort your own read of your own team: "When surveyed, 30% to 50% of developers told us that they were choosing not to submit some tasks because they did not want to do them without AI" (METR, We are Changing our Developer Productivity Experiment Design, February 24, 2026). The direction of the headline figure moved in eighteen months. What did not move is the finding underneath it — that the people doing the work are the least reliable instrument for measuring how fast the work is going, in either direction. Which is precisely the instrument your team is using when it tells you to stop and write tests.

So measure. The first measurement is your change fail rate, which DORA defines as "the ratio of deployments that require immediate intervention following a deployment. Likely resulting in a rollback of the changes or a 'hotfix' to quickly remediate any issues." You need no tooling. Open your deployment history, take the last twenty releases, and mark each that needed a same-day follow-up to fix something the release itself caused. That ratio takes twenty minutes, and it is the first honest thing anyone in this argument will have said.

The second is deployment rework rate — "the ratio of deployments that are unplanned but happen as a result of an incident in production." It sounds like the same metric and it is not. Change fail rate counts releases that hurt. Rework rate counts releases you never planned, which measures how much engineering capacity is consumed answering production rather than building. A founder who believes the team has stopped delivering features very often has a team delivering at full tilt into rework.

The third is failed deployment recovery time — "the time it takes to recover from a deployment that fails and requires immediate intervention." This is the one that decides whether a high fail rate is an emergency or an expense. A team that breaks one release in five and restores service in four minutes is in a different business from a team that breaks one in five and spends a day. It is also the cheapest of the three to improve, and improving it requires pausing nothing: it is rollback machinery, not test coverage. A 24-month continuous-delivery engagement on a home-security device platform ran on Kubernetes with blue/green deploys and full observability for exactly this reason — being able to put the previous version back is what makes shipping often safe rather than reckless.

The fourth is not a metric and you should not pretend it is one, but it is the most diagnostic of the four: find the avoidance. Somewhere in your product there is a file, a table or a service that has a reputation. Features get quietly reshaped to avoid touching it. Estimates for anything near it come back doubled with no explanation. Ask each engineer, separately, to name the change they would refuse to make on a Friday afternoon, and what they would need before making it on a Tuesday. If three people name the same thing, you have found your problem, and it has a name and a location.

Read the four together. High fail rate with fast recovery and no avoidance is a team shipping aggressively with a working safety net — leave them alone. Low fail rate with high rework usually means the failures are not caused by releases at all, so the problem is in data, load or a dependency rather than in your code, and writing tests would have addressed none of it. Slow recovery with heavy avoidance is the genuine version of what you are feeling, and even there the instrument is not a freeze. It is a named piece of work with a deadline, against the specific thing three people just named.

There is exactly one legitimate version of "stop shipping features", and its defining property is that it is not decided in the moment. Google's SRE practice calls it an error budget policy, and the mechanism is written down in advance: "This policy should cover the specific actions that must be taken when a service has consumed its entire error budget for a given period of time, and specify who will take them." One of those actions is a production freeze — "to reduce the risk of more outages, a production freeze halts certain changes to the system until there is sufficient error budget to resume changes." The test the SRE workbook applies to the policy is the one worth stealing: "If the product manager feels that the SLO will result in a bad experience for a significant number of users before the error budget policy prompts anyone to address an issue, the SLOs are likely not tight enough" (Google, Implementing SLOs, The Site Reliability Workbook).

Compare that with the founder version. It has no numeric trigger, so it starts when somebody is frightened. It has no owner, so everyone is partly responsible. Worst of all it has no exit condition, so it ends when commercial pressure becomes unbearable rather than when the system is safe — the freeze buys a few weeks of test-writing and hands the team back the situation it started in, minus the runway. If you cannot state the number that would start a freeze and the number that would end it, you are not proposing a policy, you are proposing a mood.

What you do instead starts on the next commit rather than after a pause: test what you touch. Do not retrofit a suite across a codebase nobody has a map of; put tests around the functionality you are already in the file changing, every time. That rule and the five decisions underneath it are in testing an application when you have no QA team. It is deliberately not a pause — it is a tax on work you were doing anyway, which is why it survives contact with a roadmap and a freeze does not.

The AI-era wrinkle changes one half of this and not the other. Generating a test is now almost free, which removes the usual excuse. But a test generated against the code as it stands asserts what the code currently does, and if the code currently does the wrong thing you have ratified the bug and made it harder to fix. The judgement — what this is supposed to do, what a wrong answer looks like, which behaviour is a promise to a customer and which is an accident — is not generated. It is the judgement catalogued in what breaks when an AI-built prototype meets real users, and for anything with a model in the loop it is a standing measurement problem rather than a one-off, which is the work AI evaluation and observability exists to do.

Two conditions genuinely justify stopping, and neither is the one you asked about. The first: you cannot deploy without a specific person. If releases need somebody's laptop, a hand-run script or a password one human has, your recovery time is that person's availability and no amount of testing improves it. Fix it first; it is days of work and it moves every other number here. The second: nobody can state what the system is supposed to do. With no written definition of correct, tests cannot be written — only transcribed — and the argument is not about testing at all. If what you are staring at is a codebase you are no longer sure is worth extending, that is a different and testable question, and the three conditions that settle it are in rewrite or repair an AI-built codebase.

On Monday: pull the last twenty deployments and compute the three DORA ratios; ask each engineer privately for their Friday-afternoon refusal; write one paragraph naming the trigger, the owner and the exit condition for a freeze you hope never to call. If those four artefacts say the system is fine and the team is simply tired, that is worth knowing too. A startup software development company worth talking to will run those numbers with you before proposing anything, and will tell you when the answer is that nothing needs to change. To have the failure modes of your own system named before you hold the meeting, the free production-readiness diagnostic reads a description of what you shipped and returns the ones most likely to bite first; if the honest answer turns out to be a quarter of work rather than a fortnight, the two-week AI readiness assessment is how you get a scoped number instead of an argument.