Model drift: what changes underneath your AI system

When people say model drift they usually mean accuracy quietly degrading as the world moves away from the training data. That happens, and it is hard to detect. But the version that takes production systems down is far blunter: **the model you built on is retired on a published date, and after that date your requests fail.** This is not a hypothetical risk profile. It is a calendar, and the dates are public.

Anthropic publishes the lifecycle explicitly — Active, Legacy, Deprecated, Retired — and is unambiguous about the last one: "The model is no longer available for use. Requests to retired models will fail." Customers with active deployments get "at least 60 days' notice before model retirement for publicly released models" (Anthropic, Model deprecations). Sixty days is a reasonable notice period and a short engineering quarter, and the difference between those two readings is entirely whether you already have the machinery to swap a model safely.

The state most teams ignore is the one before the clock starts. **Legacy** means the model "will no longer receive updates and may be deprecated in the future" — no retirement date, no notification, nothing to action. It is the only free warning you get, and it is the point at which a replacement should be evaluated rather than the point at which it is noted. Teams that wait for Deprecated have converted a months-long window into a sixty-day one for no reason, because Deprecated is when the countdown is announced, not when the change became foreseeable.

The published history makes the cadence concrete rather than theoretical. Claude Opus 4.1 was deprecated on 5 June 2026 and retired on 5 August 2026 — two months. Claude Sonnet 4 and Opus 4 were deprecated on 14 April 2026 and retired on 15 June 2026. Claude Sonnet 3.7 was deprecated on 28 October 2025 and retired on 19 February 2026, a little under four months. Every one of those was announced, dated, and had a recommended replacement named. None of it was a surprise to anyone reading the page. The systems that broke were the ones where nobody was.

There is a second-order fact here that catches teams out and almost nobody writes down: **retirement dates depend on where you run the model, not just which model it is.** Anthropic's dates apply to its own platforms, and the documentation says plainly that partner-operated platforms — Amazon Bedrock and Google Cloud — "set their own retirement schedules, so a model's lifecycle status and dates can differ." If your architecture diagram says Bedrock and your calendar says the first-party API, one of them is wrong about your deadline.

Models are not the only thing that gets deprecated, either. Request parameters do too, and that failure looks like a code bug rather than a model change. `temperature`, `top_p` and `top_k` now "return a 400 error when set to a non-default value on Claude 4.7 and later models" — so a team that migrates to a newer model to escape a retirement can find the new model rejecting a request shape the old one accepted. The migration was the easy part; the parameter was the outage.

This is where the absence of an evaluation harness stops being a hygiene problem and becomes an operational one. When a replacement model is named, someone has to answer whether it is better or worse **on your task** — not on a public benchmark. Without a harness, that question gets answered by shipping and waiting for complaints, which is the most expensive form of testing available.

The 2025 Stack Overflow developer survey suggests how that goes when it is done by feel: **66%** of developers name "AI solutions that are almost right, but not quite" as their biggest frustration, and **45.2%** say debugging AI-generated code takes longer than writing it (Stack Overflow, 2025 Developer Survey: AI). Almost-right is exactly the failure a model swap produces, and it is the hardest kind to notice from a dashboard. Nothing errors. The outputs are simply a bit worse in a way nobody can point at.

Four things make a model change routine rather than a project. **Know your model IDs** — Anthropic's Console has a usage export that breaks spend down by API key and model, which is the fastest way to find the one service still calling something deprecated. **Pin versions explicitly** rather than relying on an alias, so an upgrade is a deliberate commit and not a silent change. **Own an evaluation harness** that runs real past examples, so "is the replacement acceptable" is a command you run and not a debate. And **make sure the deprecation email reaches an engineer**, not a shared inbox nobody reads — notice is worthless if it lands where notices go to die.

The honest test is a single question: if your provider announced today that your model retires in sixty days, is that a Tuesday's work or a quarter's? A team with pinned versions, an eval harness and a known inventory answers Tuesday. A team without them has just discovered that its AI feature was a research project with customers attached.

None of this is an argument for pinning to an old model and hoping. Deprecation exists partly to free capacity for better models, and the replacements are generally better — the point is that *better on average* is not the same as *better for your task*, and only your own examples can tell you which. That is what the AI evaluation and observability work is for, and it is the specific reason the harness described in the evaluation suite exists before it is needed rather than after.

Model drift is the clearest illustration of why enterprise AI solutions need an operating layer rather than a working feature. A feature is a bet that nothing underneath it will change. Everything underneath it is on a published schedule to change. A multi-assistant AI platform we built treats the model as a swappable component behind an interface for exactly this reason: the day a provider retires something, the question is which evaluation run to execute, not which weekend to cancel.