What a streaming AI response breaks in React

Six things break in a React front-end when the response arrives one token at a time, and not one of them is the model. A token is not a state update, so the obvious implementation re-renders the whole tree hundreds of times per answer. Suspense does not help here, and React's documentation says so outright. The transport has a hard connection limit and a silent reconnect, and both change what the user sees. Cancellation stops being cleanup and becomes a feature, because an abandoned stream keeps generating and keeps billing. Optimistic UI has to be reconciled against a system that is sometimes wrong, which is a different contract from optimistic UI over a database. And loading is no longer a boolean — there are eight states and a boolean holds two of them.

This is the engineering half of an AI feature, not the design half. Where a confidence marker goes, whether a correction propagates, when an approval step earns its friction — those are decisions we have written up separately, and they assume the runtime signals below already exist. What follows is the code: an unfamiliar data source, arriving incrementally, cancellable, occasionally wrong, over a connection the user's network can drop mid-sentence. Ordinary React work, and almost none of it in the tutorial you find by searching for how to stream a chat response.

A token is not a state update. The first version everyone writes appends each chunk to a state variable held near the root of the tree, because that is where the conversation lives. Every chunk then re-renders the provider and everything beneath it, several hundred times per answer, and the interface stops responding while the answer arrives — the exact opposite of what streaming was for. Two moves fix it, both architectural rather than clever. Put the buffer in the smallest component that displays it, or outside React in a ref the display subscribes to, and commit to state on a cadence rather than per token. Then mark everything that is not the user's own typing as non-urgent: React's Transitions exist for this, and "state updates marked as Transitions will be non-blocking and will not display unwanted loading indicators" (React, useTransition, react.dev).

The caveat on that page is the part a stream trips over. "The function you pass to startTransition is called immediately, marking all state updates that happen while it executes as Transitions. If you try to perform state updates in a setTimeout, for example, they won't be marked as Transitions," and so "you must wrap any state updates after any async requests in another startTransition to mark them as Transitions." A token callback is precisely a state update after an async request, so the marking has to be re-applied inside the callback rather than once around the fetch. React also notes that "Transition updates can't be used to control text inputs" — so the composer stays urgent while the answer streaming beside it does not.

Suspense does not help here, and React's documentation says so outright. "Suspense does not detect when data is fetched inside an Effect or event handler. It only activates in the cases listed below" — and those cases are lazy component code, reading a promise with the use hook, stylesheets, fonts, images, and "waiting for a large boundary's HTML to arrive during streaming server rendering" (React, Suspense, react.dev). A token stream is not one promise that resolves; it is many chunks over a connection that stays open. A boundary wrapped around it resolves on the first byte and has no further opinion, which is why teams reach for Suspense, see the fallback flash once, and conclude it works.

Two further caveats on that page decide where boundaries belong. "React does not preserve any state for renders that got suspended before they were able to mount for the first time. When the component has loaded, React will retry rendering the suspended tree from scratch" — so a half-accumulated buffer inside a tree that suspends again is gone. And the reveal has a cadence: "React reveals suspended content at most once every 300ms, measured from the last reveal." That is right for a route and for the conversation history beside it, and wrong for text that should appear as it is produced. Use Suspense for the shell and the history; give the stream its own state.

The transport has a hard connection limit and a silent reconnect. Server-sent events are the usual choice, and MDN is blunt about the ceiling: when "not used over HTTP/2", the protocol "suffers from a limitation to the maximum number of open connections, which can be especially painful when opening multiple tabs, as the limit is per browser and is set to a very low number (6)." That limit "is per browser + domain, which means that you can open 6 SSE connections across all of the tabs" to one host, while "when using HTTP/2, the maximum number of simultaneous HTTP streams is negotiated between the server and the client (defaults to 100)" (MDN, Using server-sent events, MDN Web Docs). Whether the ceiling is 6 or 100 is a deployment question with no automatic answer: the HTTP Archive's Web Almanac found 22% of desktop home pages still served over HTTP/1.1 against 71% on HTTP/2 and 7% on HTTP/3 in its 2024 edition (HTTP Archive, Web Almanac 2024: HTTP). A user with several tabs open on an HTTP/1.1 origin gets a seventh stream that never starts — no error, no console message, nothing to debug from.

The reconnect is the subtler half. MDN states that "by default, if the connection between the client and server closes, the connection is restarted." For a feed of static events that is a feature. For a half-written answer it means the browser silently asks again and the user watches the answer start over, or sees the second attempt appended to the first. Either the server gives each chunk an identifier it can resume from, or the client discards what it already holds. Choosing neither is choosing duplicated text on every flaky train journey.

Cancellation stops being cleanup and becomes a feature. MDN describes the tool exactly: AbortController "represents a controller object that allows you to abort one or more Web requests as and when desired", and its abort method "is able to abort fetch requests, consumption of any response bodies, and streams" (MDN, AbortController, MDN Web Docs). Three moments need it and only the first is usually built: the user presses stop, navigates away, or edits the question and asks again. React's convention of ignoring a stale response is not sufficient here, because ignoring output you are still paying to generate is not cancellation — the signal has to reach the request, and the Effect's cleanup function is where it belongs. Strict Mode is then the cheapest test available: it runs one extra setup-and-cleanup cycle per Effect in development, so an unaborted stream shows up immediately as two streams.

Optimistic UI has to be reconciled against a system that is sometimes wrong. React's useOptimistic gives a precise contract worth reading before designing around it: "optimistic state only renders while an Action is in progress, otherwise value is rendered", there is "no extra render to 'clear' the optimistic state" because "the optimistic and real state converge in the same render when the Transition completes", and on failure "the Transition still ends, and React renders with whatever value currently is", so "a failure means value hasn't changed, so the UI shows what it showed before the optimistic update" (React, useOptimistic, react.dev). The setter is also constrained: it "must be called inside an Action", and outside one "React will show a warning and the optimistic state will briefly render."

That contract is honest about exactly one half of a chat interface. You know what the user typed, so their message can render optimistically and converge with the server's copy unnoticed. You do not know what the model will say, so nothing about the answer can be shown ahead of it — an optimistic result the server then contradicts is not a snappier interface, it is a system that told the user something and took it back. Optimism for the input, never for the output.

Loading is no longer a boolean. Enumerate what a streaming answer can actually be doing and there are eight states: queued, connected but no first token yet, streaming, stopped by the user, interrupted with a partial answer, complete, complete but flagged for review, and failed before any token arrived. Two booleans give four combinations, three of which are impossible and one of which your code will eventually produce. Model it as one discriminated union so an impossible state cannot be represented, and take the pending signal from React rather than a flag you keep in sync — useTransition returns "the isPending flag that tells you whether there is a pending Transition." Then answer the product question the enumeration exposes: what happens to two hundred words of a four-hundred-word answer when the connection dies? Saved, discarded, or kept with an honest marker — and whether the user is charged for the half they got.

The afternoon that finds all six. Type in the composer while an answer streams; lagging keystrokes mean the buffer is too high in the tree. Press stop and watch the network panel to see whether the request actually terminates. Turn the network off at the fortieth token and write down which of the eight states the interface lands in. Open six tabs against an HTTP/1.1 origin and see whether the seventh stream ever starts. Navigate away mid-answer, then check the server log for a generation still running. Switch Strict Mode on and count the duplicate streams. Force the action to throw and confirm the optimistic message disappears instead of persisting. Seven tests, one afternoon, and every failure points at a line rather than a refactor.

None of this is model work, which is the point. A team that has shipped a streaming interface will have opinions about where the buffer lives and what happens on reconnect; a team that has only built request-and-response screens calls all six of these edge cases, and they are the ordinary path. An AI seller assistant we built put a React front-end over a custom-trained model and a scoped integration with a marketplace's own APIs, with an admin surface for uploading and prioritising the documents behind the answers — the interface and the system were specified together, because each constrains the other.

The design half of this argument is designing an interface for a system that is sometimes wrong, which covers confidence display, where corrections go, and when a confirmation step is theatre; it assumes the runtime signals above exist, and this piece is how they come to exist. What an AI app development company must get right is the same question one layer out, for a shipped mobile binary rather than a browser. And the eighth state — complete but flagged for review — exists only if somebody has defined what a wrong answer is, which is AI evaluation and observability work, not front-end work.

Streaming is now the default shape of an AI product's interface, and it is the part of the system the user actually experiences: the wait, the interruption, the stop button, the answer that started over. That makes it the wrong place to discover any of the six above. A React development company working on AI products should be able to answer the seven afternoon questions without opening the code — and if you would rather have the whole system's readiness ranked before committing to a build, a fixed-fee AI readiness assessment produces that roadmap whether or not you build with us.