What breaks when AI meets a system of record

Five things break when AI meets a system of record, and none of them is the model. **Identity** — the agent acts through a service account with far more reach than the person who asked it for anything. **Freshness** — it answers from an index that is hours behind the record. **Write semantics** — a retried request issues the same refund twice. **Permission inheritance** — the index has flattened access controls the source system was carefully enforcing. **Reconciliation** — nobody can prove afterwards what the system actually changed. Connecting to the CRM is a week of work. These five are the project.

Connectivity misleads precisely because it is easy. The API is documented, authentication is a token in an environment variable, and a first successful write usually happens the same afternoon. That afternoon is what gets demonstrated, and it is what most integration plans are unconsciously sized against. The real gate is not whether the system can reach your records. It is what happens on the ten-thousandth request, when the token turns out to be broader than anyone remembers granting, the index is stale, the network retried, and the record now has two conflicting versions of the truth.

**Identity is the first break, and the oldest problem in the list.** An AI system almost never acts as the user. It acts as itself, holding one credential that has to satisfy every request any user might make — which means it is provisioned for the union of everyone's access, and that union is usually indistinguishable from administrator. The discipline for this was written down long before agents existed. NIST's Zero Trust Architecture, published in August 2020, states that in a zero trust model "there is no implicit trust granted to assets or user accounts based solely on their physical or network location (i.e., local area networks versus the internet) or based on asset ownership," and that the approach "focuses on protecting resources (assets, services, workflows, network accounts, etc.), not network segments, as the network location is no longer seen as the prime component to the security posture of the resource" (NIST, SP 800-207, Zero Trust Architecture, August 2020).

The same document is specific that "authentication and authorization (both subject and device) are discrete functions performed before a session to an enterprise resource is established." An agent is a subject. It needs its own identity, authorised per resource, and — this is the part that gets skipped — a way to carry the requesting human's authorisation alongside its own. A system that cannot answer *on whose behalf* a write happened has an audit problem it has not discovered yet.

**Permission inheritance is the break that surprises people most,** because it is invisible until it is a disclosure. Your source systems enforce access at the row and often the field: a recruiter sees this candidate's record, a support agent sees the order but not the payment method, a regional manager sees their region. A retrieval index does not inherit any of that for free. Documents get embedded, chunks get stored, and the filter that used to sit in the source system's query planner is now something a developer has to reimplement in the vector store — or, more commonly, does not. The test is uncomfortable and takes ten minutes: give two employees with genuinely different entitlements the same question, and compare the answers. If they match, the index is not enforcing anything.

**Freshness is a definitional break, not a bug.** A record is authoritative because it is current; an index is a copy, and a copy has an age. That is fine for discovery — finding which contract mentions a clause — and dangerous for anything a decision hangs on. The workable split is to read through to the source system, live, for any value that drives an action or gets quoted to a customer, and to use the index only to find *where to look*. Where a cached number does get shown, show its timestamp next to it. A stale balance labelled with the time it was read is a usable answer. The same number unlabelled is a liability.

**Write semantics is the break that costs actual money.** Networks retry, queues redeliver, and a language model asked to complete a task twice will happily complete it twice. Every write path into a system of record needs an idempotency key derived from the intent rather than the attempt, so the second delivery of "issue a refund for order 4471" is recognised as the same instruction and not a second refund. Multi-system writes need more than that: if the ticket closes but the ledger entry fails, something has to either complete or reverse the first step, and "something" is a compensating action somebody has to design. Most pilots have neither, which is survivable at demo volume and not at production volume.

**Reconciliation is the break you only notice during the incident.** The requirement is an append-only record of every action the system took: what changed, the value before and after, under which identity and on whose behalf, which rule or threshold sent it to a human instead of acting, and what the human decided. The failure mode we see in practice is not a rogue agent — it is a permission that looked scoped and was not, discovered weeks later, with no trail to bound the damage. An audit log is what converts "we think it only touched contact fields" into a query anyone can run.

It is worth being clear that the people building these systems already sense this. In Stack Overflow's 2025 developer survey, among developers asked about AI agents, **87%** reported concern about the accuracy of agent output and **81%** about the security and privacy of data, while **37.9%** said they do not plan to adopt agents at all and roughly **52%** either avoid them or prefer simpler tools (Stack Overflow, 2025 Developer Survey: AI). The same survey found **66%** naming "AI solutions that are almost right, but not quite" as their biggest frustration. Almost-right is exactly what a system of record cannot absorb: a slightly wrong summary is a bad afternoon, and a slightly wrong ledger entry is a restatement.

None of the five breaks is novel enough to need a novel framework, which is reassuring rather than disappointing. NIST's AI Risk Management Framework, released on 26 January 2023, organises this territory into four functions — Govern, Map, Measure and Manage — and describes itself as intended "for voluntary use and to improve the ability to incorporate trustworthiness considerations into the design, development, use, and evaluation of AI products, services, and systems" (NIST, AI Risk Management Framework, version 1.0, January 2023). Voluntary is the operative word again. Identity design, index-level authorisation, idempotent writes and an audit trail are all things a team can simply not do, and the system will look identical until the day it does not.

The practical version of all this is a five-question checklist you can run against a design review rather than a vendor deck. Whose identity does the write happen under, and can you prove it? Does retrieval filter by the requester's entitlements at query time, or afterwards? For every number the system shows, is it read live or is its age displayed? Does every write carry an idempotency key, and what reverses a half-completed multi-system change? And can you reconstruct, from a log rather than from memory, what the system did last Tuesday afternoon and why? A design that answers five out of five is ready for production traffic. A design that answers two is a demo with database credentials.

This is also why the sequencing that works is unglamorous: build the identity and audit layer around the first workflow, at whatever size that workflow is, then add the second workflow onto a layer that already exists. The enterprise AI solutions that survive contact with a CRM are the ones where this layer predates the feature. It is the same discipline behind AI integration services enterprise, and the reason a 150-branch student information system we built was designed with per-branch data isolation and audit trails from the first sprint rather than retrofitted once 150 branches were live — retrofitting isolation under production traffic is the most expensive version of that work.

If you want the security-review version of this argument, connecting AI to systems of record covers the scoped-credential, audit-trail and human-approval controls in more depth, and what an enterprise AI operating system actually includes puts the same layer in the wider frame. The short version: the integration is not the hard part, and treating it as the hard part is how a working demo becomes an unrecoverable incident.