Skip to content
Insights · Engineering

What to keep when you subcontract AI

Subcontract the engineering. Keep the model account, the production keys, the golden dataset, and a score you can re-run without the partner.

Scroll
EngineeringOct 13, 202613 min readBy Salman Naqvi, Founder & CEO
What to keep when you subcontract AI

Subcontract the engineering. Keep four things in your own name before the first token is spent: the model-vendor organization, the production API keys, the golden dataset that defines a correct answer, and a run of that evaluation you can execute with the partner out of the room. Those four are what your client is paying you to stand behind. The repository stores the code. A non-solicit deters a person. Neither one holds the account, and the account is where the prompts in production, the spend, the rate limits and the training settings live. If the partner created the organization, they remain the vendor's customer on the day you stop paying them.

Buying the build from someone else is already the ordinary path, which is why the paperwork is behind the practice. Eurostat's 2026 statistical report uses 2024 figures for this question, because "there is no data for the EU as a whole for 2025, as not all countries collected this data." Among EU enterprises with at least ten employees that used at least one AI technology, 57.9% acquired it by purchasing ready-to-use commercial software or systems, 28.6% used technologies developed or modified by external providers, and 21.8% developed their own. The categories overlap, so they are not slices of one pie. Among large enterprises using AI, the external-provider share rises to 45.8% (Eurostat, The use of artificial intelligence (AI) technologies in the European Union, Key results 2026 edition, KS-01-26-009-EN-N, online data code isoc_eb_ai). The denominator is enterprises already using AI, not every firm in the EU. Read it as a description of the market you sell into. Commissioning the system is normal. Building it entirely inside the firm is the smaller path. Normal still leaves the account in somebody's name, and the default somebody is whoever clicked "create organization" on a Monday.

A software subcontract's habits miss that click, and the standards body had already separated the problem from ordinary supply chain. NIST's AI Risk Management Framework gives third-party software and data their own governance category: "Policies and procedures are in place to address AI risks and benefits arising from third-party software and data and other supply chain issues," and pairs it with a contingency requirement — "Contingency processes are in place to handle failures or incidents in third-party data or AI systems deemed to be high-risk" (National Institute of Standards and Technology, Artificial Intelligence Risk Management Framework (AI RMF 1.0), NIST AI 100-1, January 2023, GOVERN 6 and GOVERN 6.2). The same publication defines the third party in terms that describe a delivery partner exactly: entities "responsible for AI design and development tasks, in whole or in part," and "external to the design, development, or deployment team of the organization that acquires its technologies or services." Then the sentence that belongs in the statement of work: "The technologies acquired from third-party entities may be complex or opaque, and risk tolerances may not align with the deploying or operating organization."

Opacity here is not a verdict on the partner's character. It is what you have when the number that means "good enough" was produced by the people being measured, on cases you have not seen, inside an account you cannot open. NIST notes the same gap from the other direction: the organization developing the system "may not be transparent about the risk metrics or methodologies it used." In an agency engagement that sentence has a commercial translation. You sold your client a result. Your partner sold you engineering. If the only evidence the result is acceptable is a figure they computed on a machine you will never log into, you have adopted their metric as yours. The metric has to be an artefact you hold. That artefact is the golden dataset. The account has to be an artefact you hold. That artefact is the vendor organization.

The organization is the deliverable agencies skip, because it does not look like one. A model vendor does not contract with a project. It contracts with the organization that created the workspace: billing, members, usage history, rate limits, and the settings that decide what happens to prompts. One vendor's published rule shows why the name on that organization matters, and it is that vendor's rule — do not copy the details onto a different provider without reading theirs. Anthropic states, for its commercial products: "By default, we will not use your inputs or outputs from our commercial products (e.g. Claude for Work, Anthropic API, Claude Gov, etc.) to train our models." The next sentence is the exception that makes the account holder the decision-maker: "If you explicitly report feedback or bugs to us (e.g. via our thumbs up/down feedback button), or otherwise choose to allow us to use your data, then we may use your chats and coding sessions to train our models." Feedback is stored, not discarded. "When you provide us feedback via our thumbs up / down button, we will store the entire related conversation, including any content, custom styles, conversation preferences, or model settings, in our secured back-end for up to 5 years" (Anthropic, Is my data used for model training?, Privacy Center, 18 August 2026). On a Team or Enterprise plan, the page says a Primary Owner or Owner can disable members' ability to send that feedback. The "you" in all of this is the customer organization.

Follow that through an engagement where the partner opened the workspace on their own card. They can see the spend and the shape of your client's traffic. A person on their side can submit a production conversation as feedback, and the control that would have stopped it sits in an organization you do not administer — where that control exists at all. Anthropic describes the disable switch for Team and Enterprise plans, not as a universal console feature, so the honest move with any other vendor is to ask where the equivalent switch lives and whose login can reach it, rather than assuming a toggle you have not seen. And when the engagement ends, removing the partner is a migration: a new organization, new keys, new limits, a cutover. Removing a member from an organization you already own is an afternoon. Put the organization in your name. When your client's security review requires them to be the contracting party with the model vendor, put it in their name instead, and take a key they issue. The partner's engineers are members you can remove in either arrangement. They are not the owners.

That is the same boundary as access to the client's cloud, applied to a system that is not inside the client's cloud. What access to give a subcontractor covers repositories, staging and production write. The model account is the grant that list does not include. Treat production keys the way that article treats a deploy credential: named humans, issued by you, revocable without the partner's cooperation, and absent from any shared login. Development keys can be looser, because they should be pointed at synthetic data and a separate workspace. A key that can spend production money and read production prompts is not a development convenience. A tool the partner wants to connect, which will send the client's text to a vendor you have not opened, is a different decision: before a tool reads client text.

The golden dataset is how you accept the work. A pull request can be read. A generative system cannot be accepted by reading a handful of outputs and feeling that they look right. The artefact is a set of cases drawn from work like your client's, each with the correct answer recorded, a scorer per case type, a threshold that blocks a release, and a schedule that reruns the set when your code has not changed — because the vendor can change the model underneath you. The evaluation suite is that artefact written out, and AI evaluation and observability is the discipline of keeping the score honest after launch. The partner should help design the cases. They should not be the only party who can run them. A score you cannot reproduce is a status report with a percentage on it. Put the dataset in your repository, versioned, with the command that runs it written in the same place. If the cases contain client records, they inherit the production-data rule: mask them where a masked row still tests the behaviour, and treat the set as production where it does not.

NIST's point about misaligned metrics is the reason the threshold is yours to set. The partner has an incentive to call the system done at the score they can hit this month. Your client has an incentive to hear a higher one in the sales conversation you already had. Those two numbers are allowed to differ, and the difference has to be written down before anyone is grading their own homework. Agree the threshold in the statement of work, name who may change it, and require the last two scores — the release you are accepting, and the one before it — as part of the handover of every milestone. A partner who cannot show the second number has not been measuring. They have been demonstrating.

The prompt that is served is a different object from the prompt in a deck. Teams keep a tidy copy in the repository and another copy in a playground, a vendor console, or a notebook on a laptop that belongs to the partner's firm. Production runs the second one. It includes the system prompt, the tool definitions, the retrieval configuration, and whatever the console holds that the repository does not. Require that production read those from your repository, through your pipeline, the same way a release of ordinary software does. A dashboard that shows the prompt actually in use is how you notice the two copies diverging. We built that monitoring into Inflectiv Helios, a multi-assistant platform: prompt monitoring with dashboards and alerts, on a direct engagement under our own name. It is not an agency subcontract and it should not be read as one. Engagements of this kind sit under the same non-disclosure we would sign with you, so they are not on this site. A partner willing to publish the last agency's customer in order to win you will publish yours to win the next one.

Client content leaves your network on every request, which is the part a software subcontract's confidentiality clause usually fails to describe. The content is in the prompt. The prompt is sent to a model vendor. If the workspace is the partner's, that vendor is downstream of them, and your client's contract may require written authorisation before a further processor exists. If the workspace is yours, the chain is shorter and the authorisation is still a document, not an assumption. The clauses live in a separate piece — what belongs in a white-label subcontract — and none of this is legal advice. The operating rule that sits on top of the paper is short enough to enforce: no production key is issued, and no client row is pasted into a console, until that authorisation is in the file. Development runs on synthetic data. That is slower on the first Monday. It is also the only arrangement you can describe in a security questionnaire without reconstructing it from memory.

The fear principals negotiate first is still poaching: the partner takes the call and the account walks. A non-solicit makes that approach expensive, and it is worth having. It does not stop a quieter move, which is authority migrating to whoever can answer "what did this cost last month" and "what happens when this model is retired." Who your client trusts when you subcontract is that migration in full. The model account is the version of it that leaves no awkward email address behind. If those two questions can only be answered by opening a console at the partner, your client has already learned who operates the system, and they learned it from the invoice and the outage rather than from a solicitation. Keeping the organization in your name is how those answers stay on your letterhead.

None of this is an argument for building the system with your own bench. You are buying the engineering because that bench has not shipped this, which is a respectable reason and a common one. The partner writes the integration, proposes the cases, and sits in the working session. What they do not take home is the organization, the only copy of the dataset, or the right to change what the vendor does with your client's prompts. A white label software development partner who requires the workspace to live in their name, as a condition of starting, is telling you the engagement cannot be ended without their cooperation. That condition is the one to decline. The staffing shape around the work — a person you direct every day, or a team whose own lead directs the engineering while you keep the client — is a separate decision, set out in dedicated team or staff augmentation. Either shape can sit on top of these four things. Neither shape moves them.

Four situations where the subcontract is the wrong purchase even after the account is in your name. You cannot say what a wrong answer is. Then you cannot accept the work, and a build contract prices a guess. Buy a short scoped sprint whose output is the golden dataset and a written definition of correct, and sign the build after you have seen a score on cases you recognise. Your client's agreement forbids a subcontractor, or a further processor, and the authorisation is not yet in writing. The signature you need is that one, and a repository does not substitute for it. You need the partner described to the client as your employee. That is a misrepresentation, and it is a reason to stop. Or the system is the client's, they already hold the vendor organization, and their review names them as the party the model vendor contracts with. The account stays theirs. You hold a key they issued. The partner holds neither the key nor the organization, and your job is to keep it that way when someone asks for a shortcut on a Friday.

There is a version of the shortcut that feels like professionalism and is the opposite. The partner offers to "own the AI platform" so your team is not bothered with vendor administration. Billing in their name, a single invoice to you, keys they rotate, a status line about spend. It reads as service. It is custody. You have hired an administrator for the one asset that decides whether you can fire the administrator. Pay for administration if you want it — a named person who files the deprecation emails and reconciles the invoice — and keep the login that appoints that person. Administration is a task. Ownership of the organization is the right to withdraw the task.

The test is one sitting, before kickoff, and it is pass or fail. Name the organization the production keys belong to, and the inbox that receives the vendor's retirement notice. Name the repository path of the golden dataset and the command that scores it, and confirm that someone on your payroll has run that command once. Name who can turn training feedback off, or confirm in writing that the vendor's current terms do not train on commercial inputs unless the account holder opts in, and that the account holder is you. If any of the three comes back as the partner, you have rented the system you already sold your client. Where the scope is understood and those three are already yours, the build can sit with someone else, and the margin on that arrangement is real. Where they are not, the next signature is the one that moves them. A sprint that starts before that signature is how an agency becomes a pass-through with a nicer deck.

Find this useful? Tell Google to show you more of it.

Let's put AI to work in your business.

A 30-minute call. You bring the workflow or the roadmap — we'll tell you what's feasible, what it costs, and what we'd build first.

Book a call→