Buying an AI product or building around the systems you already run

Three options, usually

The question is framed as build or buy, but most teams choose between three things. A finished product comes with its own interface, workflow and model. An assembled solution buys the model access and platform pieces and builds only the process layer that connects them to your systems. A deeper build adds your own models or inference infrastructure. For process automation the middle option deserves a test, because several vendors sell comparable model access while the connection to your claims, ERP or ticket system exists nowhere else.

Questions that decide it

• Does the product's workflow match yours? Adopting a vendor's process is fine for commodity work such as meeting notes. It is expensive when your process carries rules the product cannot express.
• Can it reach the data without copies? A product that needs exports of your records adds a data flow to approve and keep in sync. The AWS Generative AI Lens tells teams to understand the data usage policies of model hosting providers during model selection.
• Who decides what happens when it is wrong? Check whether the product lets you set confidence thresholds and escalation paths, and how far. If your error handling must follow rules the product cannot express, you need control over that layer.
• What does the vendor actually commit to? The Google SRE book offers a quick test: if missing the target has no consequence, it is an objective, not an agreement.
• Will the vendor run your held-back cases before you sign? Without that, you are comparing demos.
• What does a handled case cost at your volume? The FinOps Foundation describes mature unit economics as the point where cost per unit starts to shape build or buy decisions, rather than being reported after the bill arrives.

What you maintain after the contract is signed

Sculley and colleagues at Google described in 2015 how machine learning systems collect ongoing maintenance costs through data dependencies, changes in the outside world and consumers nobody declared. How much of that work moves to a vendor depends on the product and the contract, so treat the list below as questions to answer for each candidate.

• Model and prompts: when you buy, check who controls them and whether the contract requires notice before the vendor changes them. When you build, they are yours, including the move to a new model when the old one is retired.
• Integration: check whether the vendor maintains the connectors to your systems or you do. Either way, someone has to keep them working when those systems change.
• Test set: yours in both cases. When you buy, it is how you check that a vendor update did not make results worse.
• Review process and error handling: yours in both cases. Check how far the product lets you shape them before you commit.

A hypothetical claims intake decision

Assume an insurer receives claims by email and portal. Each needs its documents classified, key fields extracted, completeness checked against the policy and the case created in a claims system that has an API. Suppose the document processing product it trials extracts fields well, but its review queue sits outside the claims system and it cannot see policy terms. Claims handlers would work in two tools, and completeness would still be checked by hand.

A reasonable split is to buy the extraction, which is generic, and build the process layer: the completeness check against policy data, the routing and the review inside the claims system where handlers already work. The insurer then maintains the process layer and the test set, and the vendor maintains extraction. Whether this beats either pure option depends on volumes and prices the insurer has to measure, not assume.

How each choice goes wrong

A bought product that does not fit the process tends to fail quietly through workarounds: staff copy results between tools, keep side spreadsheets, and nobody can see why a case went wrong. A build fails through upkeep that nobody planned, or through framework layers that hide the prompts and responses, a risk Anthropic's 2024 article on agents warns about. Both fail when the project team leaves and nobody owns the system.

A time-boxed comparison before you sign

For a fixed period you agree in advance, run the same held-back cases through the vendor's trial and through a thin prototype built on a hosted model and your own API. Compare quality, review effort, cost per handled case and the list of what you would maintain. If the product wins on all four, buy it and keep the test set. Our AI engineering work covers the build side, with the process layer running in your environment.

Planning how to leave an AI vendorWho owns an AI system after the pilot