Guides for teams putting AI to work

Practical answers to the questions that come up when a company funds, builds and runs AI, from choosing the first workflow and rolling out coding agents to retrieval, model costs and platform security.

Planning and funding AI work

Choosing where AI goes first, what a pilot must prove, what a workflow really costs, who owns the result and when to change course.

PostTopic

AI pilot success criteriaWhat an AI pilot has to show before the next stage is funded, and how to write acceptance criteria before anyone chooses a tool.Business case Choosing the first AI workflowHow to choose the first process to automate with AI by scoring every team's proposal on volume, variation, system access, review effort and known outcomes.AI strategy AI build or buyHow to decide between buying an AI product and building around the systems you already run, and what your team owns and maintains in each case.Procurement AI project data readinessThe data owners, system access, representative cases and expert time an AI project needs before it starts, with a start checklist and what can wait.Delivery Full cost of an AI workflowHow to compare AI options on the cost of a handled case, including review, escalations, rework and upkeep, with a worked example and its assumptions.Business case Human review budgetHow to budget reviewer hours before automating a process: review modes, a capacity sheet with peak days, faster reviews and when the review share may fall.Operations AI pilot to productionWhat has to be in place before a successful AI pilot goes live: regression tests, versioning and rollback, monitoring, an owner, a handover and a staged rollout.Delivery AI system ownershipWho owns the prompts, models, data, test set and failures of an AI system once the pilot team leaves, who approves which change and who can switch it off.Operations AI vendor exit planWhat to keep in your own hands so that changing an AI vendor does not mean starting again: tests, prompts, rules, data, a dependency inventory and contract terms.Procurement When to stop an AI projectHow to decide from measured quality and cost per case whether an AI project should continue, narrow its scope, switch to a simpler design or stop.AI strategy AI SDLCHow coding agents change each phase of the software development lifecycle, who owns each handoff, and how to measure and govern the change.Process Autonomous software factoryWhat it takes for coding agents to carry scoped work from issue to merge, which work fits, and where people stay in control.Strategy

Rolling out coding agents

Pilots, repository setup, review, CI and team instructions for engineering teams that hand more of their work to coding agents.

PostTopic

Agentic engineeringWhat agentic engineering means for an engineering team: how it differs from vibe coding, where the work moves, what the research says about results and which foundations decide whether it pays off.Delivery practice Coding agent pilot designHow to compare coding agents on the same work, with random task assignment, a replay of merged changes and a decision rule agreed in advance.Rollout Repository readiness for coding agentsWhat a repository needs before coding agents can work in it reliably, and how to turn the setup instructions into a check that runs in CI.Harness Branch protection for agent pull requestsThe GitHub branch protection and ruleset settings that should apply when coding agents open pull requests, and how to check that they actually hold.Governance Agent changes failing in CIHow to narrow down why a coding agent's change passed in its own environment and failed in CI, and which harness fixes stop it from recurring.Harness Claude Code team instructionsHow a team assigns owners to CLAUDE.md files and rules, reviews them, and retires entries that no longer change what Claude Code does.Claude Code Cursor rules in a monorepoHow to keep Cursor rules relevant to each package in a monorepo, with path-scoped rules, nested AGENTS.md files and an owner for every rule.Cursor GitHub Copilot pilot metricsWhat GitHub's Copilot usage metrics measure, where they stop, and how to combine them with your own review and rework data to decide whether to continue.GitHub Copilot Handing tasks to cloud coding agentsWhat a task should contain before it goes to a cloud coding agent, with a reusable brief, three readiness questions and rules for handing work back.Workflow Coding agent context budgetsHow to reduce the context a coding agent carries without dropping constraints that matter, and how to check on your own tasks that quality held.Cost Testing coding agent upgradesHow to check whether a new model or agent version is better on your repositories before it becomes the team default, with pinned versions and a paired comparison.Evaluation Spec-driven developmentHow to write specs coding agents can follow, take one change from spec to merge, and choose between Spec Kit, OpenSpec, BMad, GSD and Kiro.Delivery practice AI code reviewHow to set up AI code review and keep a person accountable for every merge when coding agents write more of your pull requests.Quality pstack workflowsWhat the pstack plugin for Cursor contains, how its playbooks and model roles work, and how a team adopts it under its own controls.Tooling

Process agents in operation

Testing, approvals, escalation, monitoring and recovery for agents that handle business cases and act in your systems.

PostTopic

Agent test sets from past casesHow to pick past business cases for a process agent's test set and keep a held-out part that nobody tunes against.AI agents Outcome versus trace evaluationWhy an agent run can finish without errors and still handle the case wrongly, and how to check business outcomes separately from traces.AI agents Human approval in agent workflowsWhich agent actions on business records need a person's approval, what the approver must see and how to keep approvals from stalling every case.AI agents Recovering from agent tool failuresWhat a process agent should do when an API times out or a tool returns partial data, with typed failures, bounded retries and a clean handoff.AI agents Idempotency for agent actionsWhere duplicate payments and records come from in agent systems, how to key each business action and why the receiving system must enforce it.AI agents Monitoring a live process agentThe signals that show a live process agent getting more expensive or handling cases worse, measured per case and tied to objectives you agree in advance.AI agents Designing agent escalationsWhen a process agent should hand a case to a person, what the handoff must contain and how to run the queue that receives it.AI agents Shadow mode for process agentsHow to run a process agent on live cases without letting it change anything, compare its proposals with people's decisions and decide when it may act.AI agents Versioning a process agentWhat needs a version when a process agent changes, how to stamp it on every case and how to explain a result months later without rerunning it.AI agents Reviewing agent exceptionsHow to review a production agent's exceptions with a fixed time budget, triage them to an owner and turn the right ones into regression cases.AI agents

Answers from your documents

Preparing sources, keeping permissions intact, checking citations and keeping retrieval assistants accurate as documents change.

PostTopic

RAG source readinessHow to decide which documents a retrieval assistant should use, who owns each source and how to find missing answers before anything is indexed.RAG RAG document permissionsHow access rules travel from source systems into a retrieval index, where they get lost, and which negative tests show that a user cannot retrieve what they may not open.RAG RAG citation qualityHow to test whether the passages an assistant cites actually support each claim it makes, with a claim-level check, a worked example and acceptance criteria.RAG Retrieval or generation?A diagnostic procedure for finding out whether a wrong RAG answer started in ingestion, search, reranking or the model, using labelled evidence and per-question traces.RAG Stale documents in RAGHow to keep a retrieval assistant current when documents are edited, renamed, restricted or deleted, with a sync design and canary tests that prove old content is gone.RAG Chunking business documentsHow to split contracts, policies and long procedures so that an answer stays with its exceptions and definitions, and how to choose a strategy by experiment on your own questions.RAG RAG or fine-tuning?Decision criteria for choosing retrieval, fine-tuning or both: whether the task needs changing knowledge or consistent behaviour, with a worked table and an order of work.RAG Tables and spreadsheets in RAGWhen an assistant should retrieve table rows as text and when it should run a query instead, with routing rules, row formatting for search and safeguards for generated SQL.RAG Multilingual RAG testingHow to test whether an assistant finds the same evidence when people ask in English, German or Polish, with a parallel test set, a language matrix and terminology fixes.RAG RAG production handoverWhat a RAG supplier should hand over so your team can run and change the system: source connectors, configuration, evaluation assets, runbooks, owners and a rehearsal.RAG

Models, inference and cost

When to fine-tune or use a smaller model, how to test it against the current one, and where latency and model spend come from.

PostTopic

Fine-tuning data qualityFour checks that show whether your examples are ready for fine-tuning: label errors, coverage of real traffic, provenance and leakage into the test set.Models Fine-tuned model evaluationHow to show that a fine-tuned model should replace the current one: acceptance criteria written in advance, a fair held-out comparison, and cost and latency in the same table.Models Small model task selectionHow to find the high-volume tasks a small language model can handle reliably: a screening checklist, a test before any training and a cost comparison at your own volume.Models Model distillation trade-offsWhen distilling a large model into a smaller one is worth the training work: what the teacher costs, which method needs what, the quality you can lose and how to test for it.Models LLM latency budgetHow to find out why an AI feature is slow: split each request into queueing, retrieval, model and tool spans, then fix the largest one you can change and measure the tail under load.Performance LLM routing quality controlsHow to send easy requests to cheaper models without hiding quality losses: test the router like a classifier, keep fallbacks separate and log which model actually answered.Costs Prompt caching savingsWhy prompt caching saves less than expected and how to measure it: eligibility rules, the usual causes of cache misses, per-request usage fields and reconciliation with the bill.Costs GPU serving capacity planningHow to size GPU capacity for a model you serve yourself: what limits a replica, how to benchmark it at your latency target, how many replicas peak traffic needs and how to avoid paying for idle hours.Infrastructure Quantisation quality checksWhich tests to rerun before a quantised model replaces the original: task quality, structured output, long inputs, behaviour checks and serving performance, with cost compared per GPU.Models LLM cost allocationHow to explain an LLM bill by team, task and accepted outcome: tag requests at the gateway, price them as you are invoiced, allocate shared costs by a stated rule and reconcile with finance every month.Costs

Platform, integration and security

Connecting AI to enterprise systems through MCP and identity, containing prompt injection, handling credentials and running AI services reliably.

PostTopic

MCP server permissionsHow to connect an assistant to internal systems without giving every MCP tool broad access: scopes, checks inside each tool and separate downstream identities.Security MCP tool designWhy an MCP server should usually offer a few tools shaped around business actions instead of mirroring every API endpoint, with a worked example and a tool contract.Integration MCP and enterprise identityWhose identity an MCP tool should use when it calls a business system: user delegation, token exchange or a service identity, with the token boundaries to check.Security MCP server test planA test plan for an MCP server before users connect: contract tests, negative authorisation tests, failure tests and state-change tests, with checks that run in CI.Quality Prompt injection boundariesHow to limit the damage when an agent follows instructions hidden in a document, email or tool result: map untrusted inputs to actions, cut the paths and test the boundaries.Security Self-hosted AI or managed APIHow to decide whether an AI workload should run on your own infrastructure or stay on a managed model endpoint, using access requirements, operating capacity and measured costs.Infrastructure AI trace data retentionWhat to log to debug agents without keeping every sensitive prompt: three tiers of trace data, content capture off by default, the limits of redaction and retention set by your policy owners.Operations Runbooks for AI featuresWhat on-call engineers should do when an AI feature is slow, unavailable, expensive, wrong or acting outside its limits: symptom classes, runbook entries, safe switches and owners.Reliability Model provider failoverHow to fail over to another model during an outage without silently changing quality, permissions or data terms: a fallback contract, switch triggers, equivalence tests and rollback.Reliability Agent secrets and credentialsWhere credentials used by agents and model integrations should live, how to give each agent its own identity, how to rotate without breaking running workflows and what to do after a leak.Security Coding agent securityPractical security and governance for coding agents: prompt injection, secrets, permissions, sandboxing, unattended runs and ownership.Security

Dates and sources

Models, tools and vendor terms change often, so each guide shows the date it was last checked and lists the sources behind it. What a guide says about a product applies to the versions and documentation it cites.

If something has changed since, tell us and we will update the guide.

Talk to an engineer

Bring one piece of work you would like to improve. The call takes 30 minutes and needs no access to your systems or your data.

Book a 30-minute call

Booking runs on Google Calendar.

Prefer to write first? Leave your email and a few lines, and we reply with times.