RAG development for assistants that answer from your documents

Suppose a support engineer asks which firmware versions of a device need a manual reset after an update. The answer sits in two approved runbooks and one release note. The system has to find the release note by its exact version number, pick the runbook section that applies and answer with a link to each document. If the release note was withdrawn last week, it must not be used.

We start by reviewing your sources and who may access them. Every passage keeps a reference to its original document, owner and version, so an answer can be traced back. After that review we connect to the systems where the documents already live.

Permissions are checked before any text reaches the model. Someone who cannot open a document in its original system should not get an answer built from it.

• Access matched to the permissions of the original system
• Edits and deletions synchronised, so removed content leaves the index

Documents are split into chunks for search, and a chunk cut out of its document can lose the context that gave it meaning, so we keep that context with it. Search by meaning copes well with everyday phrasing but can miss exact strings such as error codes or part numbers. Where those matter, we add lexical search.

Each answer cites the passages it used. When nothing relevant turns up or the sources disagree, the system says so and can hand the question to a person.

Before launch we build a test set with your team. Retrieval and answers are scored separately, because a better prompt cannot fix a passage that never reached the model. We also track latency and cost per question.

When the system goes live, your team receives the tests, the sync jobs and a runbook for adding sources, changing permissions and investigating a wrong answer.

• Questions the documents answer, with the expected sources
• Questions they cannot answer, where the system should decline
• Questions whose answer changed in a newer document
• Questions from users without access to the source

Some cases need less. Short documents that rarely change can go to the model directly. If your existing search already finds the right document, improving it may be enough, and a managed knowledge base product covers common sources with simple permissions. When one of these fits, we tell you in the first call.

Fine-tuning changes how a model behaves, for example its output format or how it classifies a request. Facts that change every week belong in retrieval, since otherwise each change would need a new training run.