How to stop a RAG assistant from revealing documents people cannot open

Where permissions get lost

In the source system, a document and its access list are stored together. A RAG pipeline copies the text out, splits it into chunks, embeds them and stores them somewhere else. Each step can break the link between a chunk and the people allowed to read it. The failure points are predictable:

• Ingestion runs under a service account that can read everything, and the index stores the text without the access list
• Chunks take the permissions a document had at indexing time, and later changes in the source never reach them
• The application queries the index with its own credentials and never passes the user's identity
• Group membership is resolved once and cached, so someone who left a team keeps access
• Logs, caches and evaluation datasets keep restricted text outside any access control

OWASP lists unauthorised access and leakage between groups sharing a vector store among the main weaknesses of embedding-based systems. Its recommendation is permission-aware retrieval with data partitioned by user or group.

Filter at retrieval time

The design rule is short: the search query itself returns only chunks the user may read, so restricted text never enters the prompt. Search engines implement this in different ways, and the details change what you have to test.

Elasticsearch document-level security attaches a query to a role, and that query can be templated with attributes of the current user. When a user holds several roles, the role queries are combined with OR, so one broad role overrides the narrow ones. Azure AI Search offers query-time ACL and RBAC enforcement, which Microsoft marks as preview, without an SLA and not recommended for production workloads. The application passes the user's token, and the service builds a security filter from permission fields stored in the index.

Whichever engine you use, the identity must come from the authenticated session. It should never come from a parameter that the user, the model or a tool call can set.

Permission changes arrive late

Copying access lists into an index creates a second copy that can go stale. The Azure documentation is explicit: a scheduled SharePoint indexer picks up item-level permission changes on each run, but changes inherited from a parent site, library or folder need a resync, and pushed content must be reingested. Until then, someone whose access was removed can still retrieve the document.

Decide how long that window may be for each source, and build the sync schedule around it. For the most sensitive material, you can also check access against the source system at query time for the few documents about to be used, at the cost of extra latency.

Read the current limits of your engine as well. Azure documents a maximum of 32 ACL entries per file for ADLS Gen2 and 1,000 permission entries per file for SharePoint, and entries beyond the limit might not be enforced. Elastic notes that users restricted by document-level security can still run aggregations that reveal terms or counts from documents they cannot read.

Negative tests for each kind of user

Most RAG test sets check that the right answer comes back. Permission tests check the opposite, and they need real test identities with known access. Assert on the retrieved documents as well as on the final answer, because a model can paraphrase a restricted passage without naming it.

# Permission test cases (hypothetical identities and files)
# Each case runs through the full application path as a real test user.

- id: salary-bands-denied
  user: test-engineer-01        # member of Engineering only
  question: "What is the salary band for senior engineers?"
  must_not_retrieve: [hr/compensation-bands.pdf]
  expect: decline

- id: removed-member
  user: test-former-finance     # removed from Finance in the source system
  question: "Summarise the Q3 board pack"
  must_not_retrieve: [finance/board-pack-q3.pptx]
  expect: decline
  run_after: permission change has synced

- id: missing-identity
  user: none                    # request without a user token
  question: "List open incident reports"
  must_not_retrieve: [any restricted document]
  expect: error or public results only

- id: allowed-control
  user: test-hr-02              # HR member, should succeed
  question: "What is the salary band for senior engineers?"
  must_retrieve: [hr/compensation-bands.pdf]

The last case matters as much as the others, because a filter that blocks everyone passes every negative test. Run the suite after each change to ingestion, group mapping or search configuration, and schedule the removed-member case after a real permission change in the source system.

Fail closed when identity is unclear

When identity resolution fails, the safe outcome is no restricted results. Azure states that if ACL evaluation fails, for example because Microsoft Graph is unavailable, the service returns an error rather than a partially filtered result. Microsoft also changed the behaviour in November 2025: earlier preview API versions returned all documents to calls made with a service key and no user token. Applications built against those versions deserve a check.

Apply the same rule in your own code. A missing token, an unknown group or a timeout in the permission lookup ends in a refusal and a log entry, and the log records the identity and documents involved without storing the restricted text itself.

If you are assessing an assistant you already run, start with the missing-identity case: call the retrieval layer without a user token and look at what comes back.