Why retrieval is a data access event, not a search event
The mental model most teams use when building a RAG pipeline treats retrieval as a search step: the query goes in, relevant chunks come back, the LLM synthesizes them into an answer. The security framing is usually applied to the output: add a filter that blocks certain content from appearing in the response, and the problem is solved. That framing misses where the exposure actually happens.
When a restricted document enters the LLM context window, the model has already read it. Filtering the response afterward prevents the user from seeing the text in the UI, but the LLM has processed the content, may have used it in its reasoning, and in some configurations will retain it in conversation memory. For regulated industries, what matters under the access framework is whether the system retrieved, processed, and exposed PHI or confidential financial data to an unauthorized context. Whether the user saw the restricted text in the final response is a secondary question. HIPAA's minimum necessary standard, SOC 2's access control criteria, and the EU AI Act's data governance requirements all apply at the point of data access, which in a RAG system is the retrieval step, not the display step.
The five decisions below address the architecture points where enterprise RAG pipelines most commonly create unauthorized data access. Each one corresponds to a gap between how the pipeline was designed and how the source system's access model actually works.
Output filters prevent users from reading restricted text. They do not prevent the LLM from processing it. In regulated environments, those are two different compliance requirements.
The 5 access control decisions and the breach scenario each one prevents
These decisions apply to any RAG pipeline that queries documents containing personal health information, customer financial records, internal legal or compliance content, or any data whose access is governed by organizational policy or regulation. A RAG system that only queries public or fully open internal content faces fewer of these concerns, but any pipeline that handles differentiated access across users or roles should evaluate all five.
| Access control decision | What most teams do | The compliance gap it creates | How it surfaces | Risk level |
|---|---|---|---|---|
| Enforce permissions at retrieval time: filter the candidate chunk set before it reaches the context window, not after the response is generated | A post-generation output filter scans the LLM's response for restricted content and redacts or blocks it before display. The retrieval step itself does not check the requesting user's permissions against each candidate chunk. All documents in the index are equally reachable at query time | Exposure before redaction. The LLM has already processed the restricted document by the time the output filter runs. For HIPAA, the minimum necessary standard applies to access, not display. The organization accessed PHI it was not authorized to surface for that user, even if the display was redacted. For SOC 2, the access control criterion requires that access to data be restricted to authorized users, which is not satisfied by filtering the visible output of a system that retrieved the data without authorization | A hospital's RAG system for clinical staff uses post-generation filtering to prevent patient records from appearing in responses to billing queries. During a HIPAA audit, the organization's IT team demonstrates that the vector index retrieves patient records as context for billing queries before filtering runs. The auditor determines that PHI was accessed outside the minimum necessary standard on every billing query that retrieved clinical context, regardless of whether the filtered output displayed it. Finding: systematic minimum-necessary violation in RAG retrieval architecture | Critical |
| Isolate tenant data with separate vector namespaces or indexes, not just metadata filters on a shared index | A single vector index holds embeddings from all tenants or organizational units. Tenant separation is enforced by adding a tenant-id metadata field to each chunk at ingestion and including a metadata filter in every query. The embeddings themselves live in the same vector space | Metadata filter bypass. Semantic similarity operates across the full embedding space, not within the metadata-filtered subset. A query from Tenant A can retrieve chunks from Tenant B if the embedding similarity is high enough to overcome the metadata filter, particularly when the filter is applied post-ranking rather than pre-ranking. In multi-tenant SaaS platforms or healthcare systems where multiple organizations share an instance, this creates cross-tenant data exposure. The 2025 ConfusedPilot research (arXiv 2408.04870) demonstrated that production RAG systems can be manipulated to surface documents outside their intended access boundary through adversarial query construction, even with metadata controls in place | A fintech SaaS platform uses a single Pinecone index with a client-id metadata filter to segregate customer data across its RAG-powered document assistant. A security researcher constructs a query embedding that is semantically similar to both their own account documents and a competitor's loan terms, and the similarity scoring returns competitor document chunks before the metadata filter removes them from the final context. The researcher submits a responsible disclosure. Finding: metadata-only tenant isolation in a shared embedding space does not provide adequate data segregation for financial services customer data | Critical |
| Sync permissions from source systems on a schedule: access metadata set at ingestion goes stale as roles change and people leave | When a document is ingested into the vector index, its permissions are read from the source system once and stored as metadata on the chunk. That metadata is not updated unless the document is re-ingested. User role changes, terminations, and policy updates in the source system are not propagated to the index | Stale access grants. A user who has been removed from a document's access list in SharePoint or the EHR system can still retrieve that document via RAG if the index was not updated after the permission change. In healthcare, a clinician who transfers departments or leaves the organization retains retrieval access to patient records from their previous role until the index is re-indexed. HIPAA requires that access be terminated when no longer authorized; a stale permission cache in the vector index does not satisfy that requirement. The same gap applies in fintech when a broker or analyst changes desks and their access to customer portfolio documents is updated in the source system but not in the RAG index | A hospital system terminates a clinical staff member's employment and immediately revokes their EHR access. The RAG-based clinical assistant, however, was indexed with permissions captured during deployment. The former employee's account, which was not fully deprovisioned in the RAG system's identity store, can still query the assistant and retrieve patient records for 11 days until the next scheduled full re-index. The incident is discovered during a security review. Finding: RAG access control system not synchronized with HR offboarding workflow, creating a HIPAA breach window | High |
| Screen retrieved chunks for prompt injection before inserting them into the LLM context window | Retrieved chunks are inserted directly into the prompt template without any preprocessing. Document content is treated as trusted retrieval output. No check is performed for injected instructions embedded in document text | Retrieval-path prompt injection. Documents in the index can carry instructions directed at the LLM, embedded as plain text in the document body. When those documents are retrieved and placed in the context window, the instructions execute as part of the LLM's reasoning chain. This is the attack vector used in the ConfusedPilot research: a crafted instruction embedded in a Confluence page or SharePoint document was retrieved by Copilot and caused it to surface and summarize documents the querying user had not directly accessed. In a healthcare setting, an injected instruction in a patient-submitted form or a note could cause the clinical assistant to retrieve and leak records it would otherwise not have surfaced. The injection requires no access to the application layer or the vector index | A financial services firm uses a RAG assistant to help advisors answer compliance questions from internal policy documents. A document submitted by an external vendor as part of a procurement response is ingested into the compliance knowledge base. The vendor has embedded an instruction in the document body directing the LLM to append recent deal summaries to its next response. When a compliance advisor queries the assistant, the retrieved chunk executes the instruction and the response includes internal deal data that the vendor's document was intended to surface. Finding: unscreened document ingestion pipeline with no prompt injection detection prior to context insertion | High |
| Log the retrieval event as a data access event: audit the query identity, the retrieved document IDs, and the timestamp, not just the generated response | Application logs capture the user's query and the LLM's generated response. The intermediate retrieval step, including which documents were retrieved, which chunks were selected, and the identity under which retrieval was performed, is not logged. Access audit trails for the RAG system cover only application-level activity | Invisible data access. Under HIPAA, the access of PHI must be logged with the user's identity, the records accessed, the date, and the action taken. Under SOC 2 CC6.1 and CC6.3, access to sensitive data must be traceable. In a RAG system, the retrieval event is the data access event, since that is when the system reads and processes document content on behalf of a user. Logging only the generated response does not capture which records were accessed, which is what both audit frameworks require. When a HIPAA breach investigation asks which records were accessed by a specific user on a specific date, an application log that shows only queries and responses cannot answer the question | A healthcare organization's RAG-based clinical assistant experiences a suspected insider data access incident. The security team pulls application logs for the user in question and finds query text and response content. No log captures which patient records were retrieved as context for those queries. The organization cannot reconstruct the patient-level access record needed to notify patients under HIPAA's breach notification rule, because the retrieval layer was never instrumented. Finding: inadequate PHI access logging in RAG retrieval layer, breach notification scope cannot be determined | Moderate |
The two Critical-level gaps share an architecture assumption: that security controls at the application or response layer can substitute for security controls at the retrieval layer. They cannot. The LLM context window is a data processing event, and what enters it is already accessed. All controls that matter for access authorization need to operate before the retrieval step places content in the context window.
Not sure where your RAG pipeline's access control gaps are?
10decoders reviews enterprise RAG security architecture for healthcare and fintech organizations: retrieval-time enforcement design, tenant isolation architecture, permission sync pipeline setup, prompt injection screening, and audit logging instrumentation. We produce a gap report against your regulatory framework before the first security incident forces the question.
Book a Free AI Assessment →Why the metadata filter alone is a weaker control than it appears
Metadata filtering on a shared vector index feels like a solid access control because it looks like the access control model in every other enterprise system: assign attributes to data, check attributes at query time. The difference in a vector database is that similarity search and metadata filtering are two separate operations, and the order in which they run matters.
When a query arrives, most vector databases compute similarity scores across all vectors and then apply metadata filters to the top-k results. This means the similarity computation has already processed vectors from outside the permitted set before the filter removes them. In many implementations, this is handled correctly, with pre-filtering narrowing the candidate set before similarity scoring. But the default behavior of several widely deployed vector databases applies post-filter ranking, and teams building their first RAG pipeline often do not know which mode their database is using or whether their query configuration is set correctly. Verifying this requires a deliberate test: construct a query as a tenant that has no access to documents in another tenant's namespace, confirm the similarity scores are computed only within the permitted namespace, and check the database documentation against the actual query plan being used.
Separate namespaces or separate indexes solve this cleanly, because the similarity computation never crosses the boundary. The tradeoff is higher operational overhead: more indexes to manage, more infrastructure to provision, and query routing logic that must correctly identify which index to query for each request. For healthcare and fintech deployments where data segregation is a regulatory requirement rather than a preference, that overhead is not optional. The metadata filter approach is an engineering convenience that approximates the control, and the ConfusedPilot research showed that approximations in production RAG systems can be breached with query-level manipulation rather than infrastructure access.
No retrieval-layer controls
Every permission check runs post-generation on the output filter. The index is shared with no tenant isolation, access metadata was set once at ingestion, and retrieved chunks enter the context window without any screening. The retrieval layer writes no audit log. Any HIPAA or SOC 2 audit will find access control gaps at the retrieval layer.
Retrieval-time filtering in place
Permissions are enforced at retrieval time, not output time. Metadata filters run on a shared index with pre-filter ranking verified. Permission sync has a schedule but offboarding workflows still have gaps. Prompt injection screening is not yet in place. Retrieval events are logged with document IDs, though full user-identity attribution is missing across some connected systems.
Auditable, isolated, monitored
All five decisions are in place before go-live. Permissions are enforced at retrieval time with pre-filter ranking verified. Tenant data lives in separate namespaces or indexes. Permission sync is integrated with HR offboarding and role-change workflows. All retrieved chunks pass through injection screening before entering the context window. Retrieval events are logged with user identity, document IDs, and timestamps, satisfying HIPAA access log and SOC 2 CC6.1 requirements.
The enterprise RAG access control checklist
These are the security and compliance controls an organization should have in place before a RAG pipeline that queries regulated or access-differentiated content goes to production. Each item corresponds to one of the five decisions above, and each is the type of documentation an auditor or security reviewer would request when assessing the pipeline's access control design.
The LLM context window is a data access event. Access control frameworks that apply only at the response layer will not satisfy HIPAA or SOC 2 requirements, because the data was already accessed when retrieval ran.
What to do this week
Trace one real query through your retrieval layer and check where permissions are verified
Pick a production query from your RAG system and trace it through the pipeline, step by step, from the user's request to the chunks placed in the context window. For each step, ask whether a permission check is running and what it is checking against. If the first access control check you find is on the generated response rather than on the retrieved chunks, your retrieval layer has no enforcement and every query in that system is a potential access control violation for regulated data. Document what you find. The trace itself takes less than an hour on most pipelines and answers the question most teams do not ask until an audit makes it urgent.
Test your tenant isolation by querying across the boundary
If your RAG system serves multiple tenants or organizational units from a shared vector index, construct a test query as a tenant with no authorized access to another tenant's documents. Confirm that the retrieved chunks contain no content from outside the permitted boundary. If your database uses post-filter ranking, the similarity scores may have already considered out-of-boundary vectors even if they are excluded from the final context. Check the database documentation for whether your query configuration uses pre-filter or post-filter ranking, and document which approach is in use. For healthcare and fintech deployments where tenant data segregation is a regulatory requirement, test results confirming the isolation mechanism are the kind of evidence that an audit or a security review will request.
Check whether your permission metadata is current for users who left in the past 90 days
Pull a list of users who had their access revoked in your source systems in the past 90 days. For each one, check whether their permissions are reflected in the RAG index. If you find any user who was deprovisioned in the source system but whose access metadata in the index was not updated, you have confirmed a permission sync gap. That gap represents a window during which the user could have queried the RAG system and received content they were no longer authorized to access. The remediation is to close the sync gap, but the first step is knowing whether it exists, and that check takes a few queries against your index metadata and your HR or identity system records.
Verify that your retrieval events are logged with document IDs, not just query and response text
Pull your RAG system's logs for a known query from last week. Find the log entries associated with that query and check whether the retrieved document or chunk IDs are present. If the log shows only the query text, the user identity, and the generated response, the retrieval layer is not instrumented. Add logging to the retrieval step before the pipeline handles any regulated data at scale. The instrumentation is typically a few lines of code at the point where chunks are selected for the context window, and it converts a silent data access event into an auditable record. Without it, a breach investigation asking which documents were accessed by which users on which dates cannot be answered from the logs, which creates both a notification scope problem under HIPAA and an access control evidence gap under SOC 2.
Let 10decoders secure your RAG pipeline before the first access control incident
We review enterprise RAG security architecture for healthcare and fintech organizations: retrieval-time enforcement design, tenant isolation testing, permission sync pipeline setup, prompt injection screening, and audit log instrumentation. Our AI engineering practice has built production RAG systems for organizations where HIPAA compliance and SOC 2 auditability are requirements, not optional. We produce a gap report against your regulatory framework with specific, actionable remediation steps.
