Why this matters now: Enterprise RAG deployments in healthcare and fintech have moved from pilots to production systems that query real patient records, loan documents, internal compliance policies, and customer account data. The vector index sits behind the application, and in most implementations it has no connection to the access control model of the source systems. Documents ingested from SharePoint, Confluence, or a clinical document management system bring their permission metadata at ingestion time, if at all. Query-time retrieval then proceeds without verifying whether the requesting user is authorized to see the content. A 2026 security review of a live patient-facing healthcare RAG application found critical access control failures reachable without any authenticated access, using standard browser developer tools (arXiv 2605.00796). That was a research finding. For organizations in HIPAA-regulated or SOC 2-audited environments, the same failure mode is a breach.

Why retrieval is a data access event, not a search event

The mental model most teams use when building a RAG pipeline treats retrieval as a search step: the query goes in, relevant chunks come back, the LLM synthesizes them into an answer. The security framing is usually applied to the output: add a filter that blocks certain content from appearing in the response, and the problem is solved. That framing misses where the exposure actually happens.

When a restricted document enters the LLM context window, the model has already read it. Filtering the response afterward prevents the user from seeing the text in the UI, but the LLM has processed the content, may have used it in its reasoning, and in some configurations will retain it in conversation memory. For regulated industries, what matters under the access framework is whether the system retrieved, processed, and exposed PHI or confidential financial data to an unauthorized context. Whether the user saw the restricted text in the final response is a secondary question. HIPAA's minimum necessary standard, SOC 2's access control criteria, and the EU AI Act's data governance requirements all apply at the point of data access, which in a RAG system is the retrieval step, not the display step.

The five decisions below address the architecture points where enterprise RAG pipelines most commonly create unauthorized data access. Each one corresponds to a gap between how the pipeline was designed and how the source system's access model actually works.

Output filters prevent users from reading restricted text. They do not prevent the LLM from processing it. In regulated environments, those are two different compliance requirements.
$6.64M
Average cost of a healthcare data breach in 2026, per IBM's annual Cost of a Data Breach Report. Healthcare has ranked highest among all industries for 13 consecutive years. Financial services is second at $6.29M. Both are primary production targets for enterprise RAG
6.48%
Share of enterprise AI conversations found to contain sensitive data, per LayerX's 2026 AI usage survey. At 10,000 RAG queries per day in a large enterprise deployment, that is 648 potential access-control events daily that most pipelines do not audit at the retrieval level
0
Authentication steps required to reach critical access control failures in a production patient-facing healthcare RAG application, per a 2026 academic security review (arXiv 2605.00796). The failures were reachable with standard browser developer tools and no prior credentials

The 5 access control decisions and the breach scenario each one prevents

These decisions apply to any RAG pipeline that queries documents containing personal health information, customer financial records, internal legal or compliance content, or any data whose access is governed by organizational policy or regulation. A RAG system that only queries public or fully open internal content faces fewer of these concerns, but any pipeline that handles differentiated access across users or roles should evaluate all five.

Access control decisionWhat most teams doThe compliance gap it createsHow it surfacesRisk level
Enforce permissions at retrieval time: filter the candidate chunk set before it reaches the context window, not after the response is generatedA post-generation output filter scans the LLM's response for restricted content and redacts or blocks it before display. The retrieval step itself does not check the requesting user's permissions against each candidate chunk. All documents in the index are equally reachable at query timeExposure before redaction. The LLM has already processed the restricted document by the time the output filter runs. For HIPAA, the minimum necessary standard applies to access, not display. The organization accessed PHI it was not authorized to surface for that user, even if the display was redacted. For SOC 2, the access control criterion requires that access to data be restricted to authorized users, which is not satisfied by filtering the visible output of a system that retrieved the data without authorizationA hospital's RAG system for clinical staff uses post-generation filtering to prevent patient records from appearing in responses to billing queries. During a HIPAA audit, the organization's IT team demonstrates that the vector index retrieves patient records as context for billing queries before filtering runs. The auditor determines that PHI was accessed outside the minimum necessary standard on every billing query that retrieved clinical context, regardless of whether the filtered output displayed it. Finding: systematic minimum-necessary violation in RAG retrieval architectureCritical
Isolate tenant data with separate vector namespaces or indexes, not just metadata filters on a shared indexA single vector index holds embeddings from all tenants or organizational units. Tenant separation is enforced by adding a tenant-id metadata field to each chunk at ingestion and including a metadata filter in every query. The embeddings themselves live in the same vector spaceMetadata filter bypass. Semantic similarity operates across the full embedding space, not within the metadata-filtered subset. A query from Tenant A can retrieve chunks from Tenant B if the embedding similarity is high enough to overcome the metadata filter, particularly when the filter is applied post-ranking rather than pre-ranking. In multi-tenant SaaS platforms or healthcare systems where multiple organizations share an instance, this creates cross-tenant data exposure. The 2025 ConfusedPilot research (arXiv 2408.04870) demonstrated that production RAG systems can be manipulated to surface documents outside their intended access boundary through adversarial query construction, even with metadata controls in placeA fintech SaaS platform uses a single Pinecone index with a client-id metadata filter to segregate customer data across its RAG-powered document assistant. A security researcher constructs a query embedding that is semantically similar to both their own account documents and a competitor's loan terms, and the similarity scoring returns competitor document chunks before the metadata filter removes them from the final context. The researcher submits a responsible disclosure. Finding: metadata-only tenant isolation in a shared embedding space does not provide adequate data segregation for financial services customer dataCritical
Sync permissions from source systems on a schedule: access metadata set at ingestion goes stale as roles change and people leaveWhen a document is ingested into the vector index, its permissions are read from the source system once and stored as metadata on the chunk. That metadata is not updated unless the document is re-ingested. User role changes, terminations, and policy updates in the source system are not propagated to the indexStale access grants. A user who has been removed from a document's access list in SharePoint or the EHR system can still retrieve that document via RAG if the index was not updated after the permission change. In healthcare, a clinician who transfers departments or leaves the organization retains retrieval access to patient records from their previous role until the index is re-indexed. HIPAA requires that access be terminated when no longer authorized; a stale permission cache in the vector index does not satisfy that requirement. The same gap applies in fintech when a broker or analyst changes desks and their access to customer portfolio documents is updated in the source system but not in the RAG indexA hospital system terminates a clinical staff member's employment and immediately revokes their EHR access. The RAG-based clinical assistant, however, was indexed with permissions captured during deployment. The former employee's account, which was not fully deprovisioned in the RAG system's identity store, can still query the assistant and retrieve patient records for 11 days until the next scheduled full re-index. The incident is discovered during a security review. Finding: RAG access control system not synchronized with HR offboarding workflow, creating a HIPAA breach windowHigh
Screen retrieved chunks for prompt injection before inserting them into the LLM context windowRetrieved chunks are inserted directly into the prompt template without any preprocessing. Document content is treated as trusted retrieval output. No check is performed for injected instructions embedded in document textRetrieval-path prompt injection. Documents in the index can carry instructions directed at the LLM, embedded as plain text in the document body. When those documents are retrieved and placed in the context window, the instructions execute as part of the LLM's reasoning chain. This is the attack vector used in the ConfusedPilot research: a crafted instruction embedded in a Confluence page or SharePoint document was retrieved by Copilot and caused it to surface and summarize documents the querying user had not directly accessed. In a healthcare setting, an injected instruction in a patient-submitted form or a note could cause the clinical assistant to retrieve and leak records it would otherwise not have surfaced. The injection requires no access to the application layer or the vector indexA financial services firm uses a RAG assistant to help advisors answer compliance questions from internal policy documents. A document submitted by an external vendor as part of a procurement response is ingested into the compliance knowledge base. The vendor has embedded an instruction in the document body directing the LLM to append recent deal summaries to its next response. When a compliance advisor queries the assistant, the retrieved chunk executes the instruction and the response includes internal deal data that the vendor's document was intended to surface. Finding: unscreened document ingestion pipeline with no prompt injection detection prior to context insertionHigh
Log the retrieval event as a data access event: audit the query identity, the retrieved document IDs, and the timestamp, not just the generated responseApplication logs capture the user's query and the LLM's generated response. The intermediate retrieval step, including which documents were retrieved, which chunks were selected, and the identity under which retrieval was performed, is not logged. Access audit trails for the RAG system cover only application-level activityInvisible data access. Under HIPAA, the access of PHI must be logged with the user's identity, the records accessed, the date, and the action taken. Under SOC 2 CC6.1 and CC6.3, access to sensitive data must be traceable. In a RAG system, the retrieval event is the data access event, since that is when the system reads and processes document content on behalf of a user. Logging only the generated response does not capture which records were accessed, which is what both audit frameworks require. When a HIPAA breach investigation asks which records were accessed by a specific user on a specific date, an application log that shows only queries and responses cannot answer the questionA healthcare organization's RAG-based clinical assistant experiences a suspected insider data access incident. The security team pulls application logs for the user in question and finds query text and response content. No log captures which patient records were retrieved as context for those queries. The organization cannot reconstruct the patient-level access record needed to notify patients under HIPAA's breach notification rule, because the retrieval layer was never instrumented. Finding: inadequate PHI access logging in RAG retrieval layer, breach notification scope cannot be determinedModerate

The two Critical-level gaps share an architecture assumption: that security controls at the application or response layer can substitute for security controls at the retrieval layer. They cannot. The LLM context window is a data processing event, and what enters it is already accessed. All controls that matter for access authorization need to operate before the retrieval step places content in the context window.

Not sure where your RAG pipeline's access control gaps are?

10decoders reviews enterprise RAG security architecture for healthcare and fintech organizations: retrieval-time enforcement design, tenant isolation architecture, permission sync pipeline setup, prompt injection screening, and audit logging instrumentation. We produce a gap report against your regulatory framework before the first security incident forces the question.

Book a Free AI Assessment →

Why the metadata filter alone is a weaker control than it appears

Metadata filtering on a shared vector index feels like a solid access control because it looks like the access control model in every other enterprise system: assign attributes to data, check attributes at query time. The difference in a vector database is that similarity search and metadata filtering are two separate operations, and the order in which they run matters.

When a query arrives, most vector databases compute similarity scores across all vectors and then apply metadata filters to the top-k results. This means the similarity computation has already processed vectors from outside the permitted set before the filter removes them. In many implementations, this is handled correctly, with pre-filtering narrowing the candidate set before similarity scoring. But the default behavior of several widely deployed vector databases applies post-filter ranking, and teams building their first RAG pipeline often do not know which mode their database is using or whether their query configuration is set correctly. Verifying this requires a deliberate test: construct a query as a tenant that has no access to documents in another tenant's namespace, confirm the similarity scores are computed only within the permitted namespace, and check the database documentation against the actual query plan being used.

Separate namespaces or separate indexes solve this cleanly, because the similarity computation never crosses the boundary. The tradeoff is higher operational overhead: more indexes to manage, more infrastructure to provision, and query routing logic that must correctly identify which index to query for each request. For healthcare and fintech deployments where data segregation is a regulatory requirement rather than a preference, that overhead is not optional. The metadata filter approach is an engineering convenience that approximates the control, and the ConfusedPilot research showed that approximations in production RAG systems can be breached with query-level manipulation rather than infrastructure access.

Stage 1
Unprotected retrieval

No retrieval-layer controls

Every permission check runs post-generation on the output filter. The index is shared with no tenant isolation, access metadata was set once at ingestion, and retrieved chunks enter the context window without any screening. The retrieval layer writes no audit log. Any HIPAA or SOC 2 audit will find access control gaps at the retrieval layer.

Stage 2
Partial access control

Retrieval-time filtering in place

Permissions are enforced at retrieval time, not output time. Metadata filters run on a shared index with pre-filter ranking verified. Permission sync has a schedule but offboarding workflows still have gaps. Prompt injection screening is not yet in place. Retrieval events are logged with document IDs, though full user-identity attribution is missing across some connected systems.

Stage 3
Full access control baseline

Auditable, isolated, monitored

All five decisions are in place before go-live. Permissions are enforced at retrieval time with pre-filter ranking verified. Tenant data lives in separate namespaces or indexes. Permission sync is integrated with HR offboarding and role-change workflows. All retrieved chunks pass through injection screening before entering the context window. Retrieval events are logged with user identity, document IDs, and timestamps, satisfying HIPAA access log and SOC 2 CC6.1 requirements.

The enterprise RAG access control checklist

These are the security and compliance controls an organization should have in place before a RAG pipeline that queries regulated or access-differentiated content goes to production. Each item corresponds to one of the five decisions above, and each is the type of documentation an auditor or security reviewer would request when assessing the pipeline's access control design.

Enterprise RAG access control checklist
Retrieval-time permission enforcement: document the query path and confirm permissions are checked before the candidate set is assembled.Map the query flow from the user's request to the chunks placed in the context window. For each step, identify where permissions are checked and confirm that the check narrows the candidate set before similarity scoring, not after. If the access control check runs against the response after generation, replace it with a retrieval-time filter that queries only documents the requesting user is authorized to see. Test the enforcement by querying the system as a user who has no access to a known document and confirming the document does not appear in the retrieved set, not just the response. Document the enforcement mechanism, the access policy source it queries against, and the test results. This documentation is what a HIPAA or SOC 2 auditor will request when assessing whether access is restricted to authorized users at the point of data access.
Tenant isolation architecture: verify that similarity scoring operates within security boundaries, not across them.Determine whether your vector database applies metadata filters before or after computing similarity scores across the full index. Check the database documentation and, if the documentation is ambiguous, test it: construct a query as a tenant with no access to Tenant B's documents, observe whether the similarity computation considers Tenant B vectors, and examine the query plan if the database exposes one. If post-filter ranking is confirmed, evaluate whether separate namespaces or separate indexes per tenant are warranted given the regulatory context. For HIPAA-regulated healthcare and fintech customer data, separate namespaces with pre-filter ranking or separate indexes per security boundary are the recommended architecture. Document the isolation architecture, the rationale for the approach chosen, and the test confirming that cross-boundary retrieval is not possible through query manipulation.
Permission sync pipeline: access metadata in the vector index must be updated whenever permissions change in the source system.Map every source system the pipeline indexes against and document how permission changes in those systems reach the vector index. For each source, define the update trigger (scheduled full re-index, event-driven partial update, or change-data-capture feed), the maximum lag between a permission change in the source and its reflection in the index, and whether that lag is acceptable given the regulatory context. For healthcare offboarding workflows, the permission revocation in the EHR system must propagate to the RAG index before the former employee's next session, which typically means an event-driven update rather than a nightly re-index. Test the sync by revoking a test account's access in the source system and confirming the index reflects the change within the defined lag window. Document the sync mechanism, the lag guarantee, and the test results.
Prompt injection screening: retrieved chunks must be checked for embedded instructions before insertion into the LLM context window.Implement a screening step that processes each retrieved chunk before it is placed in the prompt template. The screen should detect common prompt injection patterns: imperative instructions addressed to the model, text that attempts to override system instructions, or content that references the model's role or previous conversation state. The screen does not need to be perfect, since injection techniques evolve, but it creates a documented control that reduces the attack surface and provides evidence of due diligence. Log each detection and the document from which the injection was found. For regulated industries where document ingestion includes externally submitted content (patient forms, vendor submissions, customer-uploaded documents), the screening step is particularly important because external parties can craft documents intended to be retrieved by the system. Document the screening logic, its update cadence, and the incident response procedure for a confirmed injection detection.
Retrieval-event audit log: every retrieval must be logged with the requesting user identity, the document or chunk IDs retrieved, and the timestamp.Instrument the retrieval layer to write an audit log entry for each query, capturing: the authenticated user identity making the request, the query text or a hash of it, the document or chunk IDs included in the retrieved set, and the timestamp. This log must be distinct from the application log that captures query and response text, because the purpose is to record which data was accessed, not what was said. Under HIPAA, this log is the access record that breach investigations require to determine which patient records were accessed by which user over a time period. Under SOC 2 CC6.1 and CC6.3, it is the access control evidence. Store the retrieval log in write-once, tamper-evident storage, and retain it for the period your regulatory framework requires. Test the log by running known queries and confirming the retrieved document IDs are captured correctly and completely.
The LLM context window is a data access event. Access control frameworks that apply only at the response layer will not satisfy HIPAA or SOC 2 requirements, because the data was already accessed when retrieval ran.

What to do this week

Trace one real query through your retrieval layer and check where permissions are verified

Pick a production query from your RAG system and trace it through the pipeline, step by step, from the user's request to the chunks placed in the context window. For each step, ask whether a permission check is running and what it is checking against. If the first access control check you find is on the generated response rather than on the retrieved chunks, your retrieval layer has no enforcement and every query in that system is a potential access control violation for regulated data. Document what you find. The trace itself takes less than an hour on most pipelines and answers the question most teams do not ask until an audit makes it urgent.

Test your tenant isolation by querying across the boundary

If your RAG system serves multiple tenants or organizational units from a shared vector index, construct a test query as a tenant with no authorized access to another tenant's documents. Confirm that the retrieved chunks contain no content from outside the permitted boundary. If your database uses post-filter ranking, the similarity scores may have already considered out-of-boundary vectors even if they are excluded from the final context. Check the database documentation for whether your query configuration uses pre-filter or post-filter ranking, and document which approach is in use. For healthcare and fintech deployments where tenant data segregation is a regulatory requirement, test results confirming the isolation mechanism are the kind of evidence that an audit or a security review will request.

Check whether your permission metadata is current for users who left in the past 90 days

Pull a list of users who had their access revoked in your source systems in the past 90 days. For each one, check whether their permissions are reflected in the RAG index. If you find any user who was deprovisioned in the source system but whose access metadata in the index was not updated, you have confirmed a permission sync gap. That gap represents a window during which the user could have queried the RAG system and received content they were no longer authorized to access. The remediation is to close the sync gap, but the first step is knowing whether it exists, and that check takes a few queries against your index metadata and your HR or identity system records.

Verify that your retrieval events are logged with document IDs, not just query and response text

Pull your RAG system's logs for a known query from last week. Find the log entries associated with that query and check whether the retrieved document or chunk IDs are present. If the log shows only the query text, the user identity, and the generated response, the retrieval layer is not instrumented. Add logging to the retrieval step before the pipeline handles any regulated data at scale. The instrumentation is typically a few lines of code at the point where chunks are selected for the context window, and it converts a silent data access event into an auditable record. Without it, a breach investigation asking which documents were accessed by which users on which dates cannot be answered from the logs, which creates both a notification scope problem under HIPAA and an access control evidence gap under SOC 2.

Let 10decoders secure your RAG pipeline before the first access control incident

We review enterprise RAG security architecture for healthcare and fintech organizations: retrieval-time enforcement design, tenant isolation testing, permission sync pipeline setup, prompt injection screening, and audit log instrumentation. Our AI engineering practice has built production RAG systems for organizations where HIPAA compliance and SOC 2 auditability are requirements, not optional. We produce a gap report against your regulatory framework with specific, actionable remediation steps.

#EnterpriseRAG#RAGSecurity#AccessControl#HIPAACompliance#AICompliance#HealthcareAI#FintechAI#DataLeakage#AIEngineering#10decoders