10 Risks of Data Leakage in the Age of AI — and How to Mitigate Them
Oct 12, 2026 · @DISCInfoSec
Your data no longer leaks through the firewall. It leaks through the prompt box, the context window, the agent’s tool call, and the memory file nobody knew existed.
For twenty years, data loss prevention meant watching a handful of exits: email, USB, web uploads, cloud storage. Generative AI added dozens of new exits overnight. Every employee with a browser now has a copy-paste channel to a third-party model. Every agent with an OAuth token can read your CRM and write to the open internet. Every embedding, log, and memory store quietly keeps a second copy of what it saw.
This post walks through the 10 data leakage risks we see most often in B2B SaaS and financial services environments, maps each to OWASP, MITRE ATLAS, ISO 42001, and NIST AI RMF, and lays out a 90-day plan to close them. The short version: you cannot block your way out of this. You have to find it, bound it, watch it, and prove it.
Why AI changes the data leakage equation
AI did not invent data leakage. It changed four things about it at once.
New channels. Prompts, file uploads to chat tools, browser extensions, copilots, and agent tool calls are all exfiltration paths. Legacy DLP and CASB were built for files moving between known endpoints, not for a paragraph of customer data pasted into a text box or an agent writing to an API.
New actors. The thing moving your data is increasingly not a person. Agents, copilots, and MCP servers act with delegated credentials at machine speed. An identity problem now has a data consequence.
New persistence. AI quietly makes copies: chat history, vendor logs, vector embeddings, agent memory files, fine-tuning sets. Each copy is a new place for data to sit, often outside your retention and deletion controls.
New scale. One overshared SharePoint folder used to expose one folder. Point a copilot at it and every user who can ask a question can now retrieve it.
The numbers back this up. IBM’s 2025 Cost of a Data Breach study found that 13% of organizations reported breaches of AI models or applications, and 97% of those lacked proper AI access controls. The same research found 63% had no AI governance policy, and high shadow AI usage added USD 670,000 to the average breach. Shadow AI incidents compromised customer PII in 65% of cases and intellectual property in 40%, both above global averages.
As we noted in our earlier post, AI Adoption Is Outpacing Governance, security leaders now rank generative AI data leakage above adversarial attacks as their top AI concern. The attacker is often not the problem. Your own data, moving through tools you do not control, is.
The 10 risks of data leakage in the age of AI
1. Shadow AI and the copy-paste exit
Employees paste source code, contracts, customer records, and board decks into public chatbots to get work done faster. Nobody approved the tool, nobody reviewed its terms, and nobody can pull the data back.
This is the original AI leak. In 2023, Samsung semiconductor engineers pasted confidential source code and meeting notes into ChatGPT, and the company banned generative AI on company devices weeks later. Bans did not work then and do not work now. Usage just moves to personal phones.
Mitigate:
- Discover usage first: SSE/CASB logs, browser telemetry, expense reports, and OAuth grant reviews.
- Give people a sanctioned, enterprise-tier alternative that is as easy as the free one.
- Publish an AI acceptable use policy with a simple data traffic light: what can never go in, what needs an approved tool, what is fine.
- Deploy AI-aware DLP at the browser or endpoint that inspects prompts and uploads, not just files.
2. Vendor retention, training, and logging of your inputs
Even sanctioned tools can keep your data. Consumer and free tiers often allow prompts to be used for model training. Enterprise tiers may still retain prompts and outputs for abuse monitoring, debugging, or legal hold. Every retained copy is subject to the vendor’s breach, subpoena, and sub-processor risk.
Mitigate:
- Use enterprise agreements with contractual no-training clauses and defined retention windows (zero data retention where offered).
- Map AI vendors as sub-processors in your DPA and records of processing; check where data is processed geographically.
- Disable chat history and logging features you do not need. Review them after every product update.
- Block consumer-tier sign-ins with corporate email through SSO enforcement.
3. Indirect prompt injection and zero-click exfiltration
Attackers no longer need to breach you. They plant instructions in content your AI will read: an email, a shared document, a web page, a support ticket. When the assistant processes it, the hidden instructions tell it to gather sensitive data and send it out, often through a rendered image link or URL.
The defining example is EchoLeak (CVE-2025-32711, CVSS 9.3), the first known zero-click AI exploit against Microsoft 365 Copilot. A single crafted email could cause Copilot to pull data from OneDrive, SharePoint, and Teams and exfiltrate it with no user click. Microsoft patched it, but the pattern (an LLM scope violation that mixes untrusted input with trusted data) applies to every RAG and agent system.
Mitigate:
- Treat all retrieved content as untrusted. Separate it from instructions and tag its provenance.
- Strip or block markdown images, auto-fetched links, and outbound URLs in model output unless allow-listed.
- Enforce egress controls on AI services and agents so they cannot reach arbitrary domains.
- Red-team your copilots and agents with injection payloads before and after every major change.
4. Over-permissioned agents and MCP connectors
Agents are given OAuth tokens to email, drive, CRM, ticketing, and code repositories, usually with far broader scopes than any single task needs. Combine broad read access with any write channel (send email, post to Slack, call a webhook) and you have built an exfiltration pipeline that only needs one injected instruction to run. This is the confused deputy problem at machine speed.
When we review MCP governance standards for clients, most of the changes we recommend address exactly this: OAuth 2.1 with PKCE, token audience validation, SSRF and egress controls, tool manifest integrity, and confused-deputy protections.
Mitigate:
- Inventory every agent and MCP server as a non-human identity with a named owner.
- Scope tokens per task and per tool, short-lived, with audience validation. No shared service accounts.
- Break the “lethal trifecta”: do not give one agent private data access, untrusted content exposure, and an external communication channel without a human checkpoint.
- Require human approval for high-impact actions (external sends, bulk exports, permission changes).
- Pin and sign tool manifests so a server cannot silently change what a tool does.
5. Copilot and RAG permission bleed
Enterprise copilots and RAG search inherit your existing permissions, and most organizations have years of oversharing baked in: “everyone” links, stale group memberships, and HR folders open to the whole company. Before AI, finding that data required knowing where to look. Now anyone can ask, “What are the salaries in the sales team?” and the assistant will find it.
Custom RAG pipelines are often worse. They index content with a service account and return it to users without checking the source system’s access controls at query time.
Mitigate:
- Run an oversharing cleanup before you turn on a copilot: sensitivity labels, “anyone” link removal, access reviews on high-risk sites.
- Enforce document-level ACLs at retrieval time, not just at ingestion.
- Exclude regulated stores (HR, legal, M&A, payroll) from indexing until they are labeled and permissioned.
- Test with “canary” questions from low-privilege accounts.
6. Agent memory and conversation history persistence
Agents that remember are agents that can leak. Long-term memory files, chat histories, and session transcripts accumulate API keys, customer names, deal terms, and personal data that users mentioned once and forgot. That memory is often stored in plaintext, shared across sessions or tenants, and outside every retention schedule you have.
We covered this in depth in The Security Risks Hiding in AI Agent Memory: poisoning, secrets at rest, cross-tenant leakage, malicious extensions, and unbounded retention.
Mitigate:
- Classify memory stores as data stores: encrypt, access-control, and include them in your data inventory.
- Scan memory writes for secrets and regulated data and redact before persisting.
- Set retention and TTLs, and make sure deletion requests (GDPR Article 17, CPRA) actually reach memory and history.
- Isolate memory per user and per tenant; never share it by default.
7. Embeddings, fine-tuning, and model memorization
Vector embeddings are not anonymization. Research has shown that text can be substantially reconstructed from embeddings, and fine-tuned models can memorize and regurgitate training records when prompted the right way. If you fine-tune on support tickets, contracts, or customer data, the model itself becomes a store of that data, and anyone with query access is a potential extractor.
Mitigate:
- Treat vector databases with the same classification as their source data. Encrypt them and restrict access.
- Minimize and de-identify data before fine-tuning. Prefer RAG with access controls over baking sensitive data into weights.
- Test fine-tuned models for memorization and membership inference before release.
- Rate-limit and monitor inference APIs for extraction patterns.
8. Secrets and credentials in AI workflows
Developers paste API keys into prompts to debug code. Coding assistants read .env files and include them in context. Agent configuration files hold long-lived tokens for every connector. AI-generated code hard-codes credentials. Each of these puts secrets into vendor logs, memory, or repositories.
Mitigate:
- Run secret scanning on repos, CI logs, agent config directories, and memory stores.
- Exclude secret files from coding-assistant context with ignore rules.
- Use a secrets manager and short-lived, scoped credentials for agents instead of static keys.
- Rotate anything that has ever touched a prompt.
9. Output leakage: system prompts, cross-tenant data, and oversharing answers
Leakage also happens on the way out. Customer-facing chatbots can reveal their system prompts, which often contain internal logic, pricing rules, or credentials. Multi-tenant AI features can return one customer’s data to another through shared caches, shared memory, or poorly scoped retrieval. And AI-generated reports, emails, and summaries can include sensitive details the author never meant to share.
For B2B SaaS, cross-tenant leakage is the nightmare scenario. It is a contract breach, a privacy incident, and a trust event in one.
Mitigate:
- Never put secrets or sensitive business logic in system prompts. Assume they will be extracted.
- Enforce tenant isolation at every layer: retrieval, cache, memory, logs, and evaluation datasets.
- Add output filtering for PII, secrets, and other tenants’ identifiers.
- Include cross-tenant tests in your release pipeline for any AI feature.
10. AI supply chain and third-party exposure
Your AI stack is mostly someone else’s: model providers, orchestration frameworks, plugins, browser extensions, MCP servers from GitHub, and SaaS vendors who quietly added AI features to tools you already approved. Each one can receive your data, and a compromise or misconfiguration at any of them becomes your incident. Malicious AI browser extensions and poisoned community tools are now a routine finding.
Mitigate:
- Add AI-specific questions to vendor due diligence (see the vendor questions below), including for existing vendors that add AI features.
- Allow-list browser extensions and MCP servers. Pin versions and review before updates.
- Require AI feature notification and opt-out clauses in SaaS contracts.
- Maintain an AI bill of materials (AI-BOM) for every system that touches customer data.
Mapping the 10 risks to the frameworks auditors use
Every one of these risks already has a home in the frameworks your customers, auditors, and regulators reference. That matters: it means you can address AI leakage inside your existing ISMS or AIMS instead of building a parallel program.
| # | Risk | OWASP LLM Top 10 (2025) | OWASP Agentic Top 10 | ISO/IEC 42001 Annex A | ISO/IEC 27001:2022 Annex A | NIST AI RMF |
| 1 | Shadow AI | LLM02 Sensitive Information Disclosure | — | A.2 AI policy, A.9 Use of AI systems | 5.10 Acceptable use, 8.12 DLP | GOVERN 1, MAP 1 |
| 2 | Vendor retention and training | LLM02, LLM03 Supply Chain | — | A.10 Third-party relationships | 5.19–5.23 Supplier and cloud | GOVERN 6 |
| 3 | Indirect prompt injection | LLM01 Prompt Injection, LLM05 Improper Output Handling | Agent goal hijack | A.6 AI lifecycle | 8.12 DLP, 8.23 Web filtering | MEASURE 2.7 |
| 4 | Over-permissioned agents and MCP | LLM06 Excessive Agency | Tool misuse, identity and privilege abuse | A.6, A.9 | 5.15 Access control, 8.2 Privileged access | MANAGE 2 |
| 5 | Copilot and RAG permission bleed | LLM08 Vector and Embedding Weaknesses | — | A.7 Data for AI systems | 5.12 Classification, 5.15 Access control | MAP 4 |
| 6 | Agent memory persistence | LLM02 | Memory and context poisoning | A.7 Data for AI systems | 8.10 Information deletion | MEASURE 2.10 |
| 7 | Embeddings and memorization | LLM02, LLM08 | — | A.7 Data for AI systems | 8.11 Data masking | MEASURE 2.10 |
| 8 | Secrets in AI workflows | LLM02 | Identity and privilege abuse | A.6 AI lifecycle | 5.17 Authentication information | MANAGE 2 |
| 9 | Output and cross-tenant leakage | LLM02, LLM07 System Prompt Leakage | — | A.6, A.8 Information for interested parties | 8.3 Information access restriction | MEASURE 2.10 |
| 10 | AI supply chain | LLM03 Supply Chain | Agentic supply chain | A.10 Third-party relationships | 5.19–5.23 Supplier and cloud | GOVERN 6 |
On the regulatory side, the same controls support GDPR Articles 5, 25, 28, and 32, CPRA and the new CPPA automated decision-making and risk assessment rules, and the EU AI Act’s data governance and transparency obligations. MITRE ATLAS covers the attacker view, including LLM Prompt Injection (AML.T0051) and LLM Data Leakage (AML.T0057).
Your 90-day plan to close AI data leakage
You do not need a two-year program to get ahead of this. You need ninety focused days and a named owner.
Days 1–30: Find it
- ☐ Name an accountable owner for AI risk (a CAIO, CISO, or vCAIO).
- ☐ Run shadow AI discovery across SSE/CASB logs, browser telemetry, OAuth grants, and expense reports.
- ☐ Build an AI inventory: tools, copilots, agents, MCP servers, vendors with AI features, and the data each touches.
- ☐ Publish an AI acceptable use policy with a clear data classification traffic light.
- ☐ Launch a sanctioned enterprise AI tool so people have a safe default.
Days 31–60: Bound it
- ☐ Clean up oversharing before enabling any copilot: labels, “anyone” links, access reviews.
- ☐ Re-scope agent and MCP tokens to least privilege; remove shared service accounts.
- ☐ Turn on AI-aware DLP for prompts and uploads on managed browsers and endpoints.
- ☐ Lock down egress for AI services and agents to allow-listed destinations.
- ☐ Confirm no-training and retention terms in every AI vendor contract and DPA.
Days 61–90: Watch it and prove it
- ☐ Log prompts, retrievals, and agent tool calls to your SIEM with alerts for bulk export and unusual destinations.
- ☐ Red-team your highest-risk copilot or agent for prompt injection and cross-tenant leakage.
- ☐ Add AI scenarios to your incident response plan and run one tabletop.
- ☐ Map controls to ISO 42001 Annex A and ISO 27001 so you can show evidence to customers and auditors.
- ☐ Report AI risk posture to leadership with a baseline score you can track quarter over quarter.
Seven questions to ask every AI vendor
If a vendor cannot answer these in writing, assume the answer is the one you would not like.
- Is our data (prompts, files, outputs) used to train or improve any model, including through human review?
- How long are prompts, outputs, and logs retained, where are they stored, and can we get zero data retention?
- Which sub-processors and model providers receive our data?
- How is our data isolated from other tenants in retrieval, caching, memory, and logs?
- Does the product enforce our source-system permissions at query time?
- What defenses exist against indirect prompt injection and data exfiltration through output links or tool calls?
- Will you notify us before adding new AI features or changing data handling, and can we opt out?
The bottom line
Most AI data leakage is not a sophisticated attack. It is a well-meaning employee, an over-scoped token, an overshared folder, or a vendor setting nobody checked. That is good news, because those are governance problems, and governance problems are fixable.
The organizations that will win customer trust in the next two years are not the ones that banned AI. They are the ones that can answer three questions with evidence: Where is our AI? What data can it touch? How do we know it stays bounded? When a customer’s security questionnaire, a regulator, or an insurer asks, “we’re working on it” is no longer an answer that closes deals.
How DISC InfoSec can help
DISC InfoSec helps B2B SaaS and financial services firms close AI data leakage gaps without slowing adoption. We led a virtual data room platform serving M&A and financial services clients, through ISO/IEC 42001 certification on the first Stage 2 audit. Financial data rooms are the hard mode of compliance, and the same controls work for you.
- AI Risk X-Ray and gap assessments: a fast baseline of your shadow AI, data leakage controls, and vendor exposure.
- AI Governance Quick-Start: policy, inventory, and a prioritized remediation plan in 7–10 days.
- ISO 42001 AIMS and ISO 27001 ISMS implementation: controls that hold up under audit and customer scrutiny.
- vCISO and vCAIO retainers: ongoing ownership of AI risk without a full-time hire.
One expert, no coordination overhead, and no “we’ll define the framework as we go.”
Book a free 30-minute AI data leakage review: calendly.com/hd-deurainfosec – info@deurainfosec.com – (707) 998-5164
DiscInfoSec, AICP, CISSP, CISM, ISO/IEC 42001 and ISO/IEC 27001 Lead Implementer
Metadata
| Field | Value |
| SEO title | 10 Risks of Data Leakage in the Age of AI (and How to Fix Them) |
| Meta description | Shadow AI, prompt injection, over-permissioned agents, and copilot oversharing. The 10 AI data leakage risks and a 90-day plan to mitigate them. |
| Slug | /ai-data-leakage-risks-mitigation |
| Focus keyword | AI data leakage |
| Secondary keywords | shadow AI, prompt injection data exfiltration, AI DLP, agentic AI security, ISO 42001, copilot oversharing |
Sources
- IBM, 2025 Cost of a Data Breach Report: Navigating the AI rush without sidelining security
- AI Business, IBM Highlights AI Security Issues in 2025 Data Breach Report
- TechRepublic, First Known Zero-Click AI Exploit: Microsoft 365 Copilot’s EchoLeak Flaw
- SC Media, Microsoft 365 Copilot zero-click vulnerability enabled data exfiltration
- GovTech, What tech company accidentally leaked trade secrets through ChatGPT?
- The Hill via AOL, Samsung bans employee use of ChatGPT after reported data leak
- DISC InfoSec, AI Adoption Is Outpacing Governance: Four Trends Every Leader Should Watch
- OWASP, Top 10 for LLM Applications
- Your biggest AI risk isn’t hackers. It’s your own data walking out the door
- The cheapest security control you’ll ever implement is the component you decided not to build
- AI Handoffs: How Uncertainty Propagates—and How to Fix It
- The Security Risks Hiding in AI Agent Memory
- AI Log Retention: Why Every AI Log Doesn’t Deserve the Same Shelf Life
