The Security Risks Hiding in AI Agent Memory
Agent memory is the least-governed data store in most enterprises, and it is filling up with secrets. An agent that remembers can be taught. An agent that can be taught can be poisoned. And whatever it has learned sits somewhere, usually in plain text, usually outside any control you would recognize from your ISMS.
Last week I argued that the harness is the contract: every consequential path an agent can take should be declared, bounded, and leave evidence another party can verify. Memory is the part of the harness that breaks that model quietly. A tool call happens once and shows up in a log. A memory write persists, gets replayed in every future session, and can change behavior weeks after the input that caused it.
That is why a new Help Net Security interview with Vectorize CEO Chris Latimer deserves the attention of every CISO running agents in production. His headline advice is blunt: if you do one security check this quarter, make it agent memory. I agree, and I want to go further, into what that check should look like and how it maps to the frameworks your auditors already use.
What “agent memory” actually means
Agent memory is any state an agent writes during one run and reads back as trusted context in a later run. That last clause is the whole security problem. Retrieval-augmented generation reads from a corpus someone curated; memory reads from a corpus the agent itself wrote, often from inputs nobody reviewed.
| Memory type | Where it usually lives | What piles up in it | Who can write to it |
|---|---|---|---|
| Session / scratchpad | Context window, temp files | Tool outputs, pasted snippets, intermediate plans | The agent, every tool it calls |
| Long-term user memory | Markdown files, local JSON, SaaS memory service | Preferences, project facts, credentials users pasted | The agent, on the user’s behalf |
| Vector memory store | Embedded DB, managed vector service | Chunked documents, conversation summaries, embeddings | Ingestion jobs, agents, sometimes other agents |
| Shared / team memory | Shared server, team workspace | Decisions, runbooks, cross-user context | Many users and many agents |
| Skill, plugin, MCP state | Extension directories, MCP server storage | Config, cached tokens, instructions | Third-party code you installed |
Notice the right-hand column. In a database you would never accept “anyone whose text the agent read” as the write-access policy. For most memory stores in production today, that is effectively the policy.
Four signals from the field
Latimer’s interview is short, but it lands four points that match what I see in agent design reviews.
- Secrets are migrating into memory. When he read through the memory banks of the coding agents he used, he found API keys, credentials, and sensitive documents that developers had fed in, and that the agents had promoted to long-term memory. The data your secure SDLC was built to protect now sits in plain text on workstations, in cloud memory services, and in markdown files.
- Extensions are the poisoning vector. He names plugins, skills, and MCP integrations as the likely entry points, because people install things that look useful without reading them. His attacker scenario targets first-time coders with a too-good-to-be-true plugin that quietly scans memory for credentials and ships them to an external endpoint.
- Provenance is the forensic question. After an incident, what you most need to know is where a memory came from: an MCP server, a tool response, or an insider. Better still is filtering hostile writes before they persist, which is the idea behind OWASP’s Agent Memory Guard.
- Access control is the gap vendors can’t close yet. Most products stop one user’s session from reading another user’s memories. Team memory and graduated access, the RBAC and ABAC patterns enterprises take for granted in databases and APIs, are mostly not there.
His quarterly audit prediction is the one to act on. He expects CISOs to find two things: no governance over which memory systems are in use, and a startling volume of keys, database passwords, and confidential business data sitting in those systems, ready to be exfiltrated.
That second finding is not an AI problem. It is a data-classification and secrets-management failure that AI has made faster and harder to see.
Six risks hiding in agent memory
Memory risk is durable risk. A prompt injection that lands in the context window dies with the session; one that lands in memory becomes a standing instruction.
1. Memory poisoning. Attacker-controlled text is written into memory and replayed as trusted context in future sessions. The entry point is rarely the chat box. It is a web page the agent summarized, a ticket it triaged, a tool response, or a plugin that writes directly to the store. The effect is delayed, so the incident and its cause show up weeks apart.
2. Secrets at rest in the wrong place. Every credential a developer pastes into a session is a candidate for promotion to long-term memory. Those stores rarely have encryption at rest, secret scanning, or retention limits. Your vault is hardened; the agent’s notes file next to it is not.
3. Cross-user and cross-tenant leakage. In multi-tenant products, the question is whether one customer’s memory can surface in another customer’s session. Tenant filters applied after retrieval can be talked around. They belong inside the vector query itself, enforced by the store, not the model.
4. Malicious extensions reading memory. A skill or MCP server runs with the agent’s access. If the agent can read its memory, so can the extension, and nothing stops it from sending what it finds to an external endpoint. This is supply-chain risk with a data-exfiltration payload.
5. Embedding and summary leakage. Teams treat embeddings as anonymized math. They are not; they can be inverted enough to leak source text. Summaries are worse, because they compress sensitive facts into short, retrievable sentences.
6. Unbounded retention. Memory stores grow without an owner, a retention schedule, or a deletion path. That turns a GDPR erasure request, a customer offboarding, or a legal hold into a question nobody can answer with confidence.
Where these risks sit in the threat frameworks
Every one of the six risks already has a home in the frameworks your security team uses. The work is connecting them to the memory store, which most threat models leave off the diagram.
| Memory risk | OWASP Agentic Top 10 | OWASP LLM Top 10 (2025) | MITRE ATLAS |
|---|---|---|---|
| Memory poisoning | ASI06 Memory & Context Poisoning; ASI01 Agent Goal Hijack | LLM04 Data and Model Poisoning | AI Agent Context Poisoning (AML.T0080); RAG Poisoning (AML.T0070) |
| Secrets at rest | ASI03 Identity & Privilege Abuse | LLM02 Sensitive Information Disclosure | LLM Data Leakage (AML.T0057) |
| Cross-tenant leakage | ASI03 Identity & Privilege Abuse | LLM08 Vector and Embedding Weaknesses | LLM Data Leakage (AML.T0057) |
| Malicious extensions | ASI04 Agentic Supply Chain Vulnerabilities; ASI02 Tool Misuse | LLM03 Supply Chain | AI Supply Chain Compromise (AML.T0010) |
| Embedding and summary leakage | ASI06 Memory & Context Poisoning | LLM08 Vector and Embedding Weaknesses | LLM Data Leakage (AML.T0057) |
| Unbounded retention | ASI10 Rogue Agents (drift over time) | LLM02 Sensitive Information Disclosure | Not a technique; a governance gap |
OWASP’s Agent Memory Guard is the reference implementation for ASI06. It sits between the agent and the store, screens every read and write, and maps each finding to allow, redact, quarantine, or block. In its published benchmark it caught 92.5% of 40 attack payloads with zero false positives, and its misses were API tokens slightly longer than its fixed-length patterns. Treat it as a strong first layer, not the whole defense.
The governance lens: memory is data, and data has owners
The fastest way to get memory under control is to stop treating it as an AI novelty. It is a data store. Your AIMS and ISMS already have controls for data stores; memory just hasn’t been put in scope.
| Framework | Where agent memory lands | What the auditor will ask for |
|---|---|---|
| ISO/IEC 42001 | A.7 Data for AI systems, especially A.7.5 data provenance; A.6.2.8 event logs; A.10.3 suppliers | Memory inventory, provenance records per write, supplier assessments for memory vendors and MCP servers |
| ISO/IEC 27001:2022 | A.5.9 asset inventory, A.5.12 classification, A.8.12 data leakage prevention, A.5.33 protection of records | Memory stores in the asset register, classified, with DLP and retention applied |
| NIST AI RMF 1.0 | MAP 4.1 third-party components, MEASURE 2.7 security and resilience, MANAGE 3.1 third-party monitoring | Evidence that memory poisoning and leakage were tested and are monitored |
| EU AI Act | Art. 10 data governance and Art. 15 robustness for high-risk systems; Art. 26 deployer obligations | Proof that memory can’t silently degrade a high-risk system’s accuracy or robustness |
| GDPR / CCPA | Storage limitation, right to erasure, deletion requests | A working deletion path that reaches memory, embeddings, and summaries |
Having taken virtual data room organization through ISO 42001 Stage 2 on the first attempt, I can tell you what certification auditors reward: a complete inventory, named owners, and evidence that reviews actually happened. Agent memory needs exactly that, and in most organizations today it has none of it.
A control framework for agent memory: govern the lifecycle, not the model
Apply the harness test to memory: every write is a declared path, every store is bounded, and every read leaves evidence. That gives five control points across the memory lifecycle.
| Lifecycle stage | Declared | Bounded | Verifiable evidence |
|---|---|---|---|
| 1. Admit (write) | Allow-list of sources that may write to memory; extensions denied write by default | Secret and PII scanning before persist; injection screening; size and growth limits | Provenance tag on every entry: source class, session, identity, timestamp |
| 2. Store | Every memory store in the asset register with an owner and classification | Encryption at rest; no plain-text markdown for anything above Internal; tenant partitioning | Asset register entry; configuration evidence; SHA-256 baselines on protected keys |
| 3. Retrieve (read) | Read scope defined per user, team, and agent role | Access filters enforced in the store’s query, not by the model; extensions get no read by default | Read logs with requesting identity and entries returned |
| 4. Retain and expire | Retention schedule per memory class | TTLs; deletion that reaches embeddings and summaries; legal-hold override | Deletion job logs; erasure request test results |
| 5. Respond | Incident playbook covering memory poisoning | Quarantine and point-in-time rollback | Snapshots; poisoning test results; post-incident provenance trace |
Two rules do most of the work. Secrets never belong in memory: detect them at write time, block them, and rotate anything you find. No extension gets memory access it did not declare: a plugin that needs to read your agent’s memory should have to say so, and you should have to say yes.
Your 90-day plan: find it, bound it, prove it
Latimer’s informal audit is the right first step. Here is how I would turn it into a quarter of work that ends with evidence an auditor will accept.
Start with discovery, because you cannot bound what you have not found. Latimer’s prediction is that you will find unvetted memory tools on developer workstations and small shared servers, so scope the inventory to endpoints, not just production.
Five questions to ask any agent memory vendor
- Can you enforce access by team and role inside the store, or only by user session?
- Is every memory entry tagged with its source, and can I query by source after an incident?
- Do you scan for secrets and PII before a write persists, and what happens when you find one?
- When I delete a user, do their embeddings and summaries go too, and can you prove it?
- Can a third-party skill or MCP server read memory it did not request, and where is that logged?
If a vendor struggles with the first question, Latimer’s experience says you are not alone. Put the gap in your risk register and compensate with network isolation and aggressive retention until the product catches up.
The bottom line
Memory is what turns an agent from a tool into a colleague, and it is what turns a one-time injection into a standing compromise. The organizations that get this right will not have better models. They will have an inventory, an owner, a write policy, and a log, the same four things that have governed every other data store for twenty years.
If your agents remember, your governance has to remember too.
Not sure what your agents have been memorizing? DISC InfoSec runs a focused Agent Memory Risk Review: we inventory memory stores across endpoints and production, scan for exposed secrets, map findings to ISO 42001, the OWASP Agentic Top 10, and NIST AI RMF, and hand you a remediation plan you can take into your next audit. Book a free 30-minute call or email info@deurainfosec.com.
DISC InfoSec, ISO/IEC 42001 and 27001 Lead Implementer, CISSP, and CISM. We led VDR organization through ISO 42001 Stage 2 certification on the first attempt.
- Meta title: The Security Risks Hiding in AI Agent Memory (and a 90-Day Fix)
- Meta description: Agent memory is filling up with secrets and poisoned context. Six risks, their OWASP and ISO 42001 mappings, and a 90-day plan to govern it.
- Primary keyword: AI agent memory security
- Secondary: memory poisoning, OWASP ASI06, agent memory governance, ISO 42001 A.7.5, MCP security, Agent Memory Guard
Sources
- Help Net Security, If you do one security check this quarter, make it agent memory, interview with Chris Latimer, Sept. 28, 2026
- Help Net Security, OWASP Agent Memory Guard, June 1, 2026
- 14 Months to Comply. 12 Months of Work. The High-Risk AI Playbook Nobody Has Started
- Your biggest AI risk isn’t hackers. It’s your own data walking out the door
- The cheapest security control you’ll ever implement is the component you decided not to build
- AI Handoffs: How Uncertainty Propagates—and How to Fix It
- The Security Risks Hiding in AI Agent Memory
