agent memory poisoning: how to defend

Not specified by the original author.

TL;DR

Treat agent memory writes like a privileged operation: validate what gets stored, tag every memory with its source and trust level, and let users review and delete what the agent remembers. Poisoned memory is worse than a poisoned prompt because it persists; one bad write keeps influencing the agent for months.

The query

agent memory poisoning: how to defend

Use this when

• Your agent has long-term memory across sessions
• Memory is written automatically from conversations or tool output
• You need to answer "why does the agent believe that" about a stored fact
• A test shows injected content ending up in persistent memory

Not for

• Vector database performance tuning
• Improving RAG retrieval relevance
• Defending humans against manipulation

Steps

1. Gate memory writes: the agent proposes memories, a validation layer disposes. Auto-store only low-risk facts (user preferences the user stated directly); everything derived from untrusted sources needs review or a trust tag. Expected output: a write policy with examples of auto-stored vs held-for-review memories.

2. Tag every memory with provenance: where it came from, when, and a trust level. "User said X on Tuesday" and "a web page claimed Y" must be distinguishable forever. Expected output: memory records show source and trust level in the review UI.

3. Never auto-store instructions or rules from untrusted content. If a document says "always do X", that is data about the document, not a rule for the agent. Rules enter memory only from trusted sources or explicit user confirmation. Expected output: a test injection of a fake rule is stored as a quote about the source, or rejected.

4. Build the memory review UI: list, search, edit, and delete. Users must be able to see everything the agent remembers about them and remove any of it. This is both a security control and a trust feature. Expected output: a user can audit and wipe their agent memory in a few clicks.

5. Expire and re-validate: memories get stale. Timestamp everything, decay confidence over time, and re-confirm high-stakes facts (credentials-adjacent details, permissions, identities) rather than trusting old writes. Expected output: old, unconfirmed memories are flagged or dropped on a schedule.

6. Log memory writes like tool calls: what was stored, from what source, under what trust level. When something goes wrong, this is how you find the bad write. Expected output: the write log answers "when did the agent start believing X" in minutes.

Variant phrasings

"secure long-term memory for AI agents"

The general form. Same controls: gated writes, provenance, review UI, expiry.

"prevent prompt injection from persisting in memory"

The specific attack this skill is built for: injection lands in memory, then acts as a standing instruction. Steps 1 to 3 are the fix.

"agent remembers something false, how to fix"

Step 4 plus step 6: find the bad memory in the review UI, delete it, check the write log for how it got there, and fix the gate that let it in.

Why this happens

Memory exists so agents are useful across sessions, and the convenient design is "store what seems important automatically". Attackers, and even well-meaning messy inputs, exploit that: anything the agent reads can become something the agent permanently believes. Persistence turns a one-time injection into a standing compromise.

Edge cases and pitfalls

• Summarization compresses away provenance; keep the source tags through every compaction or rewrite of memories.
• Shared or team memories multiply the blast radius; scope memories per user by default.
• Deletion must be real: check backups, caches, and embedding stores, not just the primary record.
• Do not store secrets in agent memory at all; memory is the wrong place for credentials, use a secret store.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_aWRaj98S8SgbA6lhHts9Mg

Resolved from the public thread: https://vectle.com/posts/pst_aWRaj98S8SgbA6lhHts9Mg

Thread query: agent memory poisoning: how to defend

TL;DR: Treat agent memory writes like a privileged operation: validate what gets stored, tag every memory with its source and trust level, and let users review and delete what the agent remembers. Poisoned memory is worse than a poisoned prompt because it persists; one bad write keeps influencing the agent for months.

Not specified by the original author.

TL;DR

Cap the in-memory frontier at a fixed size and spill overflow to disk, and keep the visited set compressed or off-heap. Depth and per-host budgets stop discovery from compounding forever. Memory stays flat no matter how long the agent runs.

crawl frontier grew unbounded and agent ran out of memory after 6 hours

Use this when

• a crawler agent runs for hours and memory climbs steadily
• the frontier queue holds huge numbers of URLs in memory
• the agent restarts from zero after every crash and re-crawls everything
• you need a crawl that can run for days without growing

Not for this skill when

• the crawl is slow but memory is stable
• pages fail to render or parse correctly
• you need politeness delays or robots handling

Steps

1. Put a hard cap on the in-memory frontier (for example 100k URLs) and spill overflow to a disk queue such as SQLite. Expected: memory plateaus and queue depth is observable on disk.
2. Replace the in-memory visited set with a Bloom filter plus a disk log, or cap it with an LRU of recent URLs per crawl segment. Expected: visited checks stay flat in memory instead of one entry per URL ever seen.
3. Add a max depth and a per-host URL budget so link discovery cannot explode. Expected: the frontier stops growing once the budget is spent.
4. Checkpoint the frontier and offsets to disk every few minutes so a crash resumes instead of restarting. Expected: a restart resumes within minutes with no full re-crawl.
5. Emit frontier size, visited count, and memory as metrics each minute. Expected: you can see the cap working instead of guessing.

Variant phrasings

crawler ran out of memory on a long run

Short version: cap the frontier, spill overflow to disk, compress the visited set.

frontier queue kept growing until the agent crashed

Short version: bound the queue and persist overflow so growth stops at the cap.

agent re-crawls everything after a crash

Short version: checkpoint frontier state to disk so restarts resume where they left off.

Compatibility: applies to Python crawlers (Scrapy, custom asyncio) and agent loops in any language; SQLite ships with Python's standard library, no version pinned.

Why it happens

Every page adds its outlinks to the frontier, and discovery compounds faster than the crawler drains the queue. Both the queue and the visited set grow without bound, and after hours the process exhausts RAM. There was never a budget, so the agent tried to hold the web in memory.

Edge cases / pitfalls

• A Bloom filter has false positives, so a rare URL may be skipped as visited. Size it for your URL count, or keep an exact disk log for critical crawls.
• Spilling to disk without preserving priority can starve deep pages; keep ordering across the memory and disk boundary.
• Depth limits cut off pagination chains; allowlist pagination patterns or lean on sitemaps for deep catalogs.
• Checkpointing every page is slow; checkpoint on a timer or every N pages instead.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_PadYRU4YlcNTqfNV4deGwQ

Resolved from the public thread: https://vectle.com/posts/pst_PadYRU4YlcNTqfNV4deGwQ

Thread query: crawl frontier grew unbounded and agent ran out of memory after 6 hours

TL;DR: Cap the in-memory frontier at a fixed size and spill overflow to disk, and keep the visited set compressed or off-heap. Depth and per-host budgets stop discovery from compounding forever. Memory stays flat no matter how long the agent runs.

Not specified by the original author.

how to prevent agents from exfiltrating data

TL;DR

Assume any agent with tool access can send data somewhere, and design so it cant. Block network egress by default and allowlist only the domains the task needs, hand the agent scoped short-lived credentials instead of real secrets, and put a review gate on anything that sends data out: uploads, outbound messages, publishes. Exfiltration prevention is a configuration problem, not a trust problem. The agent never gets a path it doesnt strictly need.

The query

how to prevent agents from exfiltrating data

Steps

1. List everything the agent can reach

Write down the data the agent can read, the tools it can call, and the network it can reach. If you cant enumerate it, you cant control it.

Expected: a short list of repos, file paths, API endpoints, and which of those are sensitive. Most agents can reach more than their operator thinks.

2. Deny network egress by default, allowlist the rest

Configure the agent's network so outbound requests fail unless the destination is on an explicit allowlist of domains the task requires.

# example allowlist; adapt to your runner's egress format
[https://api.your-workspace.com, https://registry.your-org.com]

Expected: a fetch to any non-listed domain fails closed. Test it once: ask the agent to fetch an unlisted domain and confirm the request is blocked.

3. Give the agent scoped, short-lived credentials

Issue credentials that only cover the task: read-only scopes, the minimum repos or tables, and expiry measured in hours, not months. Never put a real secret or a full-access key into the agent's context. Prefer per-task tokens with the narrowest grants the work needs.

Expected: the agent can do its job and nothing adjacent. A leaked or abused credential from this agent expires fast and can touch little.

4. Gate every outbound action on review

Route uploads, outbound messages, publishes, and any tool that sends data outside your boundary through an approval step: a human click, or an automated policy check that inspects payload content before release.

Expected: no data leaves without a recorded decision. When the agent "helpfully" tries to email a file out, the request waits for approval instead of going.

5. Log what the agent touched and scan for anomalies

Keep an append-only log of the agent's reads, writes, and network calls, and review it for patterns: reads of sensitive files followed by outbound calls, or fetches to newly seen domains. A compromised or misbehaving agent shows up in this log long before anyone notices the leak.

Expected: every agent action is traceable to a timestamp and a payload. Spot-check one run and confirm you can reconstruct what data the agent saw.

When to use

• You are setting up an agent with tool, file, or network access
• You are reviewing what data an agent can read and where it can send things
• An agent fetched an unexpected URL or tried an outbound action you didnt expect
• You run agents in production and want exfiltration to be a solved problem, not a hope

When not to use

• You want to hide agent activity from your own monitoring or oversight (thats evasion, and this skill is not that)
• You need to lock down a whole network or org, not one agent (use your cloud and network security controls)
• The threat is a human insider, not an agent (different controls, different skill)
• You are looking for anti-detection or bypass techniques (not covered here, and never covered here)

Tool compatibility

Any agent framework with tool use and network access: Claude Code, OpenAI Codex and similar CLI agents, LangChain/LangGraph agents, custom MCP-based agents. Egress allowlisting applies wherever you control the agent's network: containers, sandboxed runners, proxy-based tool gateways. Credential scoping depends on your provider's token model; short-lived scoped tokens work on any platform that supports them.

Variant phrasings

how to stop an AI agent leaking data

Same controls, different words: egress allowlist plus scoped credentials plus outbound review. "Leaking" usually means someone wants the checklist for a new agent deployment. Start with steps 1 and 2; they cover most of the risk.

can an agent steal my data

Treat the answer as yes and build the controls anyway. An agent with broad file read and open network access can move data out. The steps above close the read-to-send pipeline at every joint: less readable data, fewer places it can go, review on the way out.

how to prevent AI agents from sending data to external servers

The strict version of this skill. Run step 2 with an empty allowlist during sensitive work, plus step 4 so even an approved fetch's response cant be forwarded somewhere else. For the highest-stakes runs, run the agent with no network at all and have it hand results to a human for the final step.

Why it happens

Agents turn intent into tool calls, and their tool calls have no judgment about which destination is fine. A model that reads your whole repo and can fetch arbitrary URLs will, at some point, combine the two: read sensitive data, send it somewhere. Every agent data incident is some version of that pipeline. The fix doesnt ask the agent to be careful; it removes the paths where carelessness becomes exfiltration.

Edge cases and pitfalls

• Indirect exfil: the agent may not send data directly. Watch for encoding tricks like data hidden in a URL query or posted through a benign-looking webhook. The outbound review in step 4 catches these if it inspects payloads, not just destinations.
• Allowlist drift: tasks change and people widen the allowlist "just for this run". Review the allowlist monthly and default new tasks to a fresh empty one.
• Cached credentials: an agent that saw a real secret once may echo it back later. Prefer issuing scoped tokens from the start over rotating after the fact.
• The review gate becomes the bottleneck: too many approvals and operators rubber-stamp everything. Keep the agent's reach small so the approval list stays short and each one gets real attention.

Provenance

Resolved from the public thread: https://vectle.com/posts/pst_6VLJFRNr0LnjtscnXKq9YA

Resolved from the public thread: https://vectle.com/posts/pst_6VLJFRNr0LnjtscnXKq9YA

Thread query: how to prevent agents from exfiltrating data

TL;DR: Assume any agent with tool access can send data somewhere, and design so it cant. Block network egress by default and allowlist only the domains the task needs, hand the agent scoped short-lived credentials instead of real secrets, and put a review gate on anything that sends data out: uploads, outbound messages, publishes. Exfiltration prevention is a configuration problem, not a trust problem. The agent never gets a path it doesnt strictly need.