The Prompt Injection Hidden in Your Agent's Memory
The classic intrusion announces itself somewhere: a process running that nobody started, a file with the wrong timestamp, a line in a log that a night-shift analyst circles in red. The prompt injection described in a paper posted to arXiv this October has none of those. No process, no presence, no attacker on the line. Its entire footprint is a paragraph of text sitting in a notes file, the kind of file a personal AI assistant reads automatically at the start of every working session. The attacker left. The paragraph stays. And every morning, when the agent opens its notes to remember who it is, it reads the paragraph and obeys.[1]
Memory, with a heartbeat
To see why this works, you have to look at what modern agent software considers a feature. Frameworks in the current wave give their agents two channels of persistence. The first is memory: a markdown file, typically called MEMORY.md, whose contents are silently prepended to the system prompt at the start of each session, so the agent begins every day already knowing what it knew yesterday. One framework’s documentation describes the intent with disarming frankness: saved preferences are applied automatically as standing instructions, no confirmation needed. The second channel is scheduling: a heartbeat daemon that wakes the agent every thirty minutes by default and instructs it to read a file called HEARTBEAT.md and execute each registered task, and to reply with a token meaning all clear if nothing needs attention.[1]
Both channels exist because users asked for them. An assistant that forgets everything overnight is a toy; one that checks its task list on a schedule can monitor inboxes and price drops while you sleep. The design assumption, inherited from every previous generation of software, is that the contents of your own configuration are yours. The authors of the paper point out what that assumption is worth when a third party can write text into resources the agent reads: any webpage, email, document or tool output can carry an instruction, and once the agent writes the wrong summary of it into its own memory, the instruction becomes part of who the agent is.[1]
Three ways a prompt injection stays behind
The paper’s taxonomy sorts the possibilities into three classes. Instant attacks are the ones the industry already argues about: a payload hidden in a document hijacks the current session, does its damage now, and dies with the session. The two new classes are the ones that give the paper its name. Autonomous attacks plant the payload in HEARTBEAT.md, where it waits for the next scheduled tick and fires in an empty session, with no user present and nothing to notice. Latent attacks plant it in MEMORY.md, where it lies dormant until some future, topically relevant request reactivates it, like a letter left in a drawer that only opens when the drawer is pulled. The authors compress the mechanism into a single sentence: a one-time indirect prompt injection committed to either channel can re-fire across sessions long after the attacker leaves, threats we term carryover attacks.[1]
The benchmark that refuses to guess
Measuring this honestly is harder than it sounds, because the obvious measurement tool is broken. Most prompt injection research scores attacks with a language model acting as judge, and a judge that reads your agent’s transcript is itself a surface for prompt injection. The researchers built CarryoverBench instead: 101 attack tasks across 12 assistant scenarios, from email and banking to code review and travel booking, scored by a deterministic rule-based verifier that checks whether the agent’s tools actually performed the harmful action. Attack success becomes arithmetic, the product of two probabilities: the chance the payload gets planted into persistent state at all, and the chance it triggers once planted. Five rounds of triggering per planted payload, a fresh agent each round, five frontier models under test: GPT-5.4, Claude Sonnet 4.6, Gemini 3.1 Pro, GLM-5 and Qwen3.[1]
The results have the shape of a door found standing open. Payloads, once planted, re-fired with no measurable decay across all five rounds; there is no fatigue in a file. The trigger probability for Autonomous attacks was effectively one on every model tested; four of five models crossed 66 percent even for Latent, and two of them, GPT-5.4 and Claude Sonnet 4.6, sat near 100 percent on both channels. The paper’s conclusion is the load-bearing sentence: once the payload is committed to a persistent file, the agent acts on it almost deterministically. Planting, not triggering, is the only real obstacle left, and the two-stage barrier collapses to one.[1]
The eraser stays in the drawer
The bleaker half of the paper is the defense audit. The team tested the three remedies the community actually deploys. Teaching the agent, via its system prompt, to resist injections and to amend its memory only with consent cut the planting rate but never the trigger rate, cost one model nine points of utility, and actually amplified the attack against another. Granting the agent explicit authority to edit or delete its own memory changed nothing, because the agent never chose to use it; across every model and every opportunity, voluntary amendments were zero, in the authors’ phrase, the eraser stays in the drawer. Wrapping untrusted persistent state in warning tags, so the model can see what it is reading, closed the trigger gate in exactly one of six model-channel combinations, within the noise of rerunning the experiment.[1]
The reason is structural. All three defenses operate on the same channel the attack uses: plain text in the model’s context. The payload and the plea against it travel by identical post. The authors’ recommendation follows from the failure: gates on cross-session writes, quarantine for changes to persistent state, allowlists on what an agent may send outward, controls that live in the runtime beneath the model rather than in suggestions addressed to it. Their verdict on the current stack deserves its own line: the runtime does not contain carryover; it merely amplifies the model’s own alignment.[1]
The case for the defense
There is a fair objection to all of it, and the authors make half of it themselves. These are laboratory conditions: the same prompts replayed each round inflate the Latent trigger numbers, the deterministic verifier can miss payloads a curious human would notice, and planting still requires the attacker to get malicious content past the agent in the first place, which is the same hard problem the industry has been slowly improving on. Nothing in the paper documents a real-world carryover attack; it documents that the door is open and that five frontier models will walk through it. A separate paper published the same month reached the same summit from the other side, showing that memory systems convert prompt injection into persistent compromise, which suggests the finding is not an artifact of one benchmark.[2]
But the objection stops one door short. The architecture under attack is not a research rig; it is the local, private, memory-keeping assistant that the entire industry is currently shipping as the privacy-preserving answer to cloud AI.[3] The files that carry the payload are the product. Every capability that makes the assistant feel like a colleague, the memory that persists, the heartbeat that checks in, is a channel that survives the attacker’s exit, and the model’s own judgment, which improved steadily across the five models tested, remains the only thing standing between a planted paragraph and an executed instruction. Alignment is doing all of the work, and alignment, as the paper demonstrates with an eraser no model would pick up, is not a control. It is a hope with good grades.
Memory was the feature. Memory is the payload.
- Zhiyue Xu, Xukai Wang, Mingjie Zhan, Jian Zhao, Xu-Yao Zhang, After the Attacker Leaves: Benchmarking Carryover Attacks on LLM Agents (arXiv, October 2026; code at github.com/igloomatics/CarryoverBench). Two persistence channels: MEMORY.md “apply saved preferences automatically as standing instructions, no confirmation needed” (Nanobot) and the heartbeat daemon’s “Read HEARTBEAT.md in your workspace and execute each registered task as specified; if nothing needs attention, reply HEARTBEAT_OK.” “a one-time indirect prompt injection committed to either channel can re-fire across sessions long after the attacker leaves, threats we term carryover attacks.” 101 tasks (35 Instant / 30 Autonomous / 36 Latent), 12 task families, deterministic verifier (“a judge prompt is itself an injection surface”), ASR = Pr(Plant) x Pr(Trig|Plant), 5 trigger rounds, models GPT-5.4, Claude-Sonnet-4.6, Gemini-3.1-Pro, GLM-5, qwen3-235b-a22b; “Once the payload is committed to a persistent file, the agent acts on it almost deterministically.”; Pr(trig|plant) ~ 1 for Autonomous on all models; defenses D1 (utility cost 9.3pp on Claude, amplified attack on Qwen), D2 (“the eraser stays in the drawer”), D3 (1 of 6 cells, within rerun noise); “The runtime does not contain Carryover; it merely amplifies the model’s own alignment.” Hermes (Nous Research) ablation: ASR 2.8%-27.8%, attenuated but present on all models.
- Combining Long-Term Memory and LLMs Comes at a High Cost: From Prompt Injection to Persistent Compromise (arXiv, October 2026): independent corroboration that memory systems convert one-time prompt injection into persistent compromise.
- Compare The Moderator Never Phones Home on local-first agents: the private, offline architecture marketed as the safe option is precisely the one whose memory and scheduling channels carryover attacks use.