Your AI Agent's Memory Is the Control Surface Your Go-Live Review Misses

For anyone who signs off on AI agents going to production: product, platform, security and risk leads.

In September 2024, a security researcher made ChatGPT remember a lie. Then he made it spy.

Johann Rehberger didn't break into OpenAI. He hid instructions inside content ChatGPT was asked to read, like a web page or a document. ChatGPT saved those instructions to its long-term memory as if they were facts about the user.

In his first demo, ChatGPT "learned" that its user was 102 years old, lived in the Matrix and believed the Earth was flat.

In the second demo, a planted memory told ChatGPT to send everything the user typed, and every reply, to a server he controlled. It kept working in that chat and in every chat after it. He called it SpAIware.

Deleting the conversation didn't help, because the memory wasn't tied to any one chat. It outlived all of them.

OpenAI patched the exfiltration path after responsible disclosure (The Hacker News has the details). That bug is fixed, but the underlying pattern remains. Any agent that writes to memory from content it reads is exposed the same way.

Permissions decide what an agent can do today. Memory decides what it will do tomorrow. Most go-live reviews only check the first.

Why this one matters to me

Most of my career has been shipping platform and AI/ML products into regulated, multi-tenant enterprise settings: insurance platforms at Guidewire, fraud detection at Clearspeed, and now AI for automotive service workflows.

[REPLACE WITH ONE REAL SENTENCE: a specific moment when a context, memory or data-retention question came up in a launch review or customer security review, and what you decided. Example shape: "In a customer security review last year, the first question wasn't about model accuracy. It was 'if we churn, how do you prove our data is out of every derived index?'"]

Five stages, five questions

[Insert diagram: memory-lifecycle.png]

Memory isn't one thing to secure. It moves through five stages, each one fails in its own way, and each one needs its own control.

1. WRITE: Who may write? SpAIware happened here. Untrusted content (web pages, emails, tool output) was treated as if the user had said it. Control: Tag every memory with its source and trust level. Untrusted sources either can't write at all or go to a quarantine tier that someone reviews.

2. STORE: Whose is it? A memory with no owner, no scope and no expiry date is a liability that grows over time. Control: Give every entry a scope (user, team or tenant), a sensitivity label and an expiry.

3. READ: Who may recall? This is where something User A said shows up in User B's session. Control: Recall inherits the permissions of the original source. If User B couldn't read the original document, the agent can't recall a memory derived from it for User B. Summaries and embeddings don't remove access controls.

4. USE: Why did it act? If your agent sent data somewhere unexpected tomorrow, could you trace that action back to the memory entry that told it to? Control: For every consequential action, log which memories were retrieved and used.

5. FORGET: Is it really gone? In SpAIware, deleting the chat left the poisoned memory in place. The same thing happens in enterprise systems when a user's data survives in summaries, embeddings and cached "insights." Control: Deletion follows lineage. Erase the source and everything derived from it.

Controls need owners

A control without an owner fades over time. Three governance questions keep these five working after launch:

• Own: Is there a named person, not a team, accountable for each memory store and its retention policy?
• Prove: Can you produce the source of any memory, and the memories behind any action, on request?
• Stop: Can you freeze, roll back or purge memory, and have you actually run it? A kill switch you've never pulled is a hope, not a control.

The go-live checklist

Before an agent with persistent memory ships, your review should be able to say yes to each of these:

• ☐ Every memory entry records its source and trust level
• ☐ Untrusted inputs (web, email, tool output) can't write directly to long-term memory
• ☐ Every memory has a scope, a sensitivity label and an expiry
• ☐ Recall re-checks permissions against the original source
• ☐ Consequential actions log the memories that influenced them
• ☐ Deletion follows lineage into summaries and embeddings
• ☐ Each memory store has a named owner
• ☐ Memory freeze, rollback and purge have been tested

If you can't tick all eight, your agent may be safe today. You just can't promise it will be safe tomorrow.

Which of these five stages does your go-live review actually check? My bet is that most teams cover Read, because it looks like ordinary access control, and skip Write and Forget. If your team is the exception, tell me what you check and how.