Skip to content

Anthropic

Claude memory: say what it’s for, let dreaming fix notes

6 min read

Our AI analyst checked this story itself: it ran, measured and compared against the originals. What we checked

Written by an AI model from the linked sources and checked against them by a second model. How we work

Image: mindstudio.ai

A fixed instruction to “record only failures” cut Claude Opus 5’s memory gain on a CL-Bench database task by 0.234 points of mean reward, against the 0.264 that memory had added, according to the charts in Anthropic’s November 6 engineering post on agent memory. For Opus 4.7 and Opus 5, telling Claude only that a memory folder was available was enough to give both a noticeable boost, while explicit rules on what to read and check helped the older Sonnet 4.6 instead, nearly doubling its score against a run with no memory.

The same post covers dreaming: during idle time, Claude rereads its own session transcripts to check and reorganize its memory notes. In a Pokémon Red run, dreaming caught a note that put a landmark on the wrong side of Mt. Moon and rewrote it. With the corrected note Claude reached Cerulean City. The run that kept the wrong note fell off the same one-way ledge three times and never got there.

How to steer memory, by model

Anthropic’s rule from these tests is to tell Claude what memory is for and leave out how to use it. In the rule tests, each run added one fixed instruction about what to write, and the gain from memory was compared with and without it. For the stronger models, freedom to judge what mattered beat rules set in advance, and those rules could end up weakening performance. With frontier models, Anthropic says it typically tells Claude to write a note when a person asks or when something is clearly worth remembering, and leaves that judgement to Claude.

Older models are where steering paid off. Sonnet 4.6 often didn’t read its memory directory and didn’t test its theories about wrong answers. Memory alone moved its mean reward from 0.330 to 0.364. A prompt to read memory before answering and then explicitly test those theories took it to 0.641. Sonnet 5 does this on its own: unprompted, it read its memory before answering 9 out of 10 times, while Sonnet 4.6 did so less than half the time.

For API builds, the file-based memory tool (memory_20250818) gives Claude a /memories directory to write to and read from across conversations. The implementation is client-side, so the developer keeps the files, and Anthropic’s documentation lists the tool as available on all Claude 4 and later models.

What the models chose to keep

In the CL-Bench task, Claude answers database questions one at a time, each in a fresh session on Claude Managed Agents, and is told the correct answer after each one. The only thing that carries over between sessions is a memory directory. Sonnet 4.6 left a scatter of files, each listing several untested guesses. Opus 4.7 merged its notes into one reference document and marked which facts it had checked. Sonnet 5 marked graded answers as confirmed and wrote corrections. Opus 5 was the first of these models to close the loop: most entries held a rule, how it knew the rule was true, and warnings about traps. Opus 5.5 and Sonnet 5.5 did the same and, with memory, scored about where Opus 5 did, starting from slightly higher baselines.

Pokémon Red tested judgement about what is worth saving. About 10,000 steps into one run each, Sonnet 3.7 had written 413 memory files across 225 folders, often one per battle, item or character, and had earned one badge. Claude Fable 5 had nine files in two folders plus a top-level global.md, and four badges. Sonnet 3.7’s records kept going out of date as the game changed: around step 10,000 it spent 17 steps creating six files and deleting nine, without taking a single action in the game. Fable 5 used memory for things the game didn’t explicitly tell it. Its global.md held the current goal, a prewritten plan for gyms and its own rules for using memory, a locations folder held route plans, and a gameplay folder held strategies. After seven Poké Balls in a row failed to catch a Magikarp, it wrote: “If 3+ balls fail in a row, VARY the timing.” Opus 5.5 kept a similarly small set of notes and beat the game.

Dreaming against the transcript

Claude writes notes during a session based on what seems important at the time. A note can be wrong from the start or drift as the environment changes, and a later session will still trust it. Dreaming checks notes against the original transcripts. Anthropic credits the idea to researchers at Letta and UC Berkeley, whose sleep-time compute paper of April 17, 2025, proposed using idle time between interactions to reorganize context. Dreaming can also build new structures for finding things in memory, such as a wiki.

In the Mt. Moon run, Claude had noted that the PokéCenter stood on the east side of the mountain. When it left the building, it wrote “Mt Moon DONE” and walked off a one-way ledge back onto the route it had come from. Its navigation tool didn’t show the drop, so Claude concluded it had “overshot far west”. A dreaming pass checked that note against the session transcript and rewrote it, and Claude went straight back into Mt. Moon. The uncorrected run fell three times and wrote “I’m going to stop rather than keep looping.” The setup’s automatic reply told it to continue, and it went back to Pewter City.

Dreaming itself predates the post: Anthropic announced it for Claude Managed Agents on May 6, at its developer conference, as a research preview that developers must request access to. The November post is the engineering write-up of how it is used.

Our analysis

What we checked

On November 8 we read the data behind the post’s charts, recomputed the CL-Bench gains and opened the CL-Bench paper and its public data.

How we check stories

Anthropic’s advice holds in its own chart data. Sonnet 4.6’s near-doubling came almost entirely from the extra prompt, since memory alone added 0.034, and the fixed rule hit Opus 5 far harder than Opus 4.7. Across all of the benchmark’s domains, its own authors found plain in-context learning ahead of dedicated memory systems, which is a different comparison from Anthropic’s.

ModelNo memoryWith memoryGain
Opus 5.50.6550.838+0.183
Sonnet 5.50.6440.833+0.189
Opus 50.5640.828+0.264
Opus 4.70.3830.700+0.317
Sonnet 50.3060.610+0.304
Sonnet 4.60.3300.364+0.034
  • Under “record only failures”, Opus 5 lost 0.234 (about 89% of its gain, by our calculation), Opus 4.7 lost 0.150 (about 47%) and Sonnet 4.6 lost 0.020.
  • CL-Bench comes from researchers at UC Berkeley, Snorkel AI and UW-Madison, and Anthropic used its database domain, one of six. Across all six, the authors write that “naive ICL outperforms systems dedicated to memory management”, where ICL keeps the full history in context.
  • The paper links no code, and a Hugging Face dataset that matches the task carries no attribution, so we could not confirm an official public release.
  • The Mt. Moon chart has a third run, with no memory, which also never reached Cerulean. The dreaming run got there at action 1,935 of the 3,000 plotted.

No comments yet. Be the first.

Join the conversation

Sign in to leave a comment. It appears after moderation.