Giving my agents a shared memory

Every agent session starts blank. The model might be brilliant, but it wakes up with no idea how I work, what we decided last week, or why the deploy script looks the way it does. For a while I did what most people do. I wrote longer and longer instruction files.

It didn’t work. Agents read the first file you point them at, but they don’t reliably follow the trail of “see also this file” pointers inside it. A rule that lives three hops deep almost never loads. The fix isn’t better prose, it’s structure.

A memory made of files

So I built my agents a shared memory. Nothing exotic. A knowledge base of plain markdown notes, versioned in git. One note per durable fact. A decision and its why. A convention worth reusing. A fix that took a day to find and would take a day to find again. An index ties it together, and notes link to each other like a wiki.

Plain files turn out to be the right shape for this. Agents are good at reading and editing files, git gives history and review for free, and nothing is locked into a tool I might leave.

Injecting the core

The memory only helps if it shows up in the session. A small script slices the mandatory core, the handful of rules that must always be in force, directly into every session at start. Everything else stays on demand. The agent opens a note when the work calls for it.

I run four different agent CLIs, and each one needs its own delivery mechanism. One takes a session-start hook. One needs a plugin. One gets the rules passed through a wrapper flag because its hooks silently drop output. The script is the same, only the plumbing differs. That was a week of fiddling, and it’s the part nobody warns you about.

The discipline

The system is only half the work. The habit is the rest. Recall before starting, so a task that the memory likely covers never starts cold. Capture after, so a decision, a convention or a hard-won fix gets written down while it’s fresh. And the bar stays high. Most sessions add nothing, and that’s correct. A memory full of noise is worse than no memory.

What changed

Agents stopped re-deriving decisions we already made. Conventions survive across tools and across machines. The same rules load whether I’m at the laptop or an agent is running headless on the always-on machine. When I switch harnesses, the memory comes along.

It’s not free. Injected context has hard size caps, so the core has to stay small. Notes drift and need curation. The whole thing only works as long as the bar for writing stays high.

But the direction is clear. The models keep getting better on their own. The memory is the part that compounds, because it’s mine.