The ContextStream memory audit: 5 questions to ask your agent's recall system

The ContextStream memory audit: 5 questions to ask your agent's recall system

Your agent doesn't fail loudly when memory breaks. It fails politely — re-asks a question you answered in April, re-decides an architecture call you settled weeks ago, ships code that contradicts a convention it "knew" yesterday. Nobody notices until review. Here's a 10-minute audit: five questions, each with a test you can run today.

1. Can it recall a decision from three sessions ago — unprompted?
   Test: pick a real decision ("Postgres over Mongo", "errors return typed results, not exceptions"). Fresh session, ask it to build something that touches that decision. Don't hint.
   Pass: it applies the decision. Fail: it re-litigates. Static agents.md files break here — decisions made in conversation never land in the file. That's what a Decision memory event exists for: captured once, recalled in every session that touches it.

2. Does it separate preferences from facts?
   "Terse commit messages" and "rate limit is 100 req/min" are different memory types. Systems that blend them either nag you with your own style or treat hard constraints as optional.
   Test: state a preference in one session, a constraint in another, then ask for output touching both. Pass: preference shapes it, constraint is never violated.

3. When context gets long, what gets dropped first?
   Every recall system prunes — by recency or by relevance. Test: fill a session with unrelated work, then ask about something from the start. Recency-based systems already evicted it. Selective recall pulls back exactly what the task needs. Total recall isn't the goal; the right recall is.

4. Can it tell you why it remembers something?
   Memory without provenance is a liability. Test: ask "where did that come from?" about a recalled fact. Pass: it traces to the source event — who decided, when. Fail: it shrugs or confabulates a source.

5. What's the correction path when memory is wrong?
   Decisions get reversed. Test: reverse a documented decision, then ask the agent to build against it. Pass: the new decision supersedes the old one, which gets retired — not just outvoted by a louder prompt.

Score it:
5/5 — real memory.

3–4 — recall gaps you're papering over by re-explaining.

0–2 — you're the memory layer, and you pay that tax every session.

Run it for real: point ContextStream at one local folder, work a normal session, run these five questions the next morning. The gaps show up fast — so does the fix.