The ContextStream memory audit: 5 questions to ask your agent's recall system
Your agent doesn't fail loudly when memory breaks. It fails politely — re-asks a question you answered in April, re-decides an architecture call you settled weeks ago, ships code that contradicts a convention it "knew" yesterday. Nobody notices until review. Here's a 10-minute audit: five questions, each with a test you can run today.
-
Can it recall a decision from three sessions ago — unprompted?
Test: pick a real decision ("Postgres over Mongo", "errors return typed results, not exceptions"). Fresh session, ask it to build something that touches that decision. Don't hint.
Pass: it applies the decision. Fail: it re-litigates. Static agents.md files break here — decisions made in conversation never land in the file. That's what a Decision memory event exists for: captured once, recalled in every session that touches it. -
Does it separate preferences from facts?
"Terse commit messages" and "rate limit is 100 req/min" are different memory types. Systems that blend them either nag you with your own style or treat hard constraints as optional.
Test: state a preference in one session, a constraint in another, then ask for output touching both. Pass: preference shapes it, constraint is never violated. -
When context gets long, what gets dropped first?
Every recall system prunes — by recency or by relevance. Test: fill a session with unrelated work, then ask about something from the start. Recency-based systems already evicted it. Selective recall pulls back exactly what the task needs. Total recall isn't the goal; the right recall is. -
Can it tell you why it remembers something?
Memory without provenance is a liability. Test: ask "where did that come from?" about a recalled fact. Pass: it traces to the source event — who decided, when. Fail: it shrugs or confabulates a source. -
What's the correction path when memory is wrong?
Decisions get reversed. Test: reverse a documented decision, then ask the agent to build against it. Pass: the new decision supersedes the old one, which gets retired — not just outvoted by a louder prompt.
Score it:
5/5 — real memory.
3–4 — recall gaps you're papering over by re-explaining.
0–2 — you're the memory layer, and you pay that tax every session.
Run it for real: point ContextStream at one local folder, work a normal session, run these five questions the next morning. The gaps show up fast — so does the fix.