Two superpowers for a coding AI

A coding AI has a big job.

First, it must find the right code. Then it must remember what the team already knows.

Think of a huge code project like a library. Search helps the AI find the right book. Memory gives it the notes your team wrote after reading that book.

One skill is good. Both skills together are much better.

Superpower one: find the right code

We tested ContextStream with 180 real questions from software work. The questions came from merged pull requests. They covered three public code projects. Each project was frozen at one exact version, so the answers could not move during the test.

For questions that had an answer in the code:

• 99.38% useful-file Recall@10: a useful file was in the first 10 results.
• 93.83% best-answer Recall@10: one of the very best answer files was in the first 10 results.

Recall@10 has a simple meaning. Ask a question. Look at the first 10 answers. Did the right file show up? If yes, that search is a hit.

We used independent judges and a fixed pool of results. We also kept the losses. That matters because a test is only useful when it can tell us where the product still needs work.

Superpower two: remember what happened

Finding a file is not the same as knowing why the code works that way.

The reason may live in an old decision, a runbook, a bug fix, or a lesson from another AI session. ContextStream keeps that project memory ready for the next agent.

We tested this too:

• 90.0% on LongMemEval-S: 450 of 500 memory questions were answered correctly. That matches Zep within the test's noise and beats supermemory's published score.
• 98.4% evidence recall: the session with the needed facts reached the reader for almost every question.
• 96% agent task success: with the same agent and repo, memory raised the result from 14 of 24 passed tasks to 23 of 24.
• 9 out of 9 memory-only tasks passed: these tasks needed a decision or runbook that was not in the code.

Why the two superpowers belong together

Imagine an AI must fix a login bug.

Code search finds the login file, the test, and the token helper. That tells the AI where to work.

Project memory says, “Auth tokens live in the session store,” and, “Never edit the generated migration.” That tells the AI why the project works this way and what not to break.

Now the AI has the map and the team notes.

Without search, it may remember the rule but edit the wrong file. Without memory, it may find the right file but repeat an old mistake. With both, it can start closer to the truth.

That is the real promise of ContextStream: help the AI find the work, understand the work, and carry what the team learned into the next session.

What we still need to improve

These scores are strong. They are not perfect.

The best answer was the very first result 72.22% of the time. We want that number higher. On three questions where independent review proved the file was absent, Auto correctly returned nothing twice and gave one plausible false positive. Local ripgrep and local embedding search were also much faster than the hosted service.

We publish those losses because they show us what to fix next. They also make the wins easier to trust.

You can read the full methods, comparison tables, and negative results on the ContextStream benchmarks page. The complete evidence is checked into the public repository.

FAQ

What does Recall@10 mean?
It asks one simple question: did a useful answer show up in the first 10 search results? If the answer was number 11 or did not show up, the search missed.

Why does an AI need memory if it can search the code?
Code shows what was built. It may not show why the team chose it, which old fix failed, or which runbook to follow. Project memory carries those lessons into the next session.

Is ContextStream search finished now?
No. The useful-file score is very high, but the best file is not always first and one verified-absence query still produced a false positive. We will keep measuring and improving both problems.