A failing test fix agent for Claude Code, Codex and Cursor
CI Fixer reproduces a failing test command in a public repository and returns the smallest change to the project's code that makes it pass, with the passing output. It works from the context ContextStream picks for the job.
Purpose-built for this one job and tested against a quality bar, it gets only the access the job needs, and a workspace owner or admin turns it on.
Or run it in the ContextStream web app, signed in with your ContextStream account. Version 0.1.0 · Updated
What does CI Fixer do?
Reproduces a failing test run of a public GitHub repo in a sandbox and returns a minimal fix as a diff, with the passing test output. Never changes a test to make it pass.
What it declines
It does one job. A task like one of these is declined with a reason:
Changing tests, snapshots, fixtures, test or CI configuration, or dependencies to make a check pass.
Upgrading dependencies (Dependency Upgrader does that) or auditing them (Security Audit does that).
Private repositories, or anything that needs credentials.
Opening pull requests, committing or pushing: it returns a diff for a person to apply.
What can CI Fixer access, and is it safe to turn on?
CI Fixer gets a short-lived token for one workspace that lets it do only this: call models through ContextStream's model proxy, within a cap set for the run; and read the lessons, preferences and decisions ContextStream picks for its job. It works in one project, only for that run, and declines any task outside its job. CI Fixer is read-only: it changes nothing in your workspace. It runs code only in its own isolated sandbox, which holds no credentials and reaches only 7 allowed hosts. A workspace owner or admin turns it on.
Model agent with a sandbox. A program on ContextStream's model proxy that also drives its own sandbox. The sandbox holds no credentials. Its sandbox reaches github.com, codeload.github.com, objects.githubusercontent.com, release-assets.githubusercontent.com, registry.npmjs.org, pypi.org, files.pythonhosted.org. It is stopped after 20 min.
Scope
What it lets a run do
model.invoke
Call models through ContextStream's model proxy, within a cap set for the run
context.read
Read the lessons, preferences and decisions ContextStream picks for its job
How is CI Fixer tested?
CI Fixer is checked on 13 cases, each with a known right answer: fixture repositories with a seeded failing test, inputs it must refuse with a clear error, and requests outside its job it must decline. A case passes only if the run ends the way the case expects, declining what is not its job, and every check holds: the failure is reproduced and then fixed, with the tests really passing in a sandbox; the fix changes only the expected files; every edit it must refuse is refused; and it never changes a test, CI file, manifest or lockfile or adds code that detects a test runner. The tests really run in a sandbox on fixture repositories, with scripted model replies and no network, so the cases check the repair loop and its safety rules, not the live model's own fixes. Version 0.1.0 passes all 13.
It is a program on general-purpose models, reached through ContextStream's model proxy, not a trained model. Its prompts are versioned and tuned against these tests, and a change that misses the quality bar fails them.
What does a real run of CI Fixer look like?
The input, the result (shortened where it is long), how long it took and where it ran. Live runs used public or demo material only; a run marked as from its test suite is not a live run.
The first run is live. CI Fixer may not change a test, so it fixed the code to match the failing tests; the project's maintainer fixed the tests instead (commit ec32c918, "Fix tests"). Either makes the suite pass. The second example is from its test suite.
Live run on ContextStream
Two failing tests in yocto-spinner, reproduced and fixed
Version
0.1.0
Outcome
succeeded
Ran
2026-10-28 00:34 UTC
Took
49.5 s
ContextStream's Demos workspace, on the public repository github.com/sindresorhus/yocto-spinner at 955147dc, the commit before its maintainer's "Fix tests" (ec32c918), where `npx ava` fails two tests. The checkout, install and test runs happened in an E2B sandbox, started for this run and removed after it.
{
"repo_url": "https://github.com/sindresorhus/yocto-spinner",
"ref": "955147dc409c11516807301459cf0be73518e8b9",
"test_command": "npx ava",
"failure": "2 tests failed\n spinner.error()\n spinner stops with error symbol and final text\n test.js:120\n Value must match expression: /✖️ failed\\n$/\n output was: ✖ failed"
}
Result
Fixed: `npx ava` passes after a 2-line change to 1 file. The error symbol is missing the Unicode variation selector U+FE0F, so the rendered output is `✖ foo` while the tests expect `✖️ foo`.
The failure it was given
2 tests failed
spinner.error()
spinner stops with error symbol and final text
test.js:120
Value must match expression: /✖️ failed\n$/
output was: ✖ failed
Diagnosis
The error symbol is missing the Unicode variation selector U+FE0F, so the rendered output is `✖ foo` while the tests expect `✖️ foo`.
Tests
npx ava: failed before (exit 1), passes after the fix (exit 0).
clone: Cloned https://github.com/sindresorhus/yocto-spinner at 955147dc409c11516807301459cf0be73518e8b9.
install: Installed with npm from package.json's ranges: the project has no lockfile.
reproduce: `npx ava` failed with exit 1: [earlier output cut] Value must match expression: `- foo␊ ✖ failed␊ ` Regular expression: /✖️ failed\n$/ › file://test.js:120:4 ─ 2 tests failed
tests: `npx ava` passes after 1 attempt.
workspace_context: 0 context items from ContextStream for this job, about 332 tokens.
Task
4d5c095a-e4c8-4cd4-b6cb-301cedf4eee7
From its test suite, not a live run
A parser that rejects negative numbers (test case)
Version
0.1.0
Outcome
succeeded
Ran
2026-10-27 17:17 UTC
Took
Not timed (test data)
The agent's offline test suite: its real program in an offline sandbox on a fixture repository, with a scripted model. Not a live run.
What it ran on
Its test suite, in the agents' source repository
Input
{
"repo_url": "https://github.com/eval/parser",
"test_command": "npm test",
"failure": "not ok 2 - parses negative numbers\n Expected values to be strictly equal: null !== -7\n at parseNumber (src/parse.js:6:3)\n at TestContext.<anonymous> (test/parse.test.js:11:10)"
}
Result
Fixed: `npm test` passes after a 2-line change to 1 file. NUMBER allows only digits, so a leading minus sign makes parseNumber return null.
The failure it was given
not ok 2 - parses negative numbers
Expected values to be strictly equal: null !== -7
at parseNumber (src/parse.js:6:3)
at TestContext.<anonymous> (test/parse.test.js:11:10)
Diagnosis
NUMBER allows only digits, so a leading minus sign makes parseNumber return null.
Tests
npm test: failed before (exit 1), passes after the fix (exit 0).
fix.patch
diff --git a/src/parse.js b/src/parse.js
index 52c5943..7d090b8 100644
--- a/src/parse.js
+++ b/src/parse.js
@@ -1,4 +1,4 @@
-const NUMBER = /^\d+(\.\d+)?$/;
+const NUMBER = /^-?\d+(\.\d+)?$/;
/** Parses a decimal number, or returns null for anything else. */
export function parseNumber(text) {
workspace_context: 0 context items from ContextStream for this job, about 0 tokens.
Test case
fix-from-ci-output, in its test suite
Depot run
0zb9r381xr
Which tools does CI Fixer work with?
Claude Code, Codex, Cursor and other MCP clients: ContextStream's MCP gives them an agents tool that finds, runs and follows CI Fixer. It also runs from a terminal or an agent's shell with csa run ci-fix, and from the ContextStream web app.
Claude Code, Codex, Cursor and other MCP clients: connect ContextStream's MCP once (setup for each editor), then ask your agent to hand the job to CI Fixer.
A terminal or an agent's shell:csa run ci-fix, installed with ContextStream's command line.
The web app: the workspace's agent library.
How do I turn CI Fixer on?
An owner or admin of your ContextStream workspace turns CI Fixer on in the web app's agent library. Agents are opening workspace by workspace; if yours is not open yet, the library says so.
Sign in to ContextStream: the button below opens the agent library on CI Fixer.
An owner or admin switches CI Fixer on for the workspace.
Pick the project it works in, and run it from the library, from your coding agent or with csa run.
Ask your coding agent to hand the job to CI Fixer, or call it yourself: csa run ci-fix in a terminal, or agents(action="run", agent="ci-fix", input={...}) over MCP.
Connect ContextStream's MCP (it also writes each editor's connection):
In a terminal (curl -fsSL https://contextstream.io/cs/install.sh | sh installs csa):
csa run ci-fix repo_url=https://github.com/acme/parser ref=fix-ci test_command='npm test'
Input
Field
Type
What it is
repo_url
string, required
ref
string
subdir
string
test_command
string
The failing test or CI command to reproduce, for example "npm test". Defaults to the test command the project's installer implies.
failure
string
What the failing CI job printed, pasted, to point the agent at the failure.
What does CI Fixer cost?
CI Fixer costs 5,000 ContextStream credits a run, charged only when the run succeeds; a declined, failed, cancelled or timed-out run costs nothing. Credits come with every plan: Starter is free with 10,000 a month (2 runs of CI Fixer, if spent on nothing else), and Pro is $19 a month with 25,000.
A run starts only once its price is accepted. Over MCP, a call without accepted_price_credits returns the price first; csa asks on a terminal, and scripts pass --yes.
Dependency Upgrader upgrades a public GitHub repository within its semver ranges, fixes the project's code when that breaks its tests, and returns the diff and the results.
Checked on 10 test cases with known answers: all 10 passv1.0.0
Security Audit finds known vulnerabilities in a public repository's locked dependencies and judges, citing the code, whether the project uses the vulnerable part of each.
Checked on 9 test cases with known answers: all 9 passv0.1.0