jcode

Mission

jcode is a harness that is trying to push the frontier of intelligence. We believe that the next leaps in agent capabilities will come from the harness level. The model is just the intuitive core of an agent, and the harness around it exists to extract as much useful work out of that core as possible. Scaling test time compute via agent swarms, increasing speed through parallelism, solving continual learning through memory. We publish everything publicly, and will remain open source and open research. At the same time, we strive to be a good general purpose dev tool for day to day work.

jcode bench

Running the standard evals against these harnesses showed us how poorly they represent what agents can actually do. Existing benchmarks face a forced choice. Public benchmarks leak their solutions into training corpora, so scores measure memorization and misrepresent capability. Private benchmarks resist that, but they demand trust in a hidden test set, offer less transparency, and are hard to get access to. Beyond that choice, they share two structural flaws. They impose timeouts, so an agent is penalized for allocating more time to a problem, exactly the long-horizon behavior we want agents to develop. And they score on a coarse, discrete boundary: the gap between two very different models can show up as a single task completion. A model that solves 70% of a bench may sit next to a remaining 30% that demands a far higher capability level, and that entire delta is invisible between the discrete tasks. So we designed a new class of benchmarks with a spec:

• uncontaminatable by construction: there is no answer to memorize, so everything can be public
• hard to saturate by construction: scored on optimization depth, so the ceiling stays open past the frontier
• deterministic by construction: same submission, same score, always
• quantifiable by construction: the metric falls out of the task definition, no rubric, no judge
• continuous by construction: an analog score axis instead of pass/fail, so capability differences show up as score differences at every level, and time is recorded rather than capped
• cheat-resistant by construction: we provide a good reference implementation as the starting point, so web searching for ready-made solutions is only cheating at lower capability levels. At the frontier, there are no implementations left to copy that aren't useful recombinations

This design is immediately trustable because everything is public. Training on a previous model's transcript may even help a little, but it doesn't score better unless the model has genuinely generalized from it: replaying a transcript only matches the old score, and beating it requires optimizing past what was memorized, which is the capability the benchmark measures in the first place.

jcode bench v1 provides three tasks that follow this spec: float-print, json-unescape, and utf16-transcode. We chose these tasks because they grade in seconds and are pure coding on real world functions that are useful and genuinely benefit from optimization: a tight loop the agent can climb, on work that actually ships in libraries. The tasks are also chosen so the implementation cannot be overfit to the test suite: correctness is verified over the entire input space, not a sample. Compare measuring grep's speed by timing it on the Linux repo, where an implementation can be overfit to the Linux repo specifically. Below, we plot the score of the agent over time, from a single user prompt. Note that time is 1:1 correlated with cost and token utilization, because tool call time is negligible in these tasks. Both runs used Opus 4.8 with high thinking, identical prompts, launched concurrently, each stopping on its own judgment. On float-print both passed the full 2³² correctness gate; jcode finished at +8.64 (398x speedup) versus Claude Code's +7.17 (144x).

New: a July 19 Jcode solo matrix compares GPT-5.4, GPT-5.5, GPT-5.6 Sol, Claude Sonnet 5, Claude Fable 5, and Claude Opus 4.8 across all three tasks. All 18 matched runs finished and passed final correctness. See the finalized model matrix and audited data.

The uncontaminatable benchmark design · full results and agent transcripts

Terminal-Bench 2.1

89 terminal tasks, claude-opus-4-8, run through the Harbor harness on Modal. Our best cell is medium effort at 77.8% (k=2), against Claude Code's published 78.9%. Medium beats xhigh while costing less per trial and finishing tasks 30% sooner, so more thinking stops paying for itself past medium on this benchmark. We also read and classified 28 failing xhigh trials: 22 real misses, 3 infra flakes, 3 borderline thresholds, and 0 grader disputes.

Every run, with accuracy against cost and per-task transcripts

Confidence stepping

Jcode's todo tool asks the agent to rate its confidence in each task item, both when the item is assigned and when it's marked done. Tracing those scores across our Terminal-Bench runs showed a clear pattern: confidence is always high after, but sometimes low before, and failures rarely come from tasks the agent was confident about at assignment. The low scores at assignment are real signal, and the jump to 100 at the end is not. So when we see a large spike in confidence, we force the agent to go back and check its work instead of accepting the claim. Ideally, confidence rises incrementally as validation happens during implementation, with each passing test earning a step up. Below, both harnesses on Terminal-Bench 2.1: each line is one trial, from confidence at assignment to confidence at completion. The extra checking pays: trials that finish in time pass more often (92% vs 88%), and even trials cut off by the benchmark's 15 minute limit are more likely to already contain correct work (47% vs 42%).

the full confidence study · every trial transcript

Swarm

We believe swarms are the key to superintelligence. A single agent walks one path through a search space. A swarm explores many paths at once, while Jcode coordinates their work in the same repository.

See how Jcode swarms work

Auto-poke

Persistence is the other half of intelligence: most agent failures are not wrong answers but early exits. Models love to declare victory. Jcode checks the todo list first. When a turn ends with incomplete todos, the harness pokes the model back to work automatically. Pokes are smart about failure too: transient network errors are retried, while non-retryable errors stop the loop instead of burning tokens. The same mechanism drives headless runs, so `jcode run` keeps iterating on a task across turns until the work is actually finished.

See the persistence implementation · Read the full run transcript

Append-only context engineering keeps the model's KV cache hot turn after turn.

The single biggest lever on latency and cost is the prompt cache: providers charge a fraction for tokens the model has already seen, and reading them back is far faster than recomputing attention from scratch. Jcode is built so that cache almost never breaks. The prompt prefix is stable and the conversation is strictly append-only, so the longest possible shared prefix survives between turns instead of being silently invalidated. Things that would normally bust the cache are kept out of the prefix: tool schemas come from a fixed on-disk cache, MCP tools are advertised up front so a late connection never rewrites earlier turns, and dynamic context like memory recalls and system reminders are placed where they do the least damage. The result is consistently high cache-hit rates, lower bills, and noticeably snappier turns, especially on long sessions where naive agents pay to re-read the whole history every time.

MCP tools are ready the moment a session starts, with no prompt-cache miss.

Most agents make you wait: they block startup while every MCP server handshakes, or they connect lazily and then quietly invalidate the model's prompt cache the first time a tool shows up, costing you a full re-read of the conversation. Jcode does neither. On startup it advertises every configured MCP tool instantly from an on-disk schema cache, so the model sees the complete tool list in its very first request and the prompt cache stays warm. The actual server connections happen in the background, connect-on-first-call: if the model reaches for an MCP tool before its server has finished connecting, that single call transparently waits for the handshake, while everything else stays non-blocking. The result is that you can start typing and the agent can start working the instant a session spawns, MCP tools included, with no startup delay and no cache penalty. This works the same across the interactive TUI, the server, and headless `jcode run`.

Long-running work keeps going while the agent keeps thinking.

Ask an agent to watch a build or a test run and it usually writes a polling script: sleep 500, check once, give up. Jcode has background tools instead. Start any command with run_in_background and it becomes a task the agent can list, tail, inspect, cancel, or wait on. The wait action blocks until the task finishes or hits its next progress checkpoint, so the agent wakes on events rather than burning turns on sleep loops. A foreground command that outruns its timeout is never killed: jcode adopts it into the background, and the task survives even a reload of the agent's own binary. Jcode parses task output for progress and renders each task as a live card in the TUI: percent, counts like 6/10 tests, byte ratios, and ETA, read from structured JCODE_PROGRESS lines or inferred from ordinary output. An agent can run a dozen tasks at once and let the server wake it when one needs attention.

A good built in memory system.

Jcode embeds each turn and response as a semantic vector. Every turn queries a graph of memories to efficiently find related entries via cosine similarity. The embedding hits are fed into the conversation, or optionally a memory sideagent verifies the memories are relevant and does more retrieval work before injecting them. This results in a human-like memory system: the agent automatically recalls information relevant to the conversation without actively calling memory tools or burning tokens.

Self dev mode

Jcode is inventing a new form of customizability. One that doesn't limit you to what a plugin or extension can do. Tell your jcode agent to enter self dev mode, and it will start modifying its own source code. Jcode is optimized to iterate on itself. There is significant infrastructure around self development, which allows it to edit, build, and test its own source code, then reload its own binary and continue work in your (potentially many) sessions, fully automatically.

Built in the open.