trajectories
Integration Bench · rev 01
170 recorded runs across the first 10 released tasks. Every tool call the agent made, every request the vendor saw, every check the grader ran, and the diff it left behind. Pick a model and a task, or click any cell.
All 170 published attempts
Timeline. The agent's messages, reasoning where the harness exposes it, and every tool call with its input and output, in order. Codex CLI, Claude Code and OpenCode transcripts are normalised to one event schema so models can be compared step for step.
Vendor log. Every HTTP request the invented vendor received: method, path, status, auth outcome, whether it was rate-limited. This is the evidence the protocol checks are graded from.
Checks. All graded assertions by layer with pass or fail and the grader's detail string. Mandatory checks are marked; one failure zeroes the task.
Diff. The patch the agent left in the repository. JSON and raw views expose the normalised file and the harness's original transcript.
Secrets and hostnames are scrubbed before publication; nothing else is edited.