Skip to content
integration bench from HeyMilo
  1. Home
  2. Releases

Integration Bench released

The 50 public tasks and their official runs are published in full. Each attempt ships the ticket, diff, vendor request log, graded checks, and raw transcript. Leaderboard, matrix, and per-model and per-task indexes go up at data.integrationbench.com; this site is the public home.

2 September 2026

What shipped

  • 50 public tasks. Full tickets, graded checks, and official attempts. Leaderboard scores are the mean over all 50.
  • Runs. Gathered, checked, and published. Per attempt: problem.md, patch.diff, vendor-requests.json, verdict.json, trajectory.json, raw transcript.jsonl.
  • data.integrationbench.com. Attempt bundles plus leaderboard.json, matrix.json, manifest.json, and per-model and per-task indexes.
  • Trajectories. Claude Code, Codex, and OpenCode transcripts mapped to one event schema so runs can be compared across harnesses.
  • This site. Leaderboard, per-model and per-task pages, trajectory viewer, releases.

Leaderboard. Tasks. Trajectories.