Skip to content
integration bench from HeyMilo

Blog

Notes from building Integration Bench: how the vendors are invented, how grading reads the vendor's logs, what went wrong with our first scorers, and what the published trajectories show about how coding agents integrate.

1 Sep 2026 How we built Integration Bench The construction, in order: why a connector team started grading coding agents, how vendors that do not exist get generated from one declaration, why the grader never reads the code, and what a submission that did nothing taught us about our own scorer. 1 Sep 2026 Publishing the trajectories: the data behind this site How the raw evidence bundle from every Integration Bench run becomes the JSON on data.integrationbench.com: one event schema across three harnesses, what gets scrubbed, how scores are recomputed, and how to use the files yourself.