Legacy Integration Bench

A large share of the software HeyMilo connects to predates REST, JSON, and in some cases the idea that anyone but a browser would call it. The connector work for those vendors is slower, more brittle, and almost entirely absent from public coding benchmarks. Legacy Integration Bench applies the Integration Bench construction, invented vendors graded from their own logs, to that slice.

What the vendors will look like

• SOAP and XML-RPC endpoints with WSDLs that disagree with the server
• Cookie-session authentication with CSRF tokens and forced re-login
• Bulk exports as CSV or fixed-width files dropped over SFTP on a schedule
• Pagination by opaque continuation tokens, by date window, or not at all
• Rate limits enforced by silent truncation rather than a 429
• Character encodings that are not UTF-8 and never say so

What is different about grading

The same three layers apply: data correctness against a committed answer key, protocol discipline read from the vendor’s log, and survival under injected faults. Legacy vendors add a fourth failure class we see constantly in production: an integration that works on the sample export and breaks on the real one because a field the documentation calls optional is always present, or the reverse.

Timeline

Vendor declarations are being authored now. We will publish the first public task set with the same evidence bundle as Integration Bench: transcript, verdict, vendor requests, and diff for every attempt. If you run a legacy system and want it represented as an invented sibling, collaborate with us.