The vendors in Integration Bench are fictional, but the integration work is recognizable. Readers looking through the 50 public tasks will find candidates, requisitions, placements, notes, payroll assignments, background checks, interview reschedules, OAuth tokens, polling cursors, and signed webhooks.
That has prompted a fair question: which real applicant-tracking systems do these vendors resemble?
We published a complete vendor classes table to answer it. The useful comparison is at the level of product category, data model, and integration surface. The generated APIs remain specific to the benchmark.
Why the benchmark uses fictional vendors
A benchmark against a real ATS creates two problems. The API changes underneath the task, and models may already know the answer from SDKs, documentation, changelogs, or forum posts in their training data.
Integration Bench generates each vendor from a declaration instead. The declaration defines resources, authentication, pagination, webhooks, rate limits, fault injection, and a controlled set of documentation errors. It produces both the live API simulator and the documentation available to the agent.
This gives every model a vendor it has never encountered while retaining the shape of production integration work. The construction and validation process explains why the grader reads final state and vendor request logs after the agent stops.
The 15 synthetic service names
The current public manifest counts 14 benchmark vendors. The public tickets contain 15 service names because task-0011 connects TalentForge to Onboardly, an adjacent onboarding service. Counting every named service produces the following groups.
| Product family | Synthetic services | Familiar real products in the category |
|---|---|---|
| Staffing ATS and commercial operations | StaffLine, Placemint, GlobalHire | Bullhorn, JobDiva, Avionte, CEIPAL, Tracker RMS |
| Enterprise recruiting and HCM | TalentForge, Bullpen, RecruitOS | Workday Recruiting, SAP SuccessFactors, iCIMS, Salesforce recruiting implementations |
| Modern ATS | TalentLoop, HireWire | Greenhouse, Ashby, Lever, Teamtailor |
| Workforce management and payroll | Rosterly, Paygrade | UKG Dimensions, UKG Pro, ADP Workforce Now, Deputy, Workday HCM |
| Temporary-staffing marketplace | CrewCall | Avionte, Fountain, Indeed Flex |
| Background screening | Vettly | Checkr, Sterling, HireRight |
| Interview scheduling | Interviewly | GoodTime, Prelude, Calendly |
| Onboarding and e-signature | Onboardly | DocuSign, Adobe Acrobat Sign |
| Talent CRM and skills intelligence | SourceWell | Beamery, Avature, Eightfold |
Several rows include software outside the ATS category. That is intentional. Real hiring stacks cross system boundaries, and integration work often continues into workforce management, screening, scheduling, onboarding, payroll, and talent intelligence.
Where the likeness comes from
The closest comparisons come from the records a system owns and the workflows built around them.
StaffLine owns candidates, jobs, applications, and notes, then exposes them through polling with legacy-style authentication. That places it near staffing ATS products such as Bullhorn and JobDiva.
TalentLoop covers the same core recruiting objects but adds OAuth, webhooks, reconciliation polling, writeback, and candidate scorecard documents. Its operating model resembles modern API-first ATS products such as Greenhouse and Ashby.
RecruitOS centers requisitions, applications, reporting extracts, and warehouse reconciliation. Those tasks resemble enterprise recruiting modules in Workday, SAP SuccessFactors, and iCIMS.
Placemint owns placement outcomes, client records, notes, fees, and the commercial side of staffing. Its likeness sits closer to the back-office and placement functions in Avionte, JobDiva, and Bullhorn than to a corporate ATS.
The adjacent services are easier to recognize. Vettly manages subjects, checks, and reports, which places it alongside Checkr and Sterling. Interviewly handles reschedules and confirmation webhooks, much like interview-scheduling products. Paygrade uses employee and assignment records over an RPC-style API, a pattern familiar to teams integrating payroll and older HRIS products.
Where the likeness stops
No row in the table is a claim that a synthetic vendor reproduces a named product’s API. Authentication details, pagination rules, timestamps, error behavior, webhook semantics, and documentation defects were generated for the benchmark.
For example, StaffLine’s HMAC signing requirement helps test whether an agent signs every request correctly. It should not be read as documentation of Bullhorn or JobDiva authentication. TalentLoop’s event ordering tests measure webhook correctness without asserting that Greenhouse or Ashby emits the same event envelope.
This distinction matters when interpreting model performance. A model that solves a TalentLoop task has demonstrated useful work against a live, unfamiliar, event-first ATS-shaped API. Its score does not certify a production Greenhouse connector.
Use the map as orientation
The vendor map gives each ticket a recognizable operational setting:
- StaffLine and GlobalHire cover polling-heavy staffing systems.
- TalentForge, Bullpen, and RecruitOS cover enterprise recruiting.
- TalentLoop and HireWire cover modern ATS integration patterns.
- Rosterly and Paygrade extend the workload into workforce and payroll systems.
- Vettly, Interviewly, Onboardly, and SourceWell cover adjacent hiring services.
The task pages remain the source of truth for what each model had to build. You can inspect the public trajectories to see every tool call, vendor request, graded check, and patch, then use the vendor classes reference to place that work in a familiar product category.
If you work with an integration class the suite does not cover, send us the vendor shape and the behavior that makes it difficult.