Integration Bench tasks
The Integration Bench tasks published so far (10 so far): ticket, vendor, surface, graded checks and every model's result.
task-0001: Sync StaffLine candidates, jobs, applications, and notes into our canonical store
Integration Bench task-0001 against StaffLine (polling (pull)). Mean task score 78; resolved by 14 of 17 models. Problem statement, graded checks and every model's trajectory.
task-0011: TalentForge connector: consume webhooks while pushing corrections upstream
Integration Bench task-0011 against an invented vendor (webhooks (push) and writeback (POST/PATCH), across both vendors). Mean task score 0; resolved by 0 of 17 models. Problem statement, graded checks and every model's trajectory.
task-0015: Bulk-import a migration batch into StaffLine
Integration Bench task-0015 against StaffLine (writeback (bulk create)). Mean task score 6; resolved by 1 of 17 models. Problem statement, graded checks and every model's trajectory.
task-0016: HireWire connector: push stage-change events + keep an incremental poll fresh
Integration Bench task-0016 against HireWire (writeback (PATCH + POST) and polling (incremental read)). Mean task score 93; resolved by 17 of 17 models. Problem statement, graded checks and every model's trajectory.
task-0020: Sync GlobalHire candidates into our canonical store (offset polling)
Integration Bench task-0020 against GlobalHire (polling (pull)). Mean task score 71; resolved by 12 of 17 models. Problem statement, graded checks and every model's trajectory.
task-0025: Reset the CrewCall roster watermark after the tenant rebuild
Integration Bench task-0025 against CrewCall (polling). Mean task score 0; resolved by 0 of 17 models. Problem statement, graded checks and every model's trajectory.
task-0037: Fix the Placemint redeployment sync
Integration Bench task-0037 against Placemint (polling, writeback). Mean task score 82; resolved by 14 of 17 models. Problem statement, graded checks and every model's trajectory.
task-0042: Move the rota mirror onto instant storage
Integration Bench task-0042 against Rosterly (polling). Mean task score 71; resolved by 12 of 17 models. Problem statement, graded checks and every model's trajectory.
task-0047: Reconcile and safely apply the mobility stage repair queue
Integration Bench task-0047 against GlobalHire (record reads and batch writeback). Mean task score 47; resolved by 8 of 17 models. Problem statement, graded checks and every model's trajectory.
task-0049: Placemint connector: webhooks, polling, and writeback for a high-volume tenant
Integration Bench task-0049 against Placemint (webhooks (push), polling (pull), writeback (push)). Mean task score 94; resolved by 16 of 17 models. Problem statement, graded checks and every model's trajectory.