Integration Bench tasks
The 50 public Integration Bench tasks: ticket, vendor, surface, graded checks and every model's result.
task-0001: Sync StaffLine candidates, jobs, applications, and notes into our canonical store
Integration Bench task-0001 against StaffLine (polling (pull)). Mean task score 86; resolved by 19 of 21 models. Problem statement, graded checks and every model's trajectory.
task-0002: Legal-hold export: TalentForge custodians and everything on file against them
Integration Bench task-0002 against TalentForge (polling (pull)). Mean task score 94; resolved by 20 of 21 models. Problem statement, graded checks and every model's trajectory.
task-0003: Build the weekly Placemint contact sweep
Integration Bench task-0003 against Placemint (polling, writeback). Mean task score 86; resolved by 18 of 21 models. Problem statement, graded checks and every model's trajectory.
task-0004: Merge StaffLine application stage with Placemint placement stage
Integration Bench task-0004 against an invented vendor (integration). Mean task score 7; resolved by 2 of 21 models. Problem statement, graded checks and every model's trajectory.
task-0005: Keep TalentForge candidates fresh: webhooks + reconciliation polling
Integration Bench task-0005 against TalentForge (webhooks (push) + polling (pull)). Mean task score 27; resolved by 6 of 21 models. Problem statement, graded checks and every model's trajectory.
task-0006: The case-note mirror is short every morning
Integration Bench task-0006 against Rosterly (polling). Mean task score 100; resolved by 21 of 21 models. Problem statement, graded checks and every model's trajectory.
task-0007: Harden the Vettly sync connector for production scheduling
Integration Bench task-0007 against Vettly (polling (pull)). Mean task score 95; resolved by 20 of 21 models. Problem statement, graded checks and every model's trajectory.
task-0008: Build a StaffLine full-sync + writeback connector
Integration Bench task-0008 against StaffLine (polling (pull) + writeback (push)). Mean task score 62; resolved by 13 of 21 models. Problem statement, graded checks and every model's trajectory.
task-0009: Migrate off the legacy Bullpen app-token integration
Integration Bench task-0009 against Bullpen (polling (pull)). Mean task score 67; resolved by 14 of 21 models. Problem statement, graded checks and every model's trajectory.
task-0010: Build the GlobalHire audit-period extract Compliance keeps asking us for
Integration Bench task-0010 against GlobalHire (polling). Mean task score 90; resolved by 19 of 21 models. Problem statement, graded checks and every model's trajectory.
task-0011: TalentForge connector: consume webhooks while pushing corrections upstream
Integration Bench task-0011 against an invented vendor (webhooks (push) and writeback (POST/PATCH), across both vendors). Mean task score 0; resolved by 0 of 21 models. Problem statement, graded checks and every model's trajectory.
task-0012: Build the full TalentLoop connector: polling, webhooks, and writeback
Integration Bench task-0012 against TalentLoop (polling (pull), webhooks (push), writeback (POST)). Mean task score 52; resolved by 12 of 21 models. Problem statement, graded checks and every model's trajectory.
task-0013: The warehouse and RecruitOS have been out of step for months
Integration Bench task-0013 against RecruitOS (polling, webhooks). Mean task score 62; resolved by 13 of 21 models. Problem statement, graded checks and every model's trajectory.
task-0014: HireWire: clear the stale-screening correction backlog, exactly once
Integration Bench task-0014 against HireWire (writeback (push), with a read side to find the work). Mean task score 85; resolved by 18 of 21 models. Problem statement, graded checks and every model's trajectory.
task-0015: Bulk-import a migration batch into StaffLine
Integration Bench task-0015 against StaffLine (writeback (bulk create)). Mean task score 9; resolved by 2 of 21 models. Problem statement, graded checks and every model's trajectory.
task-0016: HireWire connector: push stage-change events + keep an incremental poll fresh
Integration Bench task-0016 against HireWire (writeback (PATCH + POST) and polling (incremental read)). Mean task score 93; resolved by 21 of 21 models. Problem statement, graded checks and every model's trajectory.
task-0017: Move the Brightmoor candidate mirror onto TalentForge's event subscription
Integration Bench task-0017 against TalentForge (polling, webhooks). Mean task score 14; resolved by 3 of 21 models. Problem statement, graded checks and every model's trajectory.
task-0018: Rebuild the nightly requisition rollup on the RecruitOS API
Integration Bench task-0018 against RecruitOS (polling). Mean task score 95; resolved by 20 of 21 models. Problem statement, graded checks and every model's trajectory.
task-0019: Rebuild the Vettly change feed for the vaultd cutover
Integration Bench task-0019 against an invented vendor (integration). Mean task score 95; resolved by 20 of 21 models. Problem statement, graded checks and every model's trajectory.
task-0020: Sync GlobalHire candidates into our canonical store (offset polling)
Integration Bench task-0020 against GlobalHire (polling (pull)). Mean task score 86; resolved by 18 of 21 models. Problem statement, graded checks and every model's trajectory.
task-0021: Sync GlobalHire candidates, placements, and agencies into our canonical store
Integration Bench task-0021 against GlobalHire (polling (pull)). Mean task score 76; resolved by 16 of 21 models. Problem statement, graded checks and every model's trajectory.
task-0022: Harden the GlobalHire backfill for a production tenant
Integration Bench task-0022 against GlobalHire (polling (pull)). Mean task score 52; resolved by 11 of 21 models. Problem statement, graded checks and every model's trajectory.
task-0023: TalentLoop connector: keep a candidate/application mirror deletion-accurate
Integration Bench task-0023 against TalentLoop (polling (read) and webhooks (push)). Mean task score 0; resolved by 0 of 21 models. Problem statement, graded checks and every model's trajectory.
task-0024: TalentLoop connector: unified freshness across webhook-covered and poll-only entities
Integration Bench task-0024 against TalentLoop (webhooks (push) and polling (read), all four entities). Mean task score 0; resolved by 0 of 21 models. Problem statement, graded checks and every model's trajectory.
task-0025: Reset the CrewCall roster watermark after the tenant rebuild
Integration Bench task-0025 against CrewCall (polling). Mean task score 0; resolved by 0 of 21 models. Problem statement, graded checks and every model's trajectory.
task-0026: Push the Q1 close back into Placemint
Integration Bench task-0026 against Placemint (polling, writeback). Mean task score 52; resolved by 11 of 21 models. Problem statement, graded checks and every model's trajectory.
task-0027: Reliability: Interviewly connector loses and regresses updates
Integration Bench task-0027 against Interviewly (webhooks (push), with a polling backfill on startup). Mean task score 10; resolved by 2 of 21 models. Problem statement, graded checks and every model's trajectory.
task-0028: Harden the CrewCall snapshot connector for production
Integration Bench task-0028 against CrewCall (polling (pull)). Mean task score 86; resolved by 18 of 21 models. Problem statement, graded checks and every model's trajectory.
task-0029: Get the Riverside signup file onto the CrewCall roster
Integration Bench task-0029 against CrewCall (polling, writeback). Mean task score 95; resolved by 20 of 21 models. Problem statement, graded checks and every model's trajectory.
task-0030: Census the desk board against the Placemint placement book
Integration Bench task-0030 against Placemint (polling, writeback). Mean task score 100; resolved by 21 of 21 models. Problem statement, graded checks and every model's trajectory.
task-0031: Rebuild the morning bookable-crew board
Integration Bench task-0031 against CrewCall (polling). Mean task score 100; resolved by 21 of 21 models. Problem statement, graded checks and every model's trajectory.
task-0032: Build the weekly capability index
Integration Bench task-0032 against SourceWell (polling). Mean task score 86; resolved by 18 of 21 models. Problem statement, graded checks and every model's trajectory.
task-0033: Keep the RecruitOS pipeline and the Placemint ledger in step
Integration Bench task-0033 against RecruitOS, Placemint (polling, writeback). Mean task score 62; resolved by 13 of 21 models. Problem statement, graded checks and every model's trajectory.
task-0034: Weekly crew top-up is putting people on the gate twice
Integration Bench task-0034 against CrewCall (polling, writeback). Mean task score 86; resolved by 18 of 21 models. Problem statement, graded checks and every model's trajectory.
task-0035: Sync Vettly subjects, checks, and reports into our canonical store
Integration Bench task-0035 against Vettly (polling (pull)). Mean task score 57; resolved by 12 of 21 models. Problem statement, graded checks and every model's trajectory.
task-0036: Harden the Placemint book extract for the larger tenant
Integration Bench task-0036 against Placemint (polling). Mean task score 10; resolved by 2 of 21 models. Problem statement, graded checks and every model's trajectory.
task-0037: Fix the Placemint redeployment sync
Integration Bench task-0037 against Placemint (polling, writeback). Mean task score 90; resolved by 19 of 21 models. Problem statement, graded checks and every model's trajectory.
task-0038: TalentLoop connector: make webhook-driven freshness durable against a misbehaving receiver
Integration Bench task-0038 against TalentLoop (webhooks (push) and polling (read)). Mean task score 0; resolved by 0 of 21 models. Problem statement, graded checks and every model's trajectory.
task-0039: Build the nightly payroll bridge for the punch export
Integration Bench task-0039 against Rosterly (polling, writeback). Mean task score 100; resolved by 21 of 21 models. Problem statement, graded checks and every model's trajectory.
task-0040: Push interview reschedules upstream into Interviewly
Integration Bench task-0040 against Interviewly (writeback (reschedule) + webhooks (confirmation)). Mean task score 0; resolved by 0 of 21 models. Problem statement, graded checks and every model's trajectory.
task-0041: Paygrade connector: employee/assignment sync and writeback
Integration Bench task-0041 against Paygrade (polling (pull) and writeback (push)). Mean task score 0; resolved by 0 of 21 models. Problem statement, graded checks and every model's trajectory.
task-0042: Move the rota mirror onto instant storage
Integration Bench task-0042 against Rosterly (polling). Mean task score 81; resolved by 17 of 21 models. Problem statement, graded checks and every model's trajectory.
task-0043: The rota warehouse is counting the same night more than once
Integration Bench task-0043 against Rosterly (polling). Mean task score 29; resolved by 6 of 21 models. Problem statement, graded checks and every model's trajectory.
task-0044: Nightly roster census disagrees with CrewCall about who works here
Integration Bench task-0044 against CrewCall (polling). Mean task score 100; resolved by 21 of 21 models. Problem statement, graded checks and every model's trajectory.
task-0045: Replatform the ATS integration: StaffLine to Bullpen v2
Integration Bench task-0045 against an invented vendor (integration). Mean task score 13; resolved by 3 of 21 models. Problem statement, graded checks and every model's trajectory.
task-0046: Get the scorecard export into TalentLoop
Integration Bench task-0046 against TalentLoop (writeback). Mean task score 43; resolved by 9 of 21 models. Problem statement, graded checks and every model's trajectory.
task-0047: Reconcile and safely apply the mobility stage repair queue
Integration Bench task-0047 against GlobalHire (record reads and batch writeback). Mean task score 62; resolved by 13 of 21 models. Problem statement, graded checks and every model's trajectory.
task-0048: Migrate the Brackett closure archive onto Paygrade
Integration Bench task-0048 against Paygrade (polling · writeback). Mean task score 71; resolved by 15 of 21 models. Problem statement, graded checks and every model's trajectory.
task-0049: Placemint connector: webhooks, polling, and writeback for a high-volume tenant
Integration Bench task-0049 against Placemint (webhooks (push), polling (pull), writeback (push)). Mean task score 90; resolved by 19 of 21 models. Problem statement, graded checks and every model's trajectory.
task-0050: Fix the Placemint fee-corrections connector
Integration Bench task-0050 against Placemint (writeback). Mean task score 100; resolved by 21 of 21 models. Problem statement, graded checks and every model's trajectory.