Skip to content
  1. Home
  2. Benchmarks
  3. Integration Bench
  4. Claude Sonnet 5

Claude Sonnet 5

Integration capability of coding agents on Claude Sonnet 5.

Rank #17 of 21 Pass 1 53.85 ±4.88 Resolved 54% of tasks Harness Claude Code Effort xhigh Total cost $182.18 Mean time 14 min Mean tool calls 71 Graded 50/50
57
polling
57% of tasks touching this surface resolved
57
writeback
57% of tasks touching this surface resolved
9
webhooks
9% of tasks touching this surface resolved

Published attempts

50 of 50 scored tasks released

One official run per task. Click a row for the full trajectory: every tool call, vendor request, graded check and the diff.

TaskTicketScoreChecksTool callsTimeCostOutcome
task-0001Sync StaffLine candidates, jobs, applications, and notes into our canonical store027/3083 (3 failed)13 min$3.79mandatory fail
task-0002Legal-hold export: TalentForge custodians and everything on file against them10064/6459 (3 failed)11 min$2.33resolved
task-0003Build the weekly Placemint contact sweep100343/34364 (4 failed)13 min$3.42resolved
task-0004Merge StaffLine application stage with Placemint placement stage032/65101 (3 failed)21 min$5.78mandatory fail
task-0005Keep TalentForge candidates fresh: webhooks + reconciliation polling016/2192 (9 failed)23 min$5.52mandatory fail
task-0006The case-note mirror is short every morning100159/15962 (4 failed)19 min$2.89resolved
task-0007Harden the Vettly sync connector for production scheduling10039/39114 (3 failed)25 min$6.56resolved
task-0008Build a StaffLine full-sync + writeback connector032/3386 (1 failed)17 min$4.40mandatory fail
task-0009Migrate off the legacy Bullpen app-token integration10040/4060 (3 failed)7 min$2.09resolved
task-0010Build the GlobalHire audit-period extract Compliance keeps asking us for10081/8169 (3 failed)9 min$2.84resolved
task-0011TalentForge connector: consume webhooks while pushing corrections upstream033/5189 (7 failed)22 min$6.13mandatory fail
task-0012Build the full TalentLoop connector: polling, webhooks, and writeback096/11993 (6 failed)24 min$5.71mandatory fail
task-0013The warehouse and RecruitOS have been out of step for months093/9692 (2 failed)16 min$4.35partial
task-0014HireWire: clear the stale-screening correction backlog, exactly once10016/1663 (4 failed)17 min$4.20resolved
task-0015Bulk-import a migration batch into StaffLine020/2546 (3 failed)11 min$2.27mandatory fail
task-0016HireWire connector: push stage-change events + keep an incremental poll fresh9245/4852 (8 failed)14 min$2.93resolved
task-0017Move the Brightmoor candidate mirror onto TalentForge's event subscription0143/160102 (6 failed)20 min$6.11partial
task-0018Rebuild the nightly requisition rollup on the RecruitOS API100142/14252 (4 failed)7 min$1.97resolved
task-0019Rebuild the Vettly change feed for the vaultd cutover100184/18464 (2 failed)11 min$2.70resolved
task-0020Sync GlobalHire candidates into our canonical store (offset polling)10034/3460 (6 failed)9 min$2.30resolved
task-0021Sync GlobalHire candidates, placements, and agencies into our canonical store10023/2371 (4 failed)11 min$2.87resolved
task-0022Harden the GlobalHire backfill for a production tenant10028/2881 (5 failed)26 min$3.38resolved
task-0023TalentLoop connector: keep a candidate/application mirror deletion-accurate018/2685 (8 failed)24 min$6.40mandatory fail
task-0024TalentLoop connector: unified freshness across webhook-covered and poll-only entities028/4065 (1 failed)17 min$3.88mandatory fail
task-0025Reset the CrewCall roster watermark after the tenant rebuild0500/50349 (2 failed)6 min$1.57partial
task-0026Push the Q1 close back into Placemint0269/26958 (3 failed)12 min$2.99partial
task-0027Reliability: Interviewly connector loses and regresses updates012/2469 (4 failed)15 min$4.32mandatory fail
task-0028Harden the CrewCall snapshot connector for production10044/4464 (7 failed)12 min$3.22resolved
task-0029Get the Riverside signup file onto the CrewCall roster100102/10255 (3 failed)10 min$2.95resolved
task-0030Census the desk board against the Placemint placement book100325/32558 (1 failed)11 min$2.61resolved
task-0031Rebuild the morning bookable-crew board100101/10156 (2 failed)10 min$2.88resolved
task-0032Build the weekly capability index100118/11853 (4 failed)6 min$1.79resolved
task-0033Keep the RecruitOS pipeline and the Placemint ledger in step082/15594 (3 failed)19 min$5.84partial
task-0034Weekly crew top-up is putting people on the gate twice100165/16567 (5 failed)15 min$4.00resolved
task-0035Sync Vettly subjects, checks, and reports into our canonical store025/2878 (8 failed)13 min$3.75mandatory fail
task-0036Harden the Placemint book extract for the larger tenant0212/42074 (5 failed)10 min$2.92partial
task-0037Fix the Placemint redeployment sync100307/30752 (2 failed)12 min$3.44resolved
task-0038TalentLoop connector: make webhook-driven freshness durable against a misbehaving receiver021/3787 (3 failed)17 min$4.52mandatory fail
task-0039Build the nightly payroll bridge for the punch export10093/9364 (2 failed)9 min$2.76resolved
task-0040Push interview reschedules upstream into Interviewly015/1769 (4 failed)13 min$3.45mandatory fail
task-0041Paygrade connector: employee/assignment sync and writeback032/3870 (6 failed)12 min$3.82mandatory fail
task-0042Move the rota mirror onto instant storage100117/11753 (1 failed)9 min$2.26resolved
task-0043The rota warehouse is counting the same night more than once083/8940 (3 failed)6 min$1.66partial
task-0044Nightly roster census disagrees with CrewCall about who works here10067/6760 (3 failed)9 min$2.33resolved
task-0045Replatform the ATS integration: StaffLine to Bullpen v2020/2480 (2 failed)12 min$3.94mandatory fail
task-0046Get the scorecard export into TalentLoop10088/8876 (5 failed)18 min$4.86resolved
task-0047Reconcile and safely apply the mobility stage repair queue098/10095 (1 failed)15 min$4.98partial
task-0048Migrate the Brackett closure archive onto Paygrade100211/21181 (3 failed)14 min$4.30resolved
task-0049Placemint connector: webhooks, polling, and writeback for a high-volume tenant10068/6868 (3 failed)18 min$3.94resolved
task-0050Fix the Placemint fee-corrections connector100254/25453 (2 failed)9 min$2.25resolved