Skip to content
  1. Home
  2. Benchmarks
  3. Integration Bench
  4. Qwen 3.8 2.4T A95B

Qwen 3.8 2.4T A95B

Integration capability of coding agents on Qwen 3.8 2.4T A95B.

Rank #7 of 21 Pass 1 61.68 ±4.57 Resolved 62% of tasks Harness OpenCode Effort default Total cost $69.50 Mean time 12 min Mean tool calls 61 Graded 50/50
69
polling
69% of tasks touching this surface resolved
71
writeback
71% of tasks touching this surface resolved
26
webhooks
27% of tasks touching this surface resolved

Published attempts

50 of 50 scored tasks released

One official run per task. Click a row for the full trajectory: every tool call, vendor request, graded check and the diff.

TaskTicketScoreChecksTool callsTimeCostOutcome
task-0001Sync StaffLine candidates, jobs, applications, and notes into our canonical store10030/3049 (7 failed)7 min$0.74resolved
task-0002Legal-hold export: TalentForge custodians and everything on file against them10064/6447 (1 failed)5 min$0.53resolved
task-0003Build the weekly Placemint contact sweep100343/34345 (3 failed)10 min$1.29resolved
task-0004Merge StaffLine application stage with Placemint placement stage049/6540 (2 failed)9 min$1.12mandatory fail
task-0005Keep TalentForge candidates fresh: webhooks + reconciliation polling018/21133 (4 failed)31 min$4.98mandatory fail
task-0006The case-note mirror is short every morning100159/15953 (6 failed)15 min$0.67resolved
task-0007Harden the Vettly sync connector for production scheduling10039/3984 (6 failed)22 min$2.10resolved
task-0008Build a StaffLine full-sync + writeback connector10033/3372 (2 failed)11 min$1.24resolved
task-0009Migrate off the legacy Bullpen app-token integration10040/4030 (4 failed)8 min$0.80resolved
task-0010Build the GlobalHire audit-period extract Compliance keeps asking us for10081/8164 (5 failed)7 min$0.88resolved
task-0011TalentForge connector: consume webhooks while pushing corrections upstream056/6078 (4 failed)19 min$2.50mandatory fail
task-0012Build the full TalentLoop connector: polling, webhooks, and writeback91120/12375 (2 failed)15 min$1.84resolved
task-0013The warehouse and RecruitOS have been out of step for months10096/9647 (3 failed)9 min$1.10resolved
task-0014HireWire: clear the stale-screening correction backlog, exactly once10016/1648 (5 failed)11 min$1.34resolved
task-0015Bulk-import a migration batch into StaffLine023/2674 (4 failed)11 min$1.20mandatory fail
task-0016HireWire connector: push stage-change events + keep an incremental poll fresh9245/4860 (7 failed)19 min$1.20resolved
task-0017Move the Brightmoor candidate mirror onto TalentForge's event subscription0144/16088 (4 failed)23 min$2.95partial
task-0018Rebuild the nightly requisition rollup on the RecruitOS API100142/14260 (4 failed)6 min$0.80resolved
task-0019Rebuild the Vettly change feed for the vaultd cutover100184/18446 (5 failed)11 min$1.43resolved
task-0020Sync GlobalHire candidates into our canonical store (offset polling)10034/3449 (5 failed)9 min$1.21resolved
task-0021Sync GlobalHire candidates, placements, and agencies into our canonical store015/2358 (2 failed)4 min$0.44mandatory fail
task-0022Harden the GlobalHire backfill for a production tenant027/2841 (6 failed)23 min$0.69mandatory fail
task-0023TalentLoop connector: keep a candidate/application mirror deletion-accurate026/3047 (7 failed)8 min$0.84mandatory fail
task-0024TalentLoop connector: unified freshness across webhook-covered and poll-only entities027/4067 (3 failed)14 min$1.46mandatory fail
task-0025Reset the CrewCall roster watermark after the tenant rebuild0500/50349 (1 failed)8 min$0.67partial
task-0026Push the Q1 close back into Placemint100268/26871 (7 failed)11 min$1.53resolved
task-0027Reliability: Interviewly connector loses and regresses updates017/2452 (4 failed)13 min$1.15mandatory fail
task-0028Harden the CrewCall snapshot connector for production10044/44109 (8 failed)25 min$3.00resolved
task-0029Get the Riverside signup file onto the CrewCall roster100102/10264 (5 failed)14 min$1.48resolved
task-0030Census the desk board against the Placemint placement book100325/32563 (5 failed)9 min$1.31resolved
task-0031Rebuild the morning bookable-crew board100101/10172 (5 failed)10 min$1.38resolved
task-0032Build the weekly capability index100118/11833 (4 failed)6 min$0.71resolved
task-0033Keep the RecruitOS pipeline and the Placemint ledger in step100153/15362 (7 failed)17 min$2.02resolved
task-0034Weekly crew top-up is putting people on the gate twice100165/16557 (4 failed)10 min$1.09resolved
task-0035Sync Vettly subjects, checks, and reports into our canonical store10028/2871 (3 failed)12 min$1.34resolved
task-0036Harden the Placemint book extract for the larger tenant0212/42043 (7 failed)7 min$0.94partial
task-0037Fix the Placemint redeployment sync100306/30692 (5 failed)20 min$3.48resolved
task-0038TalentLoop connector: make webhook-driven freshness durable against a misbehaving receiver017/3767 (3 failed)16 min$1.75mandatory fail
task-0039Build the nightly payroll bridge for the punch export10093/9354 (2 failed)10 min$1.33resolved
task-0040Push interview reschedules upstream into Interviewly014/1754 (4 failed)15 min$0.98mandatory fail
task-0041Paygrade connector: employee/assignment sync and writeback030/386218 min$1.03mandatory fail
task-0042Move the rota mirror onto instant storage100117/11751 (4 failed)6 min$0.91resolved
task-0043The rota warehouse is counting the same night more than once083/8953 (3 failed)7 min$1.00partial
task-0044Nightly roster census disagrees with CrewCall about who works here10067/6748 (2 failed)6 min$0.57resolved
task-0045Replatform the ATS integration: StaffLine to Bullpen v2020/2469 (3 failed)12 min$1.65mandatory fail
task-0046Get the scorecard export into TalentLoop084/8953 (3 failed)11 min$1.40partial
task-0047Reconcile and safely apply the mobility stage repair queue096/10075 (2 failed)7 min$1.23partial
task-0048Migrate the Brackett closure archive onto Paygrade100211/21164 (3 failed)17 min$1.81resolved
task-0049Placemint connector: webhooks, polling, and writeback for a high-volume tenant10061/6157 (3 failed)15 min$0.97resolved
task-0050Fix the Placemint fee-corrections connector100254/25452 (5 failed)18 min$1.39resolved