Skip to content
  1. Home
  2. Benchmarks
  3. Integration Bench
  4. DeepSeek V4 Pro (max)

DeepSeek V4 Pro (max)

Integration capability of coding agents on DeepSeek V4 Pro (max).

Rank #5 of 21 Pass 1 63.35 ±4.91 Resolved 64% of tasks Harness OpenCode Effort max Total cost $20.49 Mean time 18 min Mean tool calls 78 Graded 50/50
71
polling
71% of tasks touching this surface resolved
71
writeback
71% of tasks touching this surface resolved
35
webhooks
36% of tasks touching this surface resolved

Published attempts

50 of 50 scored tasks released

One official run per task. Click a row for the full trajectory: every tool call, vendor request, graded check and the diff.

TaskTicketScoreChecksTool callsTimeCostOutcome
task-0001Sync StaffLine candidates, jobs, applications, and notes into our canonical store9229/3089 (8 failed)22 min$0.44resolved
task-0002Legal-hold export: TalentForge custodians and everything on file against them10064/6461 (5 failed)11 min$0.23resolved
task-0003Build the weekly Placemint contact sweep081/34375 (7 failed)17 min$0.49partial
task-0004Merge StaffLine application stage with Placemint placement stage049/6590 (7 failed)17 min$0.46mandatory fail
task-0005Keep TalentForge candidates fresh: webhooks + reconciliation polling9428/31131 (4 failed)33 min$0.98resolved
task-0006The case-note mirror is short every morning100159/15962 (5 failed)26 min$0.34resolved
task-0007Harden the Vettly sync connector for production scheduling10039/3993 (16 failed)29 min$0.74resolved
task-0008Build a StaffLine full-sync + writeback connector10033/3386 (4 failed)19 min$0.33resolved
task-0009Migrate off the legacy Bullpen app-token integration10040/4055 (2 failed)7 min$0.24resolved
task-0010Build the GlobalHire audit-period extract Compliance keeps asking us for10081/8183 (4 failed)13 min$0.37resolved
task-0011TalentForge connector: consume webhooks while pushing corrections upstream055/59103 (8 failed)27 min$0.69mandatory fail
task-0012Build the full TalentLoop connector: polling, webhooks, and writeback89119/12375 (5 failed)18 min$0.32resolved
task-0013The warehouse and RecruitOS have been out of step for months10096/9638 (2 failed)8 min$0.16resolved
task-0014HireWire: clear the stale-screening correction backlog, exactly once10016/16119 (9 failed)22 min$0.62resolved
task-0015Bulk-import a migration batch into StaffLine019/2589 (7 failed)22 min$0.64mandatory fail
task-0016HireWire connector: push stage-change events + keep an incremental poll fresh9245/4888 (2 failed)18 min$0.44resolved
task-0017Move the Brightmoor candidate mirror onto TalentForge's event subscription0144/16088 (7 failed)20 min$0.44partial
task-0018Rebuild the nightly requisition rollup on the RecruitOS API100142/14266 (14 failed)11 min$0.28resolved
task-0019Rebuild the Vettly change feed for the vaultd cutover100183/18464 (6 failed)16 min$0.38resolved
task-0020Sync GlobalHire candidates into our canonical store (offset polling)10034/3467 (12 failed)13 min$0.35resolved
task-0021Sync GlobalHire candidates, placements, and agencies into our canonical store10021/2184 (7 failed)15 min$0.41resolved
task-0022Harden the GlobalHire backfill for a production tenant025/2789 (12 failed)69 min$0.44mandatory fail timed out
task-0023TalentLoop connector: keep a candidate/application mirror deletion-accurate018/2643 (3 failed)11 min$0.22mandatory fail
task-0024TalentLoop connector: unified freshness across webhook-covered and poll-only entities029/4089 (7 failed)19 min$0.40mandatory fail
task-0025Reset the CrewCall roster watermark after the tenant rebuild0500/50356 (4 failed)10 min$0.23partial
task-0026Push the Q1 close back into Placemint100268/26874 (4 failed)24 min$0.39resolved
task-0027Reliability: Interviewly connector loses and regresses updates017/2463 (1 failed)17 min$0.28mandatory fail
task-0028Harden the CrewCall snapshot connector for production10044/44114 (11 failed)32 min$0.44resolved
task-0029Get the Riverside signup file onto the CrewCall roster100102/10290 (4 failed)28 min$0.81resolved
task-0030Census the desk board against the Placemint placement book100325/32552 (3 failed)12 min$0.26resolved
task-0031Rebuild the morning bookable-crew board100101/10171 (8 failed)13 min$0.34resolved
task-0032Build the weekly capability index100118/11846 (6 failed)7 min$0.17resolved
task-0033Keep the RecruitOS pipeline and the Placemint ledger in step100153/153104 (3 failed)26 min$0.65resolved
task-0034Weekly crew top-up is putting people on the gate twice100165/16582 (4 failed)16 min$0.46resolved
task-0035Sync Vettly subjects, checks, and reports into our canonical store022/2692 (13 failed)19 min$0.46mandatory fail
task-0036Harden the Placemint book extract for the larger tenant0212/42047 (4 failed)7 min$0.14partial
task-0037Fix the Placemint redeployment sync100306/30681 (4 failed)15 min$0.30resolved
task-0038TalentLoop connector: make webhook-driven freshness durable against a misbehaving receiver021/37102 (3 failed)20 min$0.42mandatory fail
task-0039Build the nightly payroll bridge for the punch export10093/9355 (5 failed)10 min$0.21resolved
task-0040Push interview reschedules upstream into Interviewly014/1760 (5 failed)21 min$0.27mandatory fail
task-0041Paygrade connector: employee/assignment sync and writeback028/3891 (6 failed)15 min$0.36mandatory fail
task-0042Move the rota mirror onto instant storage100117/11766 (3 failed)12 min$0.31resolved
task-0043The rota warehouse is counting the same night more than once083/8979 (5 failed)14 min$0.37partial
task-0044Nightly roster census disagrees with CrewCall about who works here10067/6758 (7 failed)11 min$0.26resolved
task-0045Replatform the ATS integration: StaffLine to Bullpen v2021/2588 (9 failed)14 min$0.40mandatory fail
task-0046Get the scorecard export into TalentLoop080/8994 (8 failed)17 min$0.42partial
task-0047Reconcile and safely apply the mobility stage repair queue100100/10088 (7 failed)19 min$0.45resolved
task-0048Migrate the Brackett closure archive onto Paygrade100211/21153 (1 failed)12 min$0.41resolved
task-0049Placemint connector: webhooks, polling, and writeback for a high-volume tenant10064/64103 (5 failed)33 min$1.01resolved
task-0050Fix the Placemint fee-corrections connector100254/25459 (6 failed)13 min$0.26resolved