Skip to content
  1. Home
  2. Benchmarks
  3. Integration Bench
  4. GPT-6 Astra

GPT-6 Astra

Integration capability of coding agents on GPT-6 Astra.

Rank #3 of 21 Pass 1 65.40 ±5.12 Resolved 66% of tasks Harness Codex CLI Effort xhigh Total cost $247.75 Mean time 16 min Mean tool calls 31 Graded 50/50
69
polling
69% of tasks touching this surface resolved
75
writeback
76% of tasks touching this surface resolved
26
webhooks
27% of tasks touching this surface resolved

Published attempts

50 of 50 scored tasks released

One official run per task. Click a row for the full trajectory: every tool call, vendor request, graded check and the diff.

TaskTicketScoreChecksTool callsTimeCostOutcome
task-0001Sync StaffLine candidates, jobs, applications, and notes into our canonical store10030/3029 (5 failed)14 min$4.53resolved
task-0002Legal-hold export: TalentForge custodians and everything on file against them10064/6436 (14 failed)13 min$4.48resolved
task-0003Build the weekly Placemint contact sweep100343/34328 (4 failed)13 min$4.14resolved
task-0004Merge StaffLine application stage with Placemint placement stage048/6434 (6 failed)21 min$7.01mandatory fail
task-0005Keep TalentForge candidates fresh: webhooks + reconciliation polling017/2638 (6 failed)24 min$6.17mandatory fail
task-0006The case-note mirror is short every morning100159/15924 (7 failed)17 min$5.36resolved
task-0007Harden the Vettly sync connector for production scheduling9638/3926 (10 failed)20 min$5.21resolved
task-0008Build a StaffLine full-sync + writeback connector10033/3336 (6 failed)16 min$5.86resolved
task-0009Migrate off the legacy Bullpen app-token integration037/4029 (4 failed)14 min$4.15mandatory fail
task-0010Build the GlobalHire audit-period extract Compliance keeps asking us for078/8128 (4 failed)14 min$4.08partial
task-0011TalentForge connector: consume webhooks while pushing corrections upstream043/5338 (7 failed)22 min$7.12mandatory fail
task-0012Build the full TalentLoop connector: polling, webhooks, and writeback94123/12624 (7 failed)20 min$4.69resolved
task-0013The warehouse and RecruitOS have been out of step for months10096/9627 (2 failed)13 min$4.06resolved
task-0014HireWire: clear the stale-screening correction backlog, exactly once03/1032 (5 failed)21 min$6.96mandatory fail
task-0015Bulk-import a migration batch into StaffLine018/1935 (6 failed)20 min$5.71partial
task-0016HireWire connector: push stage-change events + keep an incremental poll fresh9246/4934 (9 failed)15 min$4.59resolved
task-0017Move the Brightmoor candidate mirror onto TalentForge's event subscription034/16141 (8 failed)19 min$5.56partial
task-0018Rebuild the nightly requisition rollup on the RecruitOS API100142/14219 (4 failed)8 min$2.56resolved
task-0019Rebuild the Vettly change feed for the vaultd cutover100183/18428 (4 failed)14 min$4.50resolved
task-0020Sync GlobalHire candidates into our canonical store (offset polling)10034/3421 (4 failed)11 min$3.49resolved
task-0021Sync GlobalHire candidates, placements, and agencies into our canonical store10023/2338 (7 failed)11 min$3.59resolved
task-0022Harden the GlobalHire backfill for a production tenant10028/2826 (3 failed)18 min$3.43resolved
task-0023TalentLoop connector: keep a candidate/application mirror deletion-accurate018/2646 (14 failed)17 min$5.56mandatory fail
task-0024TalentLoop connector: unified freshness across webhook-covered and poll-only entities033/4333 (3 failed)26 min$7.15mandatory fail
task-0025Reset the CrewCall roster watermark after the tenant rebuild0500/50320 (3 failed)10 min$2.87partial
task-0026Push the Q1 close back into Placemint100268/26827 (4 failed)10 min$3.17resolved
task-0027Reliability: Interviewly connector loses and regresses updates016/2827 (6 failed)17 min$5.37mandatory fail
task-0028Harden the CrewCall snapshot connector for production10044/4438 (13 failed)15 min$4.65resolved
task-0029Get the Riverside signup file onto the CrewCall roster100102/10225 (3 failed)16 min$5.18resolved
task-0030Census the desk board against the Placemint placement book100324/32428 (7 failed)14 min$5.62resolved
task-0031Rebuild the morning bookable-crew board100101/10129 (4 failed)14 min$4.58resolved
task-0032Build the weekly capability index100116/11825 (4 failed)12 min$3.78resolved
task-0033Keep the RecruitOS pipeline and the Placemint ledger in step100153/15323 (4 failed)11 min$3.80resolved
task-0034Weekly crew top-up is putting people on the gate twice100165/16520 (3 failed)11 min$3.45resolved
task-0035Sync Vettly subjects, checks, and reports into our canonical store022/2634 (5 failed)16 min$4.62mandatory fail
task-0036Harden the Placemint book extract for the larger tenant0212/42033 (5 failed)14 min$4.82partial
task-0037Fix the Placemint redeployment sync100306/30633 (5 failed)18 min$6.21resolved
task-0038TalentLoop connector: make webhook-driven freshness durable against a misbehaving receiver017/3835 (9 failed)19 min$5.09mandatory fail
task-0039Build the nightly payroll bridge for the punch export10093/9328 (10 failed)14 min$4.40resolved
task-0040Push interview reschedules upstream into Interviewly013/1732 (6 failed)21 min$6.94mandatory fail
task-0041Paygrade connector: employee/assignment sync and writeback031/3851 (10 failed)20 min$6.40mandatory fail
task-0042Move the rota mirror onto instant storage100117/11728 (5 failed)19 min$5.79resolved
task-0043The rota warehouse is counting the same night more than once10089/8934 (8 failed)16 min$4.78resolved
task-0044Nightly roster census disagrees with CrewCall about who works here10067/6732 (10 failed)14 min$4.69resolved
task-0045Replatform the ATS integration: StaffLine to Bullpen v29223/2437 (10 failed)15 min$5.21resolved
task-0046Get the scorecard export into TalentLoop10089/8931 (8 failed)13 min$5.47resolved
task-0047Reconcile and safely apply the mobility stage repair queue100103/10332 (5 failed)15 min$5.37resolved
task-0048Migrate the Brackett closure archive onto Paygrade100211/21136 (8 failed)12 min$4.16resolved
task-0049Placemint connector: webhooks, polling, and writeback for a high-volume tenant9666/6738 (13 failed)25 min$5.99resolved
task-0050Fix the Placemint fee-corrections connector100253/25332 (11 failed)16 min$5.41resolved