Skip to content
  1. Home
  2. Benchmarks
  3. Integration Bench
  4. task-0021
task-0021 · polling

Sync GlobalHire candidates, placements, and agencies into our canonical store

Vendor GlobalHire Surface polling (pull) From Integrations / Customer Success Engineering Mean score 52.9 Resolved by 9/17 22 graded checks

Results

The ticket

This is PROBLEM.md exactly as the agent received it. Vendor documentation and the starter repository ship inside the task environment and are not reproduced here.

Sync GlobalHire candidates, placements, and agencies into our canonical store

From: Integrations / Customer Success Engineering Vendor: GlobalHire (multinational staffing-agency platform) Surface: polling (pull)

Context

A staffing customer runs their pipeline on GlobalHire and wants their full candidate, placement, and agency data mirrored into our canonical store so downstream reporting and routing can run off our data instead of live calls to the vendor. This is a fresh integration for this tenant — there is no store yet, so a run has to land the complete, correct picture GlobalHire actually has for all three record types. No partial coverage, no shortcuts.

The globalhire_sync package in repo/ already talks to GlobalHire and writes output in the shape we need. Treat it as a starting point: extend or adjust whatever you need to in order to meet the contract below.

Full vendor documentation is in docs/ — start at docs/index.md.

Environment

VariableMeaning
VENDOR_BASE_URLGlobalHire sandbox base URL
GH_API_KEYstatic API key
OUTPUT_DIRwhere output JSON is written (default ./output)

What we need

The grader runs your package like this:

python -m globalhire_sync

Engineering expectations

This ships to production, so the usual bar applies:

  • Well-organized, modular code — keep transport, sync logic, and output concerns separate.
  • Tests, runnable with pytest, covering the core logic you add or change.
  • Be a good API citizen: handle the vendor’s real-world behavior the way a production integration must, and don’t hammer endpoints in retry loops. Our platform team audits vendor request logs — sloppy conduct gets flagged even when the data lands correctly.

Done when

A fresh run exits 0, and all three output files match this tenant’s actual upstream state exactly: every candidate, every placement, and every agency GlobalHire has for this tenant, correctly mapped, with no missing records and no duplicates.

Graded checks (22)

Names of every assertion the grader runs for this task. Per-attempt pass/fail and details are on each trajectory page.

  • app_exit_ok
  • candidates_rows_exact
  • candidates_row_count_6000
  • candidates_ids_exactly_once
  • agencies_rows_exact
  • agency_row_exact:agy_00011
  • agency_row_exact:agy_00005
  • candidate_beyond_cap_present:cand_00101
  • candidate_beyond_cap_present:cand_03210
  • candidate_beyond_cap_present:cand_06000
  • only_v2_used_for_candidates
  • placements_rows_exact
  • placement_row_exact:plc_00003
  • placement_row_exact:plc_00013
  • placement_row_exact:plc_00005
  • no_placement_row_carries_rpc_envelope
  • only_v1_used_for_placements
  • split_brain_composition
  • no_credentials_in_query_string
  • no_secrets_echoed_to_vendor
  • no_unnecessary_full_resync:placement
  • no_unnecessary_full_resync:agency