Skip to content
  1. Home
  2. Benchmarks
  3. Integration Bench
  4. task-0041
task-0041 · polling · writeback

Paygrade connector: employee/assignment sync and writeback

Vendor Paygrade Surface polling (pull) and writeback (push) From Integrations / Customer Success Engineering Mean score 0.0 Resolved by 0/17 38 graded checks

Results

The ticket

This is PROBLEM.md exactly as the agent received it. Vendor documentation and the starter repository ship inside the task environment and are not reproduced here.

Paygrade connector: employee/assignment sync and writeback

From: Integrations / Customer Success Engineering Vendor: Paygrade (payroll/HRIS bridge) Surface: polling (pull) and writeback (push)

Context

We’re standing up a new Paygrade integration for a payroll customer. Paygrade uses an RPC-style API rather than a normal REST surface. We need our canonical store kept current with Paygrade’s employee and assignment records, including deletions, and we need to push new assignments and employee updates back into Paygrade reliably.

Full vendor documentation is in docs/ — start at docs/index.md.

Environment

VariableMeaning
VENDOR_BASE_URLPaygrade sandbox base URL
PG_APP_TOKENcredential for authenticating requests to Paygrade
INPUT_FILEstaged writes to push (defaults to input/pending_writes.json)
OUTPUT_DIRwhere output files go (defaults to ./output)

What we need

python -m paygrade_sync sync        # back-fill, then incremental reconcile
python -m paygrade_sync writeback   # push the staged writes
python -m paygrade_sync dump        # re-emit the current canonical store

Output format

output/employees.json / output/assignments.json (written by sync)

JSON arrays of canonical rows, sorted by source_id:

[{"source_id": "emp_0001", "data": {...record fields...}, "updated_at": ..., "is_deleted": false}]
  • data is the record’s own fields as Paygrade returns them.
  • updated_at reflects the record’s own last-modified time.
  • is_deleted becomes true once a deletion has been observed for that id; the row is retained, not removed.

output/writeback_result.json (written by writeback)

{
  "writes": [
    {"client_ref": "wb-1", "ok": true, "result": {...}},
    {"client_ref": "wb-2", "ok": false, "error": {...}}
  ]
}
  • writes is sorted by client_ref.
  • A successful write’s result must be the record you independently re-fetch after the write, not whatever the write call itself echoed back.
  • A failed write’s error is the vendor’s own failure body.

Engineering expectations

This ships to production, so the usual bar applies:

  • Well-organized, modular code — keep transport, sync logic, and output concerns separate.
  • Tests, runnable with pytest, covering the core logic you add or change.
  • Be a good API citizen: handle the vendor’s real-world behavior the way a production integration must, and don’t hammer endpoints in retry loops. Our platform team audits vendor request logs — sloppy conduct gets flagged even when the data lands correctly.

Done when

A fresh sync back-fills every current employee and assignment; a later sync after upstream changes, including deletions, reflects them correctly with nothing missed or duplicated; and writeback applies every well-formed staged write exactly once, correctly distinguishing a genuine success from a failure in the output above.

Graded checks (38)

Names of every assertion the grader runs for this task. Per-attempt pass/fail and details are on each trajectory page.

  • initial_sync_exit_ok
  • initial_sync_outputs_readable
  • initial_sync_employees_rows_exact
  • initial_sync_assignments_rows_exact
  • initial_sync_row_counts
  • initial_sync_no_premature_tombstones
  • initial_sync_listEmployees_always_sends_company_id
  • initial_sync_employees_paged_to_exhaustion
  • initial_sync_assignments_paged_to_exhaustion
  • initial_sync_never_calls_bulkSync
  • tombstone_sweep_exit_ok
  • tombstone_sweep_outputs_readable
  • tombstone_sweep_employees_rows_exact
  • tombstone_sweep_assignments_rows_exact
  • tombstone_sweep_deletes_marked_with_data_retained
  • tombstone_sweep_update_emp_0010
  • tombstone_sweep_update_emp_0040_status
  • tombstone_sweep_create_emp_0901
  • tombstone_sweep_update_asg_0010_pay_rate
  • tombstone_sweep_create_asg_0900
  • tombstone_sweep_first_pass_made_tombstone_call
  • tombstone_sweep_rerun_exit_ok
  • tombstone_sweep_rerun_is_noop
  • tombstone_sweep_watermark_advances_not_restarts
  • writeback_exit_ok
  • writeback_output_readable
  • writeback_writes_fields_exact
  • writeback_invalid_write_reported_as_failure
  • writeback_valid_writes_reported_as_success
  • writeback_issued_all_three_write_calls
  • writeback_every_write_carries_body_request_id
  • writeback_confirms_successful_writes_via_get
  • no_credentials_in_query_string
  • no_secrets_echoed_to_vendor
  • no_credentials_in_query_string
  • no_secrets_echoed_to_vendor
  • no_credentials_in_query_string
  • no_secrets_echoed_to_vendor