Skip to content
  1. Home
  2. Benchmarks
  3. Integration Bench
  4. task-0030
task-0030 · polling · writeback

Census the desk board against the Placemint placement book

Vendor Placemint Surface polling, writeback From Integrations / Delivery Ops Mean score 100.0 Resolved by 17/17 325 graded checks

Results

The ticket

This is PROBLEM.md exactly as the agent received it. Vendor documentation and the starter repository ship inside the task environment and are not reproduced here.

Census the desk board against the Placemint placement book

From: Integrations / Delivery Ops Vendor: Placemint Surface: polling, writeback Category: build · Tier: 3

Context

Meridian Talent Group works its placements off a desk board — one line per engagement, in input/desk_board.csv, carrying our own board_ref, the Placemint ref the desk recorded, the candidate, the client, the role, whose desk it sits on and when it opened. Placemint is where the placements really live.

The two have drifted, and nobody has ever run them against each other in either direction. Delivery Ops want a census they can run twice a month.

Delivery Ops keep their own runbook for the job at docs/meridian-desk-board-runbook.md, written when the board first moved off the shared spreadsheet. Full vendor documentation is in docs/ — start at docs/index.md.

The rules Delivery Ops settled on

These are ours, not Placemint’s.

  • A board row and a Placemint placement are the same engagement when the placement’s candidate_name and client_id both equal the row’s. That pair is what the desk means by an engagement; anything else is a different piece of work.

  • Every board row gets exactly one verdict:

    • matched — Placemint still holds that engagement on its book.
    • retired — Placemint holds that engagement but has taken it off the book.
    • unmatched — Placemint holds no such engagement.
  • A placement Placemint still holds on its book that no matched row accounts for is unclaimed: nobody on the floor is working it. Flag every unclaimed placement with exactly one note, authored boardcensus@meridian.test, reading verbatim

    Desk board census: no Meridian board row for this placement (stage: <stage>).
    

    with <stage> the stage Placemint has the placement at.

  • Nothing is filed against any other placement, and a placement is never flagged twice.

What we need

The harness runs your code exactly as follows — this command is the contract:

python -m placemint_board_census run-census
# -> writes the output artifact listed below
  1. Resolve every board row against Placemint and give it its verdict.
  2. Flag every unclaimed placement in Placemint.
  3. Report both directions, and exit 0.
  4. Running the census twice must leave Placemint exactly as the first run left it.

The repository contains the current census implementation and its surrounding plumbing. Bring it into line with the business rules above.

Output artifacts

  • output/census_report.jsonboard_row_count, matched_count, retired_count, unmatched_count, unclaimed_count, plus rows: one entry per board row in file order, each with its board_ref, verdict, the placement_id it resolved to (null when it resolved to none) and its source_line; and unclaimed: one entry per unclaimed placement, sorted by placement_id, each with its placement_id, client_id, stage and the note_id Placemint gave the note.

Environment

VariableMeaning
VENDOR_BASE_URLVendor sandbox base URL (e.g. http://vendor:8000)
PM_CLIENT_IDVendor credential injected by the test harness
PM_CLIENT_SECRETVendor credential injected by the test harness
DESK_BOARD_FILEPath to the desk board export
OUTPUT_DIRDirectory where output artifacts land (defaults to ./output)

Engineering expectations

This ships to production, so the usual bar applies:

  • Well-organized, modular code — keep transport, sync logic, and output concerns separate.
  • Tests, runnable with pytest, covering the core logic you add or change.
  • Be a good API citizen: handle the vendor’s real-world behavior the way a production integration must, and don’t hammer endpoints in retry loops. Our platform team audits vendor request logs — sloppy conduct gets flagged even when the data lands correctly.

Done when

The entry command exits 0, every board row carries the verdict the rules give it and names the placement it is about, every unclaimed placement is listed and carries its census note, no other placement carries anything this run put there, and a second run changes nothing.

Graded checks (325)

Names of every assertion the grader runs for this task. Per-attempt pass/fail and details are on each trajectory page.

  • board_census_run_completed
  • census_accounts_for_both_directions_exactly
  • first_census_artifact_matches_the_complete_board_and_book
  • board_row_MB-0001_matched
  • board_row_MB-0002_matched
  • board_row_MB-0003_retired
  • board_row_MB-0004_matched
  • board_row_MB-0005_matched
  • board_row_MB-0006_matched
  • board_row_MB-0007_matched
  • board_row_MB-0008_matched
  • board_row_MB-0009_matched
  • board_row_MB-0010_retired
  • board_row_MB-0011_unmatched
  • board_row_MB-0012_matched
  • board_row_MB-0013_matched
  • board_row_MB-0014_retired
  • board_row_MB-0015_matched
  • board_row_MB-0016_matched
  • board_row_MB-0017_retired
  • board_row_MB-0018_matched
  • board_row_MB-0019_retired
  • board_row_MB-0020_unmatched
  • board_row_MB-0021_retired
  • board_row_MB-0022_matched
  • board_row_MB-0023_unmatched
  • board_row_MB-0024_matched
  • board_row_MB-0025_unmatched
  • board_row_MB-0026_unmatched
  • board_row_MB-0027_matched
  • board_row_MB-0028_matched
  • board_row_MB-0029_matched
  • board_row_MB-0030_matched
  • board_row_MB-0031_unmatched
  • board_row_MB-0032_retired
  • board_row_MB-0033_unmatched
  • board_row_MB-0034_matched
  • board_row_MB-0035_matched
  • board_row_MB-0036_matched
  • board_row_MB-0037_matched
  • board_row_MB-0038_matched
  • board_row_MB-0039_unmatched
  • board_row_MB-0040_unmatched
  • board_row_MB-0041_matched
  • board_row_MB-0042_matched
  • board_row_MB-0043_matched
  • board_row_MB-0044_matched
  • board_row_MB-0045_matched
  • board_row_MB-0046_unmatched
  • board_row_MB-0047_matched
  • board_row_MB-0048_matched
  • board_row_MB-0049_matched
  • board_row_MB-0050_matched
  • board_row_MB-0051_unmatched
  • board_row_MB-0052_matched
  • board_row_MB-0053_retired
  • board_row_MB-0054_matched
  • board_row_MB-0055_unmatched
  • board_row_MB-0056_matched
  • board_row_MB-0057_retired
  • and 265 more