Skip to content
  1. Home
  2. Benchmarks
  3. Integration Bench
  4. task-0003
task-0003 · polling · writeback

Build the weekly Placemint contact sweep

Vendor Placemint Surface polling, writeback From Integrations / Delivery Ops Mean score 82.3 Resolved by 14/17 343 graded checks

Results

The ticket

This is PROBLEM.md exactly as the agent received it. Vendor documentation and the starter repository ship inside the task environment and are not reproduced here.

Build the weekly Placemint contact sweep

From: Integrations / Delivery Ops Vendor: Placemint Surface: polling, writeback Category: build · Track: python · Tier: 2

Context

Meridian Talent Group runs its whole delivery pipeline in Placemint. Every open placement is supposed to get a touch from an account manager at least once a fortnight, and the way anyone knows a touch happened is that somebody filed a note against the placement in Placemint.

Right now that check is a person. Every Monday Priya exports the pipeline into a spreadsheet, eyeballs it against the notes, and pings the desk about whatever has gone quiet. She is going on secondment in three weeks and nobody wants to inherit the spreadsheet, so this is the job: a sweep that reads Placemint, works out when each open placement was last touched, and files the chase-up notes itself so the account manager sees them in their Placemint inbox.

Delivery Ops keep their own runbook for the manual version at docs/meridian-contact-sweep-runbook.md. Full vendor documentation is in docs/ — start at docs/index.md.

The rules Delivery Ops settled on

These are ours, not Placemint’s.

  • Scope. The sweep covers the placements Placemint currently holds that are still open: not retired, and in one of the stages sourced, submitted, interviewing, offered. A placement outside that scope is not reported and is never written to.
  • Last contact for a placement is the created_at of the most recent note Placemint holds against it. A placement Placemint holds no note for has no last contact.
  • Stalled means the last contact is earlier than STALE_BEFORE, or there has never been any contact.
  • Every stalled placement gets exactly one note, authored sweep@meridian.test, whose body is Chase-up: no contact since <last contact>. — with <last contact> the timestamp exactly as Placemint gave it — or, where there has never been any contact, Chase-up: no contact on record.
  • Nothing is written for a placement that is not stalled.
  • Running the sweep a second time must not file a second note anywhere.

What we need

The harness runs your code exactly as follows — this command is the contract:

python -m placemint_contact_sweep sweep
# -> writes the output artifact listed below
  1. Read the pipeline and the notes out of Placemint.
  2. Decide, per placement in scope, when it was last contacted and whether that makes it stalled.
  3. File the chase-up notes and write the report.

Output artifacts

  • output/sweep_report.jsonscope_count, stalled_count, fresh_count, and placements: one entry per placement in scope, each with its placement_id, client_id, stage, the last_note_id and last_contact_at of its last contact (both null where there is none), stalled, and the note_id Placemint gave the chase-up note (null where none was filed).

Environment

VariableMeaning
VENDOR_BASE_URLVendor sandbox base URL (e.g. http://vendor:8000)
PM_CLIENT_IDVendor credential injected by the test harness
PM_CLIENT_SECRETVendor credential injected by the test harness
STALE_BEFOREThe review horizon, an ISO 8601 Z instant
OUTPUT_DIRDirectory where output artifacts land (defaults to ./output)

Engineering expectations

This ships to production, so the usual bar applies:

  • Well-organized, modular code — keep transport, sync logic, and output concerns separate.
  • Tests, runnable with pytest, covering the core logic you add or change.
  • Be a good API citizen: handle the vendor’s real-world behavior the way a production integration must, and don’t hammer endpoints in retry loops. Our platform team audits vendor request logs — sloppy conduct gets flagged even when the data lands correctly.

Done when

The entry command exits 0, every open placement is in the report with the contact Placemint actually holds against it, every stalled one carries exactly one chase-up note saying the right thing, nothing outside the scope has been written to, and a second run files nothing further.

Graded checks (343)

Names of every assertion the grader runs for this task. Per-attempt pass/fail and details are on each trajectory page.

  • contact_sweep_run_completed
  • sweep_report_covers_the_open_pipeline_exactly
  • sweep_report_omits_every_closed_and_retired_placement
  • contact_plc_00007_is_the_latest_note_on_file
  • contact_plc_00008_is_the_latest_note_on_file
  • contact_plc_00009_is_the_latest_note_on_file
  • contact_plc_00010_is_the_latest_note_on_file
  • contact_plc_00011_is_the_latest_note_on_file
  • contact_plc_00012_is_the_latest_note_on_file
  • contact_plc_00015_is_the_latest_note_on_file
  • contact_plc_00017_is_the_latest_note_on_file
  • contact_plc_00018_is_the_latest_note_on_file
  • contact_plc_00020_is_the_latest_note_on_file
  • contact_plc_00021_is_the_latest_note_on_file
  • contact_plc_00022_is_the_latest_note_on_file
  • contact_plc_00023_is_the_latest_note_on_file
  • contact_plc_00024_is_the_latest_note_on_file
  • contact_plc_00026_is_the_latest_note_on_file
  • contact_plc_00028_is_the_latest_note_on_file
  • contact_plc_00029_is_the_latest_note_on_file
  • contact_plc_00031_is_the_latest_note_on_file
  • contact_plc_00033_is_the_latest_note_on_file
  • contact_plc_00034_is_the_latest_note_on_file
  • contact_plc_00035_is_the_latest_note_on_file
  • contact_plc_00036_is_the_latest_note_on_file
  • contact_plc_00037_is_the_latest_note_on_file
  • contact_plc_00039_is_the_latest_note_on_file
  • contact_plc_00041_is_the_latest_note_on_file
  • contact_plc_00042_is_the_latest_note_on_file
  • contact_plc_00043_is_the_latest_note_on_file
  • contact_plc_00044_is_the_latest_note_on_file
  • contact_plc_00045_is_the_latest_note_on_file
  • contact_plc_00046_is_the_latest_note_on_file
  • contact_plc_00047_is_the_latest_note_on_file
  • contact_plc_00051_is_the_latest_note_on_file
  • contact_plc_00052_is_the_latest_note_on_file
  • contact_plc_00055_is_the_latest_note_on_file
  • contact_plc_00056_is_the_latest_note_on_file
  • contact_plc_00057_is_the_latest_note_on_file
  • contact_plc_00058_is_the_latest_note_on_file
  • contact_plc_00059_is_the_latest_note_on_file
  • contact_plc_00060_is_the_latest_note_on_file
  • contact_plc_00061_is_the_latest_note_on_file
  • contact_plc_00062_is_the_latest_note_on_file
  • contact_plc_00064_is_the_latest_note_on_file
  • contact_plc_00068_is_the_latest_note_on_file
  • contact_plc_00069_is_the_latest_note_on_file
  • contact_plc_00071_is_the_latest_note_on_file
  • contact_plc_00072_is_the_latest_note_on_file
  • contact_plc_00073_is_the_latest_note_on_file
  • contact_plc_00074_is_the_latest_note_on_file
  • contact_plc_00076_is_the_latest_note_on_file
  • contact_plc_00078_is_the_latest_note_on_file
  • contact_plc_00080_is_the_latest_note_on_file
  • contact_plc_00081_is_the_latest_note_on_file
  • contact_plc_00082_is_the_latest_note_on_file
  • contact_plc_00083_is_the_latest_note_on_file
  • contact_plc_00085_is_the_latest_note_on_file
  • contact_plc_00086_is_the_latest_note_on_file
  • contact_plc_00087_is_the_latest_note_on_file
  • and 283 more