task-0049 · polling · writeback · webhooks

Placemint connector: webhooks, polling, and writeback for a high-volume tenant

Vendor Placemint Surface webhooks (push), polling (pull), writeback (push) From Integrations / Customer Success Engineering Mean score 93.6 Resolved by 16/17 69 graded checks

Results

Claude Fable 5 100 resolved

Claude Haiku 4.5 100 resolved

Claude Opus 4.8 100 resolved

Claude Sonnet 5 100 resolved

DeepSeek V4 Flash (max) 100 resolved

DeepSeek V4 Pro (max) 100 resolved

Gemini 3.7 Flash 100 resolved

GPT-5.6 Luna 100 resolved

GPT-5.6 Sol 100 resolved

GPT-5.6 Terra 100 resolved

Grok 4.6 100 resolved

Kimi K3 100 resolved

Qwen 3.8 2.4T A95B 100 resolved

Qwen 3.8 27B (xhigh) 100 resolved

Claude Opus 5 96 resolved

Muse Spark 1.2 96 resolved

GLM 5.2 (xhigh) 0 not resolved

The ticket

This is PROBLEM.md exactly as the agent received it. Vendor documentation and the starter repository ship inside the task environment and are not reproduced here.

Placemint connector: webhooks, polling, and writeback for a high-volume tenant

From: Integrations / Customer Success Engineering Vendor: Placemint (placement marketplace) Surface: webhooks (push), polling (pull), writeback (push)

Context

A high-volume tenant runs all three Placemint surfaces at once: it streams placement, client, and note changes to us over webhooks, we also poll Placemint directly, and we periodically push writeback (notes and placement updates) into the tenant.

Full vendor documentation is in docs/ — start at docs/index.md.

Environment

Variable	Meaning
`VENDOR_BASE_URL`	Placemint sandbox base URL (e.g. `http://vendor:8000`)
`PM_CLIENT_ID` / `PM_CLIENT_SECRET`	OAuth2 client credentials
`PM_WEBHOOK_SECRET`	HMAC secret for verifying inbound webhook deliveries
`SERVE_HOST` / `SERVE_PORT`	webhook listener bind address (default `0.0.0.0:4000`)
`OUTPUT_DIR`	where output files go
`STATE_PATH`	where the durable canonical store snapshot lives
`POLL_INTERVAL_S`	cadence of the background poll-reconciliation loop

Run contract

python -m placemint_summit serve

# Writeback push
python -m placemint_summit writeback

python -m placemint_summit dump

Output format

`placements.json`

JSON array of canonical rows sorted by placement id: source_id, data (all fields minus source_id), is_deleted, updated_at. Don’t change the shape.

`writeback_result.json`

{
  "writes": [
    {"client_ref": "w-1", "ok": true, "kind": "note", "record": { ...the created note... }},
    {"client_ref": "w-2", "ok": true, "kind": "placement_update", "record": { ...the updated placement... }},
    {"client_ref": "w-3", "ok": false, "error": {"status": 422, "field_errors": {"body": ["is required"]}}}
  ]
}

writes is sorted by client_ref.
A rejected write carries the vendor’s status and field_errors under error, and has no record key.

Engineering expectations

This ships to production, so the usual bar applies:

Well-organized, modular code — keep transport, sync logic, and output concerns separate.
Tests, runnable with pytest, covering the core logic you add or change.
Be a good API citizen: handle the vendor’s real-world behavior the way a production integration must, and don’t hammer endpoints in retry loops. Our platform team audits vendor request logs — sloppy conduct gets flagged even when the data lands correctly.

Done when

All invocations exit 0 and the final canonical store matches the tenant’s actual upstream state exactly, in the canonical shapes above, with the writeback contract holding throughout: each staged write lands exactly once per client_ref, and a malformed item is reported as a failure — never a crash, never a duplicate on retry.

Graded checks (69)

Names of every assertion the grader runs for this task. Per-attempt pass/fail and details are on each trajectory page.

cp0_backfill_rows_exact
cp1_webhook_delivered
cp2_webhook_delivered
cp3_webhook_delivered
webhook_freshness_rows_exact
cp1_update_applied
cp2_delete_applied
cp3_create_applied
drop_fault_actually_fired
dropped_delete_reconciled_via_poll
prior_update_also_reconciled_via_poll
reconcile_rows_exact
writeback_exit_ok
writeback_writes_fields_exact
malformed_item_reported_as_422
reauth_transparent_under_writeback
backfill_reflects_prior_writeback
resume_not_restart_under_pressure
retry_writeback_exit_ok
retry_did_not_create_new_records
retries_carry_idempotency_key
summit_writeback_exit_ok
cp2_intermediate_rows_exact
cp3_intermediate_rows_exact
cp4_intermediate_rows_exact
cp5_intermediate_rows_exact
cp6_intermediate_rows_exact
summit_rows_exact
exactly_once_end_to_end
resume_not_restart
reauth_transparent
dropped_event_reconciled
idempotent_writeback_under_faults
no_credentials_in_query_string
no_secrets_echoed_to_vendor
no_credentials_in_query_string
no_secrets_echoed_to_vendor
no_credentials_in_query_string
no_secrets_echoed_to_vendor
no_credentials_in_query_string
no_secrets_echoed_to_vendor
reauth_per_request:/oauth/token
no_unnecessary_full_resync:placement
no_unnecessary_full_resync:client
no_unnecessary_full_resync:note
no_hot_loop_on_error
reauth_per_request:/oauth/token
no_unnecessary_full_resync:placement
no_unnecessary_full_resync:client
no_unnecessary_full_resync:note
no_hot_loop_on_error
reauth_per_request:/oauth/token
retry_after_honored
no_unnecessary_full_resync:placement
resume_not_restart_on_retry:placement
no_unnecessary_full_resync:client
resume_not_restart_on_retry:client
no_unnecessary_full_resync:note
resume_not_restart_on_retry:note
no_hot_loop_on_error
and 9 more