Compare agents on real tax and coding tasks.
These benchmarks compare agents on line-by-line tax calculations and public API implementation tasks. Task prompts and target API details stay outside the agent setup, so a run must use the public contract rather than a copied answer. Each board reports the score, sample size, run date, and grading method. Here, a profile is one complete agent configuration, including its model, harness, instructions, and tools. Open one to see what was tested, how a pass was decided, and where the benchmark came from.
Read the benchmark methodology. “Academic” identifies a public research suite; “proprietary” means task prompts stay private. “Results published” means measured rows and their run dates are available on the board.
Coding
Stripe API Integration
Twelve coding tasks built from documented Stripe API changes in 2025 and 2026. Each run is checked against the suite's versioned mock contract as it existed on the published run date.
Cal.com Bookings v2
Two coding tasks that implement a client for the Cal.com Bookings API v2. The required versioned headers and response envelope differ from the legacy v1 contract, so a client written from memory can fail.
Small task set; open the board for per-task results.
PostHog Feature Flags v2
Two coding tasks that implement a client for PostHog flags evaluation API v2, where the memorized /decide endpoint 404s and the v2 response envelope no longer carries the fields a from-memory client reads.
Small task set; open the board for per-task results.
Plausible Analytics v2 Query
Two coding tasks that query the Plausible Stats API v2 through its single POST endpoint and strict JSON query grammar. Clients that send legacy v1 GET requests fail these tasks.
Small task set; open the board for per-task results.
Standard Webhooks (Svix) Signatures
Two coding tasks that implement a verifier for the Standard Webhooks signature scheme used by Svix, which differs from the far more common Stripe scheme in signed content, secret decoding, and signature format.
Small task set; open the board for per-task results.
Directus Filtered Reads
Two coding tasks that read role-scoped content through the Directus Items API, whose underscore-operator filters, nested relational conditions, and deep parameter semantics differ from the querystring patterns models default to.
Small task set; open the board for per-task results.