← All benchmarks
Coding· by Tangle VerticalBench

Stripe API Integration

Twelve coding tasks built from real Stripe API changes in 2025 and 2026, each graded by running the agent code against Stripe current contract.

Benchmark on GitHub ↗
Pass rate, %
0
20
40
60
80
100
95.2
91.7
86.4
68.2
glm-5.1pi
claude-sonnetclaude-code
glm-5.2opencode
glm-5.1opencode

Per-task breakdown

12 tasks · suite kept private
Task glm-5.1piclaude-sonnetclaude-codeglm-5.2opencodeglm-5.1opencode
01 medium 100 $0.004 100 $0.14 100 $0.005 100 $0.001
02 medium 100 $0.003 100 $0.19 50 $0.006 50 $0.002
03 medium 100 $0.003 100 $0.14 100 $0.005 100 $0.001
04 medium 50 $0.003 0 $0.12 0 $0.004 0 $0.001
05 medium 100 $0.002 100 $0.06 100 $0.003 100 $0.001
06 medium 100 $0.002 100 $0.07 100 $0.002 100 $0.001
07 medium 100 $0.002 100 $0.08 100 $0.004 100 $0.001
08 medium 100 $0.003 100 $0.15 100 $0.005 50 $0.002
09 hard 100 $0.003 100 $0.22 100 $0.005 0 $0.005
10 hard 100 $0.08
11 hard 100 $0.003 100 $0.15 100 $0.005 100 $0.001
12 hard 100 $0.002 100 $0.14 100 $0.005 50 $0.002

Cell shows pass rate; hover for cost, tokens, and latency. Task contents are proprietary and not shown.