Applied AI benchmarks

Models and agents scored on 51 tax returns and 21 legal cases.

Tax returns

0%20%40%60%80%100%Line accuracy91.4%GPT-5.5CodexGPT-5.5Codex91.2%Claude OpusClaude CodeClaude OpusClaude Code90.1%Claude SonnetClaude CodeClaude SonnetClaude Code72.8%Claude HaikuClaude CodeClaude HaikuClaude Code70.7%GPT-5.1GPT-5.1Direct API call
0%20%40%60%80%100%Line accuracy91.4%GPT-5.5CodexGPT-5.5Codex91.2%Claude OpusClaude CodeClaudeOpusClaudeCode90.1%Claude SonnetClaude CodeClaudeSonnetClaudeCode72.8%Claude HaikuClaude CodeClaudeHaikuClaudeCode70.7%GPT-5.1GPT-5.1Direct APIcall
Line accuracy by setup
SetupLine accuracyObservationsRecorded interval
GPT-5.5 on Codex91.4%51 returns87.2%–95.7% (95% Student's t)
Claude Opus on Claude Code91.2%51 returns87.1%–95.3% (95% Student's t)
Claude Sonnet on Claude Code90.1%51 returns85.8%–94.4% (95% Student's t)
Claude Haiku on Claude Code72.8%51 returns68.3%–77.2% (95% Student's t)
GPT-5.1 on Direct API call70.7%51 returns65.7%–75.7% (95% Student's t)

Legal work

Cost per return

$0.00$0.05$0.10$0.15$0.20$0.25Cost per return$0.03GPT-5.1GPT-5.1Direct API call$0.06Claude HaikuClaude CodeClaude HaikuClaude Code$0.17Claude SonnetClaude CodeClaude SonnetClaude Code$0.18GPT-5.5CodexGPT-5.5Codex$0.21Claude OpusClaude CodeClaude OpusClaude Code
$0.00$0.05$0.10$0.15$0.20$0.25Cost per return$0.03GPT-5.1GPT-5.1Direct APIcall$0.06Claude HaikuClaude CodeClaudeHaikuClaudeCode$0.17Claude SonnetClaude CodeClaudeSonnetClaudeCode$0.18GPT-5.5CodexGPT-5.5Codex$0.21Claude OpusClaude CodeClaudeOpusClaudeCode
Cost per return by setup
SetupCost per returnObservationsRecorded interval
GPT-5.1 on Direct API call$0.0351 returnsnot recorded
Claude Haiku on Claude Code$0.0651 returnsnot recorded
Claude Sonnet on Claude Code$0.1751 returnsnot recorded
GPT-5.5 on Codex$0.1851 returnsnot recorded
Claude Opus on Claude Code$0.2151 returnsnot recorded