TokensBenchReport your cost

Methodology

How TokensBench computes its numbers, what counts as evidence, and where the data comes from — including the parts that are still thin.

How we compute the numbers

Robust stats or nothing — thin or noisy cells do not get published.

  • n ≥ 5 to publish. A cell only appears once at least five reports back it, and every cell always displays its n and verified share alongside the numbers.
  • Median, IQR, and p90 — never a mean. All user-facing stats are robust to outliers: median plus the interquartile range (p25–p75) and p90. Means never appear on the site.
  • Outliers are flagged, not dropped. A modified z-score (MAD) check flags unusual reports for the moderation queue; nothing is silently excluded from the data.
  • Verified reports get their own median. Once a cell has three or more verified reports, we show a verified-only median alongside the all-data median.

Role budgets follow the same rule: 15 of the 15 roles in the taxonomy are currently seeded from published references, pending community reports that cross n ≥ 5.

Freshness

Old-model data does not quietly leak into current-gen answers.

The default view windows to reports from the last 180 days on current-generation models only. Older or superseded-model data is still there — behind a toggle, and grayed so it reads as historical, not current.

Every reference row (benchmark or pricing) carries a staleness label; several respected sources run months behind, and we say so on the row rather than pass off a stale number as live. The reference snapshot behind this build was retrieved 2026-09-15.

The fallback ladder

Answer pages disclose exactly which rung of the ladder produced the numbers shown.

Exact: family + size + model + harness→Drop model→Drop harness→Family + size→Family→No data yet — benchmark reference points + be the first to report

Moderation & verification

Nothing about trust here is automatic.

Submissions are anonymous-first, but every one passes a honeypot field and an IP-hash rate limit before it can even reach the queue. Nothing publishes without moderation approval — the write path fails closed, not open.

Reports carry a verification tier: none by default, or receipt once a submitter pastes a machine-readable usage export (Phase 2). Receipts are parsed in memory for aggregate numbers only — raw payloads are never persisted. Verification is an opt-in badge and a weighting signal, never a submission gate: requiring receipts on day one would kill cold-start volume.

A separate, later org-verified tier (Phase 3) will use admin-gated APIs from providers and tools — authoritative, but never a requirement for individual submitters.

Data & privacy

What TokensBench collects, what it never does, and what happens after you hit send.

  • Anonymous by default. Submitting a report needs no account, no email, and no name.
  • Your network address is hashed, not stored. We keep a salted, one-way hash of it solely to enforce the rate limit — never the address itself.
  • Free text renders as plain text. Nothing written in a report is ever interpreted as HTML or code.
  • Approved reports are published. They join the open aggregates TokensBench publishes — medians, percentiles, and technique notes. Include nothing in a report you do not want public.
  • Nothing publishes without a human. Every submission sits in the moderation queue until a moderator approves it.
  • Receipts are never stored raw. Once receipt parsing ships (Phase 2), a pasted receipt is parsed in memory for numeric totals only — the raw payload is never persisted.

Sources

Actively-updated sources first; stale or dormant ones grouped below, collapsed until you want them.

SourceWhat we takeStatus
LiteLLM model_prices*.json$/tok pricing including cache rates, 300+ modelsFresh
Epoch AI Benchmarking HubTokens-per-eval and cost-per-evalFresh
Google Gemini pricing pageOfficial list pricesStable
OpenRouter /api/v1/modelsLive $/tok pricingFresh
Artificial AnalysisIntelligence Index scores + blended list prices, via their Data APIFresh
Stale or dormant sources (5) — kept for the record, labeled on every row we display
SourceWhat we takeStatus
Aider polyglot leaderboardTotal $ per 225-exercise suite runStale
SWE-bench leaderboards.jsoninstance_cost, sparse (Verified split, 45 of 180 entries)Stale
SWE-rebenchCost/Problem and Tokens/Problem, livePending
HAL (Princeton)Cost-controlled agent eval resultsDormant
vals.ai / METRCost-per-test charts, narrow cost ratiosPending

Anthropic and OpenAI official pricing pages are hand-maintained fixtures, spot-checked against LiteLLM rather than scraped. We also evaluated LMArena, LiveBench, and llm-stats and chose not to ingest them: no real per-task cost data, hostile scraping terms, or a direct competitive conflict.

Currently live

The rows below are the actual reference snapshot this build shipped with, retrieved 2026-09-15. Current rows first; rows from stale sources are grouped and collapsed.

HAL — Holistic Agent Leaderboard (Princeton)
Current
cross-model
SWE-bench Verified full-run cost (median across agents): $163 median per run; $0.08–$32.00 per task

Harness archived 2026-07-01; leaderboard winding down — snapshot reference As of 2026-07-01. Source · none stated — displayed as cited fact with attribution; not redistributed in bulk.

Epoch AI Benchmarking Hub
Current
Gemini 3.1 Pro (Preview)
Cost per task (Epoch) — ARC-AGI: $0.52

Epoch AI Benchmarking Hub is actively maintained (weekly-ish updates); not a stale snapshot As of 2026-02-19. Source · CC-BY 4.0 (Epoch-run benchmarks; externally-sourced benchmarks retain original licenses).

Epoch AI Benchmarking Hub
Current
GPT-5.6 Sol (xHigh)
Cost per task (Epoch) — ARC-AGI: $0.40

Epoch AI Benchmarking Hub is actively maintained (weekly-ish updates); not a stale snapshot As of 2026-07-09. Source · CC-BY 4.0 (Epoch-run benchmarks; externally-sourced benchmarks retain original licenses).

Epoch AI Benchmarking Hub
Current
GPT-5.6 Sol (High)
Cost per task (Epoch) — ARC-AGI: $0.30

Epoch AI Benchmarking Hub is actively maintained (weekly-ish updates); not a stale snapshot As of 2026-07-09. Source · CC-BY 4.0 (Epoch-run benchmarks; externally-sourced benchmarks retain original licenses).

Epoch AI Benchmarking Hub
Current
Fable 5.1 Max
Cost per task (Epoch) — CursorBench: $9.64, 72.1k tokens/task

Epoch AI Benchmarking Hub is actively maintained (weekly-ish updates); not a stale snapshot As of 2026-09-01. Source · CC-BY 4.0 (Epoch-run benchmarks; externally-sourced benchmarks retain original licenses).

Epoch AI Benchmarking Hub
Current
Fable 5.1 Extra High
Cost per task (Epoch) — CursorBench: $6.96, 51.3k tokens/task

Epoch AI Benchmarking Hub is actively maintained (weekly-ish updates); not a stale snapshot As of 2026-09-01. Source · CC-BY 4.0 (Epoch-run benchmarks; externally-sourced benchmarks retain original licenses).

Epoch AI Benchmarking Hub
Current
Grok 4.6 Extra High
Cost per task (Epoch) — CursorBench: $2.81, 41.1k tokens/task

Epoch AI Benchmarking Hub is actively maintained (weekly-ish updates); not a stale snapshot As of 2026-08-12. Source · CC-BY 4.0 (Epoch-run benchmarks; externally-sourced benchmarks retain original licenses).

Epoch AI Benchmarking Hub
Current
Claude Opus 5
Total tokens, full eval run (Epoch) — GBAEval: 8.71M tokens

Epoch AI Benchmarking Hub is actively maintained (weekly-ish updates); not a stale snapshot As of 2026-07-26. Source · CC-BY 4.0 (Epoch-run benchmarks; externally-sourced benchmarks retain original licenses).

Epoch AI Benchmarking Hub
Current
Claude Fable 5
Total tokens, full eval run (Epoch) — GBAEval: 41.71M tokens

Epoch AI Benchmarking Hub is actively maintained (weekly-ish updates); not a stale snapshot As of 2026-06-10. Source · CC-BY 4.0 (Epoch-run benchmarks; externally-sourced benchmarks retain original licenses).

Epoch AI Benchmarking Hub
Current
Claude Opus 4.8
Total tokens, full eval run (Epoch) — GBAEval: 57.08M tokens

Epoch AI Benchmarking Hub is actively maintained (weekly-ish updates); not a stale snapshot As of 2026-05-30. Source · CC-BY 4.0 (Epoch-run benchmarks; externally-sourced benchmarks retain original licenses).

Epoch AI Benchmarking Hub
Current
Claude Opus 4.7 (max)
Total cost, full eval run (Epoch) — OSWorld 2: $3870.00

Epoch AI Benchmarking Hub is actively maintained (weekly-ish updates); not a stale snapshot As of 2026-04-16. Source · CC-BY 4.0 (Epoch-run benchmarks; externally-sourced benchmarks retain original licenses).

Epoch AI Benchmarking Hub
Current
GPT-5.5 (xhigh)
Total cost, full eval run (Epoch) — OSWorld 2: $2750.00

Epoch AI Benchmarking Hub is actively maintained (weekly-ish updates); not a stale snapshot As of 2026-04-23. Source · CC-BY 4.0 (Epoch-run benchmarks; externally-sourced benchmarks retain original licenses).

Epoch AI Benchmarking Hub
Current
Claude Sonnet 4.6 (medium)
Total cost, full eval run (Epoch) — OSWorld 2: $1550.00

Epoch AI Benchmarking Hub is actively maintained (weekly-ish updates); not a stale snapshot As of 2026-02-17. Source · CC-BY 4.0 (Epoch-run benchmarks; externally-sourced benchmarks retain original licenses).

Epoch AI Benchmarking Hub
Current
claude-fable-5-1_max
Cost per task (Epoch) — WeirdML: $11.69

Epoch AI Benchmarking Hub is actively maintained (weekly-ish updates); not a stale snapshot As of 2026-09-01. Source · CC-BY 4.0 (Epoch-run benchmarks; externally-sourced benchmarks retain original licenses).

Epoch AI Benchmarking Hub
Current
gpt-6-astra_high
Cost per task (Epoch) — WeirdML: $2.98

Epoch AI Benchmarking Hub is actively maintained (weekly-ish updates); not a stale snapshot As of 2026-09-03. Source · CC-BY 4.0 (Epoch-run benchmarks; externally-sourced benchmarks retain original licenses).

Epoch AI Benchmarking Hub
Current
claude-fable-5-1_high
Cost per task (Epoch) — WeirdML: $3.20

Epoch AI Benchmarking Hub is actively maintained (weekly-ish updates); not a stale snapshot As of 2026-09-01. Source · CC-BY 4.0 (Epoch-run benchmarks; externally-sourced benchmarks retain original licenses).

Artificial Analysis
Current
Claude Opus 5 (Adaptive Reasoning, Max Effort)
AA Intelligence Index vs. blended $/MTok cost: Index 50.7 · $10.00/MTok blended (3:1 in:out)

Live via the Artificial Analysis Data API As of 2026-09-14. Source · Displayed with attribution per AA Data API terms (free tier) — byline required; redistribution requires agreement.

Artificial Analysis
Current
Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback)
AA Intelligence Index vs. blended $/MTok cost: Index 49.7 · $20.00/MTok blended (3:1 in:out)

Live via the Artificial Analysis Data API As of 2026-09-14. Source · Displayed with attribution per AA Data API terms (free tier) — byline required; redistribution requires agreement.

Artificial Analysis
Current
GPT-5.6 Sol (max)
AA Intelligence Index vs. blended $/MTok cost: Index 47.1 · $8.00/MTok blended (3:1 in:out)

Live via the Artificial Analysis Data API As of 2026-09-14. Source · Displayed with attribution per AA Data API terms (free tier) — byline required; redistribution requires agreement.

Artificial Analysis
Current
GPT-5.6 Terra (max)
AA Intelligence Index vs. blended $/MTok cost: Index 42.3 · $4.50/MTok blended (3:1 in:out)

Live via the Artificial Analysis Data API As of 2026-09-14. Source · Displayed with attribution per AA Data API terms (free tier) — byline required; redistribution requires agreement.

Artificial Analysis
Current
Claude Sonnet 5 (Adaptive Reasoning, Max Effort)
AA Intelligence Index vs. blended $/MTok cost: Index 38.4 · $4.00/MTok blended (3:1 in:out)

Live via the Artificial Analysis Data API As of 2026-09-14. Source · Displayed with attribution per AA Data API terms (free tier) — byline required; redistribution requires agreement.

Artificial Analysis
Current
GPT-5.6 Luna (max)
AA Intelligence Index vs. blended $/MTok cost: Index 37.5 · $0.45/MTok blended (3:1 in:out)

Live via the Artificial Analysis Data API As of 2026-09-14. Source · Displayed with attribution per AA Data API terms (free tier) — byline required; redistribution requires agreement.

Artificial Analysis
Current
Gemini 3.6 Flash (high)
AA Intelligence Index vs. blended $/MTok cost: Index 34.3 · $1.50/MTok blended (3:1 in:out)

Live via the Artificial Analysis Data API As of 2026-09-14. Source · Displayed with attribution per AA Data API terms (free tier) — byline required; redistribution requires agreement.

Artificial Analysis
Current
GPT-5 (high)
AA Intelligence Index vs. blended $/MTok cost: Index 23.0 · $3.44/MTok blended (3:1 in:out)

Live via the Artificial Analysis Data API As of 2026-09-14. Source · Displayed with attribution per AA Data API terms (free tier) — byline required; redistribution requires agreement.

Artificial Analysis
Current
Gemini 2.5 Pro
AA Intelligence Index vs. blended $/MTok cost: Index 16.7 · $3.44/MTok blended (3:1 in:out)

Live via the Artificial Analysis Data API As of 2026-09-14. Source · Displayed with attribution per AA Data API terms (free tier) — byline required; redistribution requires agreement.

Artificial Analysis
Current
Claude 4.5 Haiku (Non-reasoning)
AA Intelligence Index vs. blended $/MTok cost: Index 15.4 · $2.00/MTok blended (3:1 in:out)

Live via the Artificial Analysis Data API As of 2026-09-14. Source · Displayed with attribution per AA Data API terms (free tier) — byline required; redistribution requires agreement.

Artificial Analysis
Current
Claude Fable 5.1 (Adaptive Reasoning, Max Effort, Default Fallback)
AA Intelligence Index vs. blended $/MTok cost: Index 53.4 · $20.00/MTok blended (3:1 in:out)

Live via the Artificial Analysis Data API As of 2026-09-14. Source · Displayed with attribution per AA Data API terms (free tier) — byline required; redistribution requires agreement.

Artificial Analysis
Current
Claude Opus 4.8 (Adaptive Reasoning, Max Effort)
AA Intelligence Index vs. blended $/MTok cost: Index 42.0 · $10.00/MTok blended (3:1 in:out)

Live via the Artificial Analysis Data API As of 2026-09-14. Source · Displayed with attribution per AA Data API terms (free tier) — byline required; redistribution requires agreement.

Artificial Analysis
Current
Gemini 3.8 Flash (high)
AA Intelligence Index vs. blended $/MTok cost: Index 41.2 · $1.50/MTok blended (3:1 in:out)

Live via the Artificial Analysis Data API As of 2026-09-14. Source · Displayed with attribution per AA Data API terms (free tier) — byline required; redistribution requires agreement.

Artificial Analysis
Current
Claude Opus 4.7 (Adaptive Reasoning, Max Effort)
AA Intelligence Index vs. blended $/MTok cost: Index 40.7 · $10.00/MTok blended (3:1 in:out)

Live via the Artificial Analysis Data API As of 2026-09-14. Source · Displayed with attribution per AA Data API terms (free tier) — byline required; redistribution requires agreement.

Artificial Analysis
Current
Gemini 3.7 Flash (high)
AA Intelligence Index vs. blended $/MTok cost: Index 39.4 · $1.50/MTok blended (3:1 in:out)

Live via the Artificial Analysis Data API As of 2026-09-14. Source · Displayed with attribution per AA Data API terms (free tier) — byline required; redistribution requires agreement.

Artificial Analysis
Current
Muse Spark 1.3 (max)
AA Intelligence Index vs. blended $/MTok cost: Index 48.2 · $2.00/MTok blended (3:1 in:out)

Live via the Artificial Analysis Data API As of 2026-09-14. Source · Displayed with attribution per AA Data API terms (free tier) — byline required; redistribution requires agreement.

Artificial Analysis
Current
GLM-5.3 (max)
AA Intelligence Index vs. blended $/MTok cost: Index 44.9 · $2.15/MTok blended (3:1 in:out)

Live via the Artificial Analysis Data API As of 2026-09-14. Source · Displayed with attribution per AA Data API terms (free tier) — byline required; redistribution requires agreement.

Artificial Analysis
Current
Grok 4.6 (high)
AA Intelligence Index vs. blended $/MTok cost: Index 44.4 · $3.00/MTok blended (3:1 in:out)

Live via the Artificial Analysis Data API As of 2026-09-14. Source · Displayed with attribution per AA Data API terms (free tier) — byline required; redistribution requires agreement.

Artificial Analysis
Current
Kimi K3 (max)
AA Intelligence Index vs. blended $/MTok cost: Index 43.8 · $6.00/MTok blended (3:1 in:out)

Live via the Artificial Analysis Data API As of 2026-09-14. Source · Displayed with attribution per AA Data API terms (free tier) — byline required; redistribution requires agreement.

Artificial Analysis
Current
Qwen3.8 Max
AA Intelligence Index vs. blended $/MTok cost: Index 40.3 · $3.00/MTok blended (3:1 in:out)

Live via the Artificial Analysis Data API As of 2026-09-14. Source · Displayed with attribution per AA Data API terms (free tier) — byline required; redistribution requires agreement.

43 rows from stale sources
Aider polyglot leaderboard
Stale
gpt-5 (high)
Total cost, 225-exercise polyglot suite: $29.08 (pass rate 88.0%)

Upstream data last updated 2025-10; historical reference As of 2025-08-23. Source · Apache-2.0.

Aider polyglot leaderboard
Stale
gpt-5 (medium)
Total cost, 225-exercise polyglot suite: $17.69 (pass rate 86.7%)

Upstream data last updated 2025-10; historical reference As of 2025-08-25. Source · Apache-2.0.

Aider polyglot leaderboard
Stale
o3-pro (high)
Total cost, 225-exercise polyglot suite: $146.32 (pass rate 84.9%)

Upstream data last updated 2025-10; historical reference As of 2025-06-28. Source · Apache-2.0.

Aider polyglot leaderboard
Stale
gemini-2.5-pro-preview-06-05 (32k think)
Total cost, 225-exercise polyglot suite: $49.88 (pass rate 83.1%)

Upstream data last updated 2025-10; historical reference As of 2025-06-06. Source · Apache-2.0.

Aider polyglot leaderboard
Stale
o3 (high)
Total cost, 225-exercise polyglot suite: $21.23 (pass rate 81.3%)

Upstream data last updated 2025-10; historical reference As of 2025-06-25. Source · Apache-2.0.

Aider polyglot leaderboard
Stale
gpt-5 (low)
Total cost, 225-exercise polyglot suite: $10.37 (pass rate 81.3%)

Upstream data last updated 2025-10; historical reference As of 2025-08-25. Source · Apache-2.0.

Aider polyglot leaderboard
Stale
grok-4 (high)
Total cost, 225-exercise polyglot suite: $59.62 (pass rate 79.6%)

Upstream data last updated 2025-10; historical reference As of 2025-07-11. Source · Apache-2.0.

Aider polyglot leaderboard
Stale
gemini-2.5-pro-preview-06-05 (default think)
Total cost, 225-exercise polyglot suite: $45.60 (pass rate 79.1%)

Upstream data last updated 2025-10; historical reference As of 2025-06-06. Source · Apache-2.0.

Aider polyglot leaderboard
Stale
o3 (high) + gpt-4.1
Total cost, 225-exercise polyglot suite: $17.55 (pass rate 78.2%)

Upstream data last updated 2025-10; historical reference As of 2025-06-27. Source · Apache-2.0.

Aider polyglot leaderboard
Stale
Gemini 2.5 Pro Preview 05-06
Total cost, 225-exercise polyglot suite: $37.41 (pass rate 76.9%)

Upstream data last updated 2025-10; historical reference As of 2025-05-07. Source · Apache-2.0.

Aider polyglot leaderboard
Stale
o3
Total cost, 225-exercise polyglot suite: $13.75 (pass rate 76.9%)

Upstream data last updated 2025-10; historical reference As of 2025-06-25. Source · Apache-2.0.

Aider polyglot leaderboard
Stale
DeepSeek-V3.2-Exp (Reasoner)
Total cost, 225-exercise polyglot suite: $1.30 (pass rate 74.2%)

Upstream data last updated 2025-10; historical reference As of 2025-10-03. Source · Apache-2.0.

Aider polyglot leaderboard
Stale
o4-mini (high)
Total cost, 225-exercise polyglot suite: $19.64 (pass rate 72.0%)

Upstream data last updated 2025-10; historical reference As of 2025-04-16. Source · Apache-2.0.

Aider polyglot leaderboard
Stale
claude-opus-4-20250514 (32k thinking)
Total cost, 225-exercise polyglot suite: $65.75 (pass rate 72.0%)

Upstream data last updated 2025-10; historical reference As of 2025-05-25. Source · Apache-2.0.

Aider polyglot leaderboard
Stale
DeepSeek R1 (0528)
Total cost, 225-exercise polyglot suite: $4.80 (pass rate 71.4%)

Upstream data last updated 2025-10; historical reference As of 2025-06-06. Source · Apache-2.0.

SWE-bench (Verified split)
Stale
Claude 4.5 Opus (high)
Avg cost per task (Verified): $0.75 at 76.8% resolved

Leaderboard updates limited since 2025-11 policy change; CC BY-NC — display with attribution, no commercial use As of 2026-02-17. Source · CC BY-NC 4.0.

SWE-bench (Verified split)
Stale
Gemini 3 Flash (high)
Avg cost per task (Verified): $0.36 at 75.8% resolved

Leaderboard updates limited since 2025-11 policy change; CC BY-NC — display with attribution, no commercial use As of 2026-02-17. Source · CC BY-NC 4.0.

SWE-bench (Verified split)
Stale
MiniMax M2.5 (high)
Avg cost per task (Verified): $0.07 at 75.8% resolved

Leaderboard updates limited since 2025-11 policy change; CC BY-NC — display with attribution, no commercial use As of 2026-02-17. Source · CC BY-NC 4.0.

SWE-bench (Verified split)
Stale
Claude 4.6 Opus
Avg cost per task (Verified): $0.55 at 75.6% resolved

Leaderboard updates limited since 2025-11 policy change; CC BY-NC — display with attribution, no commercial use As of 2026-02-17. Source · CC BY-NC 4.0.

SWE-bench (Verified split)
Stale
Claude 4.5 Opus (20251101) (medium)
Avg cost per task (Verified): $0.72 at 74.4% resolved

Leaderboard updates limited since 2025-11 policy change; CC BY-NC — display with attribution, no commercial use As of 2025-11-24. Source · CC BY-NC 4.0.

SWE-bench (Verified split)
Stale
Gemini 3 Pro Preview (2025-11-18)
Avg cost per task (Verified): $0.46 at 74.2% resolved

Leaderboard updates limited since 2025-11 policy change; CC BY-NC — display with attribution, no commercial use As of 2025-11-18. Source · CC BY-NC 4.0.

SWE-bench (Verified split)
Stale
GLM 5 (high)
Avg cost per task (Verified): $0.53 at 72.8% resolved

Leaderboard updates limited since 2025-11 policy change; CC BY-NC — display with attribution, no commercial use As of 2026-02-17. Source · CC BY-NC 4.0.

SWE-bench (Verified split)
Stale
GPT 5.2 (high)
Avg cost per task (Verified): $0.47 at 72.8% resolved

Leaderboard updates limited since 2025-11 policy change; CC BY-NC — display with attribution, no commercial use As of 2026-02-17. Source · CC BY-NC 4.0.

SWE-bench (Verified split)
Stale
GPT 5.2 Codex
Avg cost per task (Verified): $0.45 at 72.8% resolved

Leaderboard updates limited since 2025-11 policy change; CC BY-NC — display with attribution, no commercial use As of 2026-02-19. Source · CC BY-NC 4.0.

SWE-bench (Verified split)
Stale
GPT 5.2 (2025-12-11) (high)
Avg cost per task (Verified): $0.52 at 71.8% resolved

Leaderboard updates limited since 2025-11 policy change; CC BY-NC — display with attribution, no commercial use As of 2025-12-11. Source · CC BY-NC 4.0.

SWE-bench (Verified split)
Stale
Claude 4.5 Sonnet (high)
Avg cost per task (Verified): $0.66 at 71.4% resolved

Leaderboard updates limited since 2025-11 policy change; CC BY-NC — display with attribution, no commercial use As of 2026-02-17. Source · CC BY-NC 4.0.

SWE-bench (Verified split)
Stale
Kimi K2.5 (high)
Avg cost per task (Verified): $0.15 at 70.8% resolved

Leaderboard updates limited since 2025-11 policy change; CC BY-NC — display with attribution, no commercial use As of 2026-02-17. Source · CC BY-NC 4.0.

SWE-bench (Verified split)
Stale
Claude 4.5 Sonnet (20250929)
Avg cost per task (Verified): $0.56 at 70.6% resolved

Leaderboard updates limited since 2025-11 policy change; CC BY-NC — display with attribution, no commercial use As of 2025-09-29. Source · CC BY-NC 4.0.

SWE-bench (Verified split)
Stale
DeepSeek V3.2 (high)
Avg cost per task (Verified): $0.45 at 70.0% resolved

Leaderboard updates limited since 2025-11 policy change; CC BY-NC — display with attribution, no commercial use As of 2026-02-17. Source · CC BY-NC 4.0.

SWE-bench (Verified split)
Stale
Gemini 3 Pro
Avg cost per task (Verified): $0.96 at 69.6% resolved

Leaderboard updates limited since 2025-11 policy change; CC BY-NC — display with attribution, no commercial use As of 2026-02-26. Source · CC BY-NC 4.0.

SWE-bench (Multilingual split)
Stale
Gemini 3 Flash
Avg cost per task (Multilingual): $0.35 at 72.7% resolved

Leaderboard updates limited since 2025-11 policy change; CC BY-NC — display with attribution, no commercial use As of 2026-02-13. Source · CC BY-NC 4.0.

SWE-bench (Multilingual split)
Stale
Claude 4.6 Opus
Avg cost per task (Multilingual): $0.66 at 72.0% resolved

Leaderboard updates limited since 2025-11 policy change; CC BY-NC — display with attribution, no commercial use As of 2026-02-13. Source · CC BY-NC 4.0.

SWE-bench (Multilingual split)
Stale
Claude 4.5 Opus
Avg cost per task (Multilingual): $0.83 at 70.7% resolved

Leaderboard updates limited since 2025-11 policy change; CC BY-NC — display with attribution, no commercial use As of 2026-02-13. Source · CC BY-NC 4.0.

SWE-bench (Multilingual split)
Stale
GLM 5
Avg cost per task (Multilingual): $0.64 at 69.7% resolved

Leaderboard updates limited since 2025-11 policy change; CC BY-NC — display with attribution, no commercial use As of 2026-02-13. Source · CC BY-NC 4.0.

SWE-bench (Multilingual split)
Stale
Gemini 3 Pro
Avg cost per task (Multilingual): $1.02 at 68.7% resolved

Leaderboard updates limited since 2025-11 policy change; CC BY-NC — display with attribution, no commercial use As of 2026-02-13. Source · CC BY-NC 4.0.

SWE-bench (Multilingual split)
Stale
MiniMax M2.5
Avg cost per task (Multilingual): $0.10 at 68.3% resolved

Leaderboard updates limited since 2025-11 policy change; CC BY-NC — display with attribution, no commercial use As of 2026-02-16. Source · CC BY-NC 4.0.

SWE-bench (Multilingual split)
Stale
Kimi K2.5
Avg cost per task (Multilingual): $0.69 at 67.3% resolved

Leaderboard updates limited since 2025-11 policy change; CC BY-NC — display with attribution, no commercial use As of 2026-02-13. Source · CC BY-NC 4.0.

SWE-bench (Multilingual split)
Stale
Claude 4.5 Sonnet
Avg cost per task (Multilingual): $0.67 at 67.0% resolved

Leaderboard updates limited since 2025-11 policy change; CC BY-NC — display with attribution, no commercial use As of 2026-02-13. Source · CC BY-NC 4.0.

SWE-bench (Multilingual split)
Stale
GPT 5.2 (high)
Avg cost per task (Multilingual): $0.54 at 66.7% resolved

Leaderboard updates limited since 2025-11 policy change; CC BY-NC — display with attribution, no commercial use As of 2026-02-13. Source · CC BY-NC 4.0.

SWE-bench (Multilingual split)
Stale
GPT 5.2 Codex
Avg cost per task (Multilingual): $0.66 at 66.3% resolved

Leaderboard updates limited since 2025-11 policy change; CC BY-NC — display with attribution, no commercial use As of 2026-02-20. Source · CC BY-NC 4.0.

SWE-bench (Multilingual split)
Stale
Claude 4.5 Haiku
Avg cost per task (Multilingual): $0.38 at 64.7% resolved

Leaderboard updates limited since 2025-11 policy change; CC BY-NC — display with attribution, no commercial use As of 2026-02-13. Source · CC BY-NC 4.0.

SWE-bench (Multilingual split)
Stale
DeepSeek V3.2
Avg cost per task (Multilingual): $0.38 at 59.0% resolved

Leaderboard updates limited since 2025-11 policy change; CC BY-NC — display with attribution, no commercial use As of 2026-02-13. Source · CC BY-NC 4.0.

SWE-bench (Multilingual split)
Stale
GPT 5 mini
Avg cost per task (Multilingual): $0.05 at 39.7% resolved

Leaderboard updates limited since 2025-11 policy change; CC BY-NC — display with attribution, no commercial use As of 2026-02-13. Source · CC BY-NC 4.0.

What this does not guarantee

  • Self-reported data is directional, not audited. We publish n, verified share, and dispersion — not certainty.
  • Benchmark cost-per-task ≠ your cost: harness, repo size, and prompting style dominate.
  • We are not a price-comparison or procurement site; reference prices can lag same-day changes.
  • Thin cells fall back to broader categories and say so on the page.
  • Old-model data is not silently blended into current-gen answers.

Calibration, not a leaderboard

High token spend correlates with churn and defect rework, not higher productivity — tokens are not a productivity proxy. Every task report captures outcome (succeeded first try, succeeded with iteration, partial, or abandoned) as a first-class field, quality caveats are first-class copy, and TokensBench never ranks individuals by spend. This is a calibration tool, not a scoreboard.