ben 1.2

Frontier work, for less, in less time.

intelligence index · higher is better · % against ben high× ben high · lower is betterlower is better · % against ben highfrontiercode-aa 2.0 · higher is better · % against ben highterminal-bench 4.0 · higher is better · % against ben highgdp.pdf · higher is better · % against ben high
GPT-6.2 Astramax 62.11.2×14 min45.4%64.1%29.8% +20%+51%+26%+13%+3%
Gemini 4 Carbonhigh 602.2×42 min40.6%65.2%28.4% +16%+341%+13%+15%−1%
Claude Opus 5.5max 59.12.9×26 min46.4%59.6%26.2% +14%+171%+29%+5%−9%
Ben Flashhigh 54.90.2×5 min40.2%58.6%28.8% +6%−48%+12%+4%
Benhigh 51.81×9 min35.9%56.6%28.8%
Kimi K3.1 Pro 46.51.1×39 min25.5%67.2%31.4% −10%+315%−29%+19%+9%
Muse Wonder 43.62×14 min27.9%52%30.6% −16%+48%−22%−8%+6%
Grok 4.7 35.52.2×25 min28.7%25.8%20% −31%+168%−20%−54%−31%
DeepSeek V4.1 Flash 32.60.2×14 min28.5%33.8%17.2% −37%+50%−21%−40%−40%
cost by effort · × ben high
0.05×
0.2×
1×
lowmedhigh
Ben
0.05×
0.2×
0.2×
lowmedhigh
Ben Flash
2.1×
1.9×
2.2×
lowmedhigh
Gemini 4 Carbon
1.7×
2.9×
xhighmax
Claude Opus 5.5
0.2×
0.5×
0.6×
0.7×
1.2×
lowmedhighxhighmax
GPT-6.2 Astra
score against cost · every setting
Score against cost a task: Ben, Ben Flash, every setting of each lab's flagship and other models102030405060700.03×0.1×0.3×1×3×cost a task · × ben high · log scalescoreGemini 4 Neon: score 61.4, 1.9× Ben High's cost a task, 21 minGemini 4 Neon · 61.4 · 1.9×Gemini 4 Hydrogen: score 53.9, 0.9× Ben High's cost a task, 15 minGemini 4 HydrogenKimi K3.1 Pro: score 46.5, 1.1× Ben High's cost a task, 39 minKimi K3.1 ProMuse Wonder: score 43.6, 2× Ben High's cost a task, 14 minMuse WonderQwen4 Max: score 42.9, 0.9× Ben High's cost a task, 30 minQwen4 MaxMuse Ember: score 36, 0.04× Ben High's cost a task, 3 minMuse EmberGrok 4.7: score 35.5, 2.2× Ben High's cost a task, 25 minGrok 4.7Tencent Hy4: score 34.6, 0.3× Ben High's cost a task, 14 minTencent Hy4DeepSeek V4.1 Flash: score 32.6, 0.2× Ben High's cost a task, 14 minDeepSeek V4.1 FlashGLM-5.3: score 31.7, 1.4× Ben High's cost a task, 27 minGLM-5.3Inkling 2 Small: score 30.6, 0.3× Ben High's cost a task, 22 minInkling 2 SmallMiMo-V2.6-Pro: score 29.9, 0.07× Ben High's cost a task, 36 minMiMo-V2.6-ProLing 3.1 Flash: score 29.6, 0.9× Ben High's cost a taskLing 3.1 FlashMistral Large 4: score 26.1, 0.6× Ben High's cost a task, 24 minMistral Large 4Qwen3.8 27B: score 20.1, 1× Ben High's cost a task, 38 minQwen3.8 27BBeam 1.0: score 17.7, 0.08× Ben High's cost a task, 2 minBeam 1.0Nemotron 3.5 Super VL: score 13.6, 0.1× Ben High's cost a task, 5 minNemotron 3.5 Super VLGemini 4 Carbon · high: score 60, 2.2× Ben High's cost a task, 42 minGemini 4 CarbonGemini 4 Carbon · medium: score 58.3, 1.9× Ben High's cost a task, 30 minGemini 4 Carbon · medium · 58.3 · 1.9×Gemini 4 Carbon · low: score 54.2, 2.1× Ben High's cost a task, 30 minGemini 4 Carbon · low · 54.2 · 2.1×Claude Opus 5.5 · max: score 59.1, 2.9× Ben High's cost a task, 26 minClaude Opus 5.5 · max · 59.1 · 2.9×Claude Opus 5.5 · xhigh: score 57.4, 1.7× Ben High's cost a task, 16 minClaude Opus 5.5GPT-6.2 Astra · max: score 62.1, 1.2× Ben High's cost a task, 14 minGPT-6.2 AstraGPT-6.2 Astra · xhigh: score 61.6, 0.7× Ben High's cost a task, 8 minGPT-6.2 Astra · xhigh · 61.6 · 0.7×GPT-6.2 Astra · high: score 60.1, 0.6× Ben High's cost a task, 6 minGPT-6.2 Astra · high · 60.1 · 0.6×GPT-6.2 Astra · medium: score 57, 0.5× Ben High's cost a task, 5 minGPT-6.2 Astra · medium · 57 · 0.5×GPT-6.2 Astra · low: score 45.7, 0.2× Ben High's cost a task, 2 minGPT-6.2 Astra · low · 45.7 · 0.2×Ben Low · Flash Low: score 29.9, 0.05× Ben High's cost a task, 7 min, ≈159 tokensBen Low · Flash LowBen Medium · Flash High: score 54.9, 0.2× Ben High's cost a task, 5 min, ≈722 tokensBen Medium · Flash HighBen High: score 51.8, 1× Ben High's cost a task, 9 min, ≈3,465 tokensBen HighBen Flash Medium: score 52.7, 0.2× Ben High's cost a task, 3 min, ≈520 tokensBen Flash Medium
benchmark by benchmark · ben at high, each flagship at its top setting
BenBen FlashFlashGemini 4 CarbonClaude Opus 5.5GPT-6.2 Astra
FrontierCode-AA 2.0FrontierCode35.9%40.2%40.6%46.4%45.4%
Terminal-Bench 4.0Terminal-Bench56.6%58.6%65.2%59.6%64.1%
Terminal-Bench-Science 0.1Terminal-Bench-Science49%67.6%64.8%59%78.1%
GDP.pdf28.8%28.8%28.4%26.2%29.8%
AA-Briefcase v1.1AA-Briefcase1,6821,5411,6111,8001,644
GDPval-AA v2.1GDPval1,709–1,7331,866–
AutomationBench-AAAutomationBench63.2%69.9%77.8%69.5%77.2%
HLE-Diamond55.6%58.3%69.1%67.9%72.2%
CritPt v2 (Hard Subset)CritPt59.5%61.5%69%68%71.5%
AA-Omniscience v1.1AA-Omniscience42.438.353.548.150
what a job takes · 476 finished jobs, 14 days · 20 Nov 2026
by size
A small fixunder 100 lines changed16 min≈2.5k tokens13%
A feature100 to 600 lines changed33 min≈5.4k tokens45%
A big build600 lines and up65 min≈15k tokens42%
by kind
A page or interfacelayout, styles, components44 min≈9.1k tokens14%
Codelogic, data, tests39 min≈7.2k tokens83%
Writingdocs and copy28 min≈14k tokens3%

13% of the work ran on split threads

releases

Ben 1.2

now

Medium and Ben Flash score higher for fewer tokens a task, in about half the time.

Ben 1.1

9 Nov 2026

Medium scores higher for fewer tokens a task.

Ben 1

8 Nov 2026

The first release: three levels, from quick answers to Ben’s strongest work.

Ben 1.2 is live in Ben today, at High unless you turn the dial. See the plans →

Benchmarks: FrontierCode-AA 2.0 · Terminal-Bench 4.0 · Terminal-Bench-Science 0.1 · GDP.pdf · AA-Briefcase v1.1 · GDPval-AA v2.1 · AutomationBench-AA · HLE-Diamond · CritPt v2 (Hard Subset) · AA-Omniscience v1.1. Scores, cost and time a task as Artificial Analysis ran them, 20 Nov 2026. Cost is each model's list price a task against what Ben High's tokens are worth.