Entity trail

Terminal-Bench-Science

Source-backed findings, relationship evidence, citations, and briefing history from the public MindPattern archive.

Briefing refs

Findings

Edges

Sources

Corpus findings

2026-09-06 / thought-leaders-researcherZvi Mowshowitz's Fable 5.1 capabilities review: Terminal-Bench-Science jumps 24.7% to 52.6%, but users report 3x token burnZvi published his capabilities review of Claude Mythos 5.1 and Fable 5.1 on September 5, separate from his earlier system-card post. He pulls out Terminal-Bench-Science rising from 24.7% to 52.6%, CursorBench 3.2 at 73.4% vs 70.5%, OSWorld 2.0 at 78% partial/42% strict, HLE at 60.9%, a cache-read price drop from $1 to $0.25 per million tokens, and a roughly 60% fall in safety-classifier false positives. The catch for builders on subscriptions: multiple users report Fable 5.1 consuming up to 3x the tokens of Fable 5, so cheaper per-token API pricing coexists with faster subscription-limit exhaustion.
2026-09-03 / hn-researcherAnthropic Ships Claude Fable 5.1 and a Gated Mythos 5.1 Twin, With Cache Reads Cut 75% to $0.25/MtokFable 5.1 and Mythos 5.1 are the same underlying model at two safeguard levels: Fable is generally available, Mythos goes only to vetted cybersecurity and life-sciences organizations, currently US-only. Fable 5.1 scores 52.6% on Terminal-Bench-Science 0.1 (versus 24.7% for Fable 5 and 29.0% for Opus 5), 55.8% on Terminal-Bench 4.0, 60.9% on Humanity's Last Exam without tools, and 73.4% on CursorBench 3.2.0. Token pricing holds at $10/$50 per million in/out, but cache reads drop 75% to $0.25/Mtok, which Anthropic puts at ~25% cheaper for typical workloads and up to 45% for highly agentic runs.
2026-09-03 / sources-researcherFable 5.1's headline gain is a science benchmark, not coding: 52.6% on Terminal-Bench-Science 0.1 against 29.0% for Opus 5Simon Willison's 2026-09-01 writeup notes Anthropic's announcement spends most of its space on scientific research, citing 52.6% on the ten-day-old Terminal-Bench-Science 0.1 benchmark versus 24.7% for Fable 5, 29.0% for Opus 5 and 22.4% for GPT-5.6 Sol, while the other benchmark movements are only slightly improved. Fable 5.1 exposes five reasoning levels (low, medium, high, xhigh, max) with no way to turn reasoning off entirely. Willison also flags that at effort low the transcript showed no summarized reasoning tokens at all, and that he had to fix llm-anthropic to record reasoning traces correctly before testing.
2026-08-29 / projects-researcherTerminal-Bench-Science scores the best agent at 30% across 70 expert-written scientific workflowsA Stanford-led team from the Terminal-Bench and Harbor project announced Terminal-Bench-Science on 2026-08-28 (115 points on HN), with 70 tasks split across life sciences (19), physical (17), mathematical (17), earth (8) and engineering (9), advised by Ludwig Schmidt and Sanmi Koyejo. Claude Opus 5 leads at 30.0%, GPT-5.6 Sol at 22.4%, Claude Fable 5 at 21.4%, then a steep drop to Claude Opus 4.8 at 10.5%, GLM 5.3 at 8.1% and Kimi K3 at 7.1%. The 3x gap between the top three and everything else is the useful number here: agent harness quality on real research workflows is not yet a commodity.
2026-08-28 / hn-researcherTerminal-Bench-Science Launches With 70 Researcher-Authored Tasks and No Model Above 30%The v0.1.0 release is a continuous benchmark built from researchers' own workflows: 70 tasks across Life Sciences (19), Physical (17), Mathematical (17), Engineering (9) and Earth Sciences (8), assembled by 376 contributors across 22 countries. Claude Opus 5 leads at 30.0%, followed by GPT-5.6 Sol at 22.4%, Claude Fable 5 at 21.4%, Claude Opus 4.8 at 10.5%, GPT-5.6 Terra at 8.6%, GLM 5.3 at 8.1%, Kimi K3 and Grok 4.6 at 7.1%, and GPT-5.6 Luna at 3.3%. The spread between Opus 5 and Opus 4.8 is nearly 3x, which is a far wider generational gap than the same models show on saturated coding benchmarks.
2026-08-21 / arxiv-researcherTask-CoEvolve Cuts Agent Harness Optimization Cost 80% by Only Evaluating Tasks the Candidates Disagree OnarXiv 2608.20169 attacks the waste in harness optimization, where each iteration re-runs a fixed validation set in full even on tasks that stopped discriminating between candidates. Task-CoEvolve uses variance-weighted sampling driven by past outcomes to concentrate evaluation near the agent's capability frontier, then reweights by sampling probability to estimate full-set scores from partial runs so iterations stay comparable. On online text classification and Terminal-Bench 2.1 it beats fixed-subset baselines and matches full-set search's final performance with 80% fewer evaluations, with code promised at github.com/Agent4Science-UTokyo/Task-CoEvolve.
2026-04-26 / skill-finderOpenAI Launches GPT-5.5 and GPT-5.5 Pro: First Model Classified 'High' Cybersecurity Risk Under Preparedness Framework — $5/M Input, 82.7% Terminal-Bench 2.0OpenAI released GPT-5.5 on April 23 with GPT-5.5 Pro following for API on April 24. This is the first model OpenAI has classified as 'High' under its Preparedness Framework for cybersecurity capabilities. GPT-5.5 scores 82.7% on Terminal-Bench 2.0 for command-line tool use, with Pro showing large gains in business, legal, education, and data science. API pricing: $5/$30 per million tokens (standard), $30/$180 (Pro). Nearly 200 early-access partners tested before release.

Graph relationships

BENCHMARKED AGAINST
Fable 5.1
Fable 5.1 achieved 52.6% on Terminal-Bench-Science, up 24.7% from prior version.
Source finding
BENCHMARKED AGAINST
Kimi K3
Kimi K3 scored 7.1% on Terminal-Bench-Science.
Source finding
BENCHMARKED AGAINST
GLM-5.3
GLM 5.3 scored 8.1% on Terminal-Bench-Science.
Source finding
BENCHMARKED AGAINST
Claude Opus 4.8
Claude Opus 4.8 scored 10.5% on Terminal-Bench-Science.
Source finding
BENCHMARKED AGAINST
Claude Fable 5
Claude Fable 5 scored 21.4% on Terminal-Bench-Science.
Source finding
BENCHMARKED AGAINST
GPT-5.6 Sol
GPT-5.6 Sol scored 22.4% on Terminal-Bench-Science.
Source finding
BENCHMARKED AGAINST
Claude Opus 5
Claude Opus 5 led at 30.0% on Terminal-Bench-Science.
Source finding
BENCHMARKED AGAINST
GLM-5.3
GLM 5.3 scored 8.1% on Terminal-Bench-Science.
Source finding
BENCHMARKED AGAINST
GPT-5.6 Terra
GPT-5.6 Terra scored 8.6% on Terminal-Bench-Science.
Source finding
BENCHMARKED AGAINST
Claude Opus 4.8
Claude Opus 4.8 scored 10.5% on Terminal-Bench-Science.
Source finding
BENCHMARKED AGAINST
Claude Fable 5
Claude Fable 5 scored 21.4% on Terminal-Bench-Science.
Source finding
BENCHMARKED AGAINST
GPT-5.6 Sol
GPT-5.6 Sol scored 22.4% on Terminal-Bench-Science.
Source finding

Source trail

Don't Worry About the Vase (Zvi Mowshowitz)Anthropic / Hacker News Simon Willison's Weblog Terminal-Bench-Science / Hacker News arXiv 2608.20169 OpenAI — Introducing GPT-5.5

Graph sources

entity graphfindings textkg edgeskg entities

Fetching from the wire…