Research2026-06-29 · source-backed

MIT Tech Review warns about the inevitable weakness of metrics.

Issue

2026-06-29

Confidence

source-backed

Sources

Redaction

redacted

Story

The June 29 Download flags the structural gap between what benchmarks measure and what actually matters. Read it right after the GLM-5.2 numbers above. A 39% F1 on IDOR detection is a real signal, but a benchmark score is not the same as the model being good at your task. Ground your evals in the work, not the leaderboard.

Grid transmission is deciding where AI expands
MIT Technology Review's Download covers a transmission line fight that could reshape New York's grid alongside escalating US threats against Chinese AI (MIT Tech Review). Transmission, not generation and not silicon, is increasingly what determines data center siting. AI expan...
Codenotch reads your AI coding usage limits from disk
codenotch is a Swift menu-bar app at 1,011 stars, created September 5, that reads Claude Code's /usage output (falling back to the OAuth token in the login keychain), Cursor's local SQLite session state, Codex's ChatGPT usage endpoint for 5-hour and weekly limits, plus Antigra...
Replotting intelligence against cost on a linear axis puts GLM-5.3-Flash at 1/160th of Fable 5.1's per-task price.
A September 1 analysis rebuilds Artificial Analysis's chart on a linear rather than logarithmic cost axis and prices models at what third-party providers actually charge. The spread is roughly 250x top to bottom: Fable 5.1 at $3.69 per task for intelligence 66, GLM-5.3-Flash a...
Unisound U2 and GLM-5.2 keep pushing the cheap open-agent tier.
Unisound's U2 is a 266B-total / 10B-active MoE tuned for agents, citing 72.2% SWE-bench Verified at $0.15/$0.30 per 1M tokens, while GLM-5.2 is getting named the strongest open-weight coding model across July roundups. Both are roundup-sourced, so verify the benchmarks against...
Lumabri runs huge MoE models across untrusted peers by shipping 4KB activations instead of weights.
The project (pure C, Apache-2.0, 71 stars) lazy-fetches only the bytes an inference touches and caches them locally, sending 4 KB activations to peers holding the relevant experts rather than transferring expert weights. Local and remote paths share identical code to guarantee...
Frontier models reach only about 50-54% Pass@8 when an answer needs both SQL and web search.
Snowflake's HybridDeepResearch supplies 380 tool-dependent tasks grounded in LiveSQLBench-Base-Lite databases plus public web corpora, testing whether an agent preserves constraints while moving evidence between systems. GLM-5.2, Claude Sonnet 4.6 and GPT-5 all cluster around...

Source trail

Download More from technologyreview.com

Entities

Download GLM Ground IDOR MIT Tech Review

Provenance

Canonical issue: Ramsay Research Agent — June 29, 2026
AI generated: no
Story unit: 2026-06-29-mit-tech-review-warns-about-the-inevitable-weakness-of-metrics
Labels: source-backed, canonical briefing excerpt

Fetching from the wire…

MIT Tech Review warns about the inevitable weakness of metrics.

Story

Related stories

Grid transmission is deciding where AI expands

Codenotch reads your AI coding usage limits from disk

Replotting intelligence against cost on a linear axis puts GLM-5.3-Flash at 1/160th of Fable 5.1's per-task price.

Unisound U2 and GLM-5.2 keep pushing the cheap open-agent tier.

Lumabri runs huge MoE models across untrusted peers by shipping 4KB activations instead of weights.

Frontier models reach only about 50-54% Pass@8 when an answer needs both SQL and web search.

Source trail

Entities

Provenance

MIT Tech Review warns about the inevitable weakness of metrics.

Story

Related stories

Grid transmission is deciding where AI expands

Codenotch reads your AI coding usage limits from disk

Replotting intelligence against cost on a linear axis puts GLM-5.3-Flash at 1/160th of Fable 5.1's per-task price.

Unisound U2 and GLM-5.2 keep pushing the cheap open-agent tier.

Lumabri runs huge MoE models across untrusted peers by shipping 4KB activations instead of weights.

Frontier models reach only about 50-54% Pass@8 when an answer needs both SQL and web search.

Source trail

Entities

Provenance

Story

Related stories

Grid transmission is deciding where AI expands

Codenotch reads your AI coding usage limits from disk

Replotting intelligence against cost on a linear axis puts GLM-5.3-Flash at 1/160th of Fable 5.1's per-task price.

Unisound U2 and GLM-5.2 keep pushing the cheap open-agent tier.

Lumabri runs huge MoE models across untrusted peers by shipping 4KB activations instead of weights.

Frontier models reach only about 50-54% Pass@8 when an answer needs both SQL and web search.

Tomorrow's issue, in your inbox.

Source trail

Entities

Provenance

Story

Related stories

Grid transmission is deciding where AI expands

Codenotch reads your AI coding usage limits from disk

Replotting intelligence against cost on a linear axis puts GLM-5.3-Flash at 1/160th of Fable 5.1's per-task price.

Unisound U2 and GLM-5.2 keep pushing the cheap open-agent tier.

Lumabri runs huge MoE models across untrusted peers by shipping 4KB activations instead of weights.

Frontier models reach only about 50-54% Pass@8 when an answer needs both SQL and web search.

Tomorrow's issue, in your inbox.

Source trail

Entities

Provenance