Entity trail

RealReplicaBench

Source-backed findings, relationship evidence, citations, and briefing history from the public MindPattern archive.

Briefing refs

Findings

Edges

Sources

Corpus findings

2026-08-09 / github-pulse-researcherRealReplicaBench Makes Agents Actually Change State in Cloned Alibaba, Shopify and FreightOS Services — Best Score Is 61.7%Accio open-sourced RealReplicaBench on August 2 (1,042 stars by August 9) with 107 tasks across 14 containerized mock services replicating real commerce and logistics platforms including Alibaba, Shopify, and FreightOS. The task mix is 53 CLI, 28 browser, 16 file operations, and 10 API/MCP, split into 65 text-only, 20 browser-text, and 22 vision-requiring; twelve model families were run on two harnesses. Claude Opus 5 led both, scoring 66/107 (61.7%) on the Accio harness and 60/107 (56.1%) on OpenClaw — meaning the best available agent still fails roughly four in ten real business workflows that require mutating state rather than answering questions.
2026-08-07 / github-pulse-researcherRealReplicaBench Puts Claude Opus 5 on Top at 61.7% Across 107 Long-Horizon Tasks in Cloned Commerce ServicesAccio-org/RealReplicaBench (created 2026-08-02, 1,038 stars in 5 days) benchmarks agents against high-fidelity stateful replicas of eight commerce and logistics platforms — Alibaba-style publishing, FreightOS-style freight, Shopify-style storefront admin — over 107 tasks (53 CLI, 28 browser, 16 file, 10 API/MCP). Twelve models were run on two harnesses: Claude Opus 5 leads at 66/107 (61.7%) on the Accio harness and 60/107 (56.1%) on OpenClaw, ahead of Opus 4.8 (55.1%/51.4%) and GPT-5.6 Sol (49.5%). The harness swap moves scores by 5+ points on identical models — a reminder that agent scaffolding is a measured variable, not a footnote.
2026-08-04 / github-pulse-researcherAlibaba's Accio Team Publishes RealReplicaBench: 107 Long-Horizon Agent Tasks in Stateful Replicas of Real ServicesAccio-Lab/RealReplicaBench appeared 2026-08-02 (162 stars, release v1.3.1) from the Accio team at Alibaba International, benchmarking long-horizon agents against high-fidelity, stateful, reproducible replicas of real online commerce services rather than mocks or static transcripts. It ships 107 tasks, a reproducibility contract, a live leaderboard, and reference results run through the OpenClaw harness. The stateful-replica approach targets the specific failure mode static benchmarks can't measure: whether an agent's earlier actions leave the world in a state its later actions can still work in.

Graph relationships

BENCHMARKED AGAINST
OpenClaw
Claude Opus 5 scored 60/107 (56.1%) on OpenClaw harness
Source finding
BENCHMARKED AGAINST
Claude Opus 5
Claude Opus 5 scored 66/107 (61.7%) on RealReplicaBench Accio harness
Source finding
BUILT BY
Accio
Alibaba's Accio team released RealReplicaBench, an agent benchmark with 107 tasks.
Source finding
BENCHMARKED AGAINST
OpenClaw -> RealReplicaBench
Claude Opus 5 scored 60/107 (56.1%) on OpenClaw harness
Source finding

Source trail

GitHub Qwen3.8 Max DCAS meme mocking benchmark charts DeepSeek emailed API users GitHub

Graph sources

entity graphfindings textkg edgeskg entitiesnewsletter issues

Fetching from the wire…