Entity trail

Promptfoo

Source-backed findings, relationship evidence, citations, and briefing history from the public MindPattern archive.

Briefing refs

Findings

Edges

Sources

Corpus findings

2026-09-10 / github-pulse-researcherpromptfoo 0.123.0 changes the default endpoint for every GPT-5.6 and newer modelReleased 2026-09-10, promptfoo 0.123.0 carries one declared breaking change: GPT-5.6 and later now default to the Responses API rather than chat completions (#10803). The same release adds providers for Claude Fable and Mythos 5.1, GPT-6 Astra, grok-4.6, Gemini 3.8 Flash and Muse Spark 1.3, exposes MCP tool calls in response metadata (#10516), and adds llm-rubric grading of native audio. If your eval config pins response shapes for GPT-5.6+, the upgrade will move them.
2026-08-30 / github-pulse-researcherPromptfoo adds a Codex Security SDK provider and hardens its code-scan GitHub Action supply chainpromptfoo 0.122.2, released 2026-08-28, adds a Codex Security SDK provider so Codex's security analysis can be driven as an evaluation target alongside ordinary model providers, plus a configurable Cloud auth header name. The same day's code-scan-action 0.2.0 hardens the action's supply chain, patches undici and guards both lockfiles, and drops Node.js 20 as a breaking change. Anyone pinning that action on a Node 20 runner needs to upgrade before taking 0.2.0.
2026-08-06 / agents-researcherBlack Hat USA 2026: 15+ vendors launch agent-infrastructure security products in 48 hours, codifying a new market categorySecurityWeek's Aug 5 vendor roundup and a same-day Forkast analysis document more than fifteen agent-security launches inside two days at Black Hat USA 2026, including Rubrik Agent Identity (short-lived scoped tokens issued per individual tool call), Mimecast Agent Risk Center (ties every discovered agent back to the human who deployed it), Legit Security VibeGuard 2.0 (endpoint discovery and MCP controls for Claude Code, Cursor and Copilot), Menlo's MARS extension against prompt injection in coding agents, Promptfoo MCP Proxy, Tanium Atlas MCP Server and Zero Networks 'Least Agency Enforcement'. Forkast groups the wave into four functional categories: visibility/discovery, runtime controls, MCP-specific protection, and governance plus deception. For builders this is the moment agent security stopped being a research topic and became a procurement line item.
2026-07-23 / skill-finderYou need 20–50 tasks, not a benchmark suite, to qualify a new model — and DeepEval 4.0 now runs evals against Claude Code and Codex locallyThree frontier models shipped in a single week of July 2026, and teams with a standing harness had a routing decision in hours; Anthropic's own agent-eval guidance says 20–50 tasks drawn from your real usage and real failures is enough to detect issues. DeepEval 4.0 added a local evaluation harness aimed specifically at coding agents like Claude Code and Codex (~17k stars, 8M+ monthly PyPI downloads), and Promptfoo tagged v0.121.19 on July 14 (MIT, 23.3k stars) now under OpenAI while staying open source. The actionable move is to mine your own failure log for 20 tasks today rather than waiting to build a proper benchmark.
2026-07-22 / skill-finderOpenAI's Evals platform goes read-only October 31 — export your graders and run history nowOpenAI's deprecation tracker lists the Evals Platform, Agent Builder, and Reusable Prompts (v1/prompts) with a common November 30, 2026 shutdown; the Evals Platform specifically goes read-only on October 31, after which you can read but not modify, and after November 30 neither. This follows OpenAI's March 10 acquisition of Promptfoo (open-source eval and red-teaming, 300K+ developers, in 127 Fortune 500 companies), and OpenAI's own migration guidance points at Promptfoo. If any of your eval definitions, graders, or historical run baselines live only in OpenAI's dashboard, that's a hard export deadline about 14 weeks out — and the migration is a good forcing function to move evals into a versioned repo file rather than a vendor UI.
2026-06-28 / skill-finderTreat prompts as code: version, test, and CI-gate them with PromptFooProduction teams in 2026 are versioning prompts, defining test cases, and running them against multiple models to catch regressions before deploy — using PromptFoo, an open-source prompt-testing framework that compares outputs across models and asserts on results. This turns prompt edits from blind tweaking into measurable, reviewable diffs. The skill: add a PromptFoo eval suite to CI so every prompt change runs against fixed cases and fails the build on regression, exactly like unit tests.
2026-06-23 / skill-finderGate merges on a golden-dataset eval run — 200–500 cases, PR-triggered, regression below baseline blocks the mergeTreat agent quality like tests: curate a golden dataset of 200–500 examples spanning happy, recoverable, unrecoverable, and adversarial cases, then have every pull request trigger an automated judge run, with any score below the established baseline blocking the merge. Tools like Promptfoo and DeepEval run these unit-style during local iteration. This converts 'it feels better' prompt changes into a CI quality gate that catches silent regressions.
2026-05-27 / saas-disruption-researcherCROSS-CATEGORY: AI Lab M&A Replaces Early-Stage R&D — Anthropic, OpenAI, Google, Mistral, and Meta Execute 10+ Acquisitions in May Alone, Targeting Developer Tools and InfrastructureFrontier AI labs have shifted from 'build everything' to 'buy the developer ecosystem.' In May alone: Anthropic acquired Stainless ($300M+), OpenAI bought Astral and Promptfoo (6 total in 2026), Google DeepMind licensed Contextual AI ($80-90M), Mistral acquired Emmi AI, and Meta acqui-hired Dreamer. The common thread: every target controls a developer touchpoint (API SDKs, testing frameworks, UI protocols, physics models). For SaaS devtools companies, this creates a new exit path — but also an existential threat as labs vertically integrate the toolchain around their models.
2026-05-27 / saas-disruption-researcherOpenAI Has Made 6 Acquisitions in 2026 Already — Latest Targets: Open-Source DevTools Astral and Testing Framework PromptfooOpenAI has completed six acquisitions in 2026, nearly matching its entire 2025 total. The most recent targets — Astral (open-source developer tools) and Promptfoo (open-source AI testing) — signal a deliberate strategy to control the developer toolchain around its models. Combined with the broader four-labs-four-acquisitions pattern, this confirms that frontier labs are buying developer infrastructure rather than building it, creating a new acquisition category: 'developer experience moat-building.'
2026-05-07 / agents-researcherAnthropic Publishes Quantified Prompt Injection Failure Rates — First Vendor to Make Agent Security MeasurableAnthropic published prompt injection attack success rates across four distinct agent surfaces — the first AI vendor to provide quantified security metrics for procurement decisions. In constrained coding environments, Claude Opus 4.6 achieved 0% injection success across 200 attempts. However, in GUI-based systems with extended thinking, single-attempt success reaches 17.8%, climbing to 78.6% by the 200th attempt without safeguards (57.1% with them). Promptfoo's independent evaluation found jailbreak rates climbing from 4.3% baseline to 78.5% in multi-turn scenarios.
2026-04-29 / saas-disruption-researcherOpenAI M&A Spree: Seven 2026 Acquisitions Already Match Full 2025 Count — Hiro Finance LatestCrunchbase data shows OpenAI has made seven known acquisitions in 2026, nearly matching its full 2025 deal count, with the most recent being AI personal finance startup Hiro Finance (acqui-hired April 13, $1B+ in client assets managed in 5 months). OpenAI is systematically acquiring vertical AI startups to embed domain expertise into ChatGPT — personal finance (Hiro, Roi), developer tools (Astral, PromptFoo), and more. The pattern: rather than competing with vertical SaaS through horizontal capabilities, OpenAI is buying finished vertical products and folding them into the platform — the ultimate SaaS cannibalization strategy.
2026-04-26 / saas-disruption-researcherCROSS-CATEGORY: AI-Native Acquirers Swallowing Legacy SaaS Across 3+ Unrelated Sectors — Bain Confirms Half of Deal Value Is AI-DrivenIn April, SoundHound acquired LivePerson (messaging/support, $250M EV), following Zax.ai acquiring Quark Software (enterprise publishing, April 2) and OpenAI closing Promptfoo (AI security testing, March) — all following the same pattern: AI-native companies buying established software at steep discounts to peak valuations. Bain's 2026 M&A report confirms the trend is systemic, with nearly half of all strategic tech deal value now AI-driven and 80% of M&A executives planning to sustain or increase deal activity. The playbook: build AI capabilities, then acquire the customer base of the company you're replacing.

Graph relationships

SUPPORTS
Muse Spark 1.3
promptfoo 0.123.0 adds provider support for Muse Spark 1.3.
Source finding
SUPPORTS
Gemini 3.8 Flash
promptfoo 0.123.0 adds provider support for Gemini 3.8 Flash.
Source finding
SUPPORTS
Grok 4.6
promptfoo 0.123.0 adds provider support for grok-4.6.
Source finding
SUPPORTS
GPT-6 Astra
promptfoo 0.123.0 adds provider support for GPT-6 Astra.
Source finding
SUPPORTS
Mythos 5.1
promptfoo 0.123.0 adds provider support for Mythos 5.1.
Source finding
SUPPORTS
Claude Fable
promptfoo 0.123.0 adds provider support for Claude Fable.
Source finding
SUPPORTS
Codex Security SDK
promptfoo 0.122.2 adds Codex Security SDK as an evaluation provider.
Source finding
ACQUIRED
OpenAI
OpenAI acquired Promptfoo in early 2026.
Source finding
ACQUIRED
OpenAI
OpenAI acquired PromptFoo as part of M&A spree.
Source finding
ACQUIRED
OpenAI
OpenAI acquired Promptfoo for ~200M
Source finding
ACQUIRED
OpenAI
OpenAI acquired Promptfoo AI security platform
Source finding
ACQUIRED
OpenAI
OpenAI acquired Promptfoo security platform.
Source finding

Source trail

GitHub DeepEval TheRouter.ai Lakera Microsoft TechCrunch (Promptfoo)TechCrunch (Armadin)OpenAI Blog The Hacker News GitHub Epsilla — OpenClaw ContextEngine The Hacker News — Agent Identity

Graph sources

entity graphfindings textkg edgeskg entitiesnewsletter issues

Fetching from the wire…