Grafana Open-Sources o11y-bench: AI Agent Benchmark for Real Observability Workflows Against Live Prometheus/Loki/Tempo Stack

Grafana Labs·medium signal

Grafana Labs open-sourced grafana/o11y-bench at GrafanaCON 2026, a benchmark that runs AI agents against a real Grafana stack with Prometheus, Loki, and Tempo — not synthetic scenarios. Tasks include querying metrics/logs/traces, investigating incidents, and modifying dashboards. Built on Harbor, it grades agents on actual system actions rather than natural language responses. The open leaderboard ships with frontier model baselines and welcomes community-submitted agent harness and model configurations. Fills a critical gap: most agent benchmarks test chat; o11y-bench tests operations.