Fetching from the wire…
Skills2026-07-30 · source-backed
RAGAS-style evaluation checks correctness against a frozen snapshot, which means routine document updates and corrections can silently break production without moving a dashboard. This ASE 2026 paper defines 11 mutation operators perturbing at both the pre-chunk index level and post-chunk retrieved-context level. Across five datasets and 28,000+ mutants, metamorphic oracles detected violations at F1 0.927-1.000 versus 0.570 for the best RAGAS metric.
Each link below shares sources, entities, or timing with this story.
arXiv 2608.00765 compresses retrieved docs into query-conditioned visual representations, sidestepping the trade-off where hard compression is query-aware but weak and soft compression is strong but needs costly offline encoding. Beats both baselines across varying retrieval d...
A June 15 paper introduces 442 expert-curated Nature Portfolio meta-analyses against a 140,000-article PubMed corpus, benchmarking twelve pipeline configs. No system recovered more than 52.7% of ground-truth included literature, even at 90.9% retrieval recall at K=200. The bot...
A July 2 evaluation tested semantic chunking against simple approaches on long structured academic theses using RAGAs, and the sophisticated method didn't win. Performance varied more with document formatting, preprocessing, and query type than with chunking strategy. The auth...
D-SCAN (SIGIR 2026) found the standard guardrail returns high confidence on compromised output. Their alternative signal is document-level attention dynamics: during a poisoned generation, attention concentrates on the injected document and entropy collapses, versus dispersed...
arXiv 2607.08124 argues an agent's behavior is set as much by its harness (the program that builds context, invokes tools, verifies intermediate results, recovers from failure) as by the model. Current practice optimizes the harness on development data then freezes it at deplo...
Thinkingbox is an MCP-compatible sandbox with isolated sessions, full execution traces, and outcome evaluation against terminal backend state, carrying 507 policy-conditioned workflows across retail, hospitality, auto insurance, neobank internal IT and consulting support (arXi...
MindPattern daily
One email a day at 7 AM. Sources and a take on every story. Unsubscribe anytime.