Entity trail

NInfer

Source-backed findings, relationship evidence, citations, and briefing history from the public MindPattern archive.

Briefing refs

Findings

Edges

Sources

Corpus findings

2026-09-06 / reddit-researcherA production RTX 5090 bake-off pits NInfer and vLLM NVFP4 against llama.cpp GGUF on a six-tier custom evalA builder serving Qwen3.8-27B for a content-intelligence pipeline moved off llama.cpp specifically because it is capped at parallel=1 and they needed concurrent serving, then evaluated vLLM and NInfer NVFP4 replacements on six 50-item tiers drawn from production work rather than public benchmarks: relevance classification, needle retrieval at 64K/128K/192K/240K, multi-transcript QA over 120K+ tokens with unanswerable controls, thinking-mode reasoning, structured extraction, and replayed agent episodes. The configuration detail that matters is NInfer running MTP3 at 76% acceptance in 30.5 GB and llama.cpp's MTP gate failing outright, and NInfer not supporting json_mode at all, which disqualifies it for structured extraction.
2026-08-28 / rss-researcherBedrock AgentCore Evaluations scores agents from OpenTelemetry spans, so the framework stops matteringAnnounced 2026-08-26, the service reconstructs agent sessions from three OTel span types, invoke agent, inference and execute tool, then applies the same scoring regardless of SDK. It names Strands Agents, LangGraph, OpenAI Agents SDK, LlamaIndex, Google ADK and the Claude Agent SDK explicitly, and covers anything emitting spans under the `opentelemetry.instrumentation.*` or `openinference.instrumentation.*` scope prefixes through generic classification. Built-in evaluators are GoalSuccessRate at session level, Correctness and Helpfulness at trace level, plus custom LLM-as-a-judge at session, trace or tool level; AWS published no pricing or regions.
2026-08-20 / reddit-researcherFour 2017 Tesla V100s Match a $6,000 RTX 5090 on Qwen3.8 NVFP4 Decode, on Silicon With No FP4 Support At AllA developer published v100-skinny, hand-written NVFP4 W4A16 CUDA kernels plus chain-MTP speculative serving, and reported four Volta V100s hitting 219.1 ± 5.9 tok/s decode against an RTX 5090 running the NInfer engine at 214.7 ± 9.2 tok/s, both 5/5 correct on AIME 2026 problem 1 across five seeds. The mechanism is the honest part: the V100 round takes 26.9ms versus 19.9ms (35% slower) but commits 5.89 tokens per round versus 4.27 (38% more), because QPN lets it run Qwen3.8's own MTP at depth 7 while NInfer caps at 5. The repo (68 stars, created 2026-08-11, last pushed 2026-08-19) is a concrete argument that speculative depth can buy back a whole hardware generation.
2026-07-14 / skill-finderInstrument agents with OpenTelemetry GenAI/OpenInference conventions so traces stay portable across eval vendorsThe OpenTelemetry GenAI semantic conventions give a vendor-neutral schema for agent traces, and open-source tools (Phoenix, Langfuse) plus managed ones (LangSmith, Weave) build on OTel/OpenInference. Instrumenting to the standard schema now means your trajectory data — LLM calls, tool invocations, intermediate outputs, error states — moves between observability backends without re-instrumentation. The conventions are still marked experimental in 2026, but adopting them early avoids lock-in to a single dashboard's proprietary trace format.

Graph relationships

BENCHMARKED AGAINST
llama.cpp
NInfer and llama.cpp compared on concurrent serving; llama.cpp capped at parallel=1
Source finding
BENCHMARKED AGAINST
vLLM
NInfer and vLLM compared on Qwen3.8-27B serving performance
Source finding

Source trail

r/LocalLLaMA r/LocalLLaMA (64 upvotes)AWS Machine Learning Blog Braintrust — Agent Observability Complete Guide 2026

Graph sources

entity graphfindings textkg edgeskg entitiesnewsletter issues

Fetching from the wire…