Randomized Go map ordering in an MCP gateway was silently destroying vLLM's prefix cache, costing 26 seconds of time-to-first-token
A 2026-09-15 postmortem traces a coding agent's 26-28 second average TTFT to tool schemas arriving in different key orders on every 10-minute MCP refresh, because the Bifrost gateway decodes tool definitions into Go maps and Go deliberately randomizes map iteration order. One reordered schema broke the shared prefix after only 12,622 tokens, forcing the remaining ~107,000 to be re-read. Sorting schema keys in the chat template with tojson(sort_keys=True), plus upstream fixes to Bifrost and mcp-go, moved the cache hit rate from 55% to 95%, TTFT from 26-28s to 7.3s, and worst-case wait from 514s to 54s. Anyone serving agents on vLLM behind an MCP gateway should check for nondeterministic serialization before blaming the model.
↳ Follow the thread