Reconstructing Facts From Feed-Forward Residual Vectors Answers Single-Fact Questions in Two-Million-Token Contexts

arXiv 2609.12686·low signalXBlueskyLinkedInCopy link

This method keeps GPU memory near constant as context length grows, with no additional training, by reconstructing query-relevant facts from the residual vectors stored in the model's feed-forward layer activations instead of holding or re-reading the source document. Because reconstruction is deterministic, it sidesteps the fidelity loss of lossy prompt compression while staying inside a pretrained, size-constrained context window. The reported result is answering single-fact questions over two-million-token story contexts where previous methods fail, though the paper reports no head-to-head recall numbers against retrieval baselines.