AdaCodec: Predictive Visual Coding Cuts Video MLLM Token Counts by Exploiting Temporal Redundancy

arXiv·medium signal

Addresses the core inefficiency in video MLLMs: adjacent frames share most visual content but are encoded as independent images, inflating token counts. AdaCodec uses predictive coding to represent only the temporal delta between frames, dramatically reducing visual tokens per video. Directly relevant for anyone building video understanding agents — lower token counts mean longer videos fit in context and inference costs drop.