Letta Sleep-Time Compute: Let Agents 'Think' During Idle Periods — 5× Inference Cost Reduction at Equal Accuracy, Up to 18% Accuracy Gains

Letta Blog·medium signal

Letta and UC Berkeley's sleep-time compute research (arXiv 2504.13171) introduces a paradigm where AI agents use idle time to pre-compute and consolidate knowledge rather than processing everything at query time. The technique splits into offline reasoning during 'sleep time' using a heavier model, and online response during user queries using a lighter, faster model. Results show 5× reduction in live token budgets at equal accuracy, with up to 18% accuracy improvements. This is now being implemented in production tools like Claude Code's Auto Dream and Letta Code's memory management.