FlashOptim: 50% Training Memory Reduction (Databricks)

Reduces per-parameter training memory by 50%+ through improved master weight splitting and 8-bit optimizer state quantization. AdamW drops from 16 bytes to 7 bytes per parameter. Tested on Llama-3.1-8B finetuning with no quality degradation. Code at databricks/flashoptim. Directly unblocks fine-tuning larger models on consumer GPUs. (arXiv)