S²AE Makes Sparse Autoencoders Learn Modality-Consistent Concepts in VLMs

arXiv·medium signal

Sparse autoencoders are a leading mechanistic-interpretability tool, learning sparse latent features that each encode a distinct concept, but in vision-language models vanilla SAEs learn concepts with fragmented, disjoint coverage in the visual modality. The proposed Structured Sparse AutoEncoder (S²AE) enforces structure so concepts stay consistent across text and image modalities. Advances interpretability tooling for multimodal models specifically, not just text LLMs.