Shodh-MoE: Sparse Mixture-of-Experts Routing Eliminates Negative Transfer in Multi-Physics Foundation Models

arXiv·medium signal

Training a single dense neural operator on incompatible PDE regimes (open-channel fluids vs. porous media) causes gradient conflict, unstable optimization, and plasticity loss. Shodh-MoE introduces a sparse-activated latent transformer with a top-1 soft-semantic router that autonomously bifurcates tokens to specialized expert subnetworks. It achieves decoded physical MSEs of 2.48×10⁻⁶ and 1.76×10⁻⁶ across domains with mass-conservation divergence at ~2.8×10⁻¹⁰. The routing technique for avoiding negative transfer in multi-domain foundation models is broadly applicable beyond physics.