Hugging Face publishes open-source Mixture-of-Experts training guide based on 800k GPU hours

Hugging Face released The Ultimate MoE Training Handbook on 5 November 2026, a free 70k-word technical guide with 55 interactive figures and runnable code based on 800k GPU hours of internal experiments covering expert sizing, shared experts, hiding all-to-all communication overhead, fp32 softmax versus sigmoid and sqrt(softplus) routers, auxiliary loss versus DeepSeek loss-free bias, dense-to-MoE checkpoint upcycling and variable-expert inference serving.