Mixture of Experts

• Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer (2017)
• Mixture-of-Experts with Expert Choice Routing (2022)