Mixture of Experts (MoE)
Mixture of experts is a neural network design that splits the model into many specialised sub-networks (experts) and activates only a few of them for each input.
What is Mixture of Experts (MoE)?
MoE allows a model to have a very large total number of parameters while keeping the computation per token much lower. Many frontier and open-weight models use it.
How it works
A gating or router network chooses which experts handle each token, and their outputs are combined.
← All Artificial Intelligence & Superintelligence definitions · ← Back to the A–Z definitions