Mixture of experts is a model design in which only part of the model works on each request, like calling in the right specialists.
Example
An MoE model has 100 billion parameters but uses only about 10 billion of them per word.
For pros
A router picks a few expert networks for each token. This lets a model have far more parameters in total without compute per token rising at the same rate. All parameters still have to fit in memory, which makes local use harder.
