Scaling laws describe how AI models get predictably better when model size, data and compute increase.
Example
Before an expensive training run, researchers estimate roughly how good a model ten times larger will be.
For pros
Studies from 2020 and 2022 (“Chinchilla”) showed that the error roughly follows a power law and that parameters and training data should grow in the right ratio. These are empirical observations, not laws of nature. Newer approaches also scale the compute spent at answer time (test-time compute).
