A transformer is the blueprint behind almost all of today's language models. It looks at all the words in a text at once and works out which ones belong together.
Illustration: AI-generated · AI Compare Ninja · CC BY 4.0
Example
In “The cat drank because it was thirsty”, the transformer works out that “it” means the cat.
For pros
Its core is the attention mechanism: every token weighs all other tokens in the context. The architecture was introduced in 2017 in the paper “Attention Is All You Need”. Compute grows steeply with context length, which is why many more efficient variants exist.
