👁️ Where AI looks: the attention mechanism
To understand a phrase, AI 'looks' at related words — that's the attention mechanism. Tap the highlighted word and see which words the model pays attention to in order to grasp the meaning.
👆 Tap the word 'it' — see what it refers to
Thetrophydidn'tfitinthesuitcasebecausewastoobig.
💡 That's what 'attention' is
The attention mechanism is the heart of the transformer (the T in GPT). For every word, the model decides which other words to 'look at' to understand the meaning. That's how AI works out that 'it' is the 'trophy', not the 'suitcase'. Thousands of such links at once — and language understanding emerges.