Words only make sense in context. The word "bank" means something completely different in "river bank" versus "bank account" — and you figured that out instantly by glancing at the surrounding words. Self-attention is the mechanism that lets AI do the same thing. For every word in a sentence, it asks: "Which other words should I pay the most attention to right now?" It then draws invisible connections of relevance between them — stronger connections for words that matter more to each other's meaning. Hover over any word below to see those connections light up.
Hover over any word — the arcs show which other words it "pays attention to." Thicker, brighter arcs mean a stronger connection. Words highlighted in gold are the top targets.
Why this matters
Before self-attention was invented in 2017, AI models read text one word at a time — like reading a book through a keyhole. They constantly lost track of earlier context. Self-attention lets the model look at the entire sentence at once and weigh every word against every other word simultaneously. That single breakthrough is what made ChatGPT, Claude, and every other modern AI assistant possible. It's the engine inside every transformer model in use today.
Hover over a word to view mathematical attention breakdown