1 Each word → an embedding vector
2 Multiply by learned matrices → Query, Key, Value
Each word's embedding is multiplied by three matrices to produce a Query, a Key, and a Value vector.
3 Score = Query · Key ÷ √d
4 Softmax → attention weights
5 Output = weighted sum of Values
These weight matrices are illustrative — hit “Randomize” to see how they change the focus. In a trained Transformer, Wq/Wk/Wv are learned, so attention lands on the words that actually matter.