The mechanism in transformer models that scores how much each word in a passage matters to every other word, so meaning is read in context.
Self-Attention lets a model weigh the other tokens in a sequence when computing the representation of each token, so every word is read in context. This mechanism gives Large Language Models their ability to reason about meaning and to distinguish between senses of the same word from the words around it.
See also Large Language Model, Fine-Tuning and Model Inference.
