A mechanism that relates different positions of a single sequence to compute a representation of the sequence.
Self-Attention lets a model weigh the other tokens in a sequence when computing the representation of each token, so every word is read in context. This mechanism gives Large Language Models their ability to reason about meaning and to distinguish between senses of the same word from the words around it.
See also Large Language Model, Fine-Tuning and Model Inference.