GLOSSARY > SELF-ATTENTION
Self-Attention
A mechanism that relates different positions of a single sequence to compute a representation of the sequence.
Self-Attention allows an AI model to look at every other word in a sentence to deeply understand the context of the current word. This mechanism is what gives Large Language Models their unprecedented ability to perform semantic reasoning and disambiguate complex homonyms based on surrounding structural context.
Explore More