✳flâneur — a map of the web's best reading
What Is Attention in Language Models?
txt.cohere.ai · 79 words · saved by 2 readers
A huge roadblock for language models is when a word can be used in two different contexts. When this problem is encountered, the model needs to use the context of the sentence in order to decipher which meaning of the word to use. This is precisely what self-attention models do.
Explore this link on the map →