Why K and V are not the same in Transformer attention?

My understanding is for translation task K should be the same with V, but in Transformer K and V are generated by two different(randomly initialized) matrix $W^K, W^V$, therefore not the same. Can ...

neural networks - Why K and V are not the same in Transformer attention? - Cross Validated Stack Internal Knowledge at work Bring the best of human thought and AI automation together at your work. Explore Stack Internal Why K and V are not the same in Transformer attention? Ask Question Asked 6 years, 9 months ago Modified 2 years, 5 months ago Viewed 13k times 15 $\begingroup$ My understanding is for translation task K should be the same with V, but in Transformer K and V are generated by two different(randomly initialized) matrix $W^K, W^V$ , therefore not the same. Can any one tell me why?

related reading

• neural networks - What exactly are keys, queries, and values in attention mechanisms? - Cross Validatedstats.stackexchange.com
• The Illustrated Transformer – Jay Alammar – Visualizing machine learning one concept at a time.jalammar.github.io
• transformer_attention.pdfarxiv.org
• A Mathematical Framework for Transformer Circuitstransformer-circuits.pub
• An Intuition for Attention | Jay Modyjaykmody.com
• The Illustrated Transformer – Jay Alammar – Visualizing machine learning one concept at a time.jalammar.github.io
• Transformers Explained Visually (Part 3): Multi-head Attention, deep dive | Towards Data Sciencetowardsdatascience.com
• Some Intuition on Attention and the Transformereugeneyan.com
• Ask HN: Can someone ELI5 transformers and the “Attention is all we need” paper? | Hacker Newsnews.ycombinator.com
• The math behind Attention: Keys, Queries, and Values matricesyoutube.com
• Transformers from Scratche2eml.school
• KV Caching Explained: Optimizing Transformer Inference Efficiencyhuggingface.co