[1606.03126] Key-Value Memory Networks for Directly Reading Documents
Directly reading documents and being able to answer questions from them is an unsolved challenge. To avoid its inherent difficulty, question answering (QA) has been directed towards using Knowledge Bases (KBs) instead, which has proven effective. Unfortunately KBs often suffer from being too restrictive, as the schema cannot support certain types of answers, and too sparse, e.g. Wikipedia contains much more information than Freebase. In this work we introduce a new method, Key-Value Memory Networks, that makes reading documents more viable by utilizing different encodings in the addressing and output stages of the memory read operation. To compare using KBs, information extraction or Wikipedia documents directly in a single framework we construct an analysis tool, WikiMovies, a QA dataset that contains raw text alongside a preprocessed KB, in the domain of movies. Our method reduces the gap between all three settings. It also achieves state-of-the-art results on the existing WikiQA benchmark.
Key-Value Memory Networks for Directly Reading Documents Alexander H. Miller1 Adam Fisch1 Jesse Dodge1,2 Amir-Hossein Karimi1 Antoine Bordes1 Jason Weston1 1 Facebook AI Research, 770 Broadway, New York, NY, USA 2 Language Technologies Institute,…
saved by
related reading
- Key-value memory in the brainarxiv.org
- How can we develop transformative tools for thought?numinous.productions
- [2005.11401] Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasksarxiv.org
- Augmenting Long-term Memoryaugmentingcognition.com
- llm-wikigist.github.com
- Augmenting Long-term Memoryaugmentingcognition.com
- [2605.15156] MeMo: Memory as a Modelarxiv.org
- Language Models as Knowledge Bases?aclanthology.org
- Cerebras (@cerebras) on Xx.com
- KV Caching Explained: Optimizing Transformer Inference Efficiencyhuggingface.co
- [2002.08909] REALM: Retrieval-Augmented Language Model Pre-Trainingarxiv.org
- Andrej Karpathy on X: "LLM Knowledge Bases Something I'm finding very useful recently: using LLMs to build personal knowledge bases for various topics of research interest. In this way, a large fraction of my recent token throughput is going less into manipulating code, and more into manipulating" / Xx.com