BIG-bench/keywords_to_tasks.md at main · google/BIG-bench

Beyond the Imitation Game collaborative benchmark for measuring and extrapolating the capabilities of language models - BIG-bench/keywords_to_tasks.md at main · google/BIG-bench

This file maps descriptive keywords (and key phrases) to tasks described by that keyword. To add new keywords, directly edit keywords.md rather than this file. Summary table Keyword Number of tasks Description traditional NLP tasks contextual question-answering 22 identifying the meaning of a particular word/sentence in a passage context-free question answering 24 responses rely on model's knowledge base, but not on context provided during query time reading comprehension 36 a superset of contextual question-answering, measuring the degree to which a model understands the content of a…

saved by

• Belinda Mo

related reading

• BIG-bench/bigbench/benchmark_tasks/convinceme at main · google/BIG-bench · GitHubgithub.com
• [2005.14165] Language Models are Few-Shot Learnersarxiv.org
• Language Models can Solve Computer Tasksarxiv.org
• [2005.11401] Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasksarxiv.org
• KernelBench: Can LLMs Write GPU Kernels?scalingintelligence.stanford.edu
• TOD-Bertarxiv.org
• Generalized Language Models | Lil'Loglilianweng.github.io
• BigO(Bench) -- Can LLMs Generate Code with Controlled Time and Space Complexity?alphaxiv.org
• 1810.04805arxiv.org
• 2005.14165arxiv.org
• Continual Learning Bench 1.0 — Newscontinual-learning-bench.com
• ProgramBenchprogrambench.com