Simulating low-resource experiments - Mathias Müller

Low-resource machine translation has been an active area of research for years. On a high level, what many papers on low-resource MT have in common is that theysimulate low-resource scenarios.

Low-resource machine translation has been an active area of research for years. On a high level, what many papers on low-resource MT have in common is that they simulate low-resource scenarios . What I mean is that many papers take a data set for a high-resource language pair (such as DE-EN) and randomly subsample until they arrive at the desired size. As an example, consider Gu et al. (2018) . I believe I can single them out here because the paper makes very strong claims about “very low-resource machine translation”. As they state in the abstract: Our approach is able to achieve 23 BLEU on R

related reading

• transformer_attention.pdfarxiv.org
• [2005.14165] Language Models are Few-Shot Learnersarxiv.org
• [2605.12715] Scaling Laws for Mixture Pretraining Under Data Constraintsarxiv.org
• [2106.09685] LoRA: Low-Rank Adaptation of Large Language Modelsarxiv.org
• 2026.mellm-1.2.pdfaclanthology.org
• Fast Inference from Transformers via Speculative Decodingarxiv.org
• [1609.08144] Google’s Neural Machine Translation System: Bridging the Gap between Human and Machine Translationarxiv-vanity.com
• A Bitter Lesson for Data Filteringarxiv.org
• 1301.3781arxiv.org
• 1706.03762arxiv.org
• [2509.14786] Pre-training under infinite computearxiv.org
• bleu.dviaclanthology.org