[2606.06428] Reinforcement Learning Elicits Contextual Learning of Unseen Language Translation
Abstract:Prior work has shown that large language models (LLMs) can translate unseen or low-resource languages by undergoing continued training or even by encoding a grammar book in their context. However, both methods typically overfit specific languages, with limited zero-shot transfer at test time. To translate extremely low-resource languages at scale, we argue that LLMs must acquire the meta-skill of utilizing in-context linguistic knowledge rather than memorizing specific languages. In this paper, we propose a reinforcement learning (RL) approach to unseen language translation given rich linguistic context, using a surface-level translation metric (chrF) as the reward. Empirically, despite the lightweight reward, our RL-trained models effectively extract and apply relevant linguistic information from the provided context, leading to better translations on completely unseen languages than in-context learning or supervised fine-tuning. Our analyses suggest that outcome-based RL can extend beyond conventional reasoning tasks like math and coding to serve as a recipe for language learning from context.
View PDF HTML (experimental) Abstract:Prior work has shown that large language models (LLMs) can translate unseen or low-resource languages by undergoing continued training or even by encoding a grammar book in their context. However, both methods typically overfit specific languages, with limited zero-shot transfer at test time. To translate extremely low-resource languages at scale, we argue that LLMs must acquire the meta-skill of utilizing in-context linguistic knowledge rather than memorizing specific languages. In this paper, we propose a reinforcement learning (RL) approach to unseen…
saved by
related reading
- 2026.mellm-1.2.pdfaclanthology.org
- Illustrating Reinforcement Learning from Human Feedback (RLHF)huggingface.co
- [2305.18290] Direct Preference Optimization: Your Language Model is Secretly a Reward Modelarxiv.org
- Recursive Language Models | Alex L. Zhangalexzhang13.github.io
- [2510.13786] The Art of Scaling Reinforcement Learning Compute for LLMsarxiv.org
- Why Does Reinforcement Learning Generalize? A Feature-Level Mechanistic Study of Post-Training in Large Language Modelsarxiv.org
- [2511.08579] Training Language Models to Explain Their Own Computationsarxiv.org
- Learning without training: The implicit dynamics of in-context learningarxiv.org
- Language Models can Solve Computer Tasksarxiv.org
- Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?arxiv.org
- Self-Adapting Language Modelsarxiv.org
- [2506.19733] Breaking Barriers: Do Reinforcement Post Training Gains Transfer To Unseen Domains?arxiv.org