flâneur

[2304.10464] Learning to Program with Natural Language

arxiv.org · 6,872 words · saved by 1 readers

Large Language Models (LLMs) have shown remarkable performance in various basic natural language tasks, which raises hopes for achieving Artificial General Intelligence. To better complete complex tasks, we need LLMs to program for the task and then follow the program to generate a specific solution for the test sample. We propose using natural language as a new programming language to describe task procedures, making them easily understandable to both humans and LLMs. LLMs are capable of directly generating natural language programs, but these programs may still contain factual errors or incomplete steps. Therefore, we further propose the Learning to Program (LP) method to ask LLMs themselves to learn natural language programs from the training dataset of complex tasks and then use the learned program to guide inference. Our experiments on the AMPS (high school math) and Math (competition mathematics problems) datasets demonstrate the effectiveness of our approach. When testing ChatGPT on 10 tasks from the AMPS dataset, our LP method's average performance outperformed the direct zero-shot test performance by 18.3$\%$. We release our code at \url{this https URL}.

Learning to Plan by Updating Natural Language Yiduo Guo1, , Yaobo Liang2 , Chenfei Wu2 , Wenshan Wu2 , Dongyan Zhao1 , Nan Duan2 1 Wangxuan Institute of Computer Technology, Peking University 2 Microsoft Research Asia…

related reading