microsoft/guidance: A guidance language for controlling large language models.

github.com · 1,544 words · saved by 2 readers

A guidance language for controlling large language models.

Guidance is an efficient programming paradigm for steering language models. With Guidance, you can control how output is structured and get high-quality output for your use case—while reducing latency and cost vs. conventional prompting or fine-tuning. It allows users to constrain generation (e.g. with regex and CFGs) as well as to interleave control (conditionals, loops, tool use) and generation seamlessly. Install Features Install Guidance is available through PyPI and supports a variety of backends (Transformers, llama.cpp, OpenAI, etc.). If you already have the backend required for…

microsoft/guidance: A guidance language for controlling large language models.

saved by

related reading