✳flâneur — a map of the web's best reading

LLM Samplers Explained

gist.github.com · 764 words · saved by 1 readers

Everytime a large language model makes predictions, all of the thousands of tokens in the vocabulary are assigned some degree of probability, from almost 0%, to almost 100%. There are different ways you can decide to choose from those predictions. This process is known as "sampling", and there are various strategies you can use which I will cover here. Leave a comment We don’t support that file type. Try again with GIF, JPEG, JPG, MOV, MP4, PNG, SVG or WEBM. Attaching documents requires write permission to this repository. Try again with GIF, JPEG, JPG, MOV, MP4, PNG, SVG or WEBM. This file is empty. Try again with a file that’s not empty. This file is hidden. Try again with another file. Something went really wrong, and we can’t process that file. Try again. Nothing to preview

LLM Samplers Explained Everytime a large language model makes predictions, all of the thousands of tokens in the vocabulary are assigned some degree of probability, from almost 0%, to almost 100%. There are different ways you can decide to choose from those predictions. This process is known as "sampling", and there are various strategies you can use which I will cover here. OpenAI Samplers Temperature Temperature is a way to control the overall confidence of the model's scores (the logits). What this means is that, if you use a lower value than 1.0, the relative distance between the tokens wi

Explore this link on the map →

related reading