Top-p (Nucleus Sampling)

Top-p, or nucleus sampling, limits the model's choices to the smallest set of top tokens whose probabilities add up to p, then samples from just that set. Because it's defined by probability mass (a value between 0 and 1), the number of candidate tokens flexes automatically: when the model is confident, the nucleus might be a handful of tokens; when it's unsure, many more.

A low p (say 0.1) keeps only the most likely tokens — tight, conservative output. A high p (say 0.9) admits a wider vocabulary — more variety. It's an alternative or complement to temperature for tuning randomness.

The key contrast with top-k is that top-p adapts to the model's confidence, whereas top-k always keeps a fixed number of candidates.