Growing text

Prompt tokens have a neutral border. Tokens generated by the toy model use the accent border. Each new token becomes part of the context for the next prediction.

Current text
–

Illustrative next-token probabilities

A token is sampled from the distribution shown here, appended to the text, and then a new distribution is produced for the next step. The probabilities are hand-built for this explorable.

Context tokens
–
Generated tokens
0
Most likely next token
–
Last sampled token
–