💬

Word Predictor

A neural net reads 4 words and predicts the next — learning word embeddings as it goes. The idea behind language models.

Training data

Epoch
0
Loss
Perplexity
Log-likelihood
Loss (cross-entropy) per epoch

Predict the next word

Your 4 words
Model's next word

The network — watch the weights change as it learns

Blue = positive weight, orange = negative; thicker = stronger. Output dots light up with the predicted next word. Pause training anytime to inspect.

Word embedding map

Each word is a point. As the model learns, words it uses the same way move together.