Epoch

When studying for an exam, going through an entire prep book from the first page to the last without skipping anything is called completing one full pass. AI uses the exact same concept when learning from data.

Imagine teaching an AI to recognize cats using 10,000 photos. Once the AI has examined all 10,000 photos, compared its guesses against the correct labels, and checked its mistakes, it has completed one epoch of training. If you train it for 10 epochs, the AI has reviewed that entire set of 10,000 photos 10 times from start to finish.

Unlike humans, computers cannot fully grasp complex patterns in photos or text after seeing them just once. Therefore, in most cases, an AI needs to cycle through the entire dataset multiple times to fine-tune the connection strengths in its neural network. However, when a dataset is extraordinarily massive, training for around a single epoch can be enough.

To be more precise, an AI cannot digest 10,000 photos all at once. Computer memory has limits, and calculating the entire dataset in one single sweep would give the model only one chance per epoch to correct errors, slowing down learning.

Instead, developers break the dataset into manageable bite-sized chunks and feed them to the computer one after another. The size of each chunk is called the batch size. If you train on 10,000 photos in groups of 100, processing one group and updating the model is called a single "step" or "iteration."

After completing 100 consecutive steps—finally covering all 10,000 photos—the model achieves one single epoch. It works just like dividing a thick workbook into chapters and completing a few pages each day until the whole book is done.

You might assume that solving a workbook hundreds of times always guarantees higher scores, but machine learning does not work that way. Setting too few epochs causes underfitting, where the model fails to learn even basic patterns.

On the other hand, running too many epochs can lead the AI to memorize the workbook word for word instead of understanding the underlying rules. In technical terms, this is called "overfitting." For example, the AI might memorize random background stains or unique lighting in cat photos as requirements for being a cat, causing it to fail when presented with brand-new cat pictures.

To prevent this, developers test the model on separate mock exam data during training. If mock test scores stop improving over several epochs, training stops automatically and saves the model from its peak performance. This technique is called early stopping, and it helps developers find the optimal amount of training.