Batch Size
Batch size is the number of training examples a model processes before it updates its parameters once.
What is Batch Size?
Larger batches use parallel hardware efficiently and give smoother updates but need more memory; smaller batches add noise that can help generalisation. Batch size interacts with the learning rate.
How it works
The training set is divided into batches; the loss is averaged over each batch and one gradient update is made per batch.
← All Artificial Intelligence & Superintelligence definitions · ← Back to the A–Z definitions