Self-Supervised Learning
Self-supervised learning is a way of training models on unlabelled data by creating the training signal from the data itself, for example by hiding part of an input and asking the model to predict it.
What is Self-Supervised Learning?
Large language models are trained this way by predicting the next word (token) in vast amounts of text, and image models can learn by predicting masked parts of pictures. Because no human labelling is needed, self-supervised learning can use internet-scale datasets and underpins today's foundation models.
How it works
The data supplies its own labels: the hidden word, the next frame of video or the missing patch of an image is the target the model learns to predict.
← All Artificial Intelligence & Superintelligence definitions · ← Back to the A–Z definitions