Generative Pre-Trained Transformer (GPT)
A generative pre-trained transformer is a type of large language model that is pre-trained on large text corpora to predict the next token and then generates text one token at a time.
What is Generative Pre-Trained Transformer (GPT)?
GPT is the name of OpenAI's model family, which powers ChatGPT, but the design is now used across the industry. The name summarises the recipe: generative output, large-scale pre-training and the transformer architecture.
How it works
A decoder-only transformer reads the text so far and predicts a probability for each possible next token, samples one, appends it and repeats.
← All Artificial Intelligence & Superintelligence definitions · ← Back to the A–Z definitions