Direct Preference Optimisation
Direct preference optimisation is a method for training a language model on human preference data directly, without building a separate reward model or running reinforcement learning.
What is Direct Preference Optimisation?
Introduced in 2023, it is simpler and more stable than classic RLHF and is widely used for open models. It is often abbreviated DPO, not to be confused with a data protection officer.
How it works
The model is trained so that preferred responses become more likely and rejected ones less likely, relative to a reference model.
← All Artificial Intelligence & Superintelligence definitions · ← Back to the A–Z definitions