Reward Model

A reward model is an AI model trained to predict how highly humans would rate a given output, used as an automated judge when training another model.

What is Reward Model?

Because people cannot rate every output, a reward model scales their judgements. The model being trained may learn to exploit flaws in the reward model, a form of reward hacking.

How it works

It is trained on human preference comparisons to output a score for any response.

Related terms

• Reinforcement Learning from Human Feedback (RLHF)
• Reward Function
• Reward Hacking

← All Artificial Intelligence & Superintelligence definitions · ← Back to the A–Z definitions