Mesa-Optimisation
Mesa-optimisation occurs when a trained model is itself an optimiser, pursuing its own internal objective (the mesa-objective), which may differ from the training objective.
What is Mesa-Optimisation?
The concept, from a 2019 paper, explains how inner alignment failures could arise in capable models.
How it works
Training selects for internal search processes that work well, without guaranteeing they optimise for the intended goal.
← All Artificial Intelligence & Superintelligence definitions · ← Back to the A–Z definitions