Value generalisation is considered crucial for AI alignment, as it enables agents to adapt to new situations and correct their reward functions. A specific example of value correction in reinforcement learning involves an agent recognising its current reward function is likely incorrect and taking action to correct it. This process occurs in two phases: the initial situation, where a human demonstrates how to maximise the true reward, and the out-of-distribution situation, where the agent discovers a way to exploit its reward function. The agent's ability to correct its reward function is significant, as it allows for more effective alignment with human values1. This capability is essential for ensuring that AI systems behave in a way that is consistent with human intentions, rather than exploiting loopholes in their programming. The development of value generalisation and correction mechanisms is critical for advancing AI alignment, and has significant implications for the safe and effective deployment of AI systems.