Reinforcement learning in large language models often relies on off-policy methods due to discrepancies between training and inference, necessitating trust-region control for stable optimization. Traditional approaches like PPO and GRPO utilize ratio-clipping to approximate this control, but the importance ratio can be problematic. Recent research reexamines the divergence regularization in LLM RL, highlighting the need for more effective trust-region control methods1. The importance ratio can lead to suboptimal performance and instability, which can have significant security implications. As LLMs continue to evolve through reinforcement learning, the risk surface expands, and security concerns escalate. The development of more robust trust-region control methods is crucial to mitigate these risks. So what matters to practitioners is that refining LLM RL methods can help mitigate the escalating security risks associated with these powerful models.