Researchers have uncovered a critical flaw in the widely-used LLM-as-a-judge paradigm, which assumes that automated judges evaluate text solely on its semantic content. However, it appears that these judges are susceptible to stakes signaling, a vulnerability where knowledge of downstream consequences can influence their evaluations. This means that the context in which text is presented can significantly impact the judgment, rather than just the content itself. The study reveals that informing a judge model of the potential consequences of its evaluation can lead to biased outcomes1. This finding has significant implications for the development and deployment of AI evaluation pipelines, as it suggests that current methods may not be as objective as previously thought. The discovery of this vulnerability matters to practitioners because it highlights the need for more robust and context-aware evaluation frameworks to ensure the reliability and fairness of automated judging systems.