Pretraining Data Can Be Poisoned through Computational Propaganda

Poisoning pretraining data can compromise the integrity of large language models (LMs) by introducing harmful behaviors that are challenging to identify and rectify. Recent research has shown that computational propaganda can be used to contaminate pretraining data, potentially leading to the dissemination of misleading information¹. This vulnerability is particularly concerning given the vast scale and diversity of pretraining corpora, which can be exploited to spread malicious content. The interaction between poisoned data and data curation pipelines further exacerbates the issue, making it difficult to detect and mitigate the effects of poisoned data. As a result, LMs may inadvertently perpetuate harmful behaviors, posing significant risks to security, policy, and workforce dynamics. This highlights the need for robust data validation and curation protocols to prevent the poisoning of pretraining data, so what matters most to practitioners is the development of effective countermeasures to safeguard against such threats.