IndicSafe: A Benchmark for Evaluating Multilingual LLM Safety in South Asia

Researchers have introduced IndicSafe, a benchmark for assessing the safety of large language models (LLMs) in multilingual settings, specifically across 12 Indic languages spoken by over 1.2 billion people. This benchmark is significant because these languages are underrepresented in LLM training data, making it crucial to evaluate their safety behavior in culturally diverse contexts. The evaluation uses a dataset of 6,000 culturally grounded prompts that cover sensitive topics such as caste and religion. The study aims to systematically assess LLM safety in these languages, which is essential for ensuring that AI systems do not perpetuate harmful biases or stereotypes¹. By focusing on Indic languages, this research highlights the need for more inclusive and diverse training data to improve LLM safety in multilingual settings. This matters to practitioners because it underscores the importance of considering cultural and linguistic diversity when developing and deploying AI systems, particularly in regions with limited representation in training data.