From Values to Benchmarks: Evaluating Large Language Models for Governmental Use in Dutch

The deployment of large language models in governmental settings has sparked the need for evaluation frameworks that align with public administration values and linguistic requirements, particularly in non-English contexts. Researchers have developed the "Grip on LLMs" framework, a systematic evaluation suite designed for Dutch governmental use, in collaboration with domain experts from a major Dutch municipality. This framework aims to assess the suitability of large language models for governmental applications, considering factors such as language understanding, transparency, and accountability. The "Grip on LLMs" framework is specifically tailored to the Dutch language and context, addressing the unique challenges of non-English language models¹. The development of this framework has significant implications for the effective and responsible use of AI in governmental settings, as it enables policymakers to make informed decisions about the adoption of large language models. This matters to practitioners as it highlights the importance of context-specific evaluation frameworks in ensuring the reliable and secure deployment of AI technologies.