Does VLA Even Know the Basics? Measuring Commonsense and World Knowledge Retention in Vision-Language-Action Models

Vision-Language-Action models, which combine visual, linguistic, and action-based inputs, often struggle to retain basic commonsense and factual knowledge after being fine-tuned on robotics data. This limitation can lead to failures in tasks that require a strong understanding of the physical world, making it difficult to distinguish between knowledge gaps and poor control generalization. To address this issue, researchers have developed Act2Answer, a protocol designed to assess the knowledge retention of these models. By evaluating their performance on a range of tasks, Act2Answer provides insight into the strengths and weaknesses of Vision-Language-Action models, highlighting areas where they may require additional training or fine-tuning¹. This has significant implications for the development of more robust and reliable models, particularly in applications where accurate knowledge retention is critical. The ability of these models to understand and interact with their environment effectively is crucial, so understanding their knowledge limitations is essential for advancing the field.