Google pilots world's first double-blind AI evaluations

Google DeepMind, in partnership with the Singapore AI Safety Institute, OpenMined, AVERI, and MLCommons, has piloted the world's first double-blind evaluation of a proprietary frontier-class AI model. The evaluation uses Google Cloud's Confidential Space to create a cryptographically secure environment where external evaluators cannot see the model's weights and Google cannot see the evaluators' test prompts. This prevents benchmark contamination, where models might have seen test questions in advance, and enhances the integrity of AI benchmarks. The pilot tested a Gemini Flash Lite model against confidential benchmarks, marking a significant step in secure model evaluation and building trust in AI systems.