Google DeepMind said on 27 August 2026 that it had piloted what it calls the first double-blind evaluation of a proprietary frontier-class model. Its partners were the Singapore AI Safety Institute, OpenMined, the AI Verification and Evaluation Research Institute (AVERI) and MLCommons.[1] DeepMind says Google Cloud's Confidential Space kept the evaluators from seeing the Gemini weights and Google from seeing their test prompts.[1]
The technical report names the model as Gemini 2.5 Flash-Lite. It was served on a Google Cloud virtual machine with Intel Trust Domain Extensions (TDX) memory encryption and an NVIDIA H100 GPU in confidential mode. OpenMined's PySyft software ran only code that both parties had approved. Before sending its assets in, each party checked the enclave's hardware attestation, signed evidence of the software the enclave had loaded.[2]
For one track, MLCommons supplied prompts from a reserved set of its AILuminate safety benchmark that no Google DeepMind model had seen.[2, 4] AVERI encrypted the prompts, and the run took place in an enclave Google configured. Afterward, AVERI graded the decrypted outputs and reported confidentially to Google DeepMind.[3] A second track, with the Singapore institute, used a private prompt set on harmful content in Singapore's context. Neither the report nor AVERI's post publishes benchmark scores.[2, 3]
The report also states limits: not all of the Gemini inference code could be inspected or restricted to an allowlist of approved operations. It says AVERI accepted the setup knowing this. The guest operating system's source is open, but its builds cannot be reproduced independently because they take private signing keys as inputs. Google's services sign and verify the attestation report, which the authors say places Google in the verification path.[2] AVERI writes that the design addressed many but not all ways the developer could in theory tamper with the evaluation. It adds that less trusting settings, such as verifying an international agreement, would need more assurances.[3]
Sources
Check or correct this page
Verification record
The digest below shows whether the text you are reading matches the published source. It does not show that the reporting is correct. The build computes it from this page's Markdown source and lists it in the site manifest.
6be4aabd4591dd466ba84809e18bba19d7e2ec1afa0cf32a4c4c8c12c27ba0ff
.jpg){kind=link}