Andrew Trask and coauthors submitted Double Blind Evals: Resolving the Dual Confidentiality Dilemma in AI Safety Auditing to arXiv on 9 November 2026. The preprint reports a pilot intended to keep evaluation prompts confidential from a model owner while preserving the confidentiality of proprietary model weights and inference code.[1]
The abstract describes competing requirements: evaluators want secret prompts to avoid deliberate or incidental leakage and preserve benchmark usefulness; model owners want weights and inference code to remain within their own secure infrastructure. The authors propose secure enclaves, described as encrypting data in RAM at the hardware level, to support this arrangement.[1]
For the pilot, the authors report evaluating Google DeepMind’s Gemini 2.5 Flash Lite against a selection of the private AILuminate benchmark corpus. They identify an NVIDIA H100 secure enclave on Google Cloud Platform and OpenMined’s PySyft as components. The abstract claims cryptographic guarantees for prompt confidentiality alongside confidentiality of weights and inference code, and describes the pilot as a first. These are author claims; the saved abstract supplies no evaluation scores, detailed threat model or independent validation.[1]
[1]: Double Blind Evals: Resolving the Dual Confidentiality Dilemma in AI Safety Auditing. arXiv, 9 November 2026. https://arxiv.org/abs/2611.12604
Sources
Check or correct this page
Verification record
The digest below shows whether the text you are reading matches the published source. It does not show that the reporting is correct. The build computes it from this page's Markdown source and lists it in the site manifest.
6f84833b1f63b72ede7c63f8ddd293ffef9db26ae8babb1e461ad3688760878d