Verifiable evaluation & audits

Frames the detection of quantized (reduced-precision), watermarked or fine-tuned models behind an API as a two-sample test. Such a test asks whether two sets of outputs come from the same distribution. The paper reports that 11 of 31 tested endpoints served distributions different from Meta's reference weights.

Auditing framework that checks a small fraction of API requests with verifiable computation, in which the provider returns evidence that a result is correct. It looks for model substitution, quantization abuse and token overbilling, without relying on trusted hardware.

Zero-knowledge proof scheme built on a new sum-check protocol for convolutions. A model owner can use it to prove that a convolutional neural network's prediction came from a secret model, without revealing the model. The scheme can also prove the model's accuracy on a public dataset. For VGG16, the proof took 88.3 seconds to generate and 59.3 milliseconds to verify.

Argues that white-box and outside-the-box access allow substantially more scrutiny than black-box queries. White-box access lets auditors examine a model's internals; outside-the-box access covers information such as training data and documentation. The paper also discusses technical, physical and legal safeguards for audits with deeper access.

Describes structured access, in which developers let others interact with an AI system in controlled ways, typically through cloud interfaces, instead of releasing it openly.

Proposes rigorous third-party auditing of frontier AI developers' safety and security claims. It also proposes four AI Assurance Levels (AALs), where higher levels give more confidence in an audit's findings. It recommends AAL-1 as a baseline and AAL-2 as a near-term goal for the most advanced developers.

Runs audit benchmarks inside trusted execution environments (TEEs). Users can then verify that they interacted with a compliant model, while model weights and benchmark data stay confidential. The prototype was evaluated on Llama-3.1.

Packages zk-SNARK proofs of a model's outputs on datasets into evaluation attestations. Each attestation shows that a model with fixed, private weights achieves stated performance or fairness metrics.

ZkAudit protocol, in which providers publish commitments, cryptographic values that bind them to specific datasets and model weights. They also publish a zero-knowledge proof of training. They then answer audit queries, such as copyright checks, with further proofs.

OpenMined's open-source library in which data scientists submit computations that data owners approve and run on private data, so that only approved results leave the owner's machine.

Describes how OpenMined's infrastructure combines privacy-enhancing technologies (PETs) so that outsiders can scrutinize AI systems. PETs let parties compute on data while limiting what each side sees. Case studies involve the Christchurch Call and the UK AI Safety Institute.

Pilot in which OpenMined, with staff from the UK AI Safety Institute and Anthropic, ran an AI evaluation inside an NVIDIA H100 secure enclave. GPT-2 and a five-row dataset sample stood in for confidential assets.

Report on a pilot in which Gemini 2.5 Flash-Lite was evaluated on unpublished MLCommons AILuminate prompts inside a secure enclave. The enclave was designed so that neither side saw the other's assets. AVERI notes that the design did not close every avenue for tampering.

Finds that software-only methods cannot reliably detect covert model substitution in LLM APIs. In model substitution, a provider secretly serves a different model from the one it advertises. The paper evaluates trusted execution environments (TEEs) as a way to guarantee model integrity.

Statistical test that can prove test-set contamination, meaning that a benchmark's test items were in a model's training data. It needs neither the training data nor the weights. It checks whether the model finds the benchmark's examples more likely in their original order than in shuffled orders.

Documents undisclosed private testing and selective score disclosure on Chatbot Arena, including 27 private variants that Meta tested before the Llama 4 release.

Open-source framework from the UK AI Security Institute and Meridian Labs for building and running LLM evaluations. An evaluation combines a dataset, a solver that gets the model's answers and a scorer that grades them. The framework supports agents, tool calling and sandboxed execution of model-written code, and offers more than 200 prebuilt evaluations.

Stanford CRFM benchmark that evaluated 30 language models on 42 scenarios under standardized conditions. It measured seven metrics, including accuracy, calibration, robustness, fairness and efficiency. It also released all raw prompts and completions.

EleutherAI's open-source framework for testing generative language models on more than 60 standard academic benchmarks with publicly available prompts. It aims to make results reproducible and comparable across papers.

Argues that AI companies' terms of service and their enforcement deter good-faith safety research. It proposes that major developers commit to legal and technical safe harbors. These would protect public-interest evaluation and red teaming from account suspension and legal reprisal.

Proposes three measures. Researchers would use standardized AI flaw reports and rules of engagement. Providers of general-purpose AI systems would run flaw disclosure programs with legal safe harbors. Infrastructure would coordinate how flaw reports reach the many stakeholders who may be affected.

International Standard, published in July 2025, for bodies that audit and certify AI management systems against ISO/IEC 42001. It adds requirements to those of ISO/IEC 17021-1. It can serve as a criteria document for accrediting such bodies.

Not-for-profit subsidiary of Singapore's Infocomm Media Development Authority (IMDA), launched in June 2023. It develops open-source AI testing tools, such as the AI Verify toolkit and Project Moonshot. It also runs an accreditation program for third-party AI testing firms.

NIST's Center for AI Standards and Innovation (CAISI) summarizes its evaluation of DeepSeek R1, R1-0528 and V3.1. It compared them with U.S. reference models on 19 public and private benchmarks. CAISI reports that the DeepSeek models performed worse, cost more and were more susceptible to agent hijacking and jailbreaking.

Multi-author report on how AI developers could make verifiable claims about the safety, security, fairness and privacy of their systems. It analyzes ten institutional, software and hardware mechanisms, such as third-party auditing, red teaming, audit trails and secure hardware for machine learning. It recommends steps to implement or improve them.

White House policy plan. Its recommended actions include exploring location verification features on advanced AI chips to enforce export controls. They also include having NIST's Center for AI Standards and Innovation (CAISI) evaluate frontier AI systems for national security risks and assess risks from adversaries' AI systems.