Organizers of the International Verification of Neural Networks Competition (VNN-COMP) presented its 2026 results in Lisbon on 24 July 2026.[1] The competition compares tools that check whether a neural network has a given property. A property is a precise statement of how the network must behave for every input in a defined range. Each tool tries to prove that the property holds or to find an input that violates it.[3, 4]
Thirteen tools were scored. The regular track had 24 benchmarks and 3,373 instances, each instance a single verification problem. α,β-CROWN ranked first there with a score of 2,344.0. Vibecheck followed with 2,278.6 and PyRAT with 1,813.6. The three finished in the same order in the six-benchmark extended track. The scores and scoring report went up on GitHub on 3 August.[2]
Each correct answer earns 10 points and each incorrect one costs 150. Each benchmark's points are converted to a percentage of the best tool's score on that benchmark. A tool's total is the sum of those percentages.[3]
A tool that reports a violation must supply a counterexample, an input that violates the property. The organizers check it by running the network. An honor code asks tools to report that a property holds only when they have proved it. The scoring script penalizes such an answer only when another tool supplies a valid counterexample.[2, 3] The scoring report lists one incorrect result for α,β-CROWN, a counterexample that failed the check, and none for Vibecheck or PyRAT.[2]
Vibecheck came from a team led by Stanley Bak of Stony Brook University. The university says the tool also won the competition's Best Newcomer Tool award. According to the university, the team generated the code by prompting Anthropic's Claude. Bak says he started about two months before the competition. He likens the work to supervising a team whose code he does not read line by line: "Instead I demand evidence of correctness."[4] Bak is one of this year's organizers. The rules let organizers enter tools but not win cash prizes.[3]
Sources
Check or correct this page
Verification record
The digest below shows whether the text you are reading matches the published source. It does not show that the reporting is correct. The build computes it from this page's Markdown source and lists it in the site manifest.
ebf70141d46e657c50d505ef5d47c0257faf439c2edbf87f5995dc584531b490
.jpg){kind=link}