The White House has outlined a voluntary system for testing the cybersecurity capabilities of advanced AI models before release, but the benchmarks themselves will remain classified. That combination—government oversight with limited public scrutiny—has created concern that the framework may privilege the largest AI companies while leaving researchers and smaller developers unable to judge how risk is measured.
According to Wired, the administration invited representatives from OpenAI, Anthropic, Google, Meta, Nvidia and other companies to hear about the plan. Developers could submit models to the federal government up to 30 days before public release. Agencies would test cyber capabilities and may share models with trusted government and corporate partners. Open models are reportedly excluded.
A voluntary gate with secret criteria
Pre-release testing could give federal defenders early warning about systems able to automate vulnerability discovery or offensive operations. A common benchmark might also make comparisons more consistent than separate company evaluations. The source does not establish which models will qualify, which agencies will participate or what happens when a model performs poorly.
Voluntary submission is the first limitation. Companies facing intense commercial pressure can decide whether the benefits of participation outweigh delay or scrutiny. Thirty days may also be short for investigating complex behavior and agreeing on mitigations, particularly if testing reveals a new capability rather than a known vulnerability.
Secrecy has a legitimate security rationale: publishing detailed offensive benchmarks could reveal attack techniques. Yet classifying the entire standard prevents outsiders from evaluating whether the tests are rigorous, representative or applied equally.
Smaller labs may face an uneven field
The invited companies possess extensive security teams and established government relationships. A framework designed around their models and infrastructure could become difficult for startups to navigate. One person familiar with the discussions told Wired that the approach risks entrenching providers already treated as frontier leaders.
Excluding open models introduces another gap. Open releases can be adapted by users beyond the developer’s control, making cyber evaluation important, but pre-release access and enforcement work differently when weights are published. Ignoring that category does not remove its risks; it leaves them to a separate and still unclear process.
The exclusion may also distort comparisons. A closed provider can offer controlled access to a government team, whereas an open developer may publish components through a distributed community. The framework needs criteria based on capability and deployment risk rather than corporate structure alone.
Third-party researchers also need enough information to challenge government and company conclusions. Independent testing has repeatedly exposed failures that internal teams missed. A closed circle of agencies and selected corporate partners may protect sensitive results, but it can also narrow the range of assumptions being tested.
Oversight needs accountable boundaries
The government does not need to publish exploit instructions to explain its governance. It could disclose capability categories, submission thresholds, evaluation procedures, conflict rules and aggregated findings. Companies should know what evidence is expected, while the public should know how decisions are made.
The framework also needs a response ladder. Testing has little value without clear consequences, whether those involve additional safeguards, restricted access, delayed deployment or monitoring. Because participation is voluntary, transparency about compliance becomes even more important.
Appeals and correction procedures matter too. Benchmarks can produce false positives or miss a capability that appears only under a particular tool configuration. Developers should be able to challenge a result without negotiating private exceptions unavailable to competitors.
Industry figures have warned that advanced models may change the balance between attackers and defenders. Those projections remain uncertain, but recent agent incidents show why controlled cyber evaluation deserves urgency. The challenge is to build a system that improves security without converting privileged access into permanent regulatory advantage.
A classified benchmark may be one component of that system, not its entire foundation. Credible oversight requires secret technical details where necessary and public accountability everywhere else. Without that division, the framework may test models while leaving the policy itself insufficiently tested.
