The Gap Between Open and Closed AI Models in Cyber Attacks Keeps Shrinking

The latest data points in one direction: the gap between closed and open AI models in cyber attack tasks keeps narrowing. According to The Decoder, in the Vals AI benchmark, which simulates attacks on corporate networks, OpenAI's closed model GPT-6 Sol Daybreak reached 80. percent of attack objectives. The open-weight model DeepSeek V4. Flash reached 72. percent at a tenth of the cost. The difference in results is therefore small, and a cheap alternative is close. Model names and results come from The Decoder's report. Before making decisions, check the original Vals AI report.

Why open models change the rules

A model with open weights can be downloaded, run locally and modified. The provider therefore cannot withdraw access or switch off restrictions built into the model. According to The Decoder, analysts believe the time window for defenders is shrinking fast. For companies, this means a change in thinking: security based on the assumption that the best attack tools are closed and expensive is no longer reliable.

EU regulation points in a similar direction. The AI Act (Regulation EU 2024/1689) requires providers of general-purpose AI models with systemic risk to assess and mitigate that risk, including risks related to cybersecurity. The exemptions foreseen for open-source models do not, however, cover models with systemic risk. Regulation also does not relieve companies of responsibility for their own defense.

How to prepare the defense

Buying another tool alone is not enough. It is worth starting with four steps:

1. Test your own infrastructure with automated penetration tests, not only with manual audits once a year. If an attack can be run at scale, the defense must be repeatable too.
2. Shorten the time from discovering a vulnerability to patching it. Set priorities based on whether the vulnerable system is reachable from the internet and on the impact on data.
3. Limit the consequences of a breach. Network segmentation and the principle of least privilege mean that an automated attack runs into a dead end instead of searching the whole organization.
4. Monitor machine-like traffic, meaning very fast series of requests, repeated login attempts and unusual account activity speeds.

The simplest trap is assuming that a provider's closed model guarantees security. The benchmark shows that the advantage is small. The second trap is buying tools without changing processes, responsibilities and team training.

Summary

Data from the Vals AI benchmark, reported by The Decoder, suggests that the advantage of closed models in cyber attacks is shrinking quickly. Companies that treat this as a signal to act gain time. Strengthening defense does not require predicting every threat. Fast testing, efficient patching and limiting the impact of a possible breach are enough to start.

Lub System helps B2B companies implement AI, automation and IT solutions end-to-end - from strategy to deployment. See our services or get in touch to discuss your case.