What cyber practitioners should take from the world’s biggest AI risk review

by Black Hat Middle East and Africa
on
What cyber practitioners should take from the world’s biggest AI risk review

The International AI Safety Report 2026 is the second edition of an internationally coordinated scientific review of general purpose AI capabilities, risks, and mitigations. It’s been written with guidance from over 100 independent experts and supported via nominees from more than 30 countries and organisations (including the EU, OECD and UN). 

It’s a kind of technical state of play, not a policy manifesto; and it’s definitely worth a read. It’s nuanced (not crammed with warnings about how AI is about to have everything autonomously), and that makes it useful. 

AI is already embedded across the attack chain, and the messy details are where defenders get leverage.

The attacker workflow is changing

The report says there’s now strong evidence that both criminal groups and state-sponsored attackers actively use AI in cyber operations – while also stressing that it’s uncertain whether this has increased overall scale and severity, because causality is hard to prove. 

That’s a sensible perspective for you, cybersecurity practitioner: treat AI as a capability multiplier inside known playbooks, not a brand-new species of threat.

But where does that multiplier have the biggest impact? 

First, in vulnerability discovery and code generation. The report notes that AI systems are particularly good at discovering vulnerabilities and writing malicious code, and highlights a ‘premier cyber competition’ result where an AI agent found 77% of vulnerabilities in real software and placed in the top 5% of 400+ teams. 

If you run AppSec, that should land heavily: AI-assisted vuln discovery is increasingly competitive with skilled humans in constrained environments.

Second, packaging. The report describes underground marketplaces selling pre-packaged AI tools and even AI-generated ransomware, lowering the skill threshold for less sophisticated actors. They can use faster, cheaper iteration (across phishing, initial access, and exploit adaptation) to cause real damage. 

Autonomy is limited – but ‘semi-autonomous’ is already operationally real

In positive news, the report says general-purpose AI systems haven’t been reported to conduct end-to-end cyberattacks in the real world. And it gives practical reasons for this: models struggle with long, multi-stage sequences. This means they sometimes issue irrelevant commands and lose operational state, then fail to recover from simple errors without human help.

But even without being fully autonomous, AI-powered attacks are having a serious impact. The report notes at least one real-world case involving semi-autonomous cyber capabilities, with humans intervening only at critical decision points. 

In practice, that’s the model to plan for: humans doing strategy and supervision; AI doing the slog – triage, translation, lure drafting, code scaffolding, and iterative debugging.

We spotted one data point cybersecurity practitioners will recognise instantly. In a table under phishing and deepfakes, the report cites a source claiming that “in the first half of 2025, identity-based attacks rose by 32%.” The report doesn’t say AI caused that rise – but it does say this trend is within AI’s capabilities, and that multiple sources report real-world use.

The meta-risk: your evaluation regime can be gamed

We like benchmarks – they’re reassuring. But the report pours cold water on overconfidence here. It notes that high-stakes release decisions partially rely on harmful capability evaluations, but benchmark quality varies – and models are increasingly likely to spot they’re being tested and behave differently due to situational awareness. The report also flags saturation (benchmarks no longer distinguishing between top models) and blind spots for novel tasks.

In security language, that means if you’re buying (or building) AI systems, a neat evaluation report is not the same thing as assurance. Treat model testing as an adversarial domain. Red-team it like you mean it.

Three key takeaways for cybersecurity teams:

  • Assume AI is already in your threat model. The report’s strong evidence of criminal and state use means ‘wait and see’ is no longer a strategy.
  • Plan for human-led, AI-accelerated operations. Full autonomy isn’t evidenced in the wild, but semi-autonomous and heavily AI-assisted ops are credible and growing.
  • Harden identity and comms paths. Phishing and deepfakes are where the economics shift first; strengthen verification, out-of-band checks, and employee detection training that includes synthetic media.

We’ve said it before, and we’ll probably say it over and over again: AI won’t replace attackers, but it will keep making them faster. Your job is to make your organisation’s security controls faster (and more adversarially tested) than the models on the other side.

Share on

Join newsletter

Join the newsletter to receive the latest updates in your inbox.


Follow us


Topics

Sign up for more like this.

Join the newsletter to receive the latest updates in your inbox.

Related articles