Why vibe coding creates hidden risk

Discover exclusive interviews and inspiration from the global Black Hat MEA community, in your inbox every week. 

This week we’re focused on…

Vibe coding. 

What’s that?

This week we spoke to upcoming BHMEA speaker Dimitri Van Zantvliet (CISO at Dutch Railways). Our chat was set to a backdrop of AI and automation – AI tools are becoming embedded in the fabric of modern cybersecurity, and we’re entering the era of autonomous decision-making. 

But Van Zantvliet shared a cautionary take: 

“We now have autonomous agents and generative tools writing code faster than people can validate it – welcome to the era of vibe coding. A line of AI-generated logic can now silently bypass controls or misconfigure access in ways no one notices; until a train doesn’t stop or a signal fails silently.”

That term, ‘vibe coding’, points to a new reality in security operations: machines increasingly write code not based on first principles, deep domain understanding, or context; but on patterns, predictions, and probabilities. 

And sometimes, those vibes are all wrong.

Automation doesn’t equal assurance 

AI-generated code offers new efficiency. But speed comes at a price: human oversight can’t keep up.

Security teams now face a new category of risk, in the form of machine-generated misconfigurations. These are often subtle and hard to trace, and they’re deeply embedded in systems that otherwise look like they're working just fine…until they aren’t.

In the railway sector, that might mean a silent failure in a signaling system. In healthcare, it could be an access control misfire that delays treatment. In energy, it might be a misrouted command that trips a grid.

“In theory, AI helps reduce human error. In practice, it introduces autonomous unpredictability,” Van Zantvliet said.

This unpredictability is a feature of generative systems. The point of AI is to adapt, to respond dynamically, to predict. But unlike traditional rule-based automation, these models don’t always leave clear audit trails. So “when something goes wrong, we often don't know whether to blame the developer, the AI, or the vibe.” 

The human-AI trust equation 

The challenge for CISOs now is to balance efficiency with assurance.

That means:

• Building governance frameworks around AI-generated actions
• Implementing real-time validation and alerting for AI-authored changes
• Creating clear accountability models – who signs off, who audits, and who ultimately owns the outcome

As Van Zantvliet put it, “AI offers solutions; but only if we govern it with the same care and rigour we apply to braking systems.”

It’s a reminder that while cybersecurity is evolving, the foundational principles of trust and control remain the same.

Defensive acceleration for the future 

“The more powerful and complex our systems become, the more fragile they can get,” Van Zantvliet added; “unless we design them with resilience, not just efficiency, in mind. In rail, we call that defensive acceleration.” 

We’d never heard the term before, but we like it. Speed must be matched by resilience, and systems should be built for safe failure (not just performance). Especially in sectors where silent digital errors can cause real-world harm.

We’re still in the early days of AI-native security tooling. But already, the risks are real: misconfigurations hidden in code we didn’t write, executed by systems we didn’t fully train, in contexts we might not understand until after something breaks.

Rejecting AI isn’t an option. So we need to build in friction, validation, and human awareness. Because in a world of vibe coding, oversight is going to be the difference between safety and danger.

Want to hear more from the leading minds in cybersecurity? Register now for Black Hat MEA 2025 and learn how the world’s most critical systems are securing their future.