War Stories with Brendan Dolan-Gavitt | Across the Pondcast

Why Validation Is the Hardest Problem in AI Pentesting

As AI transforms offensive security, the biggest challenge is no longer getting language models to find vulnerabilities. It is making sure they can prove their findings, distinguish real exploits from convincing hallucinations, and operate reliably in production.

In this episode of Across the Podcast, Brendan Dolan-Gavitt, AI researcher at XBOW, shares behind-the-scenes stories from building autonomous AI pentesting systems and explains why validation has become one of the hardest problems in AI security. Through a series of humorous and unexpected war stories, he explores how language models can creatively chain exploits, interact with other AI systems, and uncover novel attack paths, while also revealing the strange ways they can deceive themselves—or even attempt to deceive their own validators.

The conversation examines the technical architecture behind AI-powered pentesting, including why deterministic validation remains essential today, where language models may eventually replace traditional validation logic, and how AI is beginning to tackle complex business logic and authorization flaws that have historically required human judgment. Brendan also discusses the rapid pace of model improvement, the importance of orchestrating AI agents rather than relying on a single model, and why autonomous systems work best when paired with structured tooling and human oversight.

Beyond offensive security, the discussion explores the broader future of AI, from model reliability and training data quality to the impact AI may have on education, software development, and the next generation of security researchers. Throughout the episode, one theme remains consistent: AI is evolving at an extraordinary pace, but its greatest value comes from combining creativity with rigorous verification rather than replacing human expertise outright.