Security Testing at Superhuman Speed
See how Superhuman uses XBOW for continuous security testing that keeps pace with hourly deployments and gives engineers validated proof they can act on.
Superhuman ships code all day. Some services deploy the moment a build passes; others several times an hour. Against that pace, a once-a-year assessment describes an application that no longer exists by the time the report lands. Giles Douglas, CISO and Engineering Director, tests continuously with XBOW so coverage keeps up with the code and his engineers get proof they can act on.
Key takeaways
- Superhuman deploys continuously, some services hourly. An annual assessment is out of date the week it lands. Instead, Giles' team tests the security of each release with XBOW as the code ships.
- XBOW shows Superhuman's engineers the proof of concept and the full attack chain, so they can fix the bugs that break the chain first.
- XBOW works alongside the team's existing scanners, turning a growing pile of findings into a prioritized plan by chaining vulnerabilities into the attack paths his team can break.
Background
Giles Douglas came up on the engineering side. When he started building software, releases ran on six-month cycles that a team of manual security testers could keep up. He's now CISO and Engineering Director at Superhuman, where some services deploy hourly. Superhuman is the AI productivity platform formerly known as Grammarly, used by more than 40 million people and 50,000 organizations. Giles runs security across an engineering organization of about 650 people, with a security team of 25.
Running both security and engineering, Giles owns both sides of the same tension: keeping security from slowing releases while keeping the release pace from outrunning what his team can cover. Superhuman ships client software that runs on its customers' own machines, and Fortune 500 buyers with strict compliance programs want proof the product is not carrying risk onto their systems.
Challenge
Superhuman ships fast, and clients update frequently on top of it, so risk keeps surfacing between security assessments. Giles keeps these assessments for the audits that require them, like Google CASA and SOC 2, and understands their usefulness.
“Pentests are important for compliance, but they aren't necessarily built for operational security at our deployment cadence. A finding from Monday might already be fixed by Wednesday, so we pair them with continuous testing.”
Getting fixes made was the other half of the challenge. Giles' engineers hold a high bar for what they ship while moving fast, so they weigh every security finding against the roadmap in front of them.
“If we hand engineers a random security bug, they ask whether it's even exploitable. Show them a chain with a working proof of concept and a clear consequence, and it gets fixed, because it's concrete instead of abstract.”
Why XBOW
Giles wanted two things that a once-a-year engagement left on the table: testing that keeps pace with how often Superhuman ships, and proof his engineers trust enough to act on.
Bug bounty reports already moved through his team that way.
“A bug bounty finding gets fixed fast because it comes verified, with proof and evidence. XBOW gives us that same value chain on our own terms, without waiting on a researcher or an alert in the middle of the night.”
Superhuman first bought XBOW because it fit with their vision of bringing generative AI models to practical problems; it could do things at scale and speed that were more predictable than bug bounty campaigns.
Solution
Superhuman runs XBOW continuously against its applications, so the security team has an answer ready whenever a release is moving. When an engineering team tells Giles a feature is shipping in a few days and asks him to make sure it is safe, six months of notice is not on the table. His team turns XBOW loose on the feature over the weekend and brings the results back to the engineers, so both sides work from the same evidence. Giles' phrase for the difference: data over vibes.
XBOW also changed how the application security team gets value from the tools it already runs. Scanners are cheap and run at scale, but they stop at flagging what might be risky. Business logic flaws slip past them; understanding interactions across API boundaries requires reasoning about how the application works. Giles' engineers ship more code with AI assistance, and the scanners return more findings than the team can rank.
“Scanners find plenty of potential issues, but they don't chain findings into an attack path. XBOW tells us which handful of fixes actually break the chain, so those go first.”
For Giles, the scanners stay in place. XBOW tells his team which supply-chain vulnerabilities are exploitable in their code, so that is where their attention goes.
Results
The debate used to start with "is this even real?" and now it starts with "how fast can we fix it?" Engineers act on proof that shows a clear path to impact, so fixes move faster. Continuous testing also replaced the old rhythm of an annual assessment landing with a giant burn-down list. The team now works through a steady, understandable flow of findings it can keep up with.
It also strengthened the story Giles can tell to his own customers. Superhuman sells to security-conscious buyers who ask how the product is protected, and now he has a current answer for them.
“A continuous testing story is a far stronger attestation in our sales cycles than a point-in-time report from nine months ago. I can speak to the security aspects of the build we're shipping now.”
"A continuous testing story is a far stronger attestation in our sales cycles than a point-in-time report from nine months ago. I can speak to the security aspects of the build we're shipping now."
Looking Ahead
Giles expects the pressure to keep building. He plans to keep testing at the speed that Superhuman ships. For a team deploying by the hour, he sees that as the way to stay ahead.
“Attackers have an advantage in some ways right now, because AI gave them a new set of ways to think. But defenders are catching up with the same tooling, and using AI for defense, it can have a huge impact today.”
"Attackers have an advantage in some ways right now, because AI gave them a new set of ways to think. But defenders are catching up with the same tooling, and using AI for defense, it can have a huge impact today."
About
Superhuman (formerly Grammarly) is the AI productivity platform on a mission to unlock the superhuman potential in everyone. The Superhuman suite of apps and agents brings AI wherever people work, integrating with over 1 million applications and websites. The company's products include Grammarly's writing assistance, Doc's connected surfaces for teams, Mail's inbox management, and Go, the proactive AI assistant that understands context and delivers help automatically. Founded in 2009, Superhuman empowers over 40 million people, 50,000 organizations, and 3,000 educational institutions worldwide to eliminate busywork and focus on what matters. Learn more at superhuman.com.
XBOW is the autonomous offensive security platform that proves what attackers can actually exploit. By launching real attacks and independently validating every finding, XBOW helps security teams prioritize real risk and secure applications continuously with the controls enterprises require for production environments. xbow.com
