As noted above, Mozilla’s characterization of AI-assisted vulnerability discovery as a game-changer has been met with massive and vocal skepticism in many quarters. Critics initially scoffed when Mozilla did not obtain CVE designations for any of the 271 vulnerabilities. However, like many developers, Mozilla does not obtain CVE listings for security bugs discovered internally. Instead, they are grouped into a single patch. Typically, Bugzilla reports detailing these “rollups” are hidden for several months after being fixed to protect those who are slow to apply patches. Now that Mozilla has revealed a dozen of them, the same critics will surely claim that they were also cherry-picked and will hide less accurate results.
Of the 271 bugs found using Mythos, 180 were at high seconds level, Mozilla’s highest designation for internally reported vulnerabilities. These types of vulnerabilities can be exploited through normal user behavior, such as browsing to a web page. (The only higher rating, critical in seconds, is reserved for zero days.) Another 80 were moderate in seconds and 11 were low in seconds.
The critics are right to continue to push back. Hype is a key method of inflating the already lofty valuations of AI companies. Given the many praises Mozilla has heaped on Mythos, it’s easy for even more confident people to wonder: What do you get in return? Far from resolving the debate, Thursday’s explanations are likely to only further fuel the controversy.
However, according to Grinstead, the details are clear evidence of the usefulness of AI-assisted discovery, and Mozilla’s motivation is simple.
“People are a little burned out from the last year of these sloppy commitments, so we felt it was important to show some of our work, uncover some of the mistakes and talk about it in a little more detail as a way to hopefully spur some action or continue the conversation,” he said. “There’s no kind of marketing angle here. Our team has completely embraced this approach. We’re trying to spread a message about this technique in general and not about a specific model vendor, company or anything like that.”
