Mozilla Says AI Tool Mythos Found 271 Vulnerabilities With "Almost No False Positives" Mozilla engineers revealed on Thursday that their use of Anthropic’s Mythos AI model uncovered 271 security flaws in Firefox over two months, with the results showing "almost no false positives." The findings, detailed in a behind-the-scenes post, mark a significant step in leveraging AI for vulnerability detection, though the claims have drawn skepticism from critics who question the reliability of such tools. The project, led by Mozilla’s Distinguished Engineer Brian Grinstead, focused on improving AI-assisted security testing by developing a custom "harness" to guide Mythos through Firefox’s source code. Unlike earlier attempts where AI-generated bug reports often contained hallucinations, the harness allowed the model to interact with the same tools and processes used by human developers. This included access to Firefox’s specialized build systems and testing frameworks, enabling Mythos to identify memory safety issues by triggering crashes in a sanitizer build. The harness worked by instructing Mythos to "find a bug in this file," then providing it with tools to generate test cases and evaluate results. If the AI identified a potential issue, the system would run the test case through existing fuzzing tools. A second LLM was used to grade the output, ensuring high confidence in the findings. Grinstead emphasized that this process eliminated the need for manual verification of most reports, allowing engineers to quickly confirm vulnerabilities and iterate on fixes. Mozilla’s analysis revealed that 180 of the 271 vulnerabilities were classified as "sec-high," the highest priority for internally reported bugs. These flaws could be exploited through normal user actions, such as visiting a malicious website.#anthropic #mozilla #firefox #mythos #brian_grinstead
Mozilla's AI-Powered Bug Fixes in Firefox Spark Debate Over Model vs. Middleware Effectiveness Mozilla reported that its Firefox browser received 423 security bug fixes in April, a rate more than five times higher than March’s 76 fixes and nearly 20 times the browser’s 21.5 monthly average from the previous year. The company attributed this surge to the use of AI models, particularly Anthropic’s Mythos Preview, which identified 271 of the bugs in Firefox 150. However, the effectiveness of Mythos itself remains contested, with security experts questioning whether the model’s success stemmed from its advanced capabilities or from improvements in the middleware that connects AI systems to developers. Firefox’s security team, including distinguished engineer Brian Grinstead, tech lead Christian Holler, and head of security Frederik Braun, acknowledged that AI-generated security reports have evolved from “slop” to more reliable findings over recent months. They credited this progress to advancements in AI models and the development of better techniques for guiding these models to prioritize meaningful results over noise. To demonstrate the value of their approach, Mozilla unmasked a small sample of bug reports linked to the fixes, including a high-severity heap use-after-free vulnerability that could be triggered via the XSLTProcessor DOM API without user interaction. The team emphasized that AI analysis has enhanced security coverage, particularly in identifying sandbox escapes—complex flaws that are difficult to detect with traditional methods like fuzzing. They also highlighted how AI validated prior efforts to harden Firefox against prototype pollution attacks, as audit logs showed models attempting exploitation without success. However, Mozilla’s transparency in quantifying AI’s impact has drawn scrutiny.#anthropic #mozilla #brian_grinstead #christian_holler #frederik_braun
