On May 12, 2025, researchers from Google Threat Intelligence Group (GTIG) published a report documenting what cybersecurity experts considered inevitable: a hacker group used a large language model to develop a fully functional zero-day exploit. This is the first confirmed case of its kind.
What happened
Cybercriminals discovered a previously unknown vulnerability in a popular open-source web tool for system administration — Google did not disclose the product name but notified the vendor and law enforcement. The vulnerability allowed bypassing two-factor authentication (2FA), though it required valid credentials.
The problem stemmed from a developer error: the code contained hardcoded ineffective trust exceptions that contradicted the logic of 2FA protection. This logical defect is difficult to detect manually — it is not a classic memory error or improper input handling. According to GTIG's assessment, this is where AI gained an advantage: modern LLMs can discern developer intent and find contradictions between design and implementation.
How GTIG established AI involvement
Researchers analyzed the Python script and identified characteristic markers of LLM generation:
- Excessive instructional docstring comments — a typical feature of text generated by language models
- A "hallucinated" CVSS vulnerability rating — a number that doesn't exist in any official database, but which the AI inserted as part of a structured description
- A "textbook" Python code style — formatting characteristic of educational materials in LLM training data
"The script contains a large number of instructional docstring comments, including a hallucinated CVSS score, and uses a structured textbook Python format, extremely characteristic of LLM training data"
— GTIG, report from May 12, 2025
Google emphasizes: Gemini was not involved in this attack. Which specific model the hackers used remains unknown. However, researchers ruled out the possibility that the code was written by a human without an AI assistant.
Scale: what was planned
The group was coordinated in advance and prepared a mass operation to exploit the vulnerability — not a targeted hack, but potentially thousands of targets. GTIG managed to intervene during the active deployment phase. Concurrently, researchers documented that other known groups — notably the Chinese cyber-intelligence group UNC2814, which since 2017 has attacked telecommunications and government structures in over 42 countries — attempted to break Gemini's security filters using jailbreak prompts to analyze firmware from TP-Link routers and other embedded devices.
As John Hultquist, lead analyst at GTIG, notes: "There is a misconception that the race over AI vulnerabilities still lies ahead. In reality, it has already begun".
Why this matters more than previous incidents
Previously, AI was documented as an auxiliary tool — for writing phishing emails, translating documents, or basic code analysis. This case is different: the model independently conducted logical analysis of security architecture and formulated working code for exploitation. GTIG explicitly states that LLMs "read developer intent" and find contradictions between design and implementation — a class of vulnerabilities that previously required deep human expertise.
If the next similar operation doesn't encounter active monitoring — how many systems will be compromised before the vulnerability is publicly disclosed?
