GTIG AI Threat Tracker: Advances in Threat Actor Usage of AI Tools
webGoogle Threat Intelligence Group's 2025 report documents the first observed use of LLMs embedded within malware during execution, representing a significant escalation in AI misuse by state-sponsored and criminal threat actors relevant to AI safety and deployment risks.
Metadata
Importance: 72/100organizational reportanalysis
Summary
Google Threat Intelligence Group (GTIG) identifies a new phase of AI misuse where adversaries deploy 'just-in-time' AI-enabled malware (e.g., PROMPTFLUX, PROMPTSTEAL) that dynamically generates and obfuscates malicious code during execution. State-sponsored actors from North Korea, Iran, and China continue leveraging AI across the full attack lifecycle, while a maturing underground marketplace lowers barriers for less sophisticated cybercriminals. The report also documents social engineering tactics used to bypass AI safety guardrails.
Key Points
- •First documented malware families (PROMPTFLUX, PROMPTSTEAL) using LLMs mid-execution to dynamically generate and obfuscate malicious code, evading static detection.
- •Threat actors use social engineering pretexts (e.g., posing as students or researchers) to bypass AI safety guardrails on platforms like Gemini.
- •Underground marketplace for illicit AI tools has matured in 2025, lowering entry barriers for less sophisticated cybercriminals.
- •State-sponsored actors from North Korea, Iran, and China misuse AI across the full attack lifecycle: reconnaissance, phishing, C2 development, and data exfiltration.
- •Google has taken proactive steps including disabling malicious accounts and improving model safeguards, detailed in the 'Advancing Gemini's Security Safeguards' white paper.
Cited by 1 page
| Page | Type | Quality |
|---|---|---|
| AI Cyber Damage: Bounding the Tail | Analysis | -- |
Cached Content Preview
HTTP 200Fetched May 4, 202635 KB
Threat Intelligence GTIG AI Threat Tracker: Advances in Threat Actor Usage of AI Tools
November 5, 2025
Google Threat Intelligence Group
Google Threat Intelligence
Visibility and context on the threats that matter most.
Contact Us & Get a Demo Executive Summary
Based on recent analysis of the broader threat landscape, Google Threat Intelligence Group (GTIG) has identified a shift that occurred within the last year: adversaries are no longer leveraging artificial intelligence (AI) just for productivity gains, they are deploying novel AI-enabled malware in active operations . This marks a new operational phase of AI abuse, involving tools that dynamically alter behavior mid-execution.
This report serves as an update to our January 2025 analysis, " Adversarial Misuse of Generative AI ," and details how government-backed threat actors and cyber criminals are integrating and experimenting with AI across the industry throughout the entire attack lifecycle. Our findings are based on the broader threat landscape.
At Google, we are committed to developing AI responsibly and take proactive steps to disrupt malicious activity by disabling the projects and accounts associated with bad actors, while continuously improving our models to make them less susceptible to misuse. We also proactively share industry best practices to arm defenders and enable stronger protections across the ecosystem. Throughout this report we’ve noted steps we’ve taken to thwart malicious activity, including disabling assets and applying intel to strengthen both our classifiers and model so it’s protected from misuse moving forward. Additional details on how we’re protecting and defending Gemini can be found in this white paper , “ Advancing Gemini’s Security Safeguards .”
Key Findings
First Use of "Just-in-Time" AI in Malware: For the first time, GTIG has identified malware families, such as PROMPTFLUX and PROMPTSTEAL , that use Large Language Models (LLMs) during execution. These tools dynamically generate malicious scripts, obfuscate their own code to evade detection, and leverage AI models to create malicious functions on demand, rather than hard-coding them into the malware. While still nascent, this represents a significant step toward more autonomous and adaptive malware.
"Social Engineering" to Bypass Safeguards: Threat actors are adopting social engineering-like pretexts in their prompts to bypass AI safety guardrails. We observed actors posing as students in a "capture-the-flag" competition or as cybersecurity researchers to persuade Gemini to provide information that would otherwise be blocked, enabling tool development.
Maturing Cyber Crime Marketplace for AI Tooling: The underground marketplace for illicit AI tools has matured in 2025. We have identified multiple offerings of multifunctional tools designed to support phishing, malware development, and vulnerability research, lowering the barrier to entry for less sophistic
... (truncated, 35 KB total)Resource ID:
f5c89038a1d5f7ce | Stable ID: sid_G88hfp18mQ