NEWS Gemini Now Writes Viruses "On the Fly" and Changes Code Every Hour

[ExcalibuR]

Gemini Now Writes Viruses "On the Fly" and Changes Code Every Hour

A malicious program with a keyboard in the cloud—both laughable and terrifying at the same time.​

Google's Threat Intelligence Group (GTIG) has published an updated analysis indicating that threat actors have moved beyond simply using artificial intelligence to speed up routine tasks to embedding models directly into malware. This opens a new phase of abuse: "just-in-time" code generation and dynamic modification of malicious software behavior. In its report, GTIG details the discovered malware families, methods for bypassing model restrictions, examples of state-sponsored groups, and the measures the company has already taken to neutralize this activity and strengthen protections in its own models.

GTIG has identified the first malware samples that call Large Language Models (LLMs) during execution—the most prominent examples have been given the internal trackers PROMPTFLUX and PROMPTSTEAL.

• PROMPTFLUX is an experimental VBScript dropper that uses an API to access the Gemini model, requesting obfuscation of its own code and rewriting itself in the autostart folder. It has a "Thinking Robot" module that regularly asks the model for new ways to bypass antivirus software.
• PROMPTSTEAL is a Python-based packager that uses the Hugging Face API and the Qwen2.5-Coder-32B-Instruct model to generate single-line Windows commands, which are then executed locally to collect system information and documents.

GTIG assesses PROMPTFLUX as still in development, while PROMPTSTEAL has already been used in operations and is associated with the threat group APT28 (FROZENLAKE), with some of its activity confirmed by CERT-UA under the moniker LAMEHUG.

The report also lists other samples:

• FRUITSHELL: A public PowerShell reverse shell with prompts designed to bypass LLM protections.
• PROMPTLOCK: A proof-of-concept ransomware written in Go, using an LLM to generate Lua scripts.
• QUIETVAULT: A JS-based validator that steals GitHub/NPM tokens and exfiltrates the results via public repositories.

All these examples demonstrate different approaches to AI integration: from code regeneration and obfuscation to command generation and secret discovery on the host. GTIG notes that some mechanisms are not yet active (commented out) and serve as lab prototypes for future implementations, but the signal regarding the direction of threat evolution is already clear.

The report separately examines a technique where attackers use narratives as part of social engineering in their prompts to models to bypass safety responses. GTIG documented scenarios where actors pretend to be participants in CTF competitions or students, tricking the model into providing technically useful information that would otherwise be blocked. Similar tricks were used by TEMP.Zagros (MUDDYCOAST) and other groups. One such operational security mistake led to the exposure of C2 domains and encryption keys right in the chat dialogues with the model, which helped researchers disrupt the attackers' infrastructure.

The report emphasizes that state-sponsored actors from North Korea, Iran, and China continue to integrate generative tools across the entire attack chain: reconnaissance, phishing material preparation, C2 development, and extended exfiltration. Examples include:

• UNC1069 (MASAN), targeting cryptocurrency theft and deepfake creation.
• UNC4899 (PUKCHONG), which used models to develop exploits and plan supply chain attacks.
• APT41, which used Gemini to assist with code development and obfuscation.
• APT42, which attempted to build a "data processing agent" that would translate natural language queries into SQL queries to extract sensitive information—attempts that GTIG blocked by disabling the accounts.

Beyond technical methods, GTIG notes the maturity of the underground market: in 2025, multifunctional tools and services emerged offering phishing email generation, deepfake creation, automated malware generation, and subscriptions for API access. The market is adapting the business models of legitimate services—free basic versions with paid subscriptions for advanced features and access to Discord communities. This lowers the barrier to entry and gives less experienced threat actors powerful capabilities.

GTIG details techniques for exploiting LLM interfaces: malware can embed hardcoded API keys, request the "latest stable release" of a model for resilience against the deprecation of old versions, demand "code-only" output in a format suitable for automatic execution, and log the model's responses for later analysis. Such techniques increase the malware's resilience and adaptability, making classic signature-based defenses less effective.

In response to these threats, Google reports on multi-faceted measures: disabling threat actors' assets, collaborative work between DeepMind and GTIG to strengthen model classifiers and built-in restrictions, implementation of the Secure AI Framework (SAIF), the release of toolkits for secure AI development, and the use of agents like Big Sleep for automated vulnerability hunting and experimental CodeMender for automatically fixing critical code bugs. GTIG emphasizes that many incidents were neutralized precisely thanks to operational intelligence and swift actions to disable accounts and infrastructure.

The report also provides practical recommendations for defenders:

• Monitor for anomalous API key activity.
• Detect unusual calls to external LLM services from processes.
• Control the integrity of executable files.
• Strengthen procedures for protecting secrets on hosts.
• Avoid blindly executing commands generated by external models.

GTIG points out that a crucial role remains for the operational sharing of information between providers, researchers, and law enforcement, which has already helped stop a number of campaigns.

Finally, GTIG warns that while current examples are experimental, the direction of the threat is clear: AI is making malware more adaptive and the cyber-underground more accessible. The company emphasizes the industry's responsibility and calls for the standardization of safe practices for developing and deploying AI systems to minimize the risks of these techniques being adopted en masse.