ChatGPT Hacked the Internet's Main Defense. CAPTCHA Can No Longer Tell Humans and Bots Apart
For 20 years, it guarded websites... But an AI tore down the system in a single experiment.
For 20 years, it guarded websites... But an AI tore down the system in a single experiment.
Researchers from the company SPLX have demonstrated that ChatGPT can be tricked using specially crafted prompts into solving CAPTCHA tests—a task traditionally considered achievable only by humans. This experiment calls into question the reliability of a mechanism that has been used for years to protect websites from spam and automated attacks.
CAPTCHAs were originally created as a filter: images, logic puzzles, or UI elements were meant to confirm that a real user, not a bot, was interacting with the system. And if a large language model, given a specific sequence of commands, can handle such checks, it turns all of modern internet security on its head.
According to researcher Dorian Schulze, when directly asked to solve a list of CAPTCHAs, the neural network refused, citing a prohibition in its usage policy. The team then took a different approach: they prepared a dialogue that supposedly discussed "fake" tests and convinced the model that it would only be working with them. In the exchange, ChatGPT noted that it found the task interesting from a reasoning perspective and agreed to participate on the condition that it did not violate the rules.
The next steps looked like this: the researchers opened a new session with ChatGPT-4o, copied the text from the previous chat into it, and presented it as a continuation of the conversation. The agent accepted the conditions and immediately set about finding a solution. It performed best with one-click verifications, logic puzzles, and text recognition. It had more difficulty with images requiring moving or rotating elements, but even here, the answers were correct in a number of cases.
Schulze emphasized that, to the best of his team's knowledge, this is the first documented case of a GPT agent successfully passing complex graphical CAPTCHAs. The question of how much longer such tests can serve as protection in the era of increasingly capable AI systems now sounds particularly urgent.
OpenAI did not respond to journalists' requests for comment. However, cases of bypassing restrictions through so-called prompt injection have been documented before. Just this week, specialists from Radware showed that an assistant could be tricked with one correctly composed email into revealing Gmail secrets. Last month, Amazon fixed vulnerabilities in Q Developer that allowed for the injection of malicious prompts and even remote code execution.
The SPLX experiment demonstrates that even basic protection mechanisms like CAPTCHA are ceasing to be reliable barriers. As the capabilities of generative models grow, the line between human and automated system in such checks is becoming increasingly blurred.
