Japanese cybersecurity experts have discovered that ChatGPT, an AI-powered chatbot developed by US venture OpenAI, can be tricked into writing code for malicious software applications. According to the experts, users can prompt ChatGPT to respond as if it were in developer mode, enabling them to bypass safeguards put in place to prevent criminal and unethical use of the tool.
This discovery has highlighted the ease with which AI chatbots can be exploited for malicious purposes, raising concerns about the potential for more crime and social fragmentation. In response, calls are growing for discussions on appropriate regulations at the Group of Seven summit in Hiroshima next month and other international forums.
G7 digital ministers also plan to call for accelerated research and increased governance of generative AI systems as part of their two-day meeting in Takasaki, Gunma Prefecture, at the end of this month. Meanwhile, Yokosuka, Kanagawa Prefecture, has started trial use of ChatGPT across all of its offices in a first among local governments in Japan.
While ChatGPT is trained to decline unethical uses, such as requests for how to write a virus or make a bomb, such restrictions can be evaded by telling it to act in developer mode, according to Takashi Yoshikawa, an analyst at Mitsui Bussan Secure Directions. When further prompted to write code for ransomware, a type of malware that encrypts data and demands payments in exchange for restoring access, it completed the task in a few minutes, successfully infecting an experimental PC.
“It is a threat (to society) that a virus can be created in a matter of minutes while conversing purely in Japanese. I want AI developers to place importance on measures to prevent misuse,” Yoshikawa said.
OpenAI acknowledged that it is impossible to predict all the ways ChatGPT could be abused, but said it would strive to create a safer AI based on feedback from real-world use. ChatGPT was launched in November 2022 as a prototype and is driven by a machine learning model that works much like the human brain. It was trained on massive amounts of data, enabling it to process and simulate human-like conversations with users.
Unfortunately, cybercriminals have already been studying prompts they can use to trick AI for nefarious purposes, with the information actively shared on the dark web. This underscores the urgent need for effective regulations and governance to ensure that AI chatbots are not used to perpetrate harm or undermine societal values.
