OpenAI banned a cluster of ChatGPT accounts originating from Russia for being used to support a covert online influence campaign to spread misinformation.
OpenAI said in a blog post on Tuesday that it discovered the issue after investigating AI-generated social media posts, leading to a “broader influence operation.”
The campaign was “built around websites containing copied and misattributed academic papers, ‘sovereignty’ indicators that cast a favorable light on Russia, and efforts to conceal the Russian origin of its operators,” the company added.
This is the latest incident in which pro-Russian groups have used AI to spread misinformation. OpenAI does not allow access to its models in Russia, but the company said VPNs are being used to spoof location information.
In February, OpenAI announced that the UK government had shut down ChatGPT accounts associated with pro-Russian media company Rybar, which it described as “partly coordinated by the Office of the President.” The company is funded by Russia’s state-owned company Rostec and works with members of Russian intelligence services.
CNBC has contacted the Russian Embassy in London and the Russian Ministry of Foreign Affairs for comment.
Pro-Russian influence campaign
OpenAI said the operators instructed ChatGPT to hide linguistic clues that it was Russian by prompting social media comments posted on Substack, Telegram, X, Facebook, and LinkedIn to be generated in Russian.
Much of the content generated promoted an organization called the International Burke Institute (IBI). Its website features articles copied from actual academic documents, in some cases misrepresenting their origins.
OpenAI said the influence campaign also created a “sovereignty index” praising Russia and disparaging Western countries.
While the immediate impact of the operation appears to be limited, the company added that “the significance of the operation lies in the infrastructure it built rather than the audience it reached.”
According to OpenAI, posts created by ChatGPT point to “seemingly trustworthy institutions with staff who claim to be experts, republished academic research, and their own risk metrics.”
“This shows how influencers can use AI as a supporting tool in broader efforts to fabricate power, obfuscate the sources of favorable narratives, and establish assets that can be expanded over time,” the company added. “It also shows that the use of AI to assist can expose a wider range of activities.”
