OpenAI has terminated a number of ChatGPT accounts from Russia, accusing them of participating in a clandestine online misinformation operation.
The Sam Altman-led company discovered the issue while analyzing AI-generated social media posts, leading to the identification of a broader influence operation, according to an OpenAI blog post on Tuesday.
The operation was primarily based around a website that hosted plagiarized and wrongly attributed academic work, a 'sovereignty' index favoring Vladimir Putin's Russia, and attempts to hide the operators' Russian roots, as per OpenAI. The operators leveraged Russian prompts to generate social media comments posted on platforms such as Substack, Telegram, Meta Platform Inc.'s (NASDAQ: META) Facebook, and LinkedIn. They also directed ChatGPT to mask any linguistic indications of their Russian origin.
The majority of the generated content promoted an organization called the International Burke Institute (IBI). The IBI's website features articles copied from legitimate academic sources, sometimes falsely attributed. The influence operation also created a 'sovereignty index' that lauded Russia and criticized Western nations.
While the immediate impact of the operation appears to be limited, OpenAI stressed that the operation's significance lies in the infrastructure it had established. The AI-generated posts pointed to a seemingly credible institution, complete with alleged experts, republished academic work, and a purported proprietary risk index.
OpenAI Flags Rising AI Influence Risks
OpenAI said AI can help influence actors build authority, hide the origins of narratives, and create scalable assets for long-term influence campaigns.
In June, OpenAI said it disrupted multiple influence campaigns targeting U.S. debates over AI and data centers. One campaign, dubbed "Data Center Bandwagon," spread claims that AI data centers were driving up household electricity costs.
Another group linked to a Chinese technology company with government ties allegedly sought to shape U.S. policy discussions. OpenAI also banned Chinese-speaking users who used ChatGPT to create anti-Trump slogans, cartoons and multilingual comments for social media and news platforms.