Meta
Meta has identified “likely AI-generated” content used deceptively on its Facebook and Instagram platforms, including comments praising Israel’s handling of the military campaign in Gaza.
This content was found below posts from global news organizations and U.S. lawmakers, according to Reuters.
In a quarterly security report, Meta revealed that these accounts posed as Jewish students, African Americans, and other concerned citizens, targeting audiences in the United States and Canada.
The campaign has been attributed to Tel Aviv-based political marketing firm STOIC, which did not immediately respond to requests for comment on the allegations.
This discovery is significant as it marks the first instance of Meta identifying the use of text-based generative AI technology in influence operations since its emergence in late 2022.
While basic AI-generated profile photos have been used in such operations since 2019, the incorporation of generative AI for text represents a new development.
Researchers have expressed concerns that generative AI, capable of quickly and cheaply producing human-like text, imagery, and audio, could enhance the effectiveness of disinformation campaigns and potentially sway elections.
Despite these worries, Meta’s security executives stated in a press call that the Israeli campaign was detected and removed early. They emphasized that novel AI technologies have not impeded their ability to disrupt influence networks, which are coordinated attempts to push specific messages.
Mike Dvilyanski, Meta’s head of threat investigations, noted, “There are several examples across these networks of how they use likely generative AI tooling to create content. Perhaps it gives them the ability to do that quicker or to do that with more volume. But it hasn’t really impacted our ability to detect them.”
The report highlighted six covert influence operations that Meta disrupted in the first quarter. Besides the STOIC network, Meta also shut down an Iran-based network focused on the Israel-Hamas conflict, although there was no identified use of generative AI in that campaign.
Meta and other tech giants are actively grappling with how to address the potential misuse of new AI technologies, particularly in the context of elections.
Instances have been found where image generators from companies like OpenAI and Microsoft produced photos with voting-related disinformation, despite these companies having policies against such content.
Digital labeling systems have been emphasized to mark AI-generated content at the time of its creation, but these tools do not work on text and their effectiveness remains questionable.
Meta faces significant tests of its defenses with upcoming elections in the European Union in early June and in the United States in November, highlighting the ongoing challenge of managing AI-driven disinformation.
I am a dynamic professional, specializing in Peace and Conflict Studies, Conflict Management and Resolution, and International Relations. My expertise is particularly focused on South Asian Conflicts and the intricacies of the Indian Ocean and Asia Pacific Politics. With my skills as a Content Writer, I serve as a bridge between academia and the public, translating complex global issues into accessible narratives. My passion for fostering understanding and cooperation on the national and international stage drives me to make meaningful contributions to peace and global discourse.
