Blog
Four out of every ten social media posts contain misinformation about generative AI
USP scientists analyze the quality and accuracy of content about artificial intelligence available on Instagram and TikTok
Materials were classified based on depth, reliability, safety, and applicability – Photo: Pexels
Researchers from the Understanding Artificial Intelligence (UAI) group, affiliated with USP’s Institute of Advanced Studies (IEA), are working with data related to communication about generative AI produced between 2022 and 2025. The first analysis of 5,000 posts available on Instagram and TikTok showed that the content produced on social media about artificial intelligence does not align with the actual uses of AI, being influenced by the social imaginary.
The information gathered in the survey identifies activities that generative AIs cannot perform, such as mathematical operations and creating original text. In many cases, the quality of the discussion is lost as they try to address multiple factors at once, such as security and accuracy.
The survey, published in a technical note, included both qualitative and quantitative analysis of this content, identified through data scraping combined with the use of hashtags such as “#artificialintelligence,” “#technology,” and “#chatgpt.”
Luiz Joaquim Nunes – Photo: Courtesy of Luiz Joaquim Nunes
Luiz Joaquim Nunes is a doctoral candidate in Social and Work Psychology at the Institute of Psychology (IP) and holds a bachelor’s degree in Applied and Computational Mathematics from the Institute of Mathematics and Statistics (IME), with a specialization in Scientific Communication at the School of Communications and Arts (ECA), all at USP. The author explains that he was already working on the issues surrounding AI communication during his master’s degree and that, within the context of the approach used by the UAI group, he decided to technically and socially analyze how content on the subject is formulated and discussed.
The materials were classified into four analytical dimensions, defined based on the historiographical analysis conducted during Nunes’s master’s research: depth, reliability, safety, and applicability. To carry out the survey, the researchers used generative AI itself in the process, a choice justified by the large volume of data. Subsequently, manual verification was also conducted. “The classifications, qualitatively speaking, are more manual in that sense. We can even get assistance from certain generative AI tools to perform classification within these concepts, since comparing texts is something these technologies do well”.
Although the dimensions function and are classified individually, the survey showed that all criteria are correlated, even if they do not appear simultaneously in many pieces of content. In some cases, the relationship between criteria may be more subtle, as in videos that discuss AI safety while using reliable information, for example. However, the problem arises when addressing specific aspects of AI without establishing prior assumptions about how the technology works.
Victor de Sales Alexandre – Photo: Lattes
Victor de Sales Alexandre, who holds a PhD in School Psychology and Human Development from the IP and is a co-author of the technical note, explains that it is important to understand this specific characteristic of the informative discourse surrounding artificial intelligence.
“The fact is that in recent years we’ve seen a huge adoption of various AI mechanisms, many of them embedded in tools even if we don’t intend to use them. And when we do even a superficial analysis of the different discourses surrounding AI, it’s possible to see a duality between those who praise it and those who actually understand how it works”.
Nunes comments that people’s lack of understanding of technology is the root of many of its related problems. By treating what AI is and how it is used as common knowledge, extremely relevant information is lost.
An example of this is content that recommends having generative AI perform calculations, even though it performs better at generating content, especially text. This capability is based on patterns learned from existing data, making it far from the ideal tool for mathematical calculations, researchers say.
The reach of this type of communication also concerns specialists. According to the publication, out of every ten posts, four contain misinformation; out of every 30 posts, 16 address usage and four discuss the risks of generative AI; only two address both. “In my view, the media is not helping with this problem; in fact, it is making it worse. There is a high level of misinformation on the subject, which ultimately intensifies some of these risks over time”, says Nunes.
“If our culture does not promote the full development of people, or foster a more appropriate working relationship and a better relationship with education, then when artificial intelligence is integrated into social practices, it only intensifies what is already there”.
— Victor de Sales Alexandre
Another important factor in understanding its impacts is the cultural context in which artificial intelligence has been introduced. Currently, technical and technological discourse is being shaped by this tool, whether at work, in education, or during leisure time. The integration of AI into social relations tends to exclude the human element from creative output, reflecting existing disparities within the socio-historical context of human relationships.
“AI didn’t start these fragile relationships; instead, it’s echoing a trend that’s been culturally established for a long time. Until we resolve certain historical contradictions – for instance, in the way labor itself is produced – what we’re going to see is a rehash of the conflict using new techniques and new technologies, because they follow the development of our culture,” Sales points out.
“People are not being encouraged to think critically about how technology works and how it should or should not be used.”
— Luiz Joaquim Nunes
Researchers argue that we must stop treating artificial intelligence as something “self-evident” – beyond the scope of discussion. Many technical and technological developments have major implications for how society is organized, which is why it’s crucial to examine the potential, limitations, and narratives built around AI, as well as the sociocultural context in which it is embedded.
Current high-level discussions regarding the regulation of artificial intelligence and social media align with the findings of this study. While caution should start with individual technology use, the debate must be framed within governance structures and an analysis of today’s social context. Monitoring and digital literacy could be viable solutions to curb the misinformation identified in the research.
The technical note The production of content on generative AI quantified can be accessed here.
More information: luiz.nunes@usp.br, with Luiz Joaquim Nunes, and victor.alexandre@usp.br, with Victor de Sales Alexandre.
*Intern under the supervision of Tabita Said
English version: Nexus Traduções, edited by Denis Pacheco
Política de uso
A reprodução de matérias e fotografias é livre mediante a citação do Jornal da USP e do autor. No caso dos arquivos de áudio, deverão constar dos créditos a Rádio USP e, em sendo explicitados, os autores. Para uso de arquivos de vídeo, esses créditos deverão mencionar a TV USP e, caso estejam explicitados, os autores. Fotos devem ser creditadas como USP Imagens e o nome do fotógrafo.
A reprodução de matérias e fotografias é livre mediante a citação do Jornal da USP e do autor. No caso dos arquivos de áudio, deverão constar dos créditos a Rádio USP e, em sendo explicitados, os autores. Para uso de arquivos de vídeo, esses créditos deverão mencionar a TV USP e, caso estejam explicitados, os autores. Fotos devem ser creditadas como USP Imagens e o nome do fotógrafo.