Classifying Harm in Digital Media: Opportunities and Limits of AI Tools

The rapid advancement of communication technologies and digital media has posed significant challenges for the research, mapping, and regulation of increasingly complex and opaque systems, including generative tools, recommendation systems, and artificial intelligence-based risk detection technologies. While malicious actors and researchers engage in an ongoing algorithmic race between systems of harmful content production and detection, academic and policy debates on information integrity have shifted beyond a narrow focus on content toward the structural qualities of information environments, particularly digital media ecosystems. Against this backdrop, digital governance faces persistent asymmetries related to accountability, legitimation politics, and inter actor power relations across platforms and states, setting the stage for a discussion on how artificial intelligence-based tools are being used to map and govern harmful content in digital media.

This session will bring two empirical research strands developed in Brazil that explore both the possibilities and limitations of these technologies. The first examines the development of a text-based classifier for fraudulent advertisements on Meta platforms, highlighting the extensive manual work required to curate training data and its application in identifying thousands of financial scams targeting vulnerable groups, with implications for public policy debates and law enforcement practices. The second addresses the use of large language models to classify more complex forms of discourse, focusing on research that tracks climate denial narratives in Brazilian news websites, and discusses the strengths and limitations of these systems, their frequent uncritical adoption in research and platform moderation, and the broader implications of their growing use in public services and everyday life for the future of information governance.
 

About the speaker:

Carlos Eduardo Barros is a coordination member at NetLab, the Laboratory of Social Networks and Internet Studies, and a PhD candidate and MSc in Information Science at the Federal University of Rio de Janeiro (UFRJ) and the Brazilian Institute of Information Science and Technology (IBICT). He is currently a visiting researcher at Dublin City University and at the University of Exeter, and a member of the VOX-Pol European Network of Excellence on Violent Online Political Extremism. Carlos has published in leading academic journals such as Convergence, International Review of Information Ethics, and Social Media + Society. His research focuses on information integrity, political propaganda, and digital media ecosystems.
 

To receive the emails from the Future of Technology and Society Discussion Group, including the link for the online participation, join our mailing list by sending a blank e-mail to: future-of-tech-and-society-subscribe@maillist.ox.ac.uk. 

Link for the session: https://teams.microsoft.com/l/meetup-join/19%3ameeting_YWE0Mjk3NjMtNDM4Ni00NDFhLWE1ZjgtMGJlNGEyNDIyN2Uz%40thread.v2/0?context=%7b%22Tid%22%3a%22cc95de1b-97f5-4f93-b4ba-fe68b852cf91%22%2c%22Oid%22%3a%225e1ab948-35e6-4d30-9ebc-89d6ce1b206a%22%7d