HCC: Small: Incorporating Procedural Fairness in Flagging Mechanisms on Social Media Sites

This research will analyze expectations and interactions of social media platform users with flagging tools in each phase of the flagging lifecycle. Platforms offer flagging, a technical feature that empowers users to report inappropriate posts or bad actors, to reduce online harm, such as hate speech, nonconsensual sharing of sexual photos, etc. However, prior research shows that reporting harms can be experienced as secondary victimization, especially when victims perceive a lack of procedural justice. Flags play a critical role in maintaining the feasibility of content moderation systems, an initial step to identifying content that requires careful review by moderators or automated tools. It is, therefore, vital to design flagging interfaces in ways that ease the negative experiences of reporting. To accomplish this goal, we must understand how users make sense of flagging, what information they seek, and how they navigate flagging interfaces.There will be three phases in this project: (1) Interviews will be conducted with social media users who have recently flagged a post to understand their motivations, mental models, and concerns prior to flagging. (2) A user study with interactive prototypes will examine the awareness, navigation, and usage issues encountered while using flagging interfaces to report a post. (3) Participatory design workshop sessions will develop prototypes for post-flagging information and communication systems. In each phase, the research output will be codesigned with individuals from marginalized communities, who face disproportionate online abuse. Analysis will draw from the interpretive lens of procedural justice and its notions of voice and consistency to investigate how users develop their fairness perceptions of the flagging processes. It will produce a taxonomy of flagging-decision trees, offering a generative set of design dimensions upon which platforms may alter future flagging implementations.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.