Filtering Offensive Comments

Goal: Promote healthier interactions Evidence rating (PDN): Mixed Civic signal: Welcome — Ensure user safety When to use: Reactive

What it is

Potentially offensive comments are hidden, based on user reports and algorithmic scoring, and require users to click on them to be seen.

Intended impact

Filtering offensive comments aims to reduce users’ likelihood of exposure, thus fostering a healthier online environment and, by altering the norms of a platform, potentially reducing the production of other offensive comments.

Evidence that it works

Using data from a large-scale randomized controlled trial on Nextdoor, Katsaros et al. (2024) evaluated the impact of offensive comment filtering on how - and how much - users engage on the platform. In a treatment condition comments that had been flagged as offensive were hidden from users, although they could see them by clicking to "see all comments" (only 1% of users did). The treatment led to a 12% reduction in exposure compared to a control group (the limited drop was due to the lag time between posting and flagging), but that decrease in exposure did not lead to any detectable effect on the content users created; the comments of users with less exposure to potentially offensive content were just as civil as those in the control condition. On the plus side, however, users in the treatment group showed no decrease in the amount of time they engaged on the platform.

While not the subject of Katsaros et al. (2024), a related study by Ribeiro et al. (2022) found that filtering comments - as opposed to removing them - likewise had no prosocial effect on users who authored the problematic comments.

While Katsaros et al. (2024) observed no improvement in user interactions as result of the intervention, that could be due to the delays in filtering and limited reduction in exposure. Future designs and research might explore the effects of automated algorithmic - and thus immediate - filtering on increasing prosocial outcomes.

Why it matters

As online platforms work to create healthier environments, filtering offensive comments offers a way to reduce toxic content visibility and potentially shape norms that then lead to healthier interactions. While current research does not provide evidence that filtering comments shifts norms to produce prosocial outcomes, there is evidence that it likewise does not reduce user engagement levels; thus while filtering offensive comments may not lead to more prosocial behavior, platforms can reduce risk of exposing users to offensive material without paying a cost in reduced engagement.

Special considerations

Examples



This intervention entry currently lacks photographic evidence (screencaps, &c.)

Citations

• *Offensive Comment Filtering Impact on Online Engagement: A Large-Scale Randomized Controlled Trial on Nextdoor.* — Katsaros, M., Grüning, D., & Lou, S. (ArXiV, June 20, 2024)

---

From the Prosocial Design Network's Digital Intervention Library (https://www.prosocialdesign.org/library/filtering-offensive-comments), licensed CC BY-SA 4.0. Reformatted for the RT Commons; text otherwise as published. Library citation: Prosocial Design Network (2024), Digital Intervention Library, https://doi.org/10.17605/OSF.IO/Q4RMB. Anything built from this pattern should credit the Prosocial Design Network and the researchers cited above.

Details

• goal: Promote healthier interactions
• citations: [object Object]
• image alt: An image showing negative comments being put through a filter.
• image url: https://cdn.prod.website-files.com/647f696dc653a97c26f42a0b/676724282af9ba69f09f87bd_Gratitude%20Reaction%20copy%402x.webp
• pdn rating: Mixed
• when to use: Reactive
• civic signal: Welcome
• pdn library doi: 10.17605/OSF.IO/Q4RMB
• civic signal goal: Ensure user safety