Removal Explanations

Goal: Reduce repeated rule-breaking Evidence rating (PDN): Likely Civic signal: Welcome — Ensure user safety When to use: Reactive

What it is

Provide an explanation to users when their post or comment is removed for rule infraction.

Intended impact

By providing users an explanation for why one of their comments was removed, you can reduce the likelihood that their future comments will break a forum's rules—and thus also require removal by moderators.

Evidence that it works

Jhaver et al (2019) looked at the behavior of Reddit users who had at least one of their posts removed in a given subreddit over a three month period and compared how likely they were to break that subreddit's rules again (i.e. have another post removal) depending on whether:

• They received an explanation for why their posts were removed;

• The explanations were brief "flare" message or longer comments; and

• The explanation comments were delivered by a bot or moderator.

They found that users who received any explanation were less likely to have another post removed, and even less likely if the explanation was delivered as an extensive comment as opposed to a brief "flare". It made no difference, however, if a message was delivered by a bot or a moderator.

The limitation of the Jhaver et al (2019) study is that it is observational, so we cannot know if there is a confounding variable that may explain why explanation receivers are less likely to re-offend (e.g. the types of subreddits that provide explanations may also tend to have users who respond positively to any removal). Two additional studies, however, do use quasi-experimental methods to study a related intervention, the effect of removing a post, and find that post-removals decrease the likelihood a user will re-offend. (Horta Ribeiro et al., 2022; Srivinasan et al., 2019) Although neither of those studies directly examines removal explanations, both note that users are given explanations for why their content was deleted. The combination of Jhaver et al's observational finding about the effect of explanations and those two natural experiments suggests to us that explanations are a promising intervention. See: Removal Rule-breaking Content.

It is important to note that Srinivasan et al. did not see a decrease in toxic content; this suggests that removals and explanations are an effective way to make users aware of the specific rules of a platform rather than to disincentivize toxic behavior more broadly.

In an additional study, Jhaver et al. (2024) look at the effect of removal explanations on “bystanders” - i.e. people who did not have their own comment removed but who witnessed a removal explanation on Reddit. We see tentative evidence in that study that witnessing removal explanations can increase engagement, although it does not have an apparent effect on reducing rule-breaking behavior.

‍

Why it matters

Special Considerations

Explanations may only be effective at reducing repeat rule-breaking in forums that have strict or unusual rules (i.e. not more common rules against toxic behavior), as they serve to teach new users about rules they may not be aware of.

It is possible that explanations are effective only if they also include information about why a post was removed as well as an appeal process and if the forum has a "three strikes" policy.

Citations

• *Does Transparency in Moderation Really Matter?: User Behavior After Content Removal Explanations on Reddit* — Shagun Jhaver, A. Bruckman, Eric Gilbert (Proceedings of the ACM on Human-Computer Interaction, November 7, 2019). https://doi.org/10.1145/3359252
• *Content Removal as a Moderation Strategy* — Kumar Bhargav Srinivasan, Cristian Danescu-Niculescu-Mizil, Lillian Lee, Chenhao Tan (Proceedings of the ACM on Human-Computer Interaction, November 7, 2019). https://doi.org/10.1145/3359265
• *Automated Content Moderation Increases Adherence to Community Guidelines* — Manoel Horta Ribeiro, Justin Cheng, Robert West (ArXiV, October 19, 2022). https://doi.org/10.48550/arXiv.2210.10454
• *Bystanders of online moderation: Examining the effects of witnessing post-removal explanations.* — Jhaver, Shagun, Himanshu Rathi, and Koustuv Saha. (Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems, May 11, 2024)

---

From the Prosocial Design Network's Digital Intervention Library (https://www.prosocialdesign.org/library/removal-explanations), licensed CC BY-SA 4.0. Reformatted for the RT Commons; text otherwise as published. Library citation: Prosocial Design Network (2024), Digital Intervention Library, https://doi.org/10.17605/OSF.IO/Q4RMB. Anything built from this pattern should credit the Prosocial Design Network and the researchers cited above.

Details

• goal: Reduce repeated rule-breaking
• citations: [object Object], [object Object], [object Object], [object Object]
• image url: https://cdn.prod.website-files.com/647f696dc653a97c26f42a0b/6767252f264061ca37d55952_6660a86670b697e3dd393f2b_Removal%20Reason%402x.png
• pdn rating: Likely
• when to use: Reactive
• civic signal: Welcome
• pdn library doi: 10.17605/OSF.IO/Q4RMB
• civic signal goal: Ensure user safety