The mod conversations that went into today's policy launch

https://www.reddit.com/r/modnews/comments/hi3nkr/the_mod_conversations_that_went_into_todays/

created by ggAlex on 29/06/2020 at 16:58 UTC

253 upvotes, 1 top-level comments (showing 1)

Over the last few weeks we’ve been taking a very close look at our policies, our enforcement, our intentions, and the gap between our intentions and the results. You can read more from our CEO on that here.[1] This led to the development of updated policies for the site, which have been announced today in r/announcements.

1: https://www.reddit.com/r/announcements/comments/gxas21/upcoming_changes_to_our_content_policy_our_board/

As we started to dig into these policies, we knew we wanted to involve moderators deeply in their development. We hosted several calls with Black communities as well as a few ally and activist communities and invited them to a call with all of our Community Councils - groups of mods we host quarterly calls to discuss mod issues and upcoming changes. This call was attended by 25+ moderators (representing communities across the gamut: discussion, women, gaming, beauty, Black identity, and more), 5 Reddit executives (including our CEO, Steve Huffman aka u/spez), and 20 staff total.

As promised[2], we wanted to release the summary of this call to provide some transparency into the feedback we got, which ultimately informed the final version of the new policy.

2: https://www.reddit.com/r/modnews/comments/gw5dj5/remember_the_human_an_update_on_our_commitments/

The mods who attended these calls have already seen these notes. Information was redacted only where it provided PII about moderators.

The call started with a brief overview of Steve’s feelings about where we need to step up and an overview of a draft of the policy at the time. We then split into breakout rooms (since a 45-person call usually isn’t very effective) and finally came back together to wrap up.

A HUGE thank you goes out to all the mods who participated in these calls. Everyone was passionate, thoughtful, constructive, and blunt. We feel much more confident about the new policy and enforcement because of your input. We’ve not mentioned the usernames of any moderator participants in order to protect their privacy.

Breakout Room 1 (led by u/Spez, Reddit CEO)

Themes from the mods:

• **There are pros and cons to being explicit.** Lead with the rule rather than having it in the middle. We discussed how when rules are too vague, bad faith users use vagueness in the rules to justify things like brigading. They also use these rules to accuse mods of harassing them. However, when too specific, there is no leeway to apply the rule contextually - it takes agency away from mod teams to use their judgement.
• Example: People dissect the definition of “brigade” to justify it. People will post about another subreddit and a bunch of people will flood the target subreddit, but since it wasn’t a specific call to action people think it’s fine. It’s not clear to mods how to escalate such an attack. Steve called out that if you don’t want someone in your community, it should be our burden to make sure that they are out of your community.
• **Mods asked for clarity on what “vulnerable” means.** Steve said we’re trying to avoid the “protected classes” game because there’s a problem with being too specific - what about this group? That group? Our goal is just not attacking groups of people here. But we’ve heard this feedback from our past calls and are adjusting wording.
• **Expect pushback on the term “vulnerable groups”.** Bad faith users could argue that they are a vulnerable group (i.e. minority group) within the context of a sub’s membership. For example, in one subreddit that was restricted to approved submitters, mods receive hate mail from people not able to post arguing they are the vulnerable ones because they are being censored. Mods put the restriction in place to protect the subreddit’s members. They hear people saying they promote hatred against white people - even though a lot of their approved users are white. Bad actors are quick to claim that they are the minority/vulnerable group. Steve says that’s an argument in bad faith and that we will be looking at the wording here to see if we can make it more clear. He continues that mods get to define their communities - there are insiders and outsiders, values and rules, and not everyone should be in every community. We need to do better at supporting you in enforcing that - you don’t need to be sucked into bad faith arguments.
• **Mod harassment → mod burnout → difficulties recruiting new mods.** When a bad-faith actor is banned, it's all too easy to create a new account. These people target specific mods or modmail for long stretches of time. It’s obvious to mods that these users are the same people they’ve already banned because of username similarities or content language patterns. It's obvious too that these users have harassed mods before - they aren’t new at this. Mods ban these users but don’t have any faith that Reddit is going to take further action - they’ve seen some small improvements over the last few years, but not enough. A quote - “I just want to play the game [my subreddit is about] and have fun and we get so much hate about it.”
• **Collapsing comments isn’t sufficient for keeping the conversation dynamics on course.** It can look like mods are selectively silencing users. Some users whose comments have been collapsed write in wondering if the mods are shutting down dissenters - despite comments being collapsed automatically. Some mods would prefer the option to remove the comment entirely or put it in a separate queue rather than collapsing. In general, mods should have more control over who can post in their communities - membership tenure, sub-specific karma - in addition to Crowd Control[3].
• **There’s a learning curve to dealing with tough problems.** When it’s your first time encountering a brigade, you don’t know what’s going on and it can be overwhelming. It’s hard to share strategy and learnings - to shadowban new members for a waiting period, mods have to copy automod rules from another sub or create bots.
• **Mods don’t expect us to solve everything, but want our rules to back up theirs.** One mod shares that they have rules for bad faith arguments - but also get threatened with being reported to Reddit admins when they ban someone. They have had mods suspended/banned because stalkers went through statements they’ve made and taken out of context, and reported. Steve says that it sounds like these users are at best wasting time - but more accurately harassing mods, undermining the community, and getting mods banned. There’s other things we can do here to take those teeth away - for example, adding extra measures to prevent you from being unjustifiably banned. Undermining a community is not acceptable.
• **Moderating can feel like whack a mole because mods feel they don’t have tools to deal with what they are seeing.**

Breakout Room 2 (led by u/traceroo, GC & VP of Policy)

Themes of the call:

3: https://mods.reddithelp.com/hc/en-us/articles/360038129231-Crowd-Control

• **Moderating rules consistently.** Mods asked about how we are going to implement policies around hate if only some mod teams action the content appropriately. Not everyone has the same thoughts on what qualifies as racism and what does not. They want to know how the policy will be enforced based on context and specific knowledge.
• **Differences in interpretations of words.** Mods mention that words are different for different people - and the words we use in our policies might be interpreted differently. One mod mentions that saying black lives don’t matter is violent to them. A question is brought up asking if we all are on the same page in regards to what violent speech means. u/traceroo mentions that we are getting better at identifying communities that speak hatefully in code and that we need to get better at acting on hateful speech that is directed at one person.
• Some mods also bring up the word “vulnerable” and mention that maybe “protected class” is better suited as a describer. Words like “vulnerable” can feel too loose, while words like “attack” can feel too restricted. You shouldn’t need to be attacked to be protected.
• **Allies.** Some moderators mention that they don’t necessarily experience a lot of hate or racism on their own subreddit but recognize their responsibility to educate themselves and their team on how to become a better ally. Listening to other mods experiences has given them more context on how they can do better.
• **Education** Some mods express a desire to be able to educate users who may not intentionally be racist but could use some resources to learn more. Based on the content or action by the users, it might be more appropriate to educate them than to ban them. Other mods noted that it’s not their responsibility to educate users who are racist.
• **Being a moderator can be scary.** Mods mention that with their user easily visible on the mod list of the Black communities they are on, they are easy targets to hateful messages.

Some ideas we discussed during this breakout room:

• **Hiding toxic content.** Mods felt Crowd Control does an adequate job at removing content so users can’t see it but the mods still have to see a lot of it. They mentioned that they would like to see less of that toxicity. Potentially there is a toxicity score threshold that triggers and the content is never seen by anyone. Some mods mention that it is frustrating that they have to come up with their own tactics to limit toxicity in their community.
• **Tooling to detect racist/sexist/transphobic images and text** and then deal with the user accordingly.
• **Make it easier to add Black moderators to a community.** One mod suggested the potential of r/needablackmod instead of just r/needamod.
• **Making community rules more visible.** Mods suggested that a community's individual rules should pop up before you are able to successfully subscribe or before you make your first post or comment in the community.
• **Better admin response times for hateful/racist content.** Mods would like to see much quicker reply times for racist content that is reported. They suggested that vulnerable reporters have priority.
• **A better tool to talk to each other within Reddit.** It is hard for mods to coordinate and chat between all of their mod teams through Discord/Slack. They expressed interest in a tool that would allow them to stay on the Reddit platform and have those conversations more seamlessly.
• **Education tool.** Mods asked what if there was a tool (like the current self harm tool) where they could direct people to to get more education about racism.
• **Group Account.** Some mod teams have one mod account that they can use to take actions they don't want associated with their personal account - they would like to see that be a standard feature.

Breakout Room 3 (led by u/ggAlex, VP of Product, Design, and Community)

Themes from the call:

• **Policy could be simplified and contextualized.** Mods discuss that not very many people actually read rules but it covers mods so they can action properly. It might be good to simplify the language and include some examples so everyone can understand what they mean. Context is important but intent also matters.
• **The world “vulnerable” might be problematic.** What does vulnerability mean? Not many people self-describe as vulnerable.
• **This will be all for nothing if not enforced.** There are communities that already fit the rules and should be banned today. Mods don’t want to see admins tiptoeing around, they want to see actions taken. The admins agree - every time a new policy is put in place, there is also a corresponding list of communities that will be actioned day 1. A mod mentions that if a few subreddits aren’t actioned on day one this policy will seem like it doesn’t have any teeth.
• **Distasteful vs. hateful.** Depending on where you stand on certain issues, some people will find something to be hate speech while others will think that it's just a different view on the matter. There needs to be a distinction between hate speech and speech you disagree with. “All Lives Matter” was an example being used. Admin shares that Reddit is working on giving mods more decision-making power in their own communities.
• **Taking rules and adapting them.** Mods talk about how context is important and mods need to be able to make judgement calls. Rules need to be specific but not so rigid that users use them to their advantage. Mods need some wiggle room and Reddit needs to assume that most mods are acting in good faith.
• **Teaching bad vs. good.** Mods explain that it is difficult to teach new mods coming on the team the difference between good and bad content. The admins describe a new program in the works that will provide mod training to make it easier to recruit trained mods.
• **More tools to deal with harassment.** Mods feel that there simply are not enough tools and resources to deal with the harassment they are seeing everyday. They also mention that report abuse is a big problem for them. Admins agree that this is an issue and they need to do more, on an ongoing and lasting basis. They discussed building the slur filter in chat out more.
• **People online say things they wouldn’t say IRL.** The admins discuss the fact that all of this will be a long, sustained process. And it’s a top focus of the company. We can’t fix bad behavior on the internet with just a policy change. We want to think about how we can improve discourse on the internet as a whole. We want to try to solve some of the problems and be a leader in this area.

Breakout Room 4 (led by u/KeyserSosa, CTO)

• **The word** ***vulnerable*** **in the policy leaves room for rule-lawyering.** One mod suggested replacing it with the word *disenfranchised*, which has actual definitions that make it more clear and less up to interpretation. Another mod suggested specifically calling our words like “racism” and “homophobia”. Reddit is all about context, and we need to send a clear message and not leave room for interpretation with some of these thoughts.
• In the words of one mod, “What are the groups of people that it’s okay to attack?” u/KeyserSosa agreed that this is a good point.
• **Specific examples.** While mods understood we make it vague so it covers more, it would be nice to have specific examples in there for visibility. It would be helpful to have a rule to point to people that are rule lawyering.

The group next discussed the avenues of “attacking” mods have seen so far:

• **Awards on posts.** There are secondary meanings for awards that can communicate racist and sexist thoughts.
• **Usernames.** Sometimes game devs will do an AMA, and users will harass the devs through the usernames (think - u/ihateKeyserSosa).
• **Creating onslaughts of accounts.** Mods describe seeing users come to a post from the front page and appearing to create a ton of accounts to interfere with the community. It’s tough to deal with the onslaught because they are very intense. The guess is these are a mixture of farmed accounts and users with bad intentions.
• **Username mentions.** Some mods have received harassment after having their usernames mentioned. Sometimes they don’t get pinged because users don’t always use u/ - they just get abusive messages in their inbox. People also post screenshots of ban messages that contain the mod’s name, which is another venue of attack.

Thoughts on reporting, and reporting things to admins:

• **Thoughts on ban evasion**. Mods notice the obvious ones - but if there are tons of people doing similar stuff, it’s hard for mods to tell if it is one person that we banned or this other person we banned.
• **Receipt on reports for traceability.** It would be helpful in general and to organize what we’d be reporting.
• **Reduce copy pasting.** It would make things easier if mods could report from more places - so they don’t need to copy and paste the content they are reporting.
• **Report users to admins.** The ability to easily escalate a user to admins - not just content but the user. Mods can ban them but they could be doing the same thing in lots of places. They want to be able to let admins know when the user is breaking site rules. When mods have tried to write in to report a user in the past they get asked for examples and then need to go back and find examples that they feel are immediately obvious on the profile. Mods elaborated that the context is key when reporting users - one comment by itself might not look rule violating, but the entire thread of comments can be quite harassing.
• From u/KeyserSosa: When we originally launched subreddits, we had a report button, but it just created a lot of noise. The biggest subreddits got tons of reports. * Mods: Who’s reporting makes a big difference. Trusted reporters could have prioritized reports - users that have a history of good reporting.

Some other discussions:

• **Baking karma into automod**. For example - if users have karma in one subreddit that doesn’t mesh with yours, they couldn’t post. Mods weren’t a big fan of this - this would hurt new users, however, they liked the idea of seeing a flag on these posts or comments, so they know to look deeper. Flags that appear if users have used certain words elsewhere on the site would be useful as well.
• **Should any content be deleted automatically without making mods review?** Overall, mods like being able to see the content. If the content is removed, the user who posted it is still there. Reviewing the content allows the mods to know if they should take further action i.e. banning the user, or removing other content posted by that user that might have slipped through.

Some ideas we discussed during this breakout room:

• **Tying rate limits together.** There are per context ways to do rate limit but you can’t tie it together. For example, you can mute people from modmail but that doesn’t stop them from reporting.
• **Mod Recommendations.** What if we suggested good reporters to mods as mod suggestions? Would have to be opt-in: “Can we give your name to the mods since you are a good reporter?”
• **Expanding Karma, expanding user reputation.** Mods liked this idea in terms of a built in mod-application that ties your Reddit history together. Could include things like karma from the subreddit they are applying to. Another mod brought up that this would have to happen for everyone or nobody - would be a bad experience if it was opt-in, but people were punished (not chosen) if they opted out.
• **Giving mods more insight to users.** We should make it easier for mods to see key info without having to click to profile page and search, without making mods rely on third parties (toolbox).

Breakout Room 5 (led by u/adsjunkie, COO)

• **Keeping the new rules vague vs. specific.** Sometimes if a rule is too specific mods will see users start to rule lawyer. It might be better to keep it more vague in order to cover more things. But sometimes vague words make it challenging. What does “vulnerable” actually mean? That could be different based on your identity. Maybe “disenfranchised” is a better word because it provides more historical context. Other times, if a rule is too vague, it is hard to know how they will be enforced.
• **More context and examples for enforcement.** Both groups agree that we need more examples which could allow for better alignment on how these policies look in practice, e.g., what qualifies and what doesn’t.

The admins ask if there are any thoughts around harassment pain points:

• **Hard to identify when a user comes back with a new account.** There isn’t a great way to identify ban evasion. Mods mention using age and karma rules to prevent some issues but then they have extra work to add new users that are positive contributors.
• **Crowd Control is a good start for some communities, but mods of different sized communities have different experiences.** Mods say they are using all of the tools at their disposal but it is still not enough - they need more resources and support that are better catered for their communities. Crowd control works well for medium-sized communities, but for large communities who get featured in r/all, not so much. Other mods have experienced that the tool collapses the wrong content or misses a lot of content.
• **More transparency (and speed) is needed in the reporting flow.** It’s unclear when admins are taking action on reports made by mods and oftentimes they still see the reported user taking actions elsewhere.
• **Mods getting harrassed by users and punished by admins.** There have been instances where mods are getting harassed and they say one bad thing back and the mod is the one that gets in trouble with admins. An admin recognizes that we have made mistakes around that in the past and that we have built tooling to prevent these types of mistakes from happening more. A mod says there needs to be a lot of progress there to gain mod trust again.
• **Prioritization of reporting.** Mods asked the admin what the current priorities are when reporting an issue to Reddit and expressed frustration about not understanding reviewing priorities. Mods will report the same thing several times in hopes of getting it to a queue that is prioritized. An admin tells them that there isn't a strict hierarchy but sexualization of minors, harassment, and inciting violence tend to be at the top of the list - in comparison to a spam ticket for example - and acknowledges there is room for improvement with transparency here.

Some ideas we discussed during this breakout room:

• **Being able to see what a user is doing after they are blocked.** Mods mentioned that the block feature isn't that useful for mods, because they lose the insight to see what the user is doing afterwards. If they block a user for harassment, they can’t see when they break rules in the community. There should be a better way of managing that. Admin mentions upcoming features around inbox safety that might be an helpful option.
• **Get rid of character count in report flow.** Allow mods to give more context when reporting and also allow them to check multiple boxes at once. Links take up too much of the character count.
• **More incentives for positive contribution.** Mods suggest that karma should have more weight and that maybe users could get a subreddit specific trophy after 1,000 karma for being a positive contributor. Another mod cautions that you don’t want to confuse positive contributions with hive mind. Maybe you do it based on being an effective reporter.
• **Verifying users with a unique identifier.** A mod mentions how some platforms validate accounts with a phone number, maybe Reddit could do something like that. An admin replies that this is an interesting idea but there are privacy issues to consider.
• **Filter OP edits.** A mod suggested allowing posts to be edited by the OP as usual, but edits have to go through mod approval.

Outcomes

These calls were a great starting point to inform the policy and enforcement. Thank you to everyone who participated.

These calls also identified and highlighted several things we could act on immediately:

• r/blackfathers and other similar subreddits that promoted racist ideas under innocuous names are now banned and in the RedditRequest process - extra checks are built in for these subreddits to ensure these subreddits go to the right home.
• A bug fix is underway to ensure that reply notifications are not sent when a comment is removed by automod.
• We began experimenting with rate-limiting PM's and modmail[4] to mitigate spammy and abusive messages.
• We’ve added a link to the report form to the r/reddit.com sidebar to allow for easier reporting for third party apps.
• On iOS, moderators can manage post types, community discovery, and language and disable/enable post and user flair from community settings now. There are also links to moderator resources like the help center. Android changes coming in July.
• Blocked a specific set of words and common phrases associated with harassment from being sent through Reddit Chat

There’s a lot of additional changes in progress (including a complete revamp of the report flow). We’ll be back in the next few weeks to share updates on both safety features we’ve been working on for some time as well as new projects inspired directly by these calls.

4: https://www.reddit.com/r/modnews/comments/hf4tx0/testing_new_rate_limits_for_modmail_and_private/

We know that these policies and enforcement will have to evolve and improve. In addition to getting feedback from you in this post, in r/modsupport, and via your messages[5], we will continue expanding our Community Councils and discussing what is working and what is not about this rollout.

5: https://www.reddithelp.com/en/submit-request/other-help

Note that this post is locked so we don't have two conversations we're monitoring at once, but we would love to hear your feedback over on r/announcements.

Comments

Comment by ggAlex at 29/06/2020 at 17:00 UTC

1 upvotes, 0 direct replies

Hey all - we’re locking comments to direct you all to the r/announcements post where u/spez will be answering questions[1].

1: https://www.reddit.com/r/announcements/comments/hi3oht/update_to_our_content_policy/