https://www.reddit.com/r/RedditSafety/comments/gsgg6k/improved_ban_evasion_detection_and_mitigation/
created by worstnerd on 28/05/2020 at 22:34 UTC
478 upvotes, 100 top-level comments (showing 25)
Hey everyone!
A few months ago, we mentioned[1] that we are starting to change how we handle user ban evasion in subreddits. tl;dr we’re using more signals to actively detect and action ban evaders.
1: https://www.reddit.com/r/redditsecurity/comments/f9xfmp/reddit_security_report_february_26_2019/
This work comes from the detection we have been building for admin-level bans, and we wanted to start applying it to the problems you face every day. While it’s still in an early form and we know we aren’t getting to all forms of ban evasion, some of you are starting to notice that work and how it’s affecting your users. In most cases, it has been very positively observed, but there have been some cases where the change in behavior is causing some issues, and we’d love your input.
As we mentioned in the previous post, only around 10% of ban evaders are reported by mods – which is driven by the lack of tools available to help mods proactively determine who is ban evading. This means that a large number of evaders are never actioned, but many are still causing issues in your communities. Our long-term goal and fundamental belief is that you should not have to deal with ban evasion; when you ban a user, you should feel confident that the person will not be able to come back and continue to harass you or your community. We will continue to refine what we classify as ban evasion, but as of today, we look at accounts that meet either of these criteria:
1. A user is banned from a subreddit, returns on a second account, and then is reported to us[2] by a moderator of the subreddit
2. A user is banned from a subreddit, returns on a second account, and then that second account is banned from the subreddit. For now, since it does not rely on a direct report, we will only take action if the mods of the subreddit have a history of reporting ban evasion in general.
When someone fitting either criteria 1 or 2 attempts to create yet another alt and use it in your subreddit, we permaban that alt within hours - preventing you from ever having to deal with them.
2: https://www.reddit.com/report?reason=its-ban-evasion
By the numbers:
We have also taken steps to mitigate the risks of unintended consequences. For example, we’ve whitelisted as many helpful bots as possible so as to not ban bot creators just because a subreddit doesn’t want a particular bot in their community. This applies to ModBots as well.
Because of these and other operational changes, we’ve been able to pull our average ban evasion response time from 29 hours to 4 hours, meaning you have to put up with ban evaders for a significantly shorter period of time.
Again, we want to highlight that this process is still very new and still evolving - our hope is to make ban evading users less of a burden on moderators. We’ve already been able to identify a couple of early issues thanks to feedback from moderators. If you see a user that you believe was incorrectly caught up in an enforcement action, please direct that user to go through the normal appeal flow[3]. The flow has a space for them to explain why they don’t think they should have been suspended. If you, as a moderator, are pointing them there, give them the link to your modmail conversation and ask them to include that in their appeal so we can see you’ve said ‘no, this is a user I’m fine with in my subreddit’.
3: https://www.reddit.com/appeal
As always, thanks for everything you do! We hope our work here will make your lives easier in the end.
Comment by [deleted] at 28/05/2020 at 23:01 UTC
22 upvotes, 5 direct replies
[deleted]
Comment by [deleted] at 28/05/2020 at 22:58 UTC
40 upvotes, 5 direct replies
What have you been noticing since this change?
It's been touch and go. We've had a lot of edge cases that we have to call in big guns for because it's heavily active and forcefully aggressive. We had one user go a month making new accounts, sending dozens of messages to modmail full of racism and death threats, and then moving accounts. It took about a month for that to mostly subside (it's not every 4-5 days instead of daily). We also had someone who made about a dozen accounts in an hour and then move onto doxxing a mod. He seems to have finally laid off that thankfully. But both required us reaching out. We've also had a lot of ban evaders that we realize are evading 6-7 months out that we then action. No idea on how those kinds of cases get caught up in this sort of thing.
What types of edge cases do you think we should be thinking about here?
See above, really. The unlawful evil users are a huge problem and if it's someone hyper-active, then a few hours delay is too slow. Also cases where they're not looking to post but instead get around modmail mutes.
What are your ideas on behaviors we shouldn’t be concerned about as well as ways we might be able to expand this.
I'd love for ways where people who get banned for minimally malicious reasons that might be lifted down the line to be brought to our attention as way for us to have dialog of "ban evasion isn't okay, but talking with us can get it lifted." But that requires giving us either a far more robust way of doing "permanent" (really, indefinite) bans so we could tag them, or give ways of us connecting accounts, which I know y'all have no interest doing. Because there are people who we're interested in having that conversation, but we're not going to just ignore ban evasions at the same time. It's just not a good system to have so few options with such a wide variety of rules violation extremes.
Comment by BlatantConservative at 28/05/2020 at 23:08 UTC
36 upvotes, 3 direct replies
we’ve been able to pull our average ban evasion response time from 29 hours to 4 hours
Hallelujah
I do have four questions though.
1. Is this only for permabans?
2. On /r/politicalhumor, we regularly ban users with inappropriate usernames (usually a homophobic slur or something to do with autism) but we tell them that they have our express permission to ban evade with another username (because they're kids and don't know any better). How do we make sure that these guys don't get snagged by this system? We don't want to permanently remove these users from Reddit if they're only in trouble for a relatively minor infraction and they've been banned before. I also know of other subreddits like /r/anime that ban novelty accounts but welcome people's main accounts.
3. A lot of communities have a "joke ban" system, like on /r/holdup we have a flair that says "Choose this flair to be instantly banned" and then if someone flairs a post with that we give them a one day ban. How do we make sure they aren't flagged by this system? (If this only works with permabans this question does not apply)
4. Can a subreddit opt out entirely? I run /r/modabuse which is a pro mod abuse community and we ban pretty much everyone who posts there. I got one guy to come back on 22 different accounts and if this system is implemented I won't be able to beat my high score.
Comment by superfucky at 28/05/2020 at 23:50 UTC
6 upvotes, 1 direct replies
is this why i got suspended for a week for "ban evasion" when *my husband* got banned a couple weeks prior? maybe part of your processes should involve actually looking at the activity of those "alts" to determine whether they're even the same people. an entire household should not end up de facto permabanned because one person on that IP caught a ban from a major subreddit.
Comment by SCOveterandretired at 29/05/2020 at 00:23 UTC
8 upvotes, 2 direct replies
Well, now I understand why the troll that has made 1000+ or more accounts hasn't been harassing our members for the last 2 or 3 months and for this alone I thank you. He has only been banned once this month but before I would ban him 2 or 3 times a day some weeks and he would brag that using a VPN protects him from Reddit detecting him evading our bans.
Comment by [deleted] at 28/05/2020 at 22:49 UTC
15 upvotes, 3 direct replies
Looking at those numbers, I'm suddenly shocked that r/NFL makes up such a chunk of reddit ban evasion, not gonna lie.
Comment by alittlebirdy1 at 29/05/2020 at 01:33 UTC
13 upvotes, 0 direct replies
It's difficult to say. I have one case of a user that was banned on multiple accounts from /r/sex four months back. Each of these new accounts was reported (the user had a very specific kink of gigantic women fucking themselves with skyscrapers like sex toys, obsessively posted about it in our forum, and openly attacked the mods in mod mail for banning him). He'd make alts just to attack me personally in meta posts, that sort of thing. The user then created an alt one letter off of my username and sent quite a few fake ban messages to users of our sub - and openly bragged in both PM to me and in general mod mail that he was doing this. We reported this, and the activity stopped for a while.
A couple of days ago, he reappeared with a new mutation of my name, a new rash of fake ban messages to other users, and bragging to me in PM that he was doing it again. So... it's difficult to say that the new measures have been helpful in that case.
We have another user that obsessively posts his pictures to our sub to ask for women to rate him. We have conservatively banned at least two dozen of his accounts, reporting each of them for evasion. He continues to return.
I can think of several others that seem immune to any sort of admin banning efforts.
I know that this is somewhat off topic, but I know that our mod team would be ETERNALLY grateful if we could permanently mute certain trolls from modmail. 72 hours at a time doesn't dissuade some of the really persistent loonies; we have one or two that drop in every few weeks at the most to troll us for the fun of it, and have done so for well over a year.
Comment by KKingler at 28/05/2020 at 23:08 UTC
8 upvotes, 1 direct replies
A user is banned from a subreddit, returns on a second account, and then that second account is banned from the subreddit. For now, since it does not rely on a direct report, we will only take action if the mods of the subreddit have a history of reporting ban evasion in general.
Instead of detecting history, could this be a toggle in subreddit settings? I personally like this system, but would want to ensure that a sub is opted in, and I can definitely see some people want to ensure a sub is opted out.
What types of edge cases do you think we should be thinking about here?
I'm not sure how ban evaders are detected, but what if the edge case of two separate people in a house are banned? They'd be under the same IP, and may not have maliciously ban evaded.
Comment by Bardfinn at 28/05/2020 at 23:51 UTC
31 upvotes, 2 direct replies
What have you been noticing since this change?
I no longer have to whack-a-mole. 50%+ of my mod load after being brought on to a specific subreddit that is extremely attractive to ban evaders, for several months, was whack-a-mole. I've been able to step back from that.
I've also been noticing talk in specific communities that depend on ban evasion and suspension evasion to continue their harassment agendas, that the automated suspensions are:
1: Discouraging them from continuing "the game" of serial suspension / ban evasion and harassment;
2: Causing them to strongly reconsider attempts at "playing both sides" - where they have sockpuppets and act out some manner of psychodrama between them and the established community in order to social engineer / emotionally manipulate a community. They've noted that *their entire cluster of sockpuppets* for their "operation" were being suspended.
What types of edge cases do you think we should be thinking about here?
I can think of one edge case: Someone who has been banned in one or more communities, and who *needs*, for some legitimate reason, to abandon their current account (such as if they've been doxxed or targeted by a group of harassers), and wants to abandon that account/identity and make a new Reddit account -- they should be able to privately tell Reddit "I want to migrate my infrastructural data from Account A to Account B", and that would include the subreddits they're banned from - so that they don't accidentally participate and trigger the suspension evasion heuristic. This is certainly an edge case, in that most good faith users should be both wise enough to not participate in communities that they know they're not welcome in, and capable of keeping track of the *very few* communities that they've been banned from. But.
What are your ideas on behaviors we shouldn’t be concerned about as well as ways we might be able to expand this.
I -- and every other publicly LGBTQ / woman individual on the Internet -- attract people who find purpose or fulfillment in life from abusive harassment / stalking via PM. Banning users from following our profiles / banning them from commenting on posts on our profiles should preclude them from PMing us. To build on that: Users who have been recently banned / muted from a given subreddit should not be able to successfully PM the moderators of that subreddit for [amount of time to be determined]. Also, brand new 1 post / 1 comment karma accounts should be throttled from sending more than 1 modmail to a given subreddit per day.
Thanks.
Comment by ItsRainbow at 28/05/2020 at 22:39 UTC*
12 upvotes, 0 direct replies
This is great for RPAN subs — thanks!
Comment by Kahzgul at 28/05/2020 at 22:56 UTC*
11 upvotes, 2 direct replies
On a related note, is anything being done about ban trolls? I'm talking about new accounts that actively seek out and bait other accounts into violations and then report those violations, sometimes getting long time posters banned.
It seems to me that it would be beneficial if reddit admins encouraged mods to understand the difference between banning a FRANK (Fresh Reddit Account, No Karma) and banning a multi-year poster over the same discussion.
edit: I guess the age of the accounts being baited really doesn't matter. What concerns me is that when one account baits many others into rule violations, there doesn't seem to be any acknowledgment that those other accounts were victims of someone who is almost certainly ban-evading in order to get others banned "for teh lolz."
Comment by rasherdk at 28/05/2020 at 23:18 UTC
5 upvotes, 2 direct replies
For now, since it does not rely on a direct report, we will only take action if the mods of the subreddit have a history of reporting ban evasion in general.
How would we know whether you consider us to "have a history of reporting ban evasion in general"? What's the threshold? How do we know if we're above or below the threshold?
Comment by [deleted] at 28/05/2020 at 23:30 UTC
5 upvotes, 0 direct replies
If you can detect when someone is using an alt account, could you just make so that people can't use alts to ban evade in the first place?
Comment by techiesgoboom at 29/05/2020 at 01:37 UTC
5 upvotes, 0 direct replies
What types of edge cases do you think we should be thinking about here?
The users that come into modmail bragging about evading a ban or promising to evade the current ban. There's a pervasive mindset that anyone that even tries a little bit can evade a ban, and given that number of repeat trolls we've had come back dozens and dozens of times (or in some cases hundreds) it's hard to say they're wrong.
We get easily a dozen of these a day in modmail (often with insults mixed in) and with some 20,0000+ comments a day we just have no realistic way to link the every day ban evasion accounts together.
Past responses from admins in modsupport on reporting ban evasion with only a single account have made it seem like nothing is done with these and they just get put in a garbage can that is a special filing cabinet[1].
1: https://scatteredquotes.com/garbage-can-special-filing-cabinet/
So mainly I guess just some clarity on this would be helpful. Normally I only bother reporting the ones that are paired with a higher priority report reason (which does involve many of these cases, because ban evaders love to insult). Do these reports trigger any of your systems or lead to anything at all being investigated?
Comment by Anomander at 29/05/2020 at 02:54 UTC
4 upvotes, 0 direct replies
As we mentioned in the previous post, only around 10% of ban evaders are reported by mods
I kinda want to touch on this specific, since no one else has so far.
I know I largely gave up on reporting ban evasion after the reports at least two separate users who are very clearly ban evading (one guy tells his life story on each account and uses the same catchphrases, for example), having been told “no connection exists” and being asked to report each new account ... it became fruitless & we stopped banning on sight and started automodding the new accounts instead.
Especially in the prior example case, they deleted old accounts once banned or at too low a negative karma. Keeping track of old accounts well enough to report persistent evasion felt like it required admin overhead *so* in excess of “volunteer mod” commitments that it’s general lack of results soured the whole process.
I have not noticed substantial change in our experience, but we may well be considered a non-reporting community. In the longer run, could mods perhaps be notified of evasion-related actions taken regarding our communities? Not specifics, that’s not safe to share, but overall numbers?
Comment by siouxsie_siouxv2 at 29/05/2020 at 01:03 UTC*
5 upvotes, 1 direct replies
What about subs like r/comedyhitmen where the whole point of the sub is ban evasion and trolling? I feel this kind of post[1] and things like this[2] used to be against the rules. After a while, we felt reporting them was a waste of time, the comments are still there.
2: https://www.reddit.com/r/ComedyHitmen/comments/gom6bv/reddit_good/frgygxw/?sort=confidence
Is this activity[3] against any rules? When we go to report this, there is nothing that address brigading. Is that on purpose?
Reporting [everything] (https://i.imgur.com/IBFuM2K.jpgin[4]) is impossible. Why are they still able to do this?
4: https://i.imgur.com/IBFuM2K.jpgin
Comment by soundeziner at 29/05/2020 at 01:05 UTC*
7 upvotes, 0 direct replies
When someone fitting either criteria 1 or 2 attempts to create yet another alt and use it in your subreddit, we permaban that alt within hours
This just hasn't been the case for the ban evasion going on in /r/AmazonSeller. You know this because I've reported it many many many times via the report form and via /r/modsupport modmails and even via comments to /u/woodpaneled where I pointed out that the ban evasion was not being addressed. It would be greatly appreciated if someone took the time to look into it
--------------------------------------------------------------------------------
Comment by [deleted] at 29/05/2020 at 01:22 UTC*
3 upvotes, 2 direct replies
What have you been noticing since this change?
We have noticed almost no difference. It is still hit or miss as to if we get an answer from Anti-Evil at all or in a timely manner when we report ban evasion. On the other hand, sometimes we've gotten immediate - like too fast to read the message we sent - responses telling us something was done as the same users continue to harass us. Sometimes something seems to happen and we get a reprieve. Sometimes nothing seems to happen and the alt use and harassment continues, even with super obvious alts.
What types of edge cases do you think we should be thinking about here?
Alts being used to stalk, sexually harass, harass, and threaten mods. Alts being used to threaten rape or violence. We've had multiple users in the last year alone create alts specifically just to target mods and to stalk, threaten, make rape threads/jokes at, etc. them. It has particularly been an issue for women on the mod team. This doesn't just happen with permanent bans, it happens with temp bans and sometimes even just comment or post removals.
Comment by Cowbeller at 29/05/2020 at 03:20 UTC
3 upvotes, 0 direct replies
What changes have you been noticing since this change?
Unfortunately, none. We dealt with a ban evader for nearly a month. We ended up catching their posts with automod because of admin inaction because it was the same post over and over and because the usernames were similar enough.
Comment by [deleted] at 28/05/2020 at 23:23 UTC
7 upvotes, 1 direct replies
I feel your criteria for ban evasion reaching the level of "actionable" are too lenient. Regardless of improvement, they are still derived from actions by moderators who suffer from:
a lack of tools available to help proactively determine who is ban evading.
As it is, if I ban a user, your criteria do not make me:
feel confident that the person will not be able to come back and continue to harass [me or my] community
If an account is banned from a subreddit and that user returns on an alt, that should be criteria for action whether or not a moderator finds them.
Comment by merkon at 28/05/2020 at 23:49 UTC
2 upvotes, 0 direct replies
I would love to get better feedback as to what actions are taken after ban evasion is reported. Right now, it's just a copy/paste message we receive saying "we've looked into it" essentially. Would love to know what's actually happened at this point.
Comment by Agent_03 at 29/05/2020 at 00:20 UTC
2 upvotes, 0 direct replies
This is a great improvement in general, and more automation is helpful. Especially if reddit builds on the feature over time to customize as-needed for some of the edge cases.
Is there a way to explicitly "opt in" for subreddits that do not have a history of reporting ban evasion? Especially for larger/more active subreddits, we often don't realize ban evasion is happening because it looks like normal daily trolls. We suspect it's probably happening for politically motivated astroturfing/misinformation, but people are not blatant enough for us to connect the dots between accounts.
We're also -- perhaps atypically -- slow to issue permabans. But by the time someone has earned a permaban, we'd *really* like to make sure that it *sticks.*
Comment by Dragonpixie45 at 29/05/2020 at 00:30 UTC
2 upvotes, 0 direct replies
I had wondered why I haven't had certain troll/ban evasion accounts hitting my sub lately!
Are you planning on letting mods know when these actions are taken? The reason I ask this is is because my sub had been hit with a user(s) that were harassing us on various accounts, we would ban and they would come back and during that time admins were working with us to combat the issue and a couple of users I had not banned received bans, then would come back and get banned again, they kept insisting they did nothing wrong and said they appealed but it kept happening on every account they created. They did say they once they started using TOR or something like it stopped but this was also around the same time the trolling and harassment stopped.
I only bring this up specifically because they still will bring up how I worked with reddit admins in this grand conspiracy to silence them and I have been absolutely clueless what bans they were talking about.
Comment by KokishinNeko at 29/05/2020 at 00:32 UTC
2 upvotes, 0 direct replies
Great news, in the past we had a few stubborn users, one even created alts just by adding numbers on number 10 he gave up :)
Will keep an eye on this, not happening much nowdays.
Comment by abrownn at 29/05/2020 at 01:19 UTC
2 upvotes, 1 direct replies
When someone fitting either criteria 1 or 2 attempts to create yet another alt and use it in your subreddit, we permaban that alt within hours - preventing you from ever having to deal with them.
"Permaban" from our subs for us? or do you mean suspend/shadowban from the site?