https://www.reddit.com/r/RedditSafety/comments/dp9nbg/reddit_security_report_october_30_2019/
created by KeyserSosa on 30/10/2019 at 17:16 UTC
3582 upvotes, 188 top-level comments (showing 25)
Through the year, we've shared updates on detecting[1] and mitigating content manipulation[2] and keeping your accounts safe[3]. Today we are sharing our first Reddit Security Report, which we'll be continuing on a quarterly basis. We are committed to continuously evolving how we tackle these problems. The purpose of these reports is to keep you informed about relevant events and actions.
3: https://www.reddit.com/r/redditsecurity/comments/bletrr/how_to_keep_your_reddit_account_safe/
┌──────────────────────────┬─────────────────────────┬─────────────────────────┐ │ **Category** │ **Volume (July - │ **Volume (April - │ │ │ Sept)** │ June)** │ ╞══════════════════════════╪═════════════════════════╪═════════════════════════╡ │ Content manipulation │ 5,461,005 │ 5,222,058 │ │ reports │ │ │ ├──────────────────────────┼─────────────────────────┼─────────────────────────┤ │ Admin content │ 19,149,133 │ 14,375,903 │ │ manipulation removals │ │ │ ├──────────────────────────┼─────────────────────────┼─────────────────────────┤ │ Admin content │ │ │ │ manipulation account │ 1,406,440 │ 2,520,474 │ │ sanctions │ │ │ ├──────────────────────────┼─────────────────────────┼─────────────────────────┤ │ 3rd party breach │ 4,681,297,045 │ 1,355,654,815 │ │ accounts processed │ │ │ ├──────────────────────────┼─────────────────────────┼─────────────────────────┤ │ Protective account │ 7,190,318 │ 1,845,605 │ │ security actions │ │ │ └──────────────────────────┴─────────────��───────────┴─────────────────────────┘
These are the primary metrics we track internally, and we thought you’d want to see them too. If there are alternative metrics that seem worth looking at as part of this report, we’re all ears.
Content manipulation is a term we use to combine things like spam, community interference, vote manipulation, etc. This year we have overhauled how we handle these issues, and this quarter was no different. We focused these efforts on:
1. Improving our detection models for accounts performing these actions
2. Making it harder for them to spin up new accounts
Recently, we also improved our enforcement measures against accounts taking part in vote manipulation[4] (i.e. when people coordinate or otherwise cheat to increase or decrease the vote scores on Reddit). Over the last 6 months (and mostly during the last couple of months), we increased our actions against accounts participating in vote manipulation by about 30x. We sanctioned or warned around 22k accounts for this in the last 3 weeks of September alone.
This quarter, we finished up a major effort to detect all accounts that had credentials matching historical 3rd party breaches. It's important to track breaches that happen on other sites or services because bad actors will use those same username/password combinations to break into your other accounts (on the basis that a percentage of people reuse passwords). You might have experienced some of our efforts if we forced you to reset your password as a precaution. We expect the number of protective account security actions to drop drastically going forward as we no longer have a large backlog of breach datasets to process. Hopefully we have reached a steady state, which should reduce some of the pain for users. We will continue to deal with new breach sets that come in, as well as accounts that are hit by bots attempting to gain access (please take a look at this post[5] on how you can improve your account security).
5: https://www.reddit.com/r/redditsecurity/comments/bletrr/how_to_keep_your_reddit_account_safe/
We have a lot of investigations active at any given time (courtesy of your neighborhood t-shirt spammers and VPN peddlers), and while we can’t cover them all, we want to use this report to share the results of just some of that work.
This quarter, we dealt with a highly coordinated ban evasion ring from users of r/opieandanthony. This began after we banned the subreddit for targeted harassment of users, as well as repeated copyright infringement. The group would quickly pop up on both new and abandoned subreddits to continue the abuse. We also learned that they were coordinating on another platform and through dedicated websites to redirect users to the latest target of their harassment.
This situation was different from your run-of-the-mill ~~shitheadery~~ ban evasion because the group was both creating new subreddits *and* resurrecting inactive or unmoderated subreddits. We quickly adjusted our efforts to this behavior. We also reported their offending account to the other platform and they were quick to ban the account. We then contacted the hosts of the independent websites to report the abuse. This helped ensure that the sites are no longer able to redirect automatically to Reddit for abuse purposes. Ultimately, we banned 78 subreddits (5 of which existed prior to the attack), and suspended 2,382 accounts. The ban evading activity has largely ceased (you know...until they read this).
There are a few takeaways from this investigation worth pulling out:
1. Ban evaders (and others up to no good) often work across platforms, and so it’s important for those of us in the industry to also share information when we spot these types of coordinated campaigns.
2. The layered moderation on Reddit works: Moderators brought this to our attention and did some awesome initial investigating; our Community team was then able to communicate with mods and users to help surface suspicious behavior; our detection teams were able to quickly detect and stop the efforts of the ban evaders.
3. We have also been developing and testing new tools to address ban evasion recently. This was a good opportunity to test them in the wild, and they were incredibly effective at detecting and quickly actioning many of the accounts that were responsible for the ban evasion actions. We want to roll these tools out more broadly (expect a future post around this).
The protests in Hong Kong have been a growing concern worldwide, and as always, conversation on Reddit reflects this. It’s no surprise that we’ve seen Hong Kong-related communities grow immensely in recent months as a result. With this growth, we have received a number of user reports and comments asking if there is manipulation in these communities. We take the authenticity of conversation on Reddit incredibly seriously, and we want to address your concerns here.
First, we have not detected widespread manipulation in Hong Kong related subreddits nor seen any manipulation that affected those communities or their conversations in a meaningful way.
It's worth taking a step back to talk about what we look for in these situations. While we obviously can’t share all of our tactics for investigating these threats, there are some signals that users will be familiar with. When trying to understand if a community is facing widespread manipulation, we will look at foundational signals such as the presence of vote manipulation, mod ban rates (because mods know their community better than we do), spam content removals, and other signals that allow us to detect coordinated and scaled activities (*pause for dramatic effect*). If this doesn’t sound like the stuff of spy novels, it’s because it’s not. We continually talk about foundational safety metrics like vote manipulation, and spam removals because these are the same tools that advanced adversaries use (For more thoughts on this look here[6]).
Second, let’s look at what other major platforms have reported on coordinated behavior targeting Hong Kong. Their investigations revealed attempts consisting primarily of very low quality propaganda. This is important when looking for similar efforts on Reddit. In healthier communities like r/hongkong, we simply don’t see a proliferation of this low-quality content (from users or adversaries). The story does change when looking at r/sino or r/Hong_Kong (note the mod overlap). In these subreddits, we see far more low quality and one-sided content. However, this is not against our rules, and indeed it is not even particularly unusual to see one-sided viewpoints in some geographically specific subreddits...What IS against the rules is coordinated action (state sponsored or otherwise). We have looked closely at these subreddits and we have found no indicators of widespread coordination. In other words, we do see this low quality content in these subreddits, but it seems to be happening in a genuine way.
If you see anything suspicious, please report it to us here[7]. If it’s regarding potential coordinated efforts that aren't as well-suited to our regular report system, you can also use our separate investigations report flow by emailing us[8].
7: https://www.reddit.com/report
8: mailto:investigations@reddit.zendesk.com
Finally, I would like to acknowledge the reports our peers have published during the past couple of months (or even today). Whenever these reports come out, we always do our own investigation. We have not found any similar attempts on our own platform this quarter. Part of this is a recognition that Reddit today is less international than these other platforms, with the majority of users being in the US, and other English speaking countries. Additionally, our layered moderation structure (user up/down-votes, community moderation, admin policy enforcement) makes Reddit a more challenging platform to manipulate in a scaled way (i.e. Reddit is hard). Finally, Reddit is simply not well suited to being an amplification platform, nor do we aim to be. This reach is ultimately what an adversary is looking for. We continue to monitor these efforts, and are committed to being transparent about anything that we do detect.
As I mentioned above, this is the first version of these reports. We would love to hear your thoughts on it, as well as any input on what type of information you would like to see in future reports.
I’ll stick around, along with u/worstnerd, to answer any questions that we can.
Comment by Vicegale at 30/10/2019 at 19:21 UTC
68 upvotes, 2 direct replies
TL:DR, but for real:
Content Manipulation:
Account Security:
Recent Investigations:
Also: u/KeyserSosa your first link is broken at the "and" and it makes me sad :(
1: https://www.reddit.com/report
2: mailto:investigations@reddit.zendesk.com
Comment by Halaku at 30/10/2019 at 17:40 UTC
79 upvotes, 4 direct replies
Second, let’s look at what other major platforms have reported on coordinated behavior targeting Hong Kong. Their investigations revealed attempts consisting primarily of very low quality propaganda. This is important when looking for similar efforts on Reddit. In healthier communities like r/hongkong, we simply don’t see a proliferation of this low-quality content (from users or adversaries). The story does change when looking at r/sino or r/Hong_Kong (note the mod overlap). In these subreddits, we see far more low quality and one-sided content. However, this is not against our rules, and indeed it is not even particularly unusual to see one-sided viewpoints in some geographically specific subreddits...What IS against the rules is coordinated action (state sponsored or otherwise). We have looked closely at these subreddits and we have found no indicators of widespread coordination. In other words, we do see this low quality content in these subreddits, but it seems to be happening in a genuine way.
In simpler English, the mod teams for those two subreddits suck balls, but there's no evidence they're being paid to do so?
Comment by BlogSpammr at 30/10/2019 at 17:38 UTC
23 upvotes, 2 direct replies
Comment by KeyserSosa at 30/10/2019 at 17:18 UTC*
56 upvotes, 8 direct replies
We have some labels for things that might not exactly line up with expectations, so let me try to define them with some more detail:
Comment by DecoyOne at 30/10/2019 at 18:41 UTC
22 upvotes, 1 direct replies
There are an absurd number of spam accounts that constantly post gifs of gimmicky, poorly made products or knock-off t-shirts with stolen designs. They then use alt accounts to steer Redditors to their shady websites to trick them into thinking the products are more legitimate. Is Reddit taking steps to curb this?
Comment by B1gWh17 at 30/10/2019 at 17:54 UTC
19 upvotes, 4 direct replies
This account was recently banned for violation of Reddits content policy yet I've never received a suspension or warning for violating any policy.
I submitted an appeal and was told the actions had been reviewed as well as my post history and the ban would remain in place.
After having my ban appeal denied, I received a message saying my account was banned due to an error in a new automated system.
So my questions are, has there been a remedy of the automatic action that caused my account to be banned and does the appeal process actually do anything?
I have no history of violating the content policy or user policy so an appeal should have remedied the actions taken in my account but it seems it's a delayed automated response to act like admins are actually reviewing any appeals sent to them.
Comment by Spacecowboycarl at 30/10/2019 at 18:17 UTC
38 upvotes, 1 direct replies
I was scrolling down and saw the super admin logo and thought, oh no what did we do now Reddit. It’s just a security report though. Close call.
Comment by cupcake1713 at 30/10/2019 at 18:13 UTC
22 upvotes, 1 direct replies
Thanks for sharing this report, this was an interesting look at (part of) what you've been tackling. Looking forward to seeing these quarterly.
Comment by iBleeedorange at 30/10/2019 at 18:26 UTC
12 upvotes, 3 direct replies
As a mod what information should I provide to the admins when I make a report about any of the stuff you mentioned in the report
Comment by [deleted] at 30/10/2019 at 18:36 UTC
10 upvotes, 2 direct replies
[deleted]
Comment by OKBlackBelt at 30/10/2019 at 17:32 UTC
24 upvotes, 3 direct replies
How do you determine whether or not a effort is state sponsored? That bit doesn’t make much sense.
Edit: I’m on mobile oop
Comment by [deleted] at 30/10/2019 at 18:21 UTC*
14 upvotes, 3 direct replies
[deleted]
Comment by [deleted] at 30/10/2019 at 18:18 UTC
14 upvotes, 1 direct replies
random question:do you paid for that preminium?
Comment by Ripstikerpro at 30/10/2019 at 18:27 UTC
13 upvotes, 1 direct replies
Really interesting what you did regarding 3rd party breaches.
It's a great step in the right direction and I hope the other big platforms following suit.
And also, the openness of the admins is something that I admire, keep at it.
Comment by ibm2431 at 31/10/2019 at 00:58 UTC
6 upvotes, 0 direct replies
Recently, we also improved our enforcement measures against accounts taking part in vote manipulation
When will action be taken about /r/FreeKarma4You, /r/FreeKarma4U, /r/FreeKarmaSub4Sub, and *all the other subreddits* (see how many RES autocompletes) which outright ask for votes, vote for each other in a group, and actively give new accounts a certain amount of karma in order to bypass rules that subreddits set regarding minimum karma requirements?
I reported one of these "communities" a year ago, and no action was taken, despite the *very clear* content policy[1]:
1: https://www.redditinc.com/policies/content-policy
Prohibited behavior
In addition to not submitting unwelcome content, the following behaviors are prohibited on Reddit:
And vote manipulation definition[2]:
**Asking people to vote up or down certain posts**, either on Reddit itself
Comment by ogopa at 30/10/2019 at 18:22 UTC
5 upvotes, 2 direct replies
Do you just use an alt account for regular browsing on reddit
Comment by [deleted] at 30/10/2019 at 19:17 UTC*
6 upvotes, 2 direct replies
[deleted]
Comment by poor_decisions at 30/10/2019 at 19:33 UTC
6 upvotes, 0 direct replies
What are you doing about the rampant botnet accounts spread all over reddit?
So many accounts scrape quotes text from articles, post them as if they are comments, and then copy those same comments between accounts. There is no human interaction, just bots repeating each other to farm karma.
Whenever I spot them and call them out, there is either no reply, or the comment is deleted immediately.
Comment by MordeeKaaKh at 30/10/2019 at 20:47 UTC
4 upvotes, 1 direct replies
Now that you mention it, I have noticed a significant decrease in t-shirt spammers. Well done!
Overall very happy about all this, from obviously our own security but also all the work you put into getting rid of spammers and and manipulations of various kinda. Add to all that this awesome transparecy of it all, and in my opinion Reddit is by far the best place to be online.
Thank you for all this!
Suggestions: moar data! I like that you share this data, but if possible maybe add even more? Don't know exactly what might be "missing", but I'm sure I'm not alone in my love of data like this. If not that's cool, just saying.
Another: on the mobile app atleast, it seems when you open the app it loads up your feed with posts. If you exit by pressing home or something (as in not using back button) you'll continue where you left of when you reopen, even after a couple of hours (I like this). Downside to this is if a post is removed by mods in the meantime, it'll look normal on the feed, and upon opening the post you'll see the post text for about a second before it changes to "removed". Best fix in my mind would probably be to somehow remove the post from the feed upen mod action, without need to manually update the entire feed first.
Although not a big deal, a minor annoyance, and if it's possible to fix it would improve the experience for the user.
Comment by klparrot at 30/10/2019 at 18:42 UTC
13 upvotes, 3 direct replies
Jesus, that's 2–3 admin content manipulation removals every second.
Comment by 5p33di3 at 30/10/2019 at 19:46 UTC
5 upvotes, 1 direct replies
When are we going to be able to see who's following us?
You said end of August...
Comment by LincolnshireSausage at 30/10/2019 at 19:23 UTC
7 upvotes, 3 direct replies
I had a subreddit banned about 3 months ago. I created the subreddit, let it sit for a month without doing anything and then got a message it was banned
This notice is to inform you that your subreddit has been banned due to a violation of Reddit’s rules, specifically fraudulently manipulating the subscriber count using fake accounts.
I asked the admin who banned it and in /r/ModSupport for an explanation as to why it was banned. As far as I was aware, the subreddit had 0 subscribers. I have no clue how anyone could suggest I was fraudulently manipulating subscriber counts when all I did was create the subreddit and let it sit for a month. I have received no explanation, no data, no proof. absolutely nothing to let me know why my subreddit was banned. I cannot explain it and nobody wants to take any time to explain it to me.
How can we trust the manipulation numbers in your report when I know for a fact I did nothing wrong but my subreddit was still banned?
Comment by Butterslingshot at 30/10/2019 at 18:09 UTC
11 upvotes, 1 direct replies
Nah I'd rather keep the comments on this post, not the other one *where you got downvoted into Oblivion* so I actually agree with you. *And please Reddit spare me*
Comment by greedie1 at 30/10/2019 at 18:01 UTC
6 upvotes, 1 direct replies
I'm sorry, what was that again?
Comment by mepronz at 30/10/2019 at 19:04 UTC
6 upvotes, 2 direct replies
I know reddit exists 90% celebrate obscure self referencing circle jerks, 9% to talk about popular stuff I'm unaware of and .1% for the subs I go to... so forgive my probably stupid question, but what the hell is an opieandanthony? You dedicated a whole lot of words about a group with absolutely no context for who or what they are given that the only reference was to a deleted sub. There were thousands of their kind and they are apparently no more, and youre bragging about it. But what were they?
How am I to know if they are a dedicated group of highschool musical fans sticking it to the reddit corporate overlords, or fury white supramacist hongkong cops? Whiskey traders or incels?