Advancing Community-Led Moderation: An Update on How NCRI/Pushshift and Reddit, Inc. are Working Together

https://www.reddit.com/r/pushshift/comments/13w6j20/advancing_communityled_moderation_an_update_on/

created by Pushshift-Support on 31/05/2023 at 00:05 UTC

128 upvotes, 31 top-level comments (showing 25)

Dear Reddit community

We are pleased to share an important update about our collaboration with Reddit, Inc. As an organization that maintains the Pushshift Reddit API, a key component behind several community-enabled moderation tools, we are pleased to announce that we have entered into a Memorandum of Understanding (MoU) with Reddit. This agreement establishes how Pushshift and Reddit will cooperate toward the common objective of supporting the Reddit community.

We want to express our appreciation for your support and patience during the recent challenges we have encountered and the disruptions that have occurred. In fairness to Reddit, this disruption falls on the shoulders of Pushshift, where there was a gap in our responsiveness to Reddit’s outreach. For this, we apologize. Moving forward, Pushshift will now have dedicated support staff to try to address questions about Pushshift from the Reddit community. We value Reddit's proactive approach and their dedication to collaborating with us to find constructive solutions.

To that end, we are happy to inform you that access to community-enabled moderation tools developed through the Pushshift API will be reinstated for verified Reddit moderators starting at a date soon to be determined. Note this will be contingent on moderators registering for Pushshift accounts. Each moderator will also need explicit approval from Reddit, and the use of Pushshift will be limited to moderation use cases only. This move will enable moderators to effectively use these tools to enhance community moderation and enforce guidelines, while protecting the privacy and data security of Reddit's user base.

While the main focus of the MoU lies in supporting the use of the Pushshift API for Reddit's community-enabled moderation, we also want to affirm our commitment to the academic research community. Pushshift's contributions to the academic realm have been recognized in numerous peer-reviewed papers.

Though access to Pushshift data for research purposes is not available at this time, , we are keen to explore possibilities that might allow us to provide researchers with access to datasets essential for their valuable social media research. We understand the significance of empowering the academic community, and we are dedicated to working with Reddit to develop frameworks that responsibly balance data access, data security, and user privacy.

We are excited about the potential for increased collaboration with Reddit in the months ahead and are committed to keeping you updated on our progress as we strive to create an environment where moderators, researchers, and the entire Reddit community can thrive together.

Thank you for your continued support and for being an invaluable part of the Reddit community.

Sincerely,

Pushshift and the Network Contagion Research Institute

Comments

Comment by safrax at 31/05/2023 at 01:03 UTC

49 upvotes, 1 direct replies

Please share the contents of the Memorandum of Understanding so that we as a community know the restraints Reddit has placed on PushShift and thus know its utility going forward.

Comment by inspiredby at 31/05/2023 at 02:05 UTC

18 upvotes, 1 direct replies

Are you allowed to share the text of the MoU?

Comment by Watchful1 at 31/05/2023 at 00:27 UTC

37 upvotes, 2 direct replies

You know you can edit posts right? No need to delete the other one with all the discussion and re-post it. I'll repeat my questions here.

Both you and Jason have said many times that you will be more active in the subreddit and community and then just go off and disappear for a couple weeks. How is this time going to be any different?

Note this will be contingent on moderators registering for Pushshift accounts. Each moderator will also need explicit approval from Reddit, and the use of Pushshift will be limited to moderation use cases only.

That's good information. Do you know anything about how reddit will approve users?

Comment by ExcitingishUsername at 31/05/2023 at 01:19 UTC

16 upvotes, 2 direct replies

Asking this again as it was not answered before the post was deleted-

Will the search bugs be fixed? PS isn't much use to us being unable to search by authors whose names aren't alphanumeric, or be able to include/exclude more than one subreddit, and most search queries containing numbers were broken as well.

Additionally, will content from NSFW communities still be archived?

Can you also clarify whether the new restrictions would limit data to only communities we moderate? This would of course render the service completely useless for anti-spam and similar purposes, so we'd like to know if that is or is not the case.

Comment by Btan21 at 31/05/2023 at 10:14 UTC

14 upvotes, 1 direct replies

No access to Pushshift data for research purposes? Honestly, I wasn't expecting this. If the data is being made available to Reddit mods, then why are researchers denied access?

Comment by LindyNet at 31/05/2023 at 00:17 UTC

10 upvotes, 0 direct replies

Note this will be contingent on moderators registering for Pushshift accounts

How does one go about this?

Comment by [deleted] at 31/05/2023 at 16:49 UTC

10 upvotes, 1 direct replies

[deleted]

Comment by Eusocial_Snowman at 31/05/2023 at 03:02 UTC

22 upvotes, 1 direct replies

Oh, this is bad. This is hilariously bad.

🚩

Comment by ThruBucknersLegs at 03/06/2023 at 08:15 UTC*

5 upvotes, 1 direct replies

You have most of the leverage here. Reddit needs Pushshift for moderation tools. Use that leverage to insist that Pushshift remains available for everyone. Reddit is not capable of filling the gap without Pushshift. Don't let them fleece you! Insist on access for everyone.

Comment by MathSciElec at 31/05/2023 at 11:35 UTC

5 upvotes, 1 direct replies

RIP Pushshift (unless you’re of the few approved mods IG). Guess we’ll have to continue scraping to archive Reddit…

Comment by reaper527 at 05/06/2023 at 20:21 UTC

3 upvotes, 0 direct replies

In fairness to Reddit, this disruption falls on the shoulders of Pushshift, where there was a gap in our responsiveness to Reddit’s outreach.

for what it's worth, reddit stated that their new ToS would take effect june 19th. it's june 5th today, and they pulled pushshift offline on may 1rst.

you guys not being responsive due to extenuating circumstances shouldn't have been relevant.

Comment by EntamebaHistolytica at 31/05/2023 at 00:36 UTC

5 upvotes, 1 direct replies

Does this mean sites like camas.undit will be available to the public for basic searches?

Comment by iKR8 at 31/05/2023 at 04:19 UTC

2 upvotes, 1 direct replies

So the verification of moderators will be done by Reddit side or Pushshift side?

Comment by rogerspublic at 02/06/2023 at 22:02 UTC

2 upvotes, 0 direct replies

I'm an academic and think Pushshift may be a better solution for my use given the size of my monthly downloads, which include r/conspiracy. I'd be more than happy to discuss my views on the matter with anyone from Reddit or Pushshift.

Here I'll note the following:

(1) While using social media is a gray area in human subjects research, academics could easily be asked to submit IRB paperwork, even if the research ends up being declared exempt.

(2) There are probably enough academics involved in social media research to form a user group that helps design policies and monitor compliance. Especially junior faculty who need brownie points for public service.

(3) I actually thought Twitter was on the right track with Twitter Academic, so it's sad that Elon discontinued it. It was not unlimited access, but it was enough for most uses. We academic sometimes forget that there is a real cost that we aren't absorbing when pulling data off someone's server, so Twitter Academic created some balance of interests. Having a Pushshift Academic is not a terrible idea.

Comment by Halaku at 31/05/2023 at 00:57 UTC

2 upvotes, 0 direct replies

To that end, we are happy to inform you that access to community-enabled moderation tools developed through the Pushshift API will be reinstated for verified Reddit moderators starting at a date soon to be determined. Note this will be contingent on moderators registering for Pushshift accounts. Each moderator will also need explicit approval from Reddit, and the use of Pushshift will be limited to moderation use cases only.

I'm looking forward to learning more, so I can use this while performing moderation duties.

Comment by [deleted] at 31/05/2023 at 01:19 UTC

2 upvotes, 1 direct replies

[removed]

Comment by Infrah at 04/06/2023 at 14:42 UTC

2 upvotes, 0 direct replies

Each moderator will also need explicit approval from Reddit, and the use of Pushshift will be limited to moderation use cases only.

So Reddit’s all good with it only if it’s to do their dirty work, that the thousands of unpaid moderators do. Forget about the end user who also may rely on 3rd party tools/applications!

Comment by riba2233 at 09/06/2023 at 20:55 UTC

2 upvotes, 0 direct replies

f this, it should be available to everyone just like before

Comment by brucemo at 31/05/2023 at 03:19 UTC

0 upvotes, 1 direct replies

National Council of Resistance of Iran?

National Catastrophe Restoration, Inc.?

Network Contagion Research Institute?

Okay, it actually really is Network Contagion Research Institute.

https://networkcontagion.us/

Comment by Bot-yMcBotface at 01/06/2023 at 22:13 UTC

1 upvotes, 0 direct replies

So reddit can have the cake an eat it too.

The mods have won. intersting. usually they lose.

But for researchers this is hilariously bad. I mean maybe one day the network contagion research institute offers some aggregated data. But reddit stays closed.

I am disappointed. Even more than I was before when I thought it would shut down.

Well this sub is not really worthwhile anymore :(

Comment by TRAFICANTE_DE_PUDUES at 04/06/2023 at 21:05 UTC

1 upvotes, 0 direct replies

OK boys, who's gonna scrape reddit and be a hero?

Red flag.

Also, share the MoU.

Comment by dniepr at 31/05/2023 at 14:18 UTC

0 upvotes, 0 direct replies

Lol pushshift to manage in-site business : mais oui

Pushshift for academia : denied, you peasants.

Anyway thank you , pushishift support, for all the info

Comment by exposecreepsandliars at 31/05/2023 at 22:11 UTC

0 upvotes, 1 direct replies

access to community-enabled moderation tools developed through the Pushshift API will be reinstated for verified Reddit moderators

Did y'all see the update[1] on r/modnews?

1: https://www.reddit.com/r/modnews/comments/13wshdp/api_update_continued_access_to_our_api_for/

From the update:

We're in discussions with PushShift to enable them to support moderation access. Moderators of sexually-explicit spaces will have continued access to their communities via 3rd party tooling and apps.

Only for sexually-explicit subs? So moderators of communities like r/MakeNewFriendsHere with 766k members and filled with vulnerable people who are strictly looking for platonic friendships will be left out of this for some reason?