A successor to NNTP

Recently I got into an email discussion of the Network News Transfer Protocol via the Gemini email list; I thought I’d share my contribution more widely, partly to model how a decentralised discussion system might work today. Gemini is a good model for how one might develop an NNTP successor; it might also be instructive to try *discussing* an NNTP successor *using* Gemini, rather than email and email lists.

I last used NNTP in the 1990s, but never really knew the protocol details (the way one might know enough SMTP/POP3/IMAP/HTTP/Gemini to carry on one side of the session by hand). NNTP predates the current legal architecture alongside the Internet, which was rolled out 1988-1998, covering automatic copyright, defamation, intermediary liability, notice-and-takedown; it is similar to HTTP in this respect. Automatically copying material between machines is now legally a lot harder, and that may militate against decentralised data-sharing systems, or at least significantly impact their design.

Nevertheless, I am a big fan of decentralisation, which has benefits as well as drawbacks, and think that something like NNTP should be brought back if possible. Gemini benefits immensely from hindsight and from the contemporary availability of the Web and Gopher, for which is a complement rather than substitute. So my approach would be to explore the potential characteristics of a minimalist article-sharing system, which must exist in a world where NNTP and Reddit continue to be used.

The Gemini approach consists in the judicious combination of the minimal set of pre-existing standard components sufficient to be functional. The functionality is roughly analogous to the Web: URLs, MIME types, HTTP and HTML, but with the latter two components replaced by Gemini’s protocol and conventional file format. A newsgroup system has slightly more moving parts as there is server-to-server communication, but there is the document format and transport protocol just as with the HTTP Web and Gemini. A trivial overview of the NNTP verbs can be found in its grammar a section 9.2 of RFC3977.

Building on what’s there

An inefficient NNTP-like system could be constructed on top of Gemini, at least in part. There is already a convention of using atom.xml to provide feed indexes. These just reference the Gemini URLs of articles, which must be obtained one-at-a-time in the established Gemini fashion. It is likely that these affordances will continue to develop, and may become slightly more standardised. They’re probably not exactly what a Usenet-like system requires but it would sensible to try to reuse existing infrastructure where appropriate.

One of the benefits of Gemini over the Web is the absence of visual design, at least beyond the default layout that GUI clients give you. All sites look basically the same. On the Web, a general-purpose content-management system must perforce support some means of specifying the visual style with which the writing is to be presented, either by permitting arbitrary modification of the style or the selection of one of a finite set of pre-installed themes. Sites like Wikipedia, Medium and Facebook impose a single theme for all content; Myspace, Geocities and Tumblr represent the opposite extreme. Medium looks very good, and is squarely for readers and writers, rather than designers. Wikipedia doesn’t look great on desktop, but is similarly focused on the text not on the non-textual elements, like any serious book or journal article.

The visual design elements of webpage publishing are a source of friction which Gemini avoids. This will make it easier for people to get their serious thoughts shared online, without having to go through a firm like Medium. A decentralised discussion system using Gemini or something like it could include serious formal writing, as opposed to the less serious online discussion systems which are already well-catered for via WhatsApp, IRC, Facebook and Web-based bulletin boards. Email and the Web were originally both unadorned textual mediums, and despite intransigent opposition both have been gradually transformed into mediums incorporating visual design to the point where “plain text” writing looks shabby without significant deviation from the default theming settings. NNTP dates from before this shift. I would like to see a visually minimalist textual discussion system for long-form writing, outside the Web; either this should be build *on* Gemini or built *like* Gemini.

The big difference between Usenet/NNTP and the Web/HTTP is that the data in Usenet is distributed, and copied automatically from server to server. This means if you disconnected a server from the Internet it would still have the data, ideal for intermittently connected devices such as laptops. The distribution model has many other important positive and negative features.

Some would likely prefer to have a visually much richer discussion system. I’d suggest that the content type (HTML, Gemini format, PDF, Markdown, some MIME-encapsulated system for tying attachments to text, whatever) is largely orthogonal to the distribution mechanism: you can use HTML with Gemini protocol and Gemini-formatted files with HTTPS and so on.

I would also commend Secure Scuttlebutt (“SSB”), which has tried to build something a *bit* like Usenet. Those interested in an NNTP successor simply *have* to familiarise themselves with it. SSB is a Curate’s Egg. It combines some very good designs with a clutch of superlatively terrible implementation decisions which are hard to change. What should be discussed is the successor to NNTP *and* Scuttlebutt.

Next steps

What I would propose is trying to dogfood the system we want into existence. Do this by conducting the discussion about an NNTP successor over Gemini gemlogs themselves, rather than email or IRC. This will have the benefit of forcing us to confront any limitations in Gemini and generating new useful content for Gemini. Much of the effort in Gemini is focused on clients, servers and on mailing list discussions which seem to revolve around a handful of individuals agitating *against* Gemini in principle. These are largely distractions from actually *using* Gemini for something.

There are three parts in no particular order:

1) consolidate the use of gemlogs as an organised discussion platform, getting the atom.xml and feed discovery systems working adequately

2) determine what the preferred format for articles is (Gemini format, MIME-encapsulated CommonMark, etc)

3) implement article propagation: Gemini rightly does not and will not support this at the protocol level

This article arose because of an email discussion about NNTP successors. My interlocutor over email (whom I’m not going to identify unless he agrees) raised concerns about the contemporary use of NNTP. Some of these are similar to the critique of Gopher that led to Gemini: Gopher and NNTP are still used in a limited way, but contain well-meaning 1980s features that are hilariously anachronistic today. But his other concerns are deeper and relate to hard policy tradeoffs about privacy and so on.

I say let’s discuss these over gemlogs whilst trying to build the parts of the system which are unaffected by those tradeoffs, which is probably enough of it to be useful. They’ll then be confronted in a context of “running code”, with or without “rough consesus”, as they used to say in times past.

Links

Curate’s egg

Scuttlebutt protocol

NNTP Specification (RFC3977)