💾 Archived View for auragem.letz.dev › devlog › 20240316.gmi captured on 2024-03-21 at 15:04:55. Gemini links have been rewritten to link to archived content
-=-=-=-=-=-=-
I want to announce a new project I am working on, called the Scroll Protocol.
The Scroll Protocol is a new document-retrieval protocol on port 5699, inspired by Gemini and Gopher+. One of its primary focuses is on quality-of-life additions, particularly internationalization. It adds more metadata to responses, the ability to request pages in a specific language, and a new metadata/abstract request, similar to Gopher+'s metadata request. It tries to offer a power-to-weight balance by providing basic metadata and relational data in documents with minimal effort and additions.
The additional metadata added to each response is the author, publish date, and modification date. Abstracts of resources are also provided with *metadata requests* only.
Request strings now include an additional parameter that allows the client to specify a list of desired languages. The server will respond with the document in one of the desired languages, or the default language if the requested languages are unavailabe. The language of the response is given in the mimetype.
See the "Why a New Protocol" section below for why I think Scroll's features warrant the creation of a new protocol.
The Reference Scroll Terminal Client supports Scroll, Gemini, and Nex, and has audio streaming support. It was somewhat modelled after Solderpunk's AV-98. Graphical and TUI versions are also being worked on.
The Reference Scroll Server is a very simplified version of SIS that only supports serving static files over the Scroll Protocol. It does support virtual hosting. CGI is coming soon. For more advanced features, use SIS.
Reference Scroll Terminal Client (golang)
Reference Scroll Server (golang)
You can see example images of the Scroll-Term on the repo.
I originally wasn't going to create any new protocols that mimic much of the same functionality as Gemini or Titan or Gopher. What changed my mind then?
The Scroll Protocol takes some inspiration from Gopher+. I read mozz's journal entry about Gopher+ recently:
Report: State of Gopher+, 2023
This led me to reading the Gopher+ spec:
And while reading this, even though there were a lot of convoluted things that Scroll deliberately does not take from Gopher+ (a scripts section?! yuck!!), there were a couple of interesting things that jumped out as being useful, particularly the metadata requests and the concept of abstracts.
In Gemini there is a lack of semantic detail. Gemtext files don't have a standardized way of getting the publication date or modification date or even an author metadata field. However, we got around this by using gemsub feeds, but it is very often that I come across gemlog pages where the publication date is on the gemsub feed but *not* in the gemlog page itself! And what about other filetypes that also don't include this metadata information, particularly text files? What if we just added these three extremely common metadata fields to all responses?
Gopher+'s Metadata requests are really interesting because they allow you to just get the metadata of a resource without actually getting that resource. This is actually really useful for browsers displaying information *before* getting the whole resource. This is even particularly useful for Search Engines and Crawlers to just check the existance of a resource, or whether it has been updated even, or to get its content type, before getting the actual file.
Gopher+ also adds in these "abstracts" to the Metadata requests. Abstracts offer a description of the file. This lets someone get a brief description of a file before spending resources downloading it. Abstracts in Scroll also offer the title of the resource so browsers and crawlers can get the title before downloading the resource. One use of an abstract could be to list the currently playing song of an audio stream within the abstract, and AuraGem's public radio does just this.
All of the above, however, introduces a new concept: a distinction between contextual information and non-contextual information. When you link to a resource from a page, you title the link in a way that it's contextual to that page. However, resource metadata is non-contextual; it doesn't depend on the context of another page. Both contextual and non-contextual information is useful, and this to me justifies the use of documents like in gemini and the web over file listings like in Gopher.
Contextual information lends itself towards semantic information when linking between pages. This is one of the few things that search engines really need but that nobody has effectively created a solution for: how do search engines know when a link to a page implies a positive or negative relationship? Search Engines that use pagerank currently assume all links imply a positive relationship. Scrolltext adds a very simple tag to the end of all links to identify these relationships. It is done in a human-readable way so that they are useful to users, and so that they don't distract too much when the browser doesn't support these tags.
Lastly, and the biggest addition to scroll that I think justifies and warrants its existance, is internationalization, which Gemini and Gopher sorely lack. Gemini offers a lang parameter on responses, and allows UTF-8, which is already more than Gopher does, but there is no way to *request* resources of a particular language without creating a whole different link to the resource adding some language tag to the beginning of the link's path. Scroll solves this by adding a field to all requests for the languages the user desires, and then the server responds with a match for one of the languages, or a fallback if no matches.
I chose this method because the language should not be in links but be requested by the user's client. Links assume everyone reads in the same language as the one who posted the link. Rather, it should be the client, not the content, that decides what language is requested, *and* clients can offer up multiple languages in the request instead of just one language if the user speaks multiple languages. This is such an easy addition that I think will vastly improve internationalization. It also means all error messages may be internationalized by the server!
For all of the reasons above, I felt that these additions were very simple additions, and yet powerful enough, to warrant the creation of a new protocol.