💾 Archived View for acidic.website › musings › gemgohane-release.gmi captured on 2021-12-17 at 13:26:06. Gemini links have been rewritten to link to archived content
⬅️ Previous capture (2020-10-31)
-=-=-=-=-=-=-
I've been working on GemGoHaNe a Gemini & Gopher HackerNews Server. The initial v0.1 release only supports Gemini and Gemtext output.
You can find the source code at:
I know a lot of Gemini folks are not the biggest fans of HackerNews. HackerNews to me is what I like to call "low-content entertainment". It takes the place of mindless YouTube scrolling or occupies time when I'm in line at the grocery or otherwise between meetings. I also do genuinely enjoy the diverse perspectives that occasionally surface in HackerNews comments, since software is a vast field and I only work in a portion of it.
There's a decent amount of prior art for this. Two other HackerNews mirrors I'm aware of are:
Both of these are on Gopher so I'm hoping Geminauts will enjoy this mirror. My own implementation is heavily influenced by these two, so I want to be explicit in mentioning them.
GemGoHaNe uses the HackerNews SDK to fetch its data.
Here's what's implemented now:
I don't guarantee that GemGoHaNe development will continue in any regular fashion. I'll probably chip away at features as I see fit and if others find interest in the thing.
Features I plan on implementing soon-ish:
Features that I'll get to a bit later or prioritize if others want them:
There are certainly a few bugs. This is v0.1, aka alpha software. Most Gemgohane testing was done on Darwaza, Elpher, and Dudum, so if users find bugs in other clients please feel free to reach out.
There's also a myriad of things I'd like to fix with the implementation itself but I'll get to those in time, I hope.
The HackerNews API is painful to work with. A story contains IDs of its top-level comments. From there, each comment much be fetched individually from the API, and its child comment IDs are fetched. More requests are made to the API to fetch each comment. If we wish to grab all the comments on the front page, one must make as many HTTP requests as there are comments.
The tree nature of comments and the large number of requests necessary made async a very attractive option, so Gemgohane uses async work to make fetch calls to the upstream API.
Comments are also returned in a subset of HTML (using "simple" tags such as <a>, <i>, and <code>). I had to use an HTML parser to parse each comment and transform them into Gemtext for consumption. Output in HTML and Plain Text will take more HTML transformation logic.
I was floored by how cumbersome the API is to use. Talk about a pain.
Grabbing comment trees (or threads, as I call them in Gemgohane) from the API is quite painful and very chatty over the network. To decrease network traffic I cache all of the comments that I fetch from the API. The cache is kept in memory and has a TTL of 15 min. If a new fetch of upstream threads is in progress, a client accessing Gemgohane will see a 44 response [3] asking the client to wait for 3 min. It takes roughly 2-3 min. to fetch the entire contents of top stories and their comments.
Feel free to send me feedback at meff@meff.me. Thanks for listening!
[1]: This seems to be an aspect of the API itself. I'm not sure why the API does this and if there's a good way to undo this.
[2]: More of a gotcha than a bug, but I wanted to note it.
[3]: I found that most clients (including Darwaza heh) do not support 44s well. Luckily they do work with 4x responses so the effect is good enough.