💾 Archived View for gemini.circumlunar.space › users › acdw › 2020-05-22.gmi captured on 2020-09-24 at 01:27:06. Gemini links have been rewritten to link to archived content

-=-=-=-=-=-=-

Writing a gemini client in bash

5/22/2020

I thought I'd hop on the train of (to, I'm sure, the chagrin of solderpunk) Gemini-client-writers this Memorial Day weekend. I liked the idea of dylanarap's

birch IRC client

which is written completely in bash -- no external dependencies at all. I thought I could get away with that at first, but I soon found that TLS is pretty tricky to implement (or I didn't want to implement it anyway). I was stuck.

Lucky for me,

a toot by @julienxx

gave me the seed for bollux's functionality:

The tiniest #gemini client

echo gemini://konpeito.media/index-spicy.gmi | 
openssl s_client -crlf -ign_eof -quiet -connect konpeito.media:1965

With that, I was off to the races. I plunked that one-liner into a script and added a little bit of interactivity, and I had a very basic gemini client. I couldn't navigate to other links, or page content, but I could get stuff!

State of the art

For a little bit, I toyed with building my own pager in bash, printing the raw control characters to clear the screen, switch to the alt buffer, and so on. I could not wrap my head around the flow, though, so I decided to use less. It turns out that less was a great choice, because as I was reading around in the manpages I found that less is actually quite powerful! I was able to customize keybindings and use exit codes to communicate with the parent process. Now that I'm thinking about it, though, I could use the pipe command to bollux itself with an argument that will let it open links and what-not. But I'm getting ahead of myself.

So here's what happens when you open a link in bollux:

request() gets called with the url you're looking for and begins to download the content.
request() pipes the response to handle_response(), which slurps up the stdin to the first "\r", which delineates the header (actually "\r\n", but I'm trusting servers not to serve a solo "\r" in the META field).
I have a big case block to decide what to do with the content depending on the statuscode of the request. This is where I still have the most work to do -- I need to write handlers for 40, 50, and 60 responses, which will entail history tracking and client certificate generation, at least.
For 10 responses, bollux will read input from /dev/tty and pass it along as a query back to the server. This was pretty hard to get right, as I tried messing around with file descriptors for a while before remembering I could just force reading from the tty.
For 30 responses, bollux redirects to the url directed in the META. I need to make it ask if it's redirecting to another domain, probably.
For 20 responses, I run the display() function, which figures out what to do based on the mimetype: for text/*, it prints it with less, and with anything else, it downloads it to a file using dd for progress printing. I could probably make the downloading bit a little nicer.
Less has a custom lesskey defined that makes it exit with different codes. This was the only way I could figure out to make it communicate with the parent process while keeping bollux to one file -- the pipe and shell commands in less don't work with functions. Of course, as I wrote above I just realized I could run bollux again with some 'hidden' parameters to make it do what I want.
If less exits with 0, bollux quits. Otherwise, the exit code is passed along to hande_keypress(), which decides what to do based on the exit code (for some reason less adds custom codes to 48, who knows why). It can open links from within the page, go to a new link, or refresh the page. The next thing I want to do is add history browsing, which I don't think will be too hard.

Challenges

I ran into a number of challenges while writing bollux. I was stuck on file descriptors for a while, trying to exec 3<>/dev/fd/0 and read and write from that, or read -u 4 to get user input, or many other things.. I don't remember them all. After a while, I figured out that I just had to stay on &0 and &1 (i.e., /dev/stdin and /dev/stdout) for the gemini content, and when I wanted to hold up the program for input from the user, just run `read </dev/tty`.

The other big problem came in when I tried to clean up the code by writing it from scratch (I know it's not efficient, but it's how I do things). For some reason, the read-handle pipeline was just. not. working. I worked at it for like, 4 hours, before finally re-re-writing it and fixing it somehow.

Meta-note on this section: I'm realizing that I don't really remember what challenges I faced while writing bollux, probably because they were so traumatizing I've blocked them out from my mind. Of course, that can cause its own problems, as I outline in the very next section.

Release! (and lessons learned)

Finally, I was finished. I packaged it up, put it up

online,

and told the world about it. Then I went to bed because I was tired.

This morning, I woke up to find that the repo wasn't posted. Apparently, sr.ht's projects won't display a repo that's private/unlisted, even if it's set to public after it's added to the project. But I was able to remove the repo and re-add it, so I thought it was all smooth sailing.

But then, I got to work. And I started reading the mailing list. And I checked the sourcehut project. And I started getting pinged on Mastodon ...

Bollux wasn't working for anyone!!!

I was panicked. I can't SSH anywhere or do anything at work, really, so I wasn't sure what the problem was. I finally signed in to Python Anywhere (which has bash shells that I use for light development when I'm supposed to be working), copy-and-pasted bollux, and tried to find the issue. Turns out, this is the offending code:

	ssl_cmd=(openssl s_client -crlf -quiet -connect "$server/$port")
-----------------------------------------------------------^ R I G H T. H E R E.

Yep, that's a slash. Not a colon, which is what it's supposed to be. Last night I was reading through the OpenSSL manpage and I thought it said you could use a slash instead of a colon to separate the port, for ease-of-use with IPv6. So I changed it to a slash because ... I thought it *looked* prettier.

I broke bollux because of a e s t h e t i c s that no one would ever even see. :'(

So I changed it in Python Anywhere and tried to run it, and it *seems* to work. The problem is that now the servers kick the request, I'm hoping because the system on PA is neutered in some way. I'm going to make the change when I get home and TEST it, of course, then reply in all the places people are seeing issues right now (which includes

geddit,

the cool new reddit-like in gemspace.

I'm definitely testing software before releasing it next time.