💾 Archived View for zaibatsu.circumlunar.space › ~solderpunk › phlog › progress-toward-offline-first… captured on 2020-09-24 at 01:45:38.
-=-=-=-=-=-=-
Progress towards "offline first" -------------------------------- I wrote previously about how I was going to try "computing less, but with more focus"[1], with part of this being moving toward an "offline first" mode of computing, where I connect to the internet only when I need to for as long as I need to to accomplish some specific task. My primary motivation for this was to cut down on distraction and procrastination, both of which a permanent internet connection facilitates wonderfully, and to turn sitting down behind a computer into something I do with a clear purpose, instead of an entry into some kind of default zombie-like state - precisely the way a lot of people treat sitting in front of a television, in fact. My recent musings on sustainable computing have provided some secondary motivation, but it really is mainly about being on my computer less, but getting more done behind it at the same time. In addition to being offline first, I wanted to try setting weekly "focal points" for my computing. I might try to do regular weekly recap posts where I summarise what I did, but then again I might not. But I'm going to write such a recap (you're reading it!) for my first week of this system, because this past week's focal point has been gearing up for precisely this shift, and I thought that might be of interest to some people. I have decided to experiment with an approach where I have a single script - named `~/bin/do-the-internet.sh` - which, well, "does the internet". It's a kind of batch script which handles all the non-interactive aspects of my internet usage. Obviously it cannot, for example, look something up on Wikipedia for me which I was wondering about earlier in the day, but it should be able to handle all of the mechanical stuff I need to do, freeing me up to focus on precisely things like that. So far I have managed to get it handling the publication of my Gopher and Gemini content, and the receiving and sending of mail. Handling publication was relatively straightforward using git. Up until now I have maintained a very manual and hands-on approach to all my publication. I usually write posts like this one locally, then scp them up to the appropriate server, and edit the gophermap or index.gmi by hand to link to it. Now I've changed things so that each server has a bare git repository on it containing all the files for my online presences. I have cloned these to my local machine, and make all changes there. When I push changes back to the origin repository, a post-receive hook checks out a copy of the files to the location where the appropriate server serves content from. I have Gophernicus and Molly Brown running locally pointed at my cloned repositories, so I can test that things look and work the way I intend before I push them to the wider world, and so I can easily refer back to stuff I have written previously. This all works pretty smoothly. This part of my do-the-internet.sh script is very simple:
echo "Pushing gopherhole..."
cd ~/gopher && git push
echo "Pushing gemini capsule..."
cd ~/gemini && git push
I can probably make this faster by only actually running `git push` if the local repository is known to be ahead of the remote origin. I expect this must be very easy, but my knowledge of both git and shell scripting is actually quite weak, so I have no idea how to do. Pointers are welcome. If I can indeed get these lines to run quickly (without setting up an ssh connection when there's nothing to push), I will add my most regularly updated software projects to the list. Handling email was not as straightforward, involving multiple separate tools for different parts of the process, but I think I have something quite nice in place now, based around keeping mail in local Maildir folders. When I recently switched from using SDF as a mail provider to Posteo, I also switched from using venerable old `mutt` (while ssh'ed into SDF) to using `aerc`[2], so everything I am about to describe here is done in an aerc environment, although you could just as easily do the exact same thing in mutt. I am using a program called `getmail`[3] to fetch mail from my posteo.net account via IMAP and save the messages to a local folder. It's more traditional to use POP for this kind of thing, but IMAP, along with some smart local state maintenance, allows a little more flexibility: instead of emails being removed from the server as soon as they are downloaded, they can be left on there for a scheduled number of days and then deleted. This means that recent enough emails can still be read on the server via IMAP or a webmail interface when I am not at my main machine with the offline Maildir, but if Posteo's servers are ever compromised, only a small subsample of my mail becomes available to the perpetrator. In order to handle sending mails, I am using the `msmtp` SMTP client tool[4], along with a community-provided "queuing script"[5]. I have configured aerc to use the `msmtpq` (as in "msmstp queue") script as a sendmail program. All that script does is save the email it receives, and its command line arguments, to local files in a queue directory. The process of "sending an email", from within aerc, feels instantaneous, as all it involves is writing to the disk. Within my do-the-internet.sh script, I use the corresponding `msmtpq-flush` script to actually send those emails to Posteo's SMTP server for actual delivery. Basically, the overall workflow goes like this: I connect to the internet, and run my do-the-internet.sh script. This downloads new mail to a local Maildir. Once that's done, I disconnect from the internet. I read the messages offline at my leisure. When I have the time and energy, I write replies, which are stored on the disk. The next time I connect to the internet and run my do-the-internet.sh script, those replies actually get sent to where they are going, and the next batch of new email comes in and the cycle begins again. Again, this part of the script looks very simple, with all of the details hidden in config files:
echo "Fetching Posteo mail..."
getmail --rcfile posteo.net
echo "Sending queued Posteo mail..."
~/bin/msmtpq-flush
(Apologies to everybody who grew up on dialup, for whom this approach is familiar and obvious and doesn't need careful explanation, but a lot of younger folks will have never used email this way in their life) My circumlunar email doesn't fit this model as well. Because that service is intended primarily as an internal mail system for circumlunar colonies, plus a few other pubnixes, there are no IMAP or POP services running. I am using rsync over ssh to simply pull down a copy of the Maildir which lives in my Zaibatsu home directory - this way I can at least read old emails while offline, and also know when new mail has arrived so I can ssh in and respond to it. I guess I could setup some kind of system where replies are stored locally, synced back up to the Zaibatsu and then sent via a cron job, but I'm not sure it's worth it. The biggest gap in my setup right now is consuming other people's Gopher and Gemini content. I anticipate a new software project in the very near future, which checks a small, pre-defined number of endpoints via both protocols, identifies URLs it has not seen previously, fetches each of them, saves the content to disk and generates a local index which makes it clear what has been recently fetched. In an ideal world this would just be a fairly standard Atom/RSS feed reading tool, but the reality is that on both Gopher and Gemini the majority of content is not syndicated that way, and this is unlikely to change anytime soon. So, some creativity will be required, but I've been thinking about this for a while and I think it can be made to work in a quite simple way. If I can get such a tool working and add it to my do-the-internet.sh script, then I can get my Gopher/Gemini workflow working much like my email workflow: I grab fresh batches of new content every now and then while online, read them at my leisure, write responses at my leisure and then push them to my server next time I'm online. I really look forward to having this work nicely. None of this addresses the "looking stuff up on demand" part of my internet usage. I don't think it will ever be possible to eliminate this entirely, but I do suspect it can probably be reduced substantially. There are multiple projects aimed at making offline reading of Wikipedia possible, and that would certainly go a long way. I think that this undertaking might also end up turning me into something of a fanatic about packaging documentation with software. Good quality man pages make a world of difference to being able to solve problems without going online. The Go programming language has also proven itself wonderful in this regard - there is a tool called `godoc` which you can use to run a local webserver which generates static web documentation for the standard library version which is installed on your machine, plus any third party libraries you have installed with `go get`. It is exactly the same software used to serve the official online documentation at golang.org, and all you need to do in order to be able to access it while offline is run `godoc -http localhost:8080`, simple as that. This is incredible, and all programming languages should aspire to similar heights. The only thing I'm left kind of wondering about is the Fediverse. No doubt it would be possible in principle to write some kind of tool to implement the periodic push/pull workflow for toots, but somehow this doesn't feel like a good match for the medium. Things like email, and phlog/gemlog posts, are very well suited to long-lived discussions which unfold over the course of days or weeks. Fediverse threads tend to progress more quickly and have much shorter lives. I don't really want to just up and leave, though. For all the medium's very real faults, it is the easiest way I have for casual interaction with a growing group of like-minded people who have made tremendous positive contributions to my thinking and beliefs in recent years, and I honestly think I'd be worse off for cutting the cord. Perhaps I will just assign myself roughly scheduled times to dedicate to it - a half hour session one weekday evening and on each day of the weekend, perhaps. It doesn't count as distraction or time wasting if you sit down specifically to do it! I'd log out and close all related tabs outside of those times. Or perhaps I'll only use it on my phone, I dunno. Obviously, working on this system over the past week involved being online quite a bit in order to test it out, so I wasn't really as offline as I intended. I'm looking forward to testing the new system out in earnest this coming week. As for a new focal point, as keen as I am to get to work on some kind of tool for fetching Gopher/Gemini content for offline reading, I think first I will spend some time working on AV-98, which is still lagging quite far behind some recent(ish) simplifying updates to the Gemini protocol specification. It also doesn't yet support the "Gemini stream" idea. While I am working on it, I might try to add support for some kind of lightweight solution for offline reading - something like a "stash" command which works kind of like "tour" but rather than just queuing up URLs actually grabs content and queues that up in memory... [1] gopher://zaibatsu.circumlunar.space:70/0/~solderpunk/phlog/computing-less-but-with-more-focus.txt [2] https://aerc-mail.org/ [3] http://pyropus.ca/software/getmail/ [4] https://marlam.de/msmtp/ [5] https://github.com/marlam/msmtp-mirror/tree/master/scripts/msmtpqueue