💾 Archived View for tilde.team › ~benk › dad1bf70.gmi captured on 2022-01-08 at 13:41:41. Gemini links have been rewritten to link to archived content

-=-=-=-=-=-=-

Re: awk is the coolest tool you don't know

Authors: Ben K. <benk@tilde.team>

Date: 2021-05-04

gemini://drewdevault.com/2021/05/03/awk-is-the-coolest-tool-you-dont-know.gmi

It was quite fortitous that I should come across this today, because it made me realize that awk is exactly the sort of thing that I have been needing to analyze my geminid server logs, for which I had no ready-made utility. Specifically, I wanted to know how much bandwidth my gemcast is using, or at least how much it MIGHT be using given what information is available in the logs. (I don't think geminid reports how much data was actually transferred, but it does seem to give the size of the files requested.)

I would say in just under an hour of tinkering (maybe even less than half an hour) I had the core program working in awk, helped along by sh. awk is surprisngly simple and easy to use, which makes me feel foolish for not having learned it twenty years ago when I first learned a bunch of arcane Unix things from books. (Believe it or not, my primary learning resource back then wasn't the Internet.)

So it would appear that since the gemcast debuted in August of last year, estimated monthly bandwidth usage doubled from roughly one gigabyte in the beginning to around two gigabytes since November. There was a slump in January, but the growth in traffic seems to coincide with a small growth in listenership since traffic didn't appear to be too adversely affected by the fact that I made few to no new episodes in the span of maybe three or four months. February and March were my biggest months:

Feb 2021:
146 files requested for a total of 2.6 GB

Mar 2021:
288 files requested for a total of 2.5 GB

Over the gemcast's lifetime, 1152 files were requested for a total of 16.0 GB. Way more than I was expecting!

M. Bays suggested that I try to compress my files more if I would like to reduce bandwidth usage. I can reasonably part with two gigs a month, so it should be fine. He suggested trying Speex, but I've experimented with it in the past and was not too satisfied with its quality. Opus might be a better choice, so maybe I'll do some listening tests this month and see if the bitrate can be pushed down a little bit. (I love Vorbis, though.)

One interesting quirk with FreeBSD's awk is that it appears not to use the CONVFMT variable as is standard now (probably on Linux), even though the variable is set by default. That resulted in a few minutes of frusration until I saw in some tutorial that older versions of awk used OFMT, and sure enough setting that did the trick.