💾 Archived View for zaibatsu.circumlunar.space › ~solderpunk › gemlog › roophloch-2023-explanation-p… captured on 2023-11-04 at 11:28:11. Gemini links have been rewritten to link to archived content
-=-=-=-=-=-=-
Well, it's the last day of the month already! I guess strictly speaking it's perfectly valid to make a ROOPHLOCH post itself during September and then explain exactly how you did it in a follow-up post in October, but I don't want to leave this too long. I made my post in my phlog back on the 19th, and didn't bother manually reposting it to my gemlog later, so if you are a Gemini-only reader and missed it, here's the link:
ROOPHLOCH 2023 433 MHz transmission 01
Way back when I lived in Auckland, I bought a pair of of 433 MHz radio modules made by the Chinese company Dorji, one transmitter and one receiver, from the (now sadly defunct!) Surplustronics store on Queen Street. These operate in a "Industrial, Scientific and Medical device" (ISM) frequency band which is available for license free use pretty well everywhere on Earth and is heavily used in basic wireless consumer devices like garage door openers, doorbells, hobby weather stations and stuff like that. I was pretty heavily into 8-bit homebrew computing at the time and was still extremely naive about anything to do with radio. I don't think I was even listening to shortwave broadcasts at this point in time. I vaguely recall that I bought these modules hoping they would let me easily add remote peripherals to my Z80 machine. They have a super basic interface, the transmitter has a single digital input pin and the receiver has a chip select pin (to switch the receiver in and out of a low power standby mode) and a single digital output, and aside from power connections that is all of it. I think I hoped I could just hook these up between UART or SPI/I2C devices and it would "just work", basically carrying an arbitrary 5V TTL signal from point A to point B.
It didn't "just work". These are Amplitude Shift Keying (ASK) devices. When the transmitter is turned on, it emits a 433 MHz carrier continuously at one of two different non-zero amplitudes, one corresponding to a logic low input and the other to a logic high. If you think about this a moment, it's clear that when you hold the input steady, the RF output is entirely ambiguous to a receiver. The receiver can tell there's a carrier there with some signal strength, but is that a low power carrier from a nearby device or a high power carrier from a distant device? The whole thing only works if the data signal is transitioning between the two states frequently enough that the receiver can "lock on" by holding the RF amplifier gain at the right level relative to the decoding comparator threshold. Proper devices using this system rely on things like Manchester encoding schemes to ensure that the transmitted bitstream flips between 0 and 1 constantly, even if the input bitstream includes long constant runs. Even trickier, the receiver has no squelch functionality. If the receiver is off and there's no carrier floating around, the transmitter just cranks the gain way up until random noise induces chaotic high speed bit flipping in the output. So proper devices again usually precede any actual transmissions with a kind of "wake up" sequence where a known bitstream is repeated over and over again enough times that the probability of that exact sequence appearing by chance is effectively zero, and the receiver has to constantly poll for it. This is all very workable but it's waaay more complicated than what I was looking for at the time, which was basically a magic invisible wire to carry logic level signals from point A to point B, so these guy have sat idle in my junk box ever since.
They've been in the back of my mind every time ROOPHLOCH has rolled around, but this year I finally decided to try to put them to good use. Last year, when I kind of dropped off the internet for nine months or so, I got pretty heavily into homebrew radio projects and spent a lot of time and energy in particular on decoding digital maritime utility signals, especially NAVTEX on 518 kHz, but also some HF NBDP stations and later HF DSC. This is still a subject pretty close to my heart and it's kind of astonishing that I have written *zero* words about it on Gopher/Gemini. It'll happen at some point, I promise you all! Anyway, these are all Binary Frequency Shift Keying (BFSK) signals, where the RF carrier switches between two slightly offset frequencies to represents 0s and 1s, at 100 baud in all these applications. It's exactly the RF equivalent of the AFSK schemes used to represent bit streams in audio by both early modems and cassette-based storage systems in home computers, and in fact the standard way to decode these signals, as a hobbyist at least, and I guess in the pre-SDR era that I'm still heavily invested in, is to shift the signal down to the audio range with an SSB receiver and then feed that into a soundcard. All of which just means that, unlike my radio-naive earlier self when I bought these modules, this year the question of "how do I easily transmit binary data using these things that only work with constantly alternating input streams and receive random noise in the absence of such?" had a blindingly obvious answer.
I didn't want to build anything permanent for the sake of this ROOPHLOCH experiment, both to save time soldering and avoid committing parts to what might not work very well, and I also didn't want to invest very much effort in programming. So for the first time in a long time, I pulled out my Arduino. Or, strictly speaking, I grabbed my "Eleven", a 100% Arduino Uno-compatible AVR dev board made by the Australian company Freetronics, which I purchased from a JayCar circa 2010 when I was just starting to get back into electronics again. It was my first encounter with microcontrollers, and I used it enthusiastically for a year or two before I learned how to use AVR microcontrollers directly, using avr-gcc and avrdude in conjunction with a USBtinyISP programmer. It's been pretty unloved since then, but it comes in handy for very quick-and-dirty proof-of-concept projects like this once. I set one of the Arduino's analog outputs to 128, i.e. the midpoint if the 8-bit output range. Because the Arduino doesn't have a true DAC but uses PWM, this caused a 50% duty-cycle square wave to appear on an output pin. With the default settings, the PWM frequency is a surprisingly low ~1 kHz, very much in the audio frequency range.
I fed this into the input of a 4040 divider chip, which gave me an additional 500 Hz squarewave output (also 250 Hz, 125 Hz, etc., but I made no use of these). The 1 kHz and 500 Hz waves were fed into two of the inputs of a 4052 analog multiplexer, the output of which I fed into the 433 MHz transmitter. This basically meant that by toggling a single one of the 4052's input selector pins, I could feed the transmitter with a squarewave at one of two audio frequencies. Because a squarewave is constantly alternating, it travels very cleanly through the ASK modulation/demodulation chain and basically this setup just beams audio frequency square waves from one circuit to another. By toggling a single pin you shift the pitch of the audio tone, and you can toggle the tone as slowly as you like or hold it steady for arbitrarily long periods. At this point, you could user various hardware FSK decoders like a pair of LM567s with some glue or an NJM2211 to recover logic high/low outputs and then you really could use these modules to pass the signal from a UART or SPI/I2C bus without worrying about the fact that they idle at a fixed level (in this scenario I'd probably use a quartz oscillator instead of an mcu PWM output). However, for ROOPHLOCH I wanted to keep things fast and simple, and since the two frequencies were in the audio range and I had been dreaming of using minimodem since the very first ROOPHLOCH, I decided to just treat the receiver output as audio. After all, the message had to get into a computer anyway in order for me to be able to push it into Gopherspace.
My initial plan was to use the Arduino's UART output to drive the multiplexer, so that the firmware "sketch" could be as simple as Serial.println()-ing my post. This didn't actually work out at all, and a little consideration of timescales makes it clear why. At the default baudrate of 9600, the UART switches state once every 104 microseconds, while the two squarewaves have periods of 1000 and 2000 microseconds, so the tones don't come through cleanly at all. Without changing the Atmega328's clock frequency (which would presumably stop the Arduino IDE being able to talk to the bootloader), its UART baudrate generator can't go any lower than about 240 baud. I probably could have played around the the timer prescalers and bumped the audio frequencies up an octave or two and found a baud rate that worked, but in the end I took the lazy way out and just resorted to bit-banging UART output on a digital pin. At a relaxed 100 baud, you can do it using delay() like a noob rather than bothering with interrupts and it still works.
At first I fed the 5V squarewave output from the receiver through a resistor divider and then into my stereo, planning to use the microphone in my Thinkpad to receive the audio, but the decoding from this was surprisingly noisy. So I fed the divided output directly into the input RCAs of my trusty Behringer UCA202 instead and it became absolutely bullet-proof. I could burn the text of a post into the Sketch (by keeping the post short it fit within the 2 kB of RAM and I didn't need to use PROGMEM techniques) and then recover that text completely using minimodem. I was pretty pleased at this point!
All that remained was to automate the process of receiving and publishing the post, so I could set a script running on my laptop indoors, then take the Arduino outdoors, plug it into a USB powerbank and wait, using my phone to check Gopher and see if the post had appeared. I spent a while experimenting with various minimodem options to get this as solid as I could. The --rx-one option will cause minimodem to exit after a single gain and loss of carrier event, and --quiet will stop it reporting those events and various other diagnostics. Using the --confidence option to set a high confidence threshold makes sure that minimodem is very unlikely to misinterpret a burst of noise as a very quick gain and loss of carrier, which in conjunction with --rx-one would cause the script to exit while I was still on my way down the stairs. Long periods of indoor testing confirmed that with this suite of options, I could start the script running and leave it running for even an hour with the transmitter turned off without any false starts - I had a very stable "standby" mode. When powering up the Arduino, the text of the post would usually be received reliably but there were two failure modes which happened often enough to worry me. One was simply that the first one or two characters were dropped, but minimodem would also sometimes terminate after receiving a single garbage character. I adjusted the sketch to idle with the UART pin high for ten seconds before it started actually transmitting, and I also had it transmit 10 0x02 bytes before starting to send content, so I could leverage minimodem's --sync-byte option, which will suppress carrier acquisition until it has received a certain byte multiple times. 0x02 is the ASCII control code for "Start of Text", so it seemed appropriate to use here. I used `tee` to capture the output of minimodem run with all these options into a .txt file, had the script use `wc` to double check that more than a single line had been received (to protect from publishing one of those mysterious single garbage characters), and then simply did a git add, commit and push to publish. I manually updated my Gophermap to include a link to the received file before starting the script running.
I positioned my laptop with the receive script running near a window, bundled the Arduino and the breadboard with the 4000 chips and the 433 MHz transmitter into a shoebox, with a USB cable to power the Arduino dangling out from under the lid, put a USB powerbank into my pocket and headed outside. This was by far my most technically ambitious ROOPHLOCH post yet, but also by far the least ambitious in terms of how "remote" a location I was posting from. Before I even got as far as doing anything with minimodem, I did a few rough and ready range tests where I just fed the transmitter a square wave, displayed the receiver's output on an oscilloscope screen, took the transmitter outside, phoned my wife and asked her to tell me when the scope stopped showing a nice clean, steady square wave and started flickering with transient garbage, while I wandered in various directions to figure out the plausible boundary of where I could post from. Reception seemed pretty solid out to 50 meters, but I didn't want to take any risks for the actual post, so I just sat on the doorstep of a closed cafe on the other side of the street from me and plugged the power bank in and hoped for the best. A couple of Mormon missionaries strolled along while I was waiting and I was briefly afraid they would take advantage of the unusual scene to ask me why I was trying recharge a shoebox before as an icebreaker before trying to convert me, but in the end they just passed me by. I kept furiously refreshing PocketGopher waiting for my post to appear, cursing myself for being too lazy to put an LED on the breadboard to signal when transmission was complete. Eventually, dejectedly, I admitted to myself that it obviously hadn't worked.
I came back upstairs and confirmed that minimodem had had one of its spurious receive events, but my `wc` sanity check had spared me the embarrassment of posting it. I decided to give it one more shot, started the script up and took my box outside again. Sat in the same location, did exactly the same thing. And it worked!!! It worked perfectly, with no dropped characters, you'd never know that it was uploaded in such an unusual way. I aspirationally titled the post "transmission 01", as I hoped to try a few additional experiments, including a solar powered post. In fact, I *did* try the solar post, with low expectations as I was using a very dinky little 5x10cm panel and the linear regulator on the Arduino board is not exactly highly efficient. To my surprise, the LEDs on the Arduino stayed on and solid and bright under midday sun, but I minimodem never picked up a thing, and I didn't make the time to dig into it too deep. It's nice to leave some targets for next year, anyway.
I also wanted to try to increase the range of the setup, but didn't end up making time for that either. I did make a little coil-loaded antenna for the receiver using some solid copper wire, but it didn't actually seem to help any. I didn't add an antenna to the transmitter. ISM devices are required to accept any interference they encounter because it's a shared band and nobody is guaranteed exclusive access, but I wanted to try to be a good neighbour during all my experimenting and not stomp on other people's signals, so trying to optimise the receiver and leave the transmitter modest seemed appropriate. It seemed to work pretty solidly out to 50 meters. Maybe I could have mounted the receiver outside a window to get some extra distance without too much effort. Folks online have built directional Yagi antennas for modules like this and reported good results. I'm pretty sure 100 meters would be achievable without extreme efforts, and that would be enough that I could at least sit in a grassy area with some trees behind my local supermarket. I mean, it's not exactly a salubrious locale, but at least it has a vaguely outdoors vibe. Something else to try next year, perhaps. It's slightly ridiculous in this day and age of LoRaWAN (which also functions, at least in some parts of the world, in the 433 MHz ISM band) to continue trying to coax what is basically a garage door opener into a reliable long distance communication tool, but this kind of thing is exactly my jam. Where's the fun in buying stuff that just works out of the box?