Self-Hosting on ESP32

This capsule is now self-hosted on an ESP32 development board (ESP32-DevKitC V4) connected to a phone charger and Wi-Fi. When the ESP32 application gets notified that Wi-Fi connection is established, it starts a Gemini server and uses the Duck DNS API to set the address of dimkr-esp32.duckdns.org to the response from ifconfig.me. gemini.dimakrasner.com is now a CNAME record that points to dimkr-esp32.duckdns.org, and I copied my certificate to the ESP32.

https://www.duckdns.org

(I might regret this decision and return to my VPS, if the ESP32 setup turns out to be unstable.)

It didn't take long to implement a Gemini server, but this board definitely has some disadvantages compared to a Pi:

• You must choose an ESP-IDF version, because they're API-incompatible with each other. The ESP-IDF also dictates the versions of libraries, because some "open-source" libraries included in ESP-IDF are provided as binary blobs: you can't update these libraries.
• mbedtls, the default TLS library, supports TLS 1.3 in ESP-IDF 5.0, which is great. However, the sentence "Mbed TLS supports SSL 3.0 up to TLS 1.3" is misleading, because TLS 1.3 support is limited to the client side.
• There's more than one way to do many of the things you'd probably want to do in your application. The examples provide at least 3 different ways to set up a Wi-Fi connection and get notifications about the connection status.
• ESP-TLS is a super simple wrapper around mbedtls (or wolfssl), which reduces the amount of boilerplate code you need to write if you want to implement a simple TLS client or server. This wrapper is very high-level and doesn't allow you to tweak internal configuration easily.
• The HTTPS Server component is implemented as a bunch of callbacks (new session, TX/RX) passed to the HTTP Server component. I haven't found a good, minimal example of a "raw TLS" server.
• The TCP/IP stack has an artificial (?) limitation of CONFIG_LWIP_MAX_SOCKETS (<= 16) sockets. That's enough for many Gemini capsules, but not much.
• It looks like O_NONBLOCK and FIONBIO don't work: esp_tls_server_session_create() blocks. To overcome this, I had to create a "task" for every incoming connection, and that's OK becaue I don't have to care about scale: the number of sockets is limited anyway, and I know I have enough RAM for 16 stacks.
• SO_RCVTIMEO doesn't work as it should: esp_tls_server_session_create() doesn't fail on timeout. The main "task" of my server, which accepts incoming connections, is responsible for killing slow clients and killing the oldest client to make room for a new one. It unblocks esp_tls_server_session_create() by closing the socket.
• idf.py and CMake can be painfully slow, even if ccache is enabled. I have a Windows laptop, which I used for a C# course I had to take when I started my current job: I'm transforming the backend of a product that normally runs on Windows servers into a scalable SaaS offering that runs on Linux with PostgreSql. It seems to me that every file system operation that involves many small files is extremely slow on Windows. Later, I moved my workflow to a much older Linux laptop: thankfully, ESP-IDF has a Linux version, and now it takes <5s to build my application.
• I looked at some power consumption figures and I'm not sure if the ESP32 is as power efficient as some people think. Maybe I misunderstood what I read, but it looks like power consumption is very close to that of a Raspberry Pi Zero 2 W, which has a quad-core AArch64 CPU and 512 MB of RAM. It will be much harder to recommend the ESP32 when Pi prices drop.
• I have concerns about longevity: unlike a Pi, the ESP32 uses on-board flash. I don't like the idea of flashing firmware every time I want to add a post.
• 4 MB of flash is not much.
• RAM is slow: the first version of my Gemini server used a giant blob of concatenated .gmi files and memmem() to search for a magic marker that marks the beginning of a file. It was extremely slow, and now this blob starts with a small array that specifies the offset of a .gmi file, given the CRC32 of its name. This is much faster than the previous solution, even after I zlib-compressed the individual .gmi files and added a decompression step (using miniz) before the response is sent.

https://github.com/richgel999/miniz

More to come later: I'm still cleaning up my code.