💾 Archived View for skyjake.fi › gemlog › 2021-07_lagrange-1.6.gmi captured on 2023-06-14 at 14:08:26. Gemini links have been rewritten to link to archived content
⬅️ Previous capture (2023-01-29)
-=-=-=-=-=-=-
I spent late June and early July largely in vacation mode so Lagrange's monthly release pattern was broken. I've managed to make some nice progress over the past couple of weeks, though, so the midsummer harvest is here! This version has a host of improvements to existing functionality and one brand new feature as well.
Please note the following when upgrading:
Version 1.6 takes a big leap forward in supporting the full range of languages that Unicode can encode. Bidirectional text and complex scripts (e.g., Arabic) are now properly parsed and rendered. This is largely thanks to two very nice libraries:
HarfBuzz (for converting Unicode codepoints into to an arrangement of font glyphs)
GNU FriBidi (for implementing the Unicode Bidirectional Algorithm)
In practice, this means that — if the fonts are available — one can now view text written in right-to-left (RTL) languages in Lagrange. You can have RTL words and phrases appearing inside left-to-right paragaraphs. In full-RTL paragraphs (as detected by FriBidi), the margin decorations for links, list bullets and quotes are also moved to the right side for completeness.
Screenshot of bidirectional text (artificial example for demonstration purposes)
In a complex script, characters may be combined together from multiple smaller glyphs and they may change appearance depending on the context. HarfBuzz handles all of this, and as a bonus it also handles the "regular" Unicode combining classes so one can now freely append diacritics and other combining marks and they are positioned correctly.
Typically apps support bidi text by using UI components provided by some larger UI framework that is installed on or built into the operating system. In Lagrange's case, my insistance on having a custom cross-platform UI exposes this complexity and requires handling it manually. I'm grateful that there are widely-used and well-tested solutions like HarfBuzz, because this stuff can be quite complicated to get right.
Here's how the list looks now:
☑︎ UI localization
☑︎ Content translation
☐ Font library to support all of the world's scripts
☑︎ Bidirectional Unicode text
In other words, we're almost there! 🎉 This makes me happy. Gemini is technology that should be available to everyone no matter which language you speak.
Expanding the font library remains as the last challenge. In practice, it'll require at least partially adopting operating system specific text rendering APIs, so it seems likely that Lagrange's font support will not be universally the same on every operating system where you can compile the app.
How "proper" the bidi text rendering actually is remains a bit of an open question... While HarfBuzz and FriBidi do the heavy lifting, my own code is responsible for typesetting a sequence of LTR and RTL runs inside a paragraph. For example, things get tricky when you have an LTR paragraph, but there's a sequence of RTL runs that need to wrap onto the next line. Real world testing will be required, and people fluent in various languages need to give it a go. My own testing has been limited to comparing the output to that of other apps and web browsers, so plenty of nuance may have been missed.
Please note the addition of new build dependencies. I've attempted to make things easy, but I don't have full confidence that every possible permutation of build options will work. Let me know if you have problems, and I'll keep working on it. On operating systems like Linux, where it's reasonable to assume the libraries are available, it's best to build with the system-provided ones (via pkg-config). However, HarfBuzz and FriBidi are also available as Git submodules so they can be compiled together with Lagrange as minimal static libraries, avoiding any unused dependencies (such as FreeType) or version mismatches. There's also the option of disabling support for bidi text and complex scripts, falling back to the simple text renderer used in previous versions of the app. These are not small libraries (HarfBuzz is a bunch of C++), so they have a real impact on the compilation time and executable size. Fortunately, even if you're compiling the app yourself, you shouldn't need to build the dependencies more than once.
The Gemini protocol only enables sending up to 1024 bytes of data in a request. Furthermore, the request URL also counts against that limit, and the sent data must be percent-encoded so it can be parsed as a valid URL. Consequently, Gemini clients can only send very limited amounts of data to a server. While this choice guarantees a level of simplicity for both server and client implementations, it limits how and where the protocol can be applied.
Geminispace has legitimate use cases for transferring data from a client to the server. For example, one might want to
Gemini's restrictions on uploads necessitate finding orthogonal solutions for these, like using the web, SFTP transfers, some unsecure SMTP contraption, or having the server fetch a URL that the client provides. While these can be made to work, they are not simple to set up, nor are they convenient for the user, and they require managing user authentication via alternative means like user accounts and passwords.
There is a theoretical elegance to using well-known internet protocols for these special use cases, but in practice it can lead to brittle Rube Goldbergian arrangements. Wouldn't it be nice to sidestep all that and leverage Gemini TLS connections and existing client certificates to _write_ content in addition to just _reading_ it?
I believe it is the right choice for Gemini to be limited to 1024 bytes because it establishes a sufficiently low baseline of functionality that can be shared by all clients and servers. Supporting arbitrary uploads would complicate both server and client implementations with things like requiring to have a mechanism for the user to select files to upload. Also, for many capsules there is no need to enable uploads — a simple one-user gemlog serving static pages can easily be administered manually via SSH, for example.
Therefore, using a different protocol is still warranted, but it doesn't have to be entirely divorced from Gemini.
Titan solves this by expanding the request so that the request URL is followed by a payload field. When it comes to TLS, Titan is equivalent to Gemini, so the same server and client certificates can be used with both. It is feasible for a Gemini server to accept Titan requests as well, or one can run a separate Titan server on the side, handling uploads. The separation of the protocols allows treating Gemini as the "read only" interface, available to all clients and users, and Titan as the "write only" interface, focused on admins, owners, and authors.
As far as Lagrange is concerned, Titan is just one of the supported URL schemes. Whenever you try to open a "titan://" URL, no matter if it is manually entered into the URL field, or by clicking on a link, opening a bookmark, feed entry, or via a redirect, a dialog will open where you can specify the data to upload. There's also a keyboard shortcut and a menu item for switching the current page's URL to "titan://". This will be convenient if the server supports editing pages, for example. All this should enable using Titan uploads in a variety of ways, both for manual and more guided use cases.
Screenshot of the Titan upload UI
At the time of this writing, Titan uploads are not widely supported by servers so I've only had occasion to test Lagrange against a debug server of my own. I'm very curious to hear how uploading works out for your use cases! Of course, I will release patches if something is broken and future releases may improve the UI. Personally, I plan to write some Python scripts that'll allow me to post to my gemlog on skyjake.fi via Lagrange, so I can submit new posts from iOS in the future.
Lagrange's TOFU implementation has been rather laid back, so to speak. It would check certificate fingerprints and expiration dates, but even in the case of a mismatch the only consequence was that a warning icon and a banner was shown. The end result was that one was able to browse untrusted capsules without much hindrance.
In v1.6, TOFU is more secure and more streamlined. In case the server certificate cannot be trusted, the connection is cancelled during the TLS handshake via an OpenSSL callback where the certificate gets verified. This applies to both fingerprint mismatches and expired certificates.
When it comes to certificate fingerprints, they are now generated based on public keys. This way, a server is allowed to renew their certificate without losing trust, as long as the key pair is not changed. Also, the server port number is included when matching fingerprints, so there can be multiple servers on the same domain with different certificates.
Certificate expiration is handled in a more serious manner. Expired certificates are always untrusted, and an error page is shown. However, you may grant a temporary exception to load the page regardless. This extends the local expiration timestamp of the certificate by one hour into the future, giving you time to browse pages on the server for a while before the expiration error is shown again.
Switching to public key fingerprints and adding server port numbers required changing the format of the file where information about trusted certificates is stored. Consequently, all previously trusted certificates are forgotten when you upgrade. The old "trusted.txt" file is kept around and used by older versions, though.
There are a couple of big changes in how Lagrange caches page content. In previous versions, only the page source text was cached and everything else (word-wrapped layout, images, etc.) was discarded as soon as the page was changed. If you often navigated back and forward in history, this would cause substantial amounts of processing to be redone, and of course inline images going missing means that scroll positions will shift. For instance on a mobile device, reprocessing pages is a complete waste of resources because there is tons of RAM available for the app to use. In v1.6, page contents are cached more thoroughly in memory to avoid redoing earlier work. This caching behavior can be controlled with a new preference (Network > Memory Size), and you can also see how much memory Lagrange thinks it's currently using on the "about:debug" page. (This may or may not reflect reality. 😄)
The easiest way to see the benefit of this caching is to navigate back to a very long page (hundreds of KBs). It should now happen instantly when in previous versions there may have been a stutter when the page layout was recalculated.
In addition to text rendering being more sophisticated overall, with complex text shaping, bidirectional paragraphs, and configurable line spacing, I've fixed the issue where font size would switch from big to normal somewhere in the middle of a long lede paragraph. This is thanks to an internal cleanup: I'm now using the same text wrapping functions throughout the app, so if the lede paragraph looks too long, it'll be redone using the regular font.
There's a new configuration file where you can change the UI color palette. It only affects UI controls; page themes use a separate palette. See the Help page (section 3.5, palette.txt) for details about the syntax and limitations.
Screenshot where UI colors have been tinted with magenta
Vacations tend to disrupt one's usual routines, and so it has been this summer as well. The time off from computers has been refreshing but I always eventually gravitate back to my normal patterns. This is to say, I expect to continue making monthly releases of Lagrange during the fall.
Next I'd like to work on:
📅 2021-07-26
CC-BY-SA 4.0
