💾 Archived View for geminiprotocol.net › news › 2024_07_30.gmi captured on 2024-08-24 at 23:46:30. Gemini links have been rewritten to link to archived content
⬅️ Previous capture (2024-08-18)
-=-=-=-=-=-=-
It has been brought to my attention that the new Gemini network transport protocol specification which was announced at the end of March this year, version 0.24.0, introduced a compatibility-breaking change relative to the much older version 0.16.1, and there is at least one URL in the wild serving response headers which are permitted by the 0.24.0 spec but cannot be parsed by some well known and widely used Gemini implementations. Specifically, in the latest version of the specification, status codes beginning with 4, 5 or 6 now have explicitly optional "errormsg" components (formally known as <META>), and the BNF formulation is clear that when no error message is included, no space is required between the status code and the CRLF. Some existing code, including code of my own, assumes and relies upon the unconditional presence of a space character. The need to change code to handle the new kind of response header was not mentioned in the corresponding news post, which did otherwise attempt to explicitly enumerate all changes which would be required to maintain spec compliance. This has lead to some uncertainty as to whether or not the space was supposed to be made optional or whether this was a mistake.
First of all: I am grateful to the people who let me know about this, and I accept full responsibility for the situation. The simple truth is that it was not a mistake to make the space optional, but it certainly was a mistake to not realise that doing so would break existing code and to not advise as such.
So, where do we go from here?
I am reluctant to make an immediate knee-jerk fix release reversing this change to restore compatibility with existing software. Something is going to have to change, obviously, be it spec or code or both, but I don't want to rush it. Don't panic! I know I don't move all that quickly even when I am trying to rush, but I don't mean I want to sit on this for four months until some whimsical space race anniversary. I'd like to see this fixed within a month. I think we can spare the time to reflect on this. The vast majority of Gemini servers remain fully interoperable with the vast majority of Gemini clients. This issue does not effect status code 20 responses, which are what most of us hopefully receive in response to most of our requests. It's mostly errors which are broken here.
What's there even to reflect on, though?
It's become clear to me that, no doubt as a consequence of the fact that Project Gemini has been undergoing a very slow and gradual reawakening" after a long period of stasis, not everybody in the community is necessarily on the same page with regard to how much and what kind of change they expect to happen during the specification finalisation phase of the project. Some people might be confused that stuff is changing at all. Some people might accept that clarifications and disambiguations are needed in the spec but think that even small breaking changes must be avoided at all costs. There's no clear official explanation on what to expect in this regard. I will prepare one. My opinion, at least at this moment, is that if incompleteness or ambiguities in the spec are found and the most obvious or natural or elegant way to fix them introduces small breaking changes, those might, at least some of the time, be worth accepting. There's not much appeal in a complete and formal specification full of warty compromises.
Whether or not individual cases meet that threshold is of course a subjective judgement, weighing up costs and benefits. I'm willing and ready and able to call the shots on those decisions and stick to them. But in order to make those calls in an informed way, we need to get an answer to the empirical question of just how "fossilised" the Gemini software landscape is in 2024. This particular issue might be a great opportunity to investigate that. If we can't get a small change like this fixed in short order in the majority of the software which people actually use, that should definitely inform which potential future breakages we do and don't want to accept.
To me, tough, there's something much more important for the project than the question of what to do to fix this particular issue, namely the simple fact that it took more than three months for this unintentional breakage to get noticed and reported. If I were sticking to my own schedule, there would already have been another quarterly release by now without this issue addressed. That's not good.
It seems clear to me now that the combination of the project's slow, gradual awakening process, in conjunction with the switch from an active public mailing list with the majority of implementers and admins in it to these read-only one-to-many news posts, is not working now that I've transitioned from "housekeeping" work like documentation updates to actively changing the spec. Even though I'm moving very slowly, awareness of the changes and discussion of their consequences is clealry not happening widely enough or quickly enough. I'm proposing changes almost in a vacuum, with limited peer review, and nobody at this point ought to believe for a second I'm smart enough for that to work out. Without a change, little things like this are going to keep happening. So, change is going to have to happen instead.
This is, believe me, scary, but I'm cautiously optimistic. It's also become clear to me that my mental model of the Gemini community (or rather, the subset of it which is interested in spec finalisation) and how big it is and what it wants and when it wants it, is totally outdated and inadequate, being derived mostly via some kind of temporal dead-reckoning from a several years old snapshot from the final days of the mailing list. That's a good thing, because that old mental model mostly just induced feelings of stress and pressure and guilt and obligation in me, and also made me view certain open decisions, such as the URI vs IRI thing, as having tremendous and long lasting moral weight, which today seems kind of absurd. Ditching that model will do me and the project a whole lot of good.
At this point I am not completely sure what the change in process is going to look like. I have a few ideas, and I'll write them up soon. Even though I suspect it might well be safe to do so, I'm reluctant to just fire up another public mailing list like the old days. A smaller, invite-only list (still publically readable), with maybe 20 or so people on it, a combination of "ideologically sound" Gemini old timers and modern "stakeholders" (e.g. the authors of the most widely used implementations and the admins of the most popular aggregators and hosters and apps), might work well instead, to get good, informed, practical feedback and testing of proposed changes without the distraction of requests for new features. Maybe a dedicated instance of the Bubble platform could work too, considering it has issue tracking functionality built in. Like I said, I have ideas. What I don't have is much enthusiasm for investing additional time or money of my own in setting up additional infrastructure. I will be soliciting volunteers or at least advice on reliable and trustworthy non-commercial providers which might be able to help.
I know this news feed has been quiet lately, but I haven't quite been doing nothing. I have started triaging the open issues at the old GitLab repo trackers. I have a half-written draft post open outlining all the ones I have already closed, explaining why I did so and giving thanks and credit where they are due. When I finish that post, I am going to batch the remaining ones up into thematic categories, and rank those categories in order of importance, to function as a kind of roadmap. When this roadmap is ready and posted here, and when I've also posted here some concrete ideas on ways that progress can be made on that roadmap in some way better than me just forging ahead by myself and narrating the changes to a mostly empty room as I go, then I'll be emailing links to those posts to people I'd like to see be part of a more interactive process. We'll see what it looks like can be made to work. The broader community will naturally be kept abreast of things here.
Watch this space.