💾 Archived View for geminiprotocol.net › history › phlog › status-codes.gmi captured on 2023-09-08 at 16:01:55. Gemini links have been rewritten to link to archived content
-=-=-=-=-=-=-
(originally posted in Gopherspace on 2019-06-27)
When I wrote the first speculative specification for Gemini, I included the following status codes:
2 Success, everything is fine, response follows (cf HTTP 200)
4 Not found, requested <PATH> does not exit (cf HTTP 404)
5 Server error, something went wrong, wasn't your fault (cf HTTP
500)
and said that these two additional were under consideration
3 Moved (cf HTTP 301)
9 Slow down, cowboy (cf HTTP 429)
(this last one was motivated by the fact that the gopher project mailing list sees frequent complaints about badly behaving bots which bombard servers with a lot of requests. There's little servers can do to control this, thanks to gopher's lack of status signalling. Even if the server detects that a rate limit has been exceeded and starts serving up only the text "Slow down, cowboy!" in response to all selectors from that IP, *the bots can't read that* and won't adjust their behaviour)
I also promised that there would be less than 10 status codes and even added the following:
If a server sends a status byte not from the above list, a compliant client MUST treat this as equivalent to "5" (server error) and, if the server does not close the connection after the non-standard status line, the client MUST close the connection itself and not process the accompanying response body.
This relatively radical idea was part of my deliberate and determined effort to make sure Gemini is not slowly unofficially extended down a slippery slope to end up having some of the misfeatures of HTTP that we want to avoid.
This minimalistic and unextendable system of status codes has not proven popular! Alex dismisses using single digit status codes like 2 instead of 200 as being a gimmick, and advocates just reusing the three digit codes from HTTP. Sean's Gemini server over at gemini.conman.org *does* reuse some HTTP codes, but also adds some new ones. These are the codes used by the conman server:
Alex Schroeder's 2019-06-21 post discussing Gemini status codes
200 Okay 301 Permanent redirect 401 Unauthorized 403 Forbidden 404 Not found 410 Gone 429 Slow down 460 Need certificate 461 Future certificate 462 Expired certificate 463 Rejected certificate 500 Server error
(the certificate stuff regards an idea for out-of-band authentication based on TLS client certificates which I'll explain in a future post)
Because of this, both of my Gemini clients take a conservative approach and will handle either the "official" single digit codes or the codes above. They do this by just looking at the first character of the status code (and don't implement the heavy-handed rejection of non-standard codes outlined above). Indeed, part of the argument for three digit status systems like those used in HTTP or SMTP is that they don't place the additional burden on client implementors that they seem to at first, because they are grouped in such a way (all 2xx codes are "success" codes, all 3xx codes are "redirection" codes, etc) that simple clients can do precisely this, instead of handling each and every code.
I actually still really like the idea of a small, simple, pre-defined and non-extensible system of status codes. I want to use this post to explain my position better and to solicit feedback from the community. It's possible my mind can be changed.
Along with my "50-100 lines of code for a basic client" requirement (which we still seem on track for!), one of the other criteria from the FAQ was that "it should be possible for somebody who had no part in designing the protocol to accurately hold the entire protocol spec in their head after reading a well-written description of it once or twice". If we take that very literally (and I'll admit I'm not sure how literally I meant it when I wrote it) so that it includes remembering all the status codes, a small and fixed set is pretty much essential to this.
Even if we don't think it's realistic that the set of status codes can be memorised, I would love it if people who are thinking about implementing a Gemini client or server but haven't done something like that before and are kind of uncertain about how big of a task it is, could read the spec and think "Oh, that's it?! I can do that!". Gemini should *feel* lightweight and approachable and implementable. I think few things will kill enthusiasm and convey a sense of heavy mental burden like a long list of three digit status codes. The use of three digit codes doesn't actually mean that there are literally hundreds of things which can go wrong, but I think they lend that *feel*.
I grant that the above two paragraphs are kind of touchy-feely and not solid technical arguments. I'm switching gears now...
The number of status codes that a protocol needs is basically a reflection of the number of things which can go wrong plus the number of different legitimate situations a client needs to be able to handle. This is an excellent proxy for the complexity of a protocol. A protocol which *needs* a lot of status codes to work can't really claim to be a simple protocol. In addition to my ubiquitous concern about extensibility being used by third parties to reduce the simplicity and privacy of the protocol, when it comes to status codes for handling error conditions, I think that building in extensibility is kind of an admission of failure. It says "this protocol is sufficiently difficult to reason about that we couldn't accurately foresee everything that might go wrong so we had to cover our ass and build in room for new errors". If a protocol is really simple and is well-designed, it should be possible to anticipate all possible conditions and assign them codes. Once that's done, if there's fewer than 100 codes it seems to me that it's just plain *weird* to give them three digit codes. I fully acknowledge that saving one or two bytes per response header is not worth striving for when we have the overhead of TLS to contend with. I'm not really worried about saving bytes, but why use larger codes than we need once we're sure we have all our requirements covered? An open-ended and extensible error system makes sense when a protocol is intended to grow and develop organically over a long time in response to shifting requirements and fashions, but that's not at all how I think of Gemini.
I am skeptical of looking to HTTP for too much inspiration for status codes. The reason that the one digit codes I proposed correspond to the first digit of similar HTTP codes was simply to make it easier for folks who know HTTP to remember them - saving on "mental bandwidth", as Alex put it. There's a lot of stuff in HTTP which we absolutely don't need, either because it corresponds to capabilities that Gemini doesn't have, or because it creates extra work for developers with extremely little reward.
A good example of this latter problem, IMHO, is "410 Gone" (which is actually in the Conman Gemini server!). If this is made official in the Gemini spec, it sends the message that Real Servers which have a Proper Full Implementation should remember every one of their URLs which *has* been valid in the past so it can respond to requests for them with 410 instead of 404. Similarly a Real Client should remember every 410 it gets so that it doesn't request them again. In the real world, almost nobody does this with HTTP, so it's basically dead weight in the spec.
I'm also not a fan of serving up "403 Forbidden" when unix file permissions don't allow the server to read a file. Characterising this as a "client error" makes no sense. The unix file permissions are in no way dependent on any credentials the browser might present. In the case that the webmaster intended that file to be visible, this condition is in fact a server configuration error. In the case that the webmaster did in fact intend for those files to be inaccessible, why on Earth is the web server leaking their existence by not returning a 404? Nothing about this convention makes much sense to me.
Of course, nobody has explicitly proposed adopting every single HTTP status for Gemini, and I doubt anybody would do anything but deny thinking this is a good idea. But bringing in several of them, especially without very good reason, is going to either give people the mistaken impression that we are doing precisely that, or encourage people to, eventually, do that by adding codes one by one.
In designing a set of response codes, I think it's important to remember that the response header has two parts: a status code, which is supposed to be machine readable, and then some UTF-8 text, which in some cases is more machine-readable content (a MIME type for successful requests, for example) but in other cases is just additional human-friendly information. A big motivation in having numeric status codes was to make it possible to write half-way robust non-human Gemini clients. As mentioned above, gopher bots are, by necessity, pretty dumb, and this causes headaches in the real world. I don't know if difficulties in writing a robust gopher spider are a big part of the reason that search engines for gopher are underdeveloped, but it certainly can't be making things *easier*.
Good clients should look at the status code, but they can't really be expected to do anything smart with the text that follows it outside of a few prescribed circumstances. So it seems to me that distinct status codes should only be specified when a sufficiently smart client could be imagined as doing some specific action which makes sense in response to that code. If the only reasonable thing a client can do is, in the case of an interactive client for humans, to tell the user "Error!" and show the accompanying status text, and in the case of a bot just give up, then there's not a lot of value in assigning that condition a status code different from any other condition that can only be handled in the same way. The fine-grained information in the machine-readable status code is essentially doing no work at all in this situation.
This leads to an idea of "status codes as opcodes", i.e. the status code is used to identify a function which specifies exactly what the client should do with the text part of the response. In the case of an "Okay" status, that's "interpret the text part as a MIME type and use that to inform your handling of the response body". In the case of a "Redirect" status, that's "interpret the text part as a URL and request it". For a great many situations, a function of "present the text part to the user under a generic heading like Client Error or Server Error and then just give up entirely on this request because there's nothing you can really do" is all that can be needed, and so arguably there is no need for those situations to get their own status codes. What does conveying their precise message in a dedicated machine-readable status code alongside the human-readable text achieve, if there is nothing in particular a machine can do in response?
Under this philosophy there is no point in assigning distinct codes to, e.g. Forbidden, Not found and Gone, because they are all just "display the message and give up" conditions, or to the four certificate conditions with 46x codes above, because they are all "display the message and provide the user with a way to choose a (new) certificate" conditions.
So which "opcodes" do we actually need? I think the following would cover everything which has been described so far:
From the perspective of a human user, the two error opcodes would more or less be identical "display the message and give up" conditions. The idea is to be able to machine-readably signal to something like a search engine crawler whether or not it makes sense to try that link again tomorrow (because the failure was due to e.g. an overloaded server, a CGI script taking too long, or something else which might right itself) or remove the link from its list because the failure is permanent. Arguably under this model "not found" falls under the "permanent error" umbrella, and in fact I'm having a hard time thinking of a permanent error condition *other* than not found...but goodness, it sure feels strange for "not found" to not have its own code...
I'm not 100% convinced this "statuscodes as opcodes" idea is necessarily the right way to go. I think there is a sensible logic to it, I think it is pretty much guaranteed to result in the minimum viable number of status codes because it encourages code reuse, I think it makes the protocol easier to reason about because the complete list of status codes doubles as a complete list of chunks of client functionality. These are all great things and so I'm at least 75% sold on this idea, but I do have some small, vague worries that it could prove to be underpowered in some way. All I've been able to think of so far is that it would complicate detailed server logging - to log all the details necessary to spot and debug possible problems would no longer be a matter of logging just the status code. But I'm not sure that's a strong argument. There's nothing to say Gemini server logs need to look like Apache logs. Why not just log the whole response header?
Or, heck, there's an obvious hybrid approach where we use multi-digit codes (maybe only 2 digit?), where the first digit is an opcode and the second specifies the actual error. This is kind of like the HTTP approach but instead of the first digit telling you only a vague category like "Client Error" which is not, by itself, enough information to know how to handle things, the first digit actually tells you exactly what to do. This means that simple clients *could* legitimately totally ignore anything beyond the first digit and be guaranteed to still function correctly, and we could still easily do fully detailed logging just by recording the status code. Slightly fancier clients could change minor details in e.g. the presentation of the error message or the range of options presented to the user based on the second digit. This feels like perhaps the best of both worlds, but it does leave me concerned about extensibility, because additional second digits could be added by the community. Ah, but then, it's a kind of harmless extensibility, because a new second digit *cannot* invoke a new kind of behaviour. Hey, this is sounding great! It would allow merging "Not Found" and "Permanent Error" as per the discussion above. I'm now about 95% sold on *this* idea. Can anybody see any major problems?