💾 Archived View for gemi.dev › gemini-mailing-list › 000032.gmi captured on 2023-11-04 at 12:19:40. Gemini links have been rewritten to link to archived content
-=-=-=-=-=-=-
Hi all, I have been working on a python script that can be pointed to a gemini server, and will automatically send various types of requests to surface errors and other strange server behavior. This was suggested in the mailing list a while ago by solderpunk(?). I thought it was a neat idea so I took my own stab at it. I am not attempting to cover 100% of the gemini specification. Some of the checks aren't even related to the spec, and are strictly informational. Take all results with a grain of salt. For example: - Does your server support IPv6? - What version of TLS do you negotiate? - Is your certificate's CA trusted? - How do you respond to a http:// URL? - How do you respond to a URL with an invalid UTF-8 byte sequence? - How do you respond to a request with a newline but no carriage return? The program can be found here: https://github.com/michael-lazar/jetforce/blob/master/jetforce_diagnostics.py I've included it in the jetforce repo out of convenience, but the script can be downloaded and run independently of jetforce. The only hard dependency is Python 3.7+. The python "pyca/cryptography" library is an optional requirement, if you want to inspect server TLS certificates. Usage: ./jetforce_diagnostics.py host [port] This is very much still considered a work-in-progress. If you have any suggestions or would like to contribute additional tests to run, please feel encouraged to respond here or open an issue or PR in the github repo. To save you all the trouble, I have already run this against all known gemini servers and published the results on my server. This is not an attempt to shame anybody, my own server exposed several failures that I now need to fix. But I think there are some interesting insights to be gleaned about how servers are implemented in practice, and how the gemini spec might be improved: gemini://mozz.us/diagnostics/ Some of the results: - All servers use "\n" line endings on their home page instead of "\r\n" - No server will accept a URL without a scheme specified (per the spec, this should be implied as gemini://) - Only half of servers currently support TLS version 1.3 - mozz
It was thus said that the Great Michael Lazar once stated: > Hi all, Hello. > I have been working on a python script that can be pointed to a gemini server, > and will automatically send various types of requests to surface errors and > other strange server behavior. This was suggested in the mailing list a while > ago by solderpunk(?). I thought it was a neat idea so I took my own stab at it. Thanks for doing this. I find the results interesting, and yes, you did find three bugs in GLV-1.12556. > For example: > > - Does your server support IPv6? Mine (GLV-1.12556) does, but my hosting provider doesn't have IPv6 yet (or maybe they do---it's been awhile since I last checked). > - What version of TLS do you negotiate? 1.2---it's a limitation of the library I'm using. > - Is your certificate's CA trusted? No. I'm the one using a custom CA. > - How do you respond to a http:// URL? I reject them. > - How do you respond to a URL with an invalid UTF-8 byte sequence? Here's one of the bugs, and I have no idea how I'll fix this as it's a cross-library concern. Sigh. > - How do you respond to a request with a newline but no carriage return? I respond. I made CRs optional everywhere internally---it makes it easier for me to test. Now, you did make one test labeled "RequestMissingCR" where you did send a LF. Per the spec, I would expect this to return a 59 Bad Request, but since I treat CRs as optional, that's why I passed that one. > Some of the results: > > - All servers use "\n" line endings on their home page instead of "\r\n" > - No server will accept a URL without a scheme specified (per the spec, this > should be implied as gemini://) This was another bug. I've already fixed this, and the one other bug (I allow URLs longer than 1024 characters, which has been fixed for now). Again, thanks for this. -spc
On Thu, Jan 09, 2020 at 02:11:32AM -0500, Sean Conner wrote: > > Again, thanks for this. > Yes, thanks a lot for this, because I'm continuing to do a bad job of pushing this idea to completion! If you're feeling motivated, some things which my rough stab at this tested which you didn't mention are:
On Thu, Jan 9, 2020 at 2:11 AM Sean Conner <sean at conman.org> wrote:
>
> It was thus said that the Great Michael Lazar once stated:
> > - How do you respond to a request with a newline but no carriage return?
>
> I respond. I made CRs optional everywhere internally---it makes it easier
> for me to test. Now, you did make one test labeled "RequestMissingCR" where
> you did send a LF. Per the spec, I would expect this to return a 59 Bad
> Request, but since I treat CRs as optional, that's why I passed that one.
It's interesting to hear that this was intentional on your part. You're not
alone on this, 3/5 servers that I tested didn't care about the CR at all. My
own server does require the CR, but I didn't like writing it that way and felt
like I had to jump through unnecessary hoops to conform to the specification:
data = await self.reader.readuntil(b"\r\n")
instead of the more straightforward:
data = await self.reader.readline()
I would prefer it if CR was optional. Similar to how <whitespace> is already
defined as "any non-zero number of consecutive spaces or tabs". The same
argument applies here. Content authors can't visually distinguish newline
characters in text/gemini files, and I can't tell the difference in my terminal
when I'm inspecting a gemini request.
<newline> = [<CR>]<LF>
- mozz
---
Previous Thread: [ANN] Castor, a graphical Gemini browser
Next Thread: [ANN] Announcing Molly Brown, a Gemini server in Go