💾 Archived View for mozz.us › journal › 2020-05-06.gmi captured on 2021-11-30 at 20:18:30. Gemini links have been rewritten to link to archived content
⬅️ Previous capture (2020-09-24)
-=-=-=-=-=-=-
Published 2020-05-06
There was some recent discussion on the gemini mailing list that lead to me re-think my whole authentication scheme for Astrobotany. Here's the new plan that I outlined, which uses certificate pinning instead of issuing trusted client certificates:
I have decided that I would really like to try this out. Unfortunately I'm running into some... hurdles.
The first hurdle is that the python standard library OpenSSL bindings don't support untrusted client certificates. This is outlined in a previous gemini post.
/journal/2019-08-21_transient_tls_certs.gmi
Reaching outside the standard library, the only significant python TLS library is pyOpenSSL.
I started looking into pyOpenSSL and it turns out to have a long, interesting history behind it. It's an old library (dating back to python 2.1). It evolved alongside python in order to address ongoing "limitations" in the standard library's wrapper. Over time, pretty much every major python networking library has reached for pyOpenSSL because they needed *something* that wasn't available elsewhere. requests/urllib3 needed SNI support for python 2, twisted needed the memory BIO interface. Even now, the python standard library SSL still doesn't support OSCP stapling and has buggy TLS v1.3 support.
https://github.com/psf/requests/issues/749
https://github.com/openssl/openssl/issues/7967
https://www.python.org/dev/peps/pep-0546/
Current development of pyOpenSSL appears to be iffy. They can't exactly abandon it because so many bedrock python projects depend on the bindings. But there's no cooperate backing and no one really wants to maintain it anymore.
Yeah, we'd love to kill pyOpenSSL, because the code is grody and sad-making and the APIs aren't very good. But it's still maintained and supported, so I don't have an opinion on this.
https://github.com/python-trio/trio/issues/1145#issuecomment-513041312
pyOpenSSL doesn't interface to libssl C library directly. A while ago, it was updated to use the `pyca/cryptography` backend, which in turn creates CFFI bindings around libssl.
https://cffi.readthedocs.io/en/latest/
https://cryptography.io/en/latest/
This transitive dependency on pyca/cryptography is a bonus as far as I'm concerned. Cryptography is a rock-solid python package that exposes primitives for working with x509 certificates. For example, I could use it to generate ad-hoc server certificates entirely in-memory, without needing to shell out to the openssl command-line tool like I do now.
On the surface, pyOpenSSL appears to be API compatible with the standard library SSL module. They both use objects called "Sessions" and "Contexts". Most of their methods even share the same names. pyOpenSSL might have some extra bells and whistles, but surely, I figured, it would be a simple drop-in-replacement in my code.
It was not a simple drop-in-replacement.
Asyncio (the standard library TCP server framework that jetforce uses) is very complex. It uses an ssl.SSLContext method called ```wrapbio()`` that was introduced in Python 3.5.
http://blog.povilasb.com/posts/python-ssl-memorybio-usage/
Memory BIOs are a low-level concept that I don't fully grasp yet. What I do know is that they're often used in async libraries, and they have something to do with efficiency of not needing to call select/poll on sockets.
https://www.python.org/dev/peps/pep-0546/
PyOpenSSL also supports memory BIOs. In fact, it has supported them for a much longer time. But it doesn't use the `context.wrapbio()` method; it has its own incompatible interface.
https://www.pyopenssl.org/en/stable/api/ssl.html?highlight=bio#OpenSSL.SSL.Connection.bio_read
The asyncio internals are a rats nest of winding, complicated code with no hooks for implementing any custom TLS behavior. The only way forward with asyncio + pyOpenSSL would be to write my own custom wrapper around the pyOpenSSL context to make it look & behave just like the ssl.SSLContext equivalent. I found some tickets where other people have tried this, and the road looks scary and full of dragons. Definitely not something that want to attempt.
So if I can't use asyncio, what other options do I have for async TCP frameworks? There are two general purpose async libraries that I have found.
https://github.com/dabeaz/curio
https://github.com/python-trio/trio
Both are delightful, well-written libraries and I would have no qualms using either of them for jetforce.
https://github.com/python-trio/trio/issues/1145
Next up on my list is to look into what TCP server frameworks *do* work well with pyOpenSSL. This will likely mean dropping async and moving to thread pooling or something else.
Oh well. I was never really using async to its full potential anyway. I have a pattern of trying to get fancy with concurrency, and it always ends up coming back to bite me in the ass.
I'll end with a link to the source file for trio's SSL module.
https://github.com/python-trio/trio/blob/master/trio/_ssl.py
This code is a work of art. I aspire to someday write code comments as clear and meaningful as this.