Yes… I’ve been struggling for about five evenings with this and now I finally have example code that works.
The context is my Gemini Wiki. Now, Gemini specifies a peculiar way to do authentication. It’s supported by the specification but it’s rare to see because the world of browsers uses TLS in a different way. Browsers use the concept of certificate authorities. That is, each browser comes with a list of these, and if you’re visiting a website that uses HTTPS, the server presents your browser with its own certificate, which is signed by a certificate authority. The browser trusts the site’s certificate if it has the certificate authority in its trust store. More or less, in any case. Usually there are intermediaries, and therefore sites usually present a chain of certificates, and browsers just know about the roots, and on and on.
Gemini uses TLS differently: certificates are self-signed, that is they don’t depend on a certificate authority. Thus, there’s no way of knowing whether to trust the certificate a site presents you with. What should happen is that the client uses “trust on first use” (TOFU). If this is the first time it visits the site, it simply saves the fingerprint, and if it returns to the same site (same host, same port) at a later date, it compares the fingerprint with the one it saved. If they match, no problem, no tampering. If they don’t match, the Gemini client is supposed to alert the user: “Hey! This site has a new certificate! Check out this fingerprint: gobbledigook gack gack! Do you trust it?” If you’re a security-conscientious user, you’ll look for the new fingerprint on social media, or in email signatures, or some other websites, trying to at least see whether different sources agree, or you’ll make an educated guess based on expiry dates (if the old certificate is about to expire, getting a new certificate makes sense). It’s a bit of trouble, but if certificates are valid for many years, this can work. This is how it works in the SSH world, for example.
Now, to further complicate matters, this is just half the story. In both variants, we’re talking about clients (Gemini clients or web browsers) trying to decide whether they trust the server. Has this server been tampered with? Are criminals trying to install trojan horses, ransomware, or some other malware on my machine? But sometimes servers need to decide whether they trust the clients, too. If the server is a wiki, for example, it might want to know whether the client attempting to edit a page is one of the clients that it trusts.
Again, the model we know from the web browser world is different from how Gemini handles it. In the web browser world, administrators set up a local certificate authority, put it in the server’s trust store, and then they use it to sign client certificates, which they then distribute to their trusted clients. Those clients then present these certificates to the server as they try to connect, the server tries to verify these client certificates, sees that they’ve been signed by a trusted certificate authority (namely the one the administrators set up), and off they go.
In the Gemini world, you can do this, but you can also do TOFU. Clients could present a self-signed certificate, which the server simply accepts for a particular purpose (such as a URL used to create an account), and all it does is it saves the account information and the certificate fingerprint. Now, whenever an account is required, the server looks at the client certificate, computes the fingerprint, and looks up the account it saved for this fingerprint. If there’s a match, that account is used. If there’s no match, the server complains and maybe tells the client to go register an account, first.
For the longest time, there has been one server that did this: Astrobotany 🌱. Amazing! Also, that’s how I knew it must be possible. What it did was offer a registration URL where it asked for a client certificate, and if it had a common name (CN) that it didn’t know, it would create an account for that name, and associate it with the fingerprint of your client certificate. From then on, nobody could use that account without having the appropriate client certificate. Beautiful!
This is what I was trying to do for Gemini Wiki. I wanted the option of limiting editing to known client certificates (a list of trusted fingerprints, basically), and the option of people “signing” their contributions with a username of their choosing.
All of that required my server to be able to access the client certificates, however. And all these days it simply didn’t work. Part of the problem was also that I just knew half of what I wrote above. Hopefully this write-up helps somebody.
Once I knew what I wanted, however, things still weren’t easy. By default, server code using the Perl library stack I’m using (Net::Server::SSL which uses IO::Socket::SSL which uses Net::SSLeay which has bindings into the openssl library or something like that) didn’t check client certificates because by default, “peer verification” is off in “server mode”. If verification is off, the server doesn’t ask the client for its certificate during the handshake, the client certificate isn’t sent, and then there’s nothing to fingerprint and no common name to extract.
Once I knew that I wanted peer verification I ran into another problem: the default operation of peer verification uses the browser model I described above. As my Gemini client was presenting a self-signed certificate, this verification failed. 😢
Yesterday, I got a hint of what I needed from a kind person on the #perl IRC channel: I needed to provide my own verification callback code which would allow me to tell the library that yes, I did want to verify my peer (those pesky Gemini clients), and yes, I did have my own way of verifying them, and then I simply wanted to return “true” for all certificates, just to get the damn code working. My first goal was to simply print the fingerprint of the client certificate!
It still didn’t work… 😭
Today, I decided to try and implement a simpler server and client, based on examples in the IO::Socket::SSL man page, and based on the hint I had gotten, and without using Net::Server::SSL – and that works!
Here’s how.
First, we need to create server and client certificates. These use eliptic curves and are valid for five years.
openssl req -new -x509 -newkey ec \
-pkeyopt ec_paramgen_curve:prime256v1 \
-days 1825 -nodes -out server-cert.pem -keyout server-key.pem
openssl req -new -x509 -newkey ec \
-pkeyopt ec_paramgen_curve:prime256v1 \
-days 1825 -nodes -out client-cert.pem -keyout client-key.pem
Answer the questions, or don’t, but do provide a common name for both.
Here’s a simple server:
use Modern::Perl;
use IO::Socket::SSL;
sub verify_fingerprint {
warn "Verifying: @_";
return 1;
}
my $srv = IO::Socket::SSL->new(
LocalAddr => 'localhost',
LocalPort => 1234,
Listen => 10,
SSL_cert_file => 'server-cert.pem',
SSL_key_file => 'server-key.pem',
SSL_verify_mode => SSL_VERIFY_PEER,
SSL_verify_callback => \&verify_fingerprint,
) or die "failed to listen: $!";
while (1) {
my $cl = $srv->accept;
if (!$cl) {
warn "failed to accept or ssl handshake: $!,$SSL_ERROR";
next;
}
my $input = <$cl>;
print $input;
print $cl->get_fingerprint() . "\n";
print $cl->peer_certificate('cn') . "\n";
print $cl "hanging up\r\n";
last if $input eq "quit\r\n"
}
And here’s a simple client:
use Modern::Perl;
use IO::Socket::SSL;
my $cl = IO::Socket::SSL->new(
PeerHost => 'localhost',
PeerPort => 1234,
SSL_cert_file => 'client-cert.pem',
SSL_key_file => 'client-key.pem',
SSL_verify_mode => SSL_VERIFY_NONE)
or die "error=$!, ssl_error=$SSL_ERROR";
print $cl->get_fingerprint(), "\n";
print $cl "hello\r\n";
print <$cl>;
I start the server:
perl server.pl
And I start the client:
perl client.pl
The client output is not surprising:
sha256$347e31c3b11a9b536e5e8e88533e985745f2ac79d31a0c5e9452cb129c550d84 hanging up
This is the fingerprint of the server certificate, followed by the message the server prints back to the client.
Here’s how to confirm the fingerprint of the server certificate:
$ openssl x509 -noout -fingerprint -sha256 -inform pem -in server-cert.pem SHA256 Fingerprint=34:7E:31:C3:B1:1A:9B:53:6E:5E:8E:88:53:3E:98:57:45:F2:AC:79:D3:1A:0C:5E:94:52:CB:12:9C:55:0D:84
But on to the server output. That’s the part that I kept failing at!
Verifying: 0 94052960936576 /CN=Alex/CN=Alex error:00000012:lib(0):func(0):reason(18) 94052958116368 0 at server.pl line 5. Verifying: 1 94052960936576 /CN=Alex/CN=Alex error:00000012:lib(0):func(0):reason(18) 94052958116368 0 at server.pl line 5. hello sha256$6997941140fa03bfb7f5eabd4a9d475103558b03a7479220dd4f4bd39eb4c287 Alex
Oh yes! My callback is printing the verification information! It’s getting the “hello” message from the client! It gets the fingerprint from the client certificate! It can extract the common name from the client certificate! 💯
Now I just need to get this working with Net::Server::SSL! 😅
I’m still using more or less the same client. I’m using my Gemini Wiki as the server, now. It supports a config file where I can add any Perl code I want to the request processing.
I’m basically using this:
if ($url =~ m!/do/test$!) {
say "20 text/plain\r";
say "Test";
say $self->{server}->{client}->get_fingerprint();
say $self->{server}->{client}->dump_peer_certificate();
say $self->{server}->{client}->peer_certificate('cn');
return 1;
}
The server is created using something like the following:
my %args = (
proto => 'ssl',
SSL_cert_file => 'cert.pem',
SSL_key_file => 'key.pem',
SSL_verify_mode => SSL_VERIFY_PEER,
SSL_verify_callback => \&verify_fingerprint, );
for (grep(/--(key|cert)_file=/, @ARGV)) {
$args{SSL_cert_file} = $1 if /--cert_file=(.*)/;
$args{SSL_key_file} = $1 if /--key_file=(.*)/;
}
die "I must have both --key_file and --cert_file\n"
unless $args{SSL_cert_file} and $args{SSL_key_file};
my $env = {};
my $protocols = 'https?|ftp|afs|news|nntp|mid|cid|mailto|wais|prospero|telnet|gophers?|irc|feed';
my $chars = '[-a-zA-Z0-9/@=+$_~*.,;:?!\'"()&#%]'; # see RFC 2396
$env->{full_url_regex} ="((?:$protocols):$chars+)"; # when used in square brackets
my $server = Gemini::Wiki->new($env);
$server->run(%args);
sub verify_fingerprint {
my ($ok, $ctx_store, $certname, $error, $cert, $depth) = @_;
return 1;
}
The new code is right there, at the top:
SSL_verify_mode => SSL_VERIFY_PEER, SSL_verify_callback => \&verify_fingerprint,
And the verification simply returns 1. Sadly, this won’t work. When I try to connect with my client, it reports:
error=Connection refused, ssl_error=IO::Socket::IP configuration failed at client.pl line 5.
And the server reports:
Can’t load application from file “/home/alex/src/gemini-wiki/gemini-wiki”: Could not finalize SSL connection with client handle (SSL accept attempt failed error:1417C086:SSL routines:tls_process_client_certificate:certificate verify failed)
It’s as if I had requested validation but my callback wasn’t being used!
After many hours of desperation 😭 I finally ended up looking at the source code of Net::Server::Proto::SSL. And what do I see?
my @ssl_args = qw(
SSL_use_cert
SSL_verify_mode
SSL_key_file
SSL_cert_file
SSL_ca_path
SSL_ca_file
SSL_cipher_list
SSL_passwd_cb
SSL_max_getline_length
SSL_error_callback
);
Notice how SSL_verify_callback is missing in this list! And when I add it, my code works!
The client says:
sha256$4a948f5a11f4a81d0a2e8b60b1e4b3c9d1e25f4d95694965d98b333a443a3b25 20 text/plain Test sha256$6997941140fa03bfb7f5eabd4a9d475103558b03a7479220dd4f4bd39eb4c287 Subject Name: /CN=Alex Issuer Name: /CN=Alex Alex
But, what now? 🤔
Pull request made. But there are now ten of them... 😓
In the end, all this does is it allows us to limit editing to people with known client certificates: the fingerprints of their client certificates are very long passwords (unlike the tokens) but in the end, just as easily passed on to friends (even though we usually don’t pass both certificate and private key on to others that’s still an option). The documentation now comes with an example setup.
#Perl #Cryptography #Gemini
(Please contact me if you want to remove your comment.)
⁂
I left a comment on the mailing list.
Client certificates on the mailing list
This is the short summary I provided:
In the TOFU world, the default OpenSSL setup doesn’t quite work. The default is that clients don’t send their client certificates to the server unless the server asks for them during the handshake. In the OpenSSL world, the server can be told to do this by telling it to verify the client certificate. When you do that, the server will then reject the connection because the client certificate is self-signed. In order to force the server to still accept it, you need to provide your own verification callback which simply returns 1 for all certificates in the chain of certificates the client presents you with.
“SSL_set_verify() sets the verification flags for ssl to be mode and specifies the verify_callback function to be used. If no callback function shall be specified, the NULL pointer can be used for verify_callback. In this case last verify_callback set specifically for this ssl remains.”
Keywords to look for in your SSL or TLS library’s documentation are “peer verification”, “verification mode”, “verification callback”, etc.
Once you have all that, then you can get the fingerprint of the client cert on the server side and compare it to the list of fingerprints you know (if you’re trying to only allow some people access), or save the combination of fingerprint and common name in your database if you want to create an account (like astrobotany does), or send a 60 code back if there is no certificate, or a 61 code if the fingerprint doesn’t match anything in your database, or a 62 if you decide to do further tests such as checking the validity start date or the expiry date of the client certificate.
Hope that helps somebody
Cheers
Alex
---
Only partially related, but here’s an interesting question: on the mailing list and on IRC, Luke Emmet said his clients was rejecting server certificates because the hostnames does not verify, the x509 didn’t match up.
Does a cert need a Common Name matching the domain?
What do other people think about this? My personal impression was that in a trust on first use (TOFU) world, the common name (CN) of a certificate could be anything. It could be “Alex Schroeder”, for example. Or it could be “alexschroeder.ch”. Even if it was served as the certificate for another domain, like transjovian.org. After all, the question is only whether you “trust on first use”.
My impression is that a client that tries to verify that CN and domain match is doing it wrong. What do you think? Sadly, my SSL know-how is weak, so any pointers to how things ought to work in a TOFU world are appreciated.
Cheers
Alex