💾 Archived View for gemi.dev › gemini-mailing-list › 000885.gmi captured on 2024-08-19 at 02:26:39. Gemini links have been rewritten to link to archived content
⬅️ Previous capture (2023-12-28)
-=-=-=-=-=-=-
Geminispace is (currently) small enough that we can afford to download all known capsules' TLS certificates and use them to generate trust stores for Gemini clients. If verified via multiple network perspectives, using a pre-generated trust store is a major improvement over blindly trusting-on-first-use. I wrote a set of shell scripts to do this: https://tildegit.org/nervuri/trust-store-generators This is their output (which I will update every few days): https://tildegit.org/nervuri/trust-stores The details are in the README files, so I won't repeat myself too much. The idea is: 1. download the list of hosts from gemini://geminispace.info/known-hosts 2. download those hosts' TLS certificates (and double-check each one via Tor, whenever possible); 3. generate trust stores for various Gemini clients (to start with: Agunua, Amfora and Lagrange). Contributions are welcome. Requests to add specific clients are also welcome (I don't know which ones are the most popular). Existing scripts (ex: [1]) can be used as templates to add trust stores for more clients. [1] https://tildegit.org/nervuri/trust-store-generators/src/branch/master/l agrange/generate-trust-store.sh For each client there are instructions on how to use the generated trust store and merge it with your own (ex: [2]). Merging at the moment is quite simplistic; a mismatch-aware merging method is something I'd like to develop later on. [2] https://tildegit.org/nervuri/trust-stores/src/branch/master/agunua/INSTRUCTIONS.md Of course, you don't need to trust that I am publishing the correct certificates and trust stores. The scripts should be easy to understand (if not, tell me - I consider this to be very important), so run them yourselves and check if my results are replicated from your own network perspectives. If your results don't coincide with what I've published, please let me know. And if you decide to run a public repo, tell me about it. We could set up automatic comparisons of published data to notify us of inconsistencies. What I hope to see eventually is client developers bundling pre-generated trust stores with their clients. This will go a long way towards addressing TOFU's first-connection problem. Probably the big issue with this idea is that client developers may not want to bundle, for instance, Let's Encrypt cert fingerprints, as they change every 2-3 months. They may only want to include long-lived (and long-expired) certificates. So, to allow for this, all generator scripts accept certificate expiry boundaries as arguments (see the README). As a side-effect, this project will give us a history of certificates in Geminispace. And aside from the certificates themselves, we have tables containing details about each certificate, in markdown [3] and CSV [4] formats. [3] https://tildegit.org/nervuri/trust-stores/src/branch/master/cert-details.md [4] https://tildegit.org/nervuri/trust-stores/raw/branch/master/cert-details.csv This project delivers on xq's idea of a distributed trust system [5], because the "certs" directory (containing PEM-encoded certs with filenames corresponding to the host:port [6]), can be freely shared and re-created. The main difference from xq's proposal is that no changes to Gemini clients are required. We don't require the involvement of client developers at all; anyone can write scripts to generate trust stores for any client. Of course, client developers are the ones best suited to write these scripts, so I encourage them especially to contribute and, ideally, to bundle trust stores with their clients (or have a secure, built-in method of downloading a verified Geminispace trust store). [5] gemini://random-projects.net/blog/2021-03-03-distributed-trust.gemini [6] https://tildegit.org/nervuri/trust-stores/src/branch/master/certs P.S. The "certs" directory is currently at 3.4 MB for 839 certificates. It compresses to 560 KB.
It'd be better an opt-in feature (like HSTS Preload for the web) because otherwise capsules, whose owners didn't know about the list, may just break.
On Wed, 2021-04-28, Anna “CyberTailor” wrote: > It'd be better an opt-in feature (like HSTS Preload for the web) > because otherwise capsules, whose owners didn't know about the list, > may just break. This would only happen on certificate change, in the same way it would if you had previously visited the capsule: your client would warn you that it received a different cert. The chances of that are happening can be reduced by not bundling short-lived certificates. I see the certificate's expiry date as a kind of opt-in: if a cert expires in 100 years, we can assume it's ok to bundle with clients. It's up to each client developer where to draw the line. I guess it would depend on release frequency. Ideally, clients could include the option to refresh the trust store from a chosen source (thereby providing a way to do out-of-band verification). This option could be shown to the user on every cert mismatch warning. The lag between certificate change and trust store generation is an issue here, but it's better than not having the option at all. We could eliminate the lag by using a Convergence-like approach (which I'm working on, but that's a different discussion). https://en.wikipedia.org/wiki/Convergence_(SSL)
On Wed, Apr 28, 2021 at 05:47:29PM +0000, nervuri <nervuri@disroot.org> wrote a message of 81 lines which said: > Probably the big issue with this idea is that client developers may > not want to bundle, for instance, Let's Encrypt cert fingerprints, > as they change every 2-3 months. Note that it is a reason to use only the public key for TOFU, not the entire cert, since this public key can be static.
Update: The trust stores [1] are now being generated using Lupa's list of capsules [2], merged with the list from geminispace.info [3]. The resulting hosts file contains 1124 capsules, 231 more than it did at the end of April, when this project started. The certificates directory currently contains 1100 certs. [1] https://tildegit.org/nervuri/trust-stores [2] gemini://gemini.bortzmeyer.org/software/lupa/lupa-capsules.txt [3] gemini://geminispace.info/known-hosts
Thank you very much for including my little capsule in your list! Ihaq: The certificates for my server are coming from zerossl.com and they give me free certificates that are valid for only 90 days, so my server will have a new certificate every three months (the current one expires on July 25th). Will my capsule be thrown out of your list due to that new certificate? Best regards from Charleston (WV), Frank/2 On 2021-06-04 12:48, nervuri wrote: > Update: > > The trust stores [1] are now being generated using Lupa's list of > capsules [2], merged with the list from geminispace.info [3]. The > resulting hosts file contains 1124 capsules, 231 more than it did at the > end of April, when this project started. The certificates directory > currently contains 1100 certs. > > [1] https://tildegit.org/nervuri/trust-stores > [2] gemini://gemini.bortzmeyer.org/software/lupa/lupa-capsules.txt > [3] gemini://geminispace.info/known-hosts -- ------------------------------------------------------------------------ My Gemini capsule orbits at gemini://h2903872.stratoserver.net/ ------------------------------------------------------------------------
On Fri, Jun 04, 2021 at 10:11:45AM -0700, panda-roux <contact@panda-roux.dev> wrote a message of 224 lines which said: > - What's your process for requesting the removal of a domain's certificate > from this list? I assume that capsules which no longer reply are removed after some time. This is how the Lupa list used as input works: it contains only capsules that were contacted successfully in the last weeks. > - Does the software you're using to generate this list respond to any > robots.txt directives (i.e. "don't index me" or "expire after x days")? No need to do so: just retrieving robots.txt informs the crawler about the certificate. No later connection is necessary. > - Why circumvent TOFU? I don't mean for this to come across as > antagonistic, but doesn't this defeat the purpose of having a decentralized > protocol in the first place? It is precisely the decentralization that allows any one (nervuri, you, me, anyone) to create such a list and such a repository. After that, people use it if they want.
On Fri, 2021-06-04, panda-roux wrote: > - What's your process for requesting the removal of a domain's > certificate from this list? I'll be adding code to prune certificates of capsules that have been offline for a while (2-3 weeks). Aside from this, there's a link to my contact info in the project's README, if a domain admin wants to request removal. I don't see why anyone would, though. The repo does not hold capsule contents or personally identifiable information. It just contains copies of public TLS certificates, which can help people connect more securely. > - Does the software you're using to generate this list respond to any > robots.txt directives (i.e. "don't index me" or "expire after x days")? No, because the `get-certs.sh` script is not a crawler and does not "speak" Gemini. It only works on the TLS level. All scripts are available here [1], by the way. [1] https://tildegit.org/nervuri/trust-store-generators > - Does this break if I arbitrarily decide to change my a cert on my > domain, or is it regularly kept up-to-date? If so, how often? Certificates are "updated every few days (no more than a week)", as stated in the README. I generally run the scripts once every 3-4 days. Later I might use a daily cron job, but I want a more hands-on approach in the initial stage. > - Why circumvent TOFU? I don't mean for this to come across as > antagonistic, but doesn't this defeat the purpose of having a > decentralized protocol in the first place? I think "complement" is more appropriate than "circumvent". I'm doing it because TOFU does not protect the first connection. Also, the intent here is not for this repo to be a single source of trust. I'd like other people to run these (or similar) scripts and publish certificates downloaded from their network perspectives. Having verification from multiple sources will give us a more trustworthy ecosystem. I also want to encourage client authors to bunlde pre-generated trust stores (verified from several perspectives) into their clients, to protect the first connection. However, this by itself is not enough; we also need an out-of-band verification method to help with certificate mismatch prompts. These prompts arguably do more harm than good if they are frequent and users have no information on which to base the trust decision. People may become accustomed to saying "yes" all the time, despite the warning. Here's an example of anonymous out-of-band verification using Tor and OpenSSL. In case of mismatch, run: host_and_port='rawtext.club:1965' torsocks --isolate openssl s_client -connect "$host_and_port" </dev/null 2>/dev/null \ | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' This gives you the certificate in PEM format, from the perspective of a random Tor exit node. You can run it repeatedly to check from multiple Tor perspectives. To get the fingerprint in your client's format, you can append a few conversion commands. For instance, this gives you the fingerprint in Agunua's format (spki:sha256:base64): host_and_port='rawtext.club:1965' torsocks --isolate openssl s_client -connect "$host_and_port" </dev/null 2>/dev/null \ | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' \ | openssl x509 -pubkey -noout \ | openssl pkey -pubin -outform der \ | openssl dgst -sha256 -binary \ | openssl enc -base64 -A This is cumbersome to do by hand, but it can be built into clients, to be executed automatically in case of mismatch on systems where Tor is installed.
On Fri, 2021-06-04, Frank Jüdes wrote: > Ihaq: The certificates for my server are coming from zerossl.com and > they give me free certificates that are valid for only 90 days, so my > server will have a new certificate every three months (the current one > expires on July 25th). Will my capsule be thrown out of your list due to > that new certificate? No, its cert will be updated. I usually run the scripts every 3-4 days, so there will be a delay.
Thank you very much for the update! - A delay shouldn't be a problem. I just need to keep the certificates updated then. Best regards from Charleston (WV), Frank/2 On 2021-06-05 17:55, nervuri wrote: > On Fri, 2021-06-04, Frank Jüdes wrote: >> Ihaq: The certificates for my server are coming from zerossl.com and >> they give me free certificates that are valid for only 90 days, so my >> server will have a new certificate every three months (the current one >> expires on July 25th). Will my capsule be thrown out of your list due to >> that new certificate? > > No, its cert will be updated. I usually run the scripts every 3-4 days, > so there will be a delay. -- ------------------------------------------------------------------------ My Gemini capsule orbits at gemini://h2903872.stratoserver.net/ ------------------------------------------------------------------------
nervuri <nervuri@disroot.org> writes: > I also want to encourage client authors to bunlde pre-generated trust > stores (verified from several perspectives) into their clients, to > protect the first connection. [...] from a packager point of view I fear this can break badly. On OSes that provides stable channels, the packages aren't update frequently. If you add to the mix that there are people using Let's Encrypt (or similar) and thus change the certificate frequently, there's a problem. There is also another drawback to this, that it ties client authors to frequent and periodic updates. Take elpher for example, it hasn't seen commits in a while now (since 2020-09-19 -- almost 9 months!), but it's fine because the code still works.
On 6/5/21 11:48 PM, nervuri wrote: > On Fri, 2021-06-04, panda-roux wrote: >> - What's your process for requesting the removal of a domain's >> certificate from this list? > > I'll be adding code to prune certificates of capsules that have been > offline for a while (2-3 weeks). Aside from this, there's a link to my > contact info in the project's README, if a domain admin wants to request > removal. > > I don't see why anyone would, though. The repo does not hold capsule > contents or personally identifiable information. It just contains > copies of public TLS certificates, which can help people connect more > securely. I assume somebody might have changed their certificate, but still keeps their capsule online? They might want to request you remove the old version and/or put up a new one.
On Sun, 2021-06-06, Omar Polo wrote:
>
> nervuri <nervuri@disroot.org> writes:
>
>> I also want to encourage client authors to bunlde pre-generated trust
>> stores (verified from several perspectives) into their clients, to
>> protect the first connection. [...]
>
> from a packager point of view I fear this can break badly.
>
> On OSes that provides stable channels, the packages aren't update
> frequently. If you add to the mix that there are people using Let's
> Encrypt (or similar) and thus change the certificate frequently, there's
> a problem.
>
> There is also another drawback to this, that it ties client authors to
> frequent and periodic updates. Take elpher for example, it hasn't seen
> commits in a while now (since 2020-09-19 -- almost 9 months!), but it's
> fine because the code still works.
I have mentioned a few solutions to this problem in this thread. I'll
expand on them here.
1. Clients can periodically refresh the trust store from a chosen source
(that the user can change, of course). On mismatches between the local
trust store and the downloaded one, local entries from capsules that the
user has visited should take precedence (unless they are expired,
perhaps). An option can be provided for advanced users to be notified
about such mismatches.
2. Most people don't know what an X.509 certificate is. Rather than
presenting them with a cryptic SSH-like prompt on every mismatch,
clients could do automatic out-of-band verification based on network
perspective. If Tor is available on the user's system, such
verification is simple and anonymous (I exemplified in a previous
email). If Tor is not available, we need a few servers running a CGI
program that takes a hostname[:port] as input, connects to it and
outputs the certificate it received. Clients could then validate certs
by querying these servers. We also need proxies to protect the user's
privacy when connecting to such servers (one extra hop should suffice -
see Anonymized DNSCrypt [1] or Oblivious DoH [2]). You can look at
Trust Seeker [3] for some ideas, although it's just an early sketch and
the code is embarrassing at this point. This month I hope to reboot the
project.
3. Client authors don't have to bundle *all* certificates. They may
pick only a few Gemini hubs with certificates set to expire many years
into the future. This can be done manually. Alternatively, they may
only want to exclude short-lived certs, like those signed by Let's
Encrypt. The trust store generator scripts allow for this. For
instance, you can run:
./generate-trust-store.sh 30- 90+
This excludes certificates for which:
{30 days ago} < cert_expiry < {90 days from now}
So this excludes current Let's Encrypt certs, for instance. Of course,
the numbers can be much higher - it's up to the developer.
[1] https://github.com/DNSCrypt/dnscrypt-protocol/blob/master/ANONYMIZED-DNSCRYPT.txt
[2] https://datatracker.ietf.org/doc/html/draft-pauly-dprive-oblivious-doh-06
[3] https://tildegit.org/nervuri/trust-seeker
On Sun, Jun 06, 2021 at 09:05:58AM +0200, Omar Polo <op@omarpolo.com> wrote a message of 18 lines which said: > from a packager point of view I fear this can break badly. > > On OSes that provides stable channels, the packages aren't update > frequently. Yes, they must be retrieved online. > If you add to the mix that there are people using Let's > Encrypt (or similar) and thus change the certificate frequently, there's > a problem. It depends if your check the entire certificate or just the public key. The later seems more reasonable to me, and can be kept intact when renewing the Let's Encrypt certificate.
---