💾 Archived View for bbs.geminispace.org › u › dimkr › 12344 captured on 2023-12-28 at 17:50:48. Gemini links have been rewritten to link to archived content
-=-=-=-=-=-=-
Re: "I saw a comment by @clseibold earlier (hope I spelled that..."
@Addison you can't distinguish between a troll server that sends the document x bytes at a time to annoy you, and a server that "streams" the response in chunks but sometimes network latency is higher or reliability is lower. And this is also true for streaming of audio and video, not just gemtext. The only "solution" is having socket timeouts (for sending and receiving) so the transfer fails if nothing is received (or acknowledged) for some time, but also per-session timeout (so download fails after a while if the server keeps sending more data but the file is really big or endless).
Dec 05 · 3 weeks ago
🚀 clseibold · Dec 05 at 16:08:
@dimkr I thought about this timeout system, but what happens if a streaming server (like my chat) doesn't send text over because nobody sent a message. Is there some other way to keep connections alive *without* sending over text data?
@clseibold Maybe TCP keepalives, but a client may close a session after some deadline regardless of last I/O time, or terminate sessions based on time since last I/O (last packet carrying more data to display) instead of time since the last packet (including keepalive packets). Gemini is not really designed for streaming, it can be done but you need to make assumptions about the client and that means a capsule marked 'works best with x', like old sites built with IE in mind.
🚀 clseibold · Dec 05 at 18:00:
@dimkr Right. I was trying to get at keep alive packets. I use them, but I don't know much about how they work. A system that assumes a timeout with just the data will not take into account these keepalive packets.
I do disagree that Gemini is not designed for livestreaming (which isn't the same as all streaming - streams that support seeking are very different from livestreams). It works perfectly fine for streaming and the protocol explicitly mentions that it supports streaming. I was there when it was added to the protocol spec.
The difference is support for streaming was *optional* rather than required, which is why we have clients that don't support it. But I do not consider this a real problem, because not all browsers support displaying all filetypes either, that's just how browsers work.
If we think about it, all TCP connections *are* streams. Livestreaming just requires sending bits over a TCP stream over a certain time period. This is why mp3s and oggs have their metadata at the front, so that this is possible. Mp4 file data can also be split up and rearranged so that you can stream them as well.
It is the seeking of a stream that requires specific protocol stuff, but livestreams don't use seeking, that's only streams of on-demand content.
🚀 clseibold · Dec 05 at 18:05:
@dimkr In fact, the idea of "this site works best with X" was a specific sub-goal that Solderpunk had in mind, to a limited extent. The idea was never to *limit* Gemini to only gemtext! The idea was that all browsers support a baseline but could do other things on top of that. This idea was explicitly outlined in Solderpunk's "A vision for Gemini applications" article:
gemini://gemini.circumlunar.space/users/solderpunk/gemlog/a-vision-for-gemini-applications.gmi
It is also the reason why gemini had the mimetype field - so that browsers could support different filetypes and aren't limited to gemtext only.
With this gemini applications idea, a gemini site could be tailored to a specific application (or rather, a specific application could make use of a gemini site in a specific way). He also outlines gemtext streaming here, and literally thought up Mozz's chat design originally (from chat.mozz.us).
Anyways, it looks like TCP keepalive packets solve a lot of problems.
@clseibold I see things differently, I think that supporting a MIME type is one thing, but assumptions about client behavior is another. For example, some capsules have links that align nicely in Lagrange, and Station expects a client certificate for / although it's /join that returns 60 (Lagrange defaults to certificates for /, so it "just works"). In gplaces, I had to mimic Lagrange in some corner cases - for example, only warn, not show an error, if a capsule doesn't send close_notify like the spec says. I don't like capsules that work better with a specific client, or only with a specific client.
🚀 clseibold · Dec 06 at 16:57:
@dimkr My point wasn't really whether that was good or not, but that it's how the spec was designed. Stream-like handling is optional in the spec. That's how it was designed.
The spec isn't all-encompasing - a lot of times it allows for *optional things.* A browser can support streaming, it can support different filetypes, it can even display things differently than other clients (headings in lagrange look very different from other browsers, so I make sure to tailor my headings for lagrange first).
How certificates behave is a completely different thing from supporting streaming, imo, so I think it is a false comparison. However, I believe certificates are optional in the spec too.
Peronally, I would require that all clients need to *support* TCP Keep Alive packets when they are used (and I bet many clients already do this), and then the problem we discussed about misbehaving clients would basically be fixed.
Update: Yes, I was correct. Client certificates are completely optional in the spec:
Basic clients may also choose not to support client-certificate authentication, in which case only four distinct status handling routines are required (for statuses beginning with 1, 2, 3 or a combined 4-or-5).
Also from the spec, notice how it's **optional**:
Note that clients are not obligated to wait until the server closes the connection to begin handling the response. This is shown above only for simplicity/clarity, to emphasise that responsibility for closing the connection under typical conditions lies with the server and that the connection should be closed immediately after the completion of the response body.
For example, some capsules have links that align nicely in Lagrange, and Station expects a client certificate for / although it's /join that returns 60 (Lagrange defaults to certificates for /, so it "just works").
The way I see it is this: There's nothing that says that clients cannot use certificates on links that don't request a certificate. Whetehr station expects a client cert for / or /join isn't a client problem, because the client is allowed to use certs on pages that don't request certs, and to lagrange, / encompasses /join.
Which means this is a server problem, not a client problem, because a **server** can only expect a cert to be used on the pages they are requested for.
So it should be reported to the server maintainer/developer, not the client developer, imo.
🚀 clseibold · Dec 06 at 17:20:
@dimkr Btw, and @skyjake can correct me on this if I'm wrong, but I'm pretty sure lagrange used to only enable certificates on the specific page (like "/join") when that page requested it (and the user enabled the cert from there), rather than always putting it on /. And if I recall correctly, it was the server developers (including me) who didn't realize that we couldn't expect certs on "/" when we only sent the request on "/join" (or "/register").
I ended up fixing this for some of my AuraGem stuff by using "/?register" instead, so that the client cert request always happens on "/", but not all servers fixed this problem, so clients have to adapt to the servers.
So I think the history is actually the opposite - servers started expecting cert requests on subpages to be working for the whole capsule, and clients had to adapt to make this work, instead of the other way around.
I specifically remember running into this problem with one of my AuraGem services, and so I had to fix it on my server, because it was a server bug, not a client bug.
Yes you are correct about the history @clseibold.
This reminds me I should use the "/?register" solution myself in Bubble, because having an actual "Create Account" UI action beats the current "just activate a certificate but please only at the root".
@clseibold @skyjake This history shows how much "wiggle room" the spec has, allowing clients to behave very differently from each other, until clients and servers achieve some status quo not mandated by the spec. I know that my client (and probably many others) doesn't support "chat" because it previews the document during download but shows the prompt that allows you to navigate to a link only when transfer is finished.
🚀 clseibold · Dec 07 at 17:35:
@dimkr I agree that the spec has quite a bit of wiggle room. However, I think the situation with where certs are activated is just an oversight in the spec. One could fix this by just saying under the client certs sections that "A server can only expect a client cert to be activated under the path in which the client cert request status code was sent" or something along those lines.
Lots of clients don't support streaming gemtext in general because it was marked optional. I think the best solution for this is to just go with Solderpunk's idea written in the article I linked ("A vision for Gemini applications") - I can create a chat terminal program specifically for AuraGem Live Chat.
I saw a comment by @clseibold earlier (hope I spelled that right) talking about streaming audio capabilities in mobile Gemini clients. I can't seem to find the post anymore, but it got me thinking about whether Opal (the client library that Rosy Crow uses to do it's Gemini stuff) can support that. [https link] Currently it does not support that, because the entire response is read all at once into a buffer. Thinking about this a while, I of course arrived at the classic conclusion that...