A discussion took place in August that was revisited on IRC today: https://lists.orbitalfox.eu/archives/gemini/2020/002404.html I disagree very strongly with the suggestion of gemini+stream://. It relies upon a presumption about the specification which is poorly supported: that all Gemini requests are finite. The supporting text for this is supposedly that section 1.1 lists "C: Handles response" last. However: 1. Section 1.1 reads like an informative summary of the authoritative text which follows 2. There is no explicit specification that these steps are ordered or cannot be completed in parallel, and aren't even numbered. The other relevant piece of text is in 3.3: > The server closes the connection after the final byte, there is no > "end of response" signal like gopher's lonely dot. There is no statement anywhere that there actually has to *be* a final byte; only a clarification on how the final byte is signalled if present. Nothing in the specification prohibits Gemini responses from being non-finite, and in fact if clients presume this will be the case then they'll be easily exploited by servers which simply forward /dev/zero into the connection and never terminate, leading to OOM crashes or disk space exhaustion. If you want to stream content, the protocol permits it as-is. And remember the one big rule: Do not. Extend. Gemini.
On 2020-11-16 (Monday) at 23:01, Drew DeVault <sir at cmpwn.com> wrote: > > If you want to stream content, the protocol permits it as-is. And > remember the one big rule: > > Do not. > Extend. > Gemini. > Hear, hear. Let's put this to rest. Clients either will or won't handle streaming; while some problems with UI might deserve solutions (how does a client handle a long-time request? How to download/stream/save a thing? etc.), I'm sure the smart people on this list and around Geminispace can do so. -- ~ acdw acdw.net | breadpunk.club/~breadw
On 11/16/20 11:01 PM, Drew DeVault wrote: > I disagree very strongly with the suggestion of gemini+stream://. It > relies upon a presumption about the specification which is poorly > supported: that all Gemini requests are finite. As the original author of the gemini+stream proposal I want to just jump in with my agreement of Drew's response. Everything I had stated was based on my *incorrect* assumption that the spec was clear that our connection ended before processing data. If that is not the case then any and all Gemini clients can support streaming and we have no need for a +stream add-on. Hooray for streaming! > Do not. > Extend. > Gemini. Agreed.
On Mon, Nov 16, 2020 at 11:43:13PM +0000, James Tomasino wrote: > On 11/16/20 11:01 PM, Drew DeVault wrote: > > I disagree very strongly with the suggestion of gemini+stream://. It > > relies upon a presumption about the specification which is poorly > > supported: that all Gemini requests are finite. > > As the original author of the gemini+stream proposal I want to just jump in with my agreement of Drew's response. Everything I had stated was based on my *incorrect* assumption that the spec was clear that our connection ended before processing data. If that is not the case then any and all Gemini clients can support streaming and we have no need for a +stream add-on. > > Hooray for streaming! > > > Do not. > > Extend. > > Gemini. > > Agreed. 100% agreed I'll update my server to no longer respond to gemini+stream:// requests (it was already responding to them the exact same way as gemini:// requests, so my gemini-backed streaming radio will, of course, remain "on air") bie
On Mon, Nov 16, 2020 at 6:43 PM James Tomasino <tomasino at lavabit.com> wrote: As the original author of the gemini+stream proposal I want to just jump in > with my agreement of Drew's response. Everything I had stated was based on > my *incorrect* assumption that the spec was clear that our connection ended > before processing data. If that is not the case then any and all Gemini > clients can support streaming and we have no need for a +stream add-on. > What concerns me, though, is that this suggests that it is forbidden for any client to drop a connection when nothing has been sent for a while or the client believes too much data has been sent. I think we have to make it clear that not all clients need to be able to accept streaming transmissions. John Cowan http://vrici.lojban.org/~cowan cowan at ccil.org A mosquito cried out in his pain, "A chemist has poisoned my brain!" The cause of his sorrow / Was para-dichloro- Diphenyltrichloroethane. (aka DDT)
On Mon Nov 16, 2020 at 7:02 PM EST, John Cowan wrote: > What concerns me, though, is that this suggests that it is forbidden for > any client to drop a connection when nothing has been sent for a while > or the client believes too much data has been sent. I think we have to > make it clear that not all clients need to be able to accept streaming > transmissions. Not accepting streaming transmissions is not really something the client can do and be a spec-compliant implementation. You can set a maximum size on the response, or you could set a minimum transfer throughput (I recommend being generous with that one), and you should let the user cancel an in-progress request.
On Mon, Nov 16, 2020 at 7:04 PM Drew DeVault <sir at cmpwn.com> wrote: > Not accepting streaming transmissions is not really something the client > can do and be a spec-compliant implementation. You can set a maximum > size on the response, or you could set a minimum transfer throughput (I > recommend being generous with that one), I'm sorry, all those negations break my poor monkey brain. The first sentence seems to say that the client must continue reading to the end of the response, the second sentence says that it may break it off. Malicious servers are, or will be, a fact of life; connection problems are already a fact of life. you should let the user > cancel an in-progress request. > Not all clients are interactive. John Cowan http://vrici.lojban.org/~cowan cowan at ccil.org We pledge allegiance to the penguin and to the intellectual property regime for which he stands, one world under Linux, with free music and open source software for all. --Julian Dibbell on Brazil, edited
On Mon Nov 16, 2020 at 7:11 PM EST, John Cowan wrote: > I'm sorry, all those negations break my poor monkey brain. The first > sentence seems to say that the client must continue reading to the end > of the response, the second sentence says that it may break it off. > Malicious servers are, or will be, a fact of life; connection problems > are already a fact of life. I'm saying that the server could send you a streaming response, and there's no way for you to stop it from doing so, so you have to be aware of it and do SOMETHING to make sure it's dealt with properly. And whatever you choose to do about it should probably be with the best UI applicable to your use-case.
On Mon, Nov 16, 2020 at 7:13 PM Drew DeVault <sir at cmpwn.com> wrote:
> I'm saying that the server could send you a streaming response, and
> there's no way for you to stop it from doing so, so you have to be aware
> of it and do SOMETHING to make sure it's dealt with properly. And
> whatever you choose to do about it should probably be with the best
> UI applicable to your use-case.
Ah, then we agree. Good.
John Cowan http://vrici.lojban.org/~cowan cowan at ccil.org
Raffiniert ist der Herrgott, aber boshaft ist er nicht.
--Albert Einstein
> [...] It > relies upon a presumption about the specification which is poorly > supported: that all Gemini requests are finite. > [...] > Nothing in the specification prohibits Gemini responses from being > non-finite, and in fact if clients presume this will be the case then > they'll be easily exploited by servers which simply forward /dev/zero > into the connection and never terminate, leading to OOM crashes or disk > space exhaustion. This is interesting. I'm writing a python lib for handling gemini requests, to simplify browser development. I just assumed that responses are finite, and as my code works now a method initiates a request, receives the response, builds a response object and returns it to the caller. If responses can be infinite this will indeed cause OOM errors and crash, and the caller won't know what hit 'em. Should I be reading the header and return a response object that contains an open socket instead of a byte array? Can we assume that browser developers understand this ambiguity and handle it correctly? - ew0k
> Should I be reading the header and return a response object that > contains an open socket instead of a byte array? Yes. This is what I've seen from Go libraries as well, and it provides a simple solution that gives the developer the most power. makeworld
When I read the spec I did not understand this. I'd be nice if this was stated explicitly in the the spec if that was in fact the intention.
Bj?rn W?rmedal <bjorn.warmedal at gmail.com> writes: > This is interesting. I'm writing a python lib for handling gemini > requests, to simplify browser development. I just assumed that > responses are finite, and as my code works now a method initiates a > request, receives the response, builds a response object and returns > it to the caller. If you get this lib to a good place, let me know, and I will deprecate `gusmobile'. I've done a pretty terrible job maintaining it because I'm not actually using it for anything. -- +-----------------------------------------------------------+ | Jason F. McBrayer jmcbray at carcosa.net | | A flower falls, even though we love it; and a weed grows, | | even though we do not love it. -- Dogen |
---