💾 Archived View for bbs.geminispace.org › s › Gemini › 1518 captured on 2023-12-28 at 18:02:01. Gemini links have been rewritten to link to archived content
⬅️ Previous capture (2023-11-14)
-=-=-=-=-=-=-
A few servers, when visited in Rosy Crow, will respond with an Invalid URL error. This problem had been a head-scratcher for a bit, but I've found the cause: some servers don't wait for the client's request to finish before they respond to it.
The root cause is the result of how the Opal client library makes requests. In order to avoid allocating an extra string (a misguided, premature optimization), it sends requests by writing twice to the open TLS socket: one write for the URL, and another for the CRLF terminator:
private static async Task SendRequestAsync(Uri uri, SslStream stream)
{
// first write the URI... more to come...
var body = ConvertToUtf8(uri.ToString());
await stream.WriteAsync(body, 0, body.Length);
// ... and the second write terminates the request
var newline = ConvertToUtf8("\r\n");
await stream.WriteAsync(newline, 0, newline.Length);
...
}
What I believe is happening on the server-side is: the server reads from the socket *once* and assumes it has the entire request. Then, because it stopped reading before the CRLF had been received, it determines that the request was invalid and responds as such.
Servers should instead continue to read from the TLS socket until they stop receiving data.
As expected, writing the entire request at once succeeds:
private static async Task SendRequestAsync(Uri uri, SslStream stream)
{
var request = ConvertToUtf8($"{uri}\r\n");
await stream.WriteAsync(request, 0, request.Length);
}
I really should have been sending requests this way all along, as it's simpler and in fact makes one *fewer* allocation than the prior method. I just thought this was an interesting quirk that some server implementers might want to keep in mind.
I'll be using this method in Opal 1.6.1 and Rosy Crow 1.1.3.
Here's a capsule that's failing to load in Rosy Crow due to this issue
This issue's tracker on s/Rosy-Crow-Issues
Jun 04 · 7 months ago
🚀 skyjake · 2023-06-04 at 04:51:
Yeah, this topic is good to note if you're writing a server. Dealing with sockets isn't quite as simple as one would like/expect.
While writing my server I found you will need to read until a CRLF is received (up to 1024 bytes), the client terminates the connection, or there's a timeout because the client never finishes writing a proper request. This timeout situation seems to happen more often than you'd think. If not handled properly, your server will eventually hang waiting for requests to complete.
🤖 alexlehm · 2023-06-05 at 14:18:
Funny enough, I had an issue like that when writing a misfin server in Java and handled the data received from the socket as it appeared and didn't wait for the \r\n to finish
🦀 jeang3nie · 2023-06-05 at 15:52:
Nice detective work there. That would be a great thing to mention in a "best practices" document for implementors.