💾 Archived View for gemi.dev › gemini-mailing-list › 000446.gmi captured on 2024-05-12 at 16:06:36. Gemini links have been rewritten to link to archived content
⬅️ Previous capture (2023-12-28)
-=-=-=-=-=-=-
Does anyone have any concerns about amending the spec to state that a TLS
close_notify message should be sent before closing the TCP connection?
While TLS guarantees the integrity of the data from the server, it does
not guarantee completeness until a close_notify is received by the client.
Interested and able clients could then determine that they received a
complete response.
The sending of close_notify is discussed in section 6.1 of RFC 8446 [1].
This approach was previously discussed on the list by Michael Lazar and
solderpunk [2] [3] and kooda [4].
I have been testing various software and configurations to find out the
feasibility of this approach. The ability to send close_notify is part of
the TLS 1.2 and 1.3 specs, and is widely available for server authors to
use. The OpenSSL C library provides the SSL_shutdown function which sends
close_notify. The LibreSSL C library has had the same function since its
first release. The libtls library included with LibreSSL provides this
functionality in the tls_close function. I've verified the effectiveness of
Python's SSLSocket.unwrap method. Ruby seems to handle this with
SSLObject.sysclose and Go with the Close method in crypto/tls/conn.go.
Every TLS 1.2 or 1.3 client I've seen will automatically handle TLS
close_notify messages. Non-blocking clients will make a poll/select/etc
call and wait for data from the server. When the close_notify message is
received by the TLS library used by the client software, it will indicate
that data is ready to be read/received. So whether the client is blocking
or non-blocking, it will eventually perform a read/receive call since it
cannot yet know that there is no more data from the server. The
close_notify message from the server will causes the TLS library to return
0 bytes from the read/receive call. Some libraries will also return an
error message or status code indicating that a close_notify was received.
In any case, the connection with the server is automatically closed
(completely in TLS 1.2, and at a minimum from further reads in TLS 1.3).
So, why can't we just use a read/receive of 0 bytes to always know that a
close_notify was received? Because if the TCP connection is closed before a
close_notify, the read/receive call will also return 0 bytes. Some clients
do not have a way to distinguish between TCP disconnect and close_notify.
But should a client author wish to use this feature, those written in C and
using OpenSSL/LibreSSL can use the SSL_read and SSL_get_error functions to
obtain the SSL_ERROR_ZERO_RETURN value. Those written in C and using libtls
can use the tls_read and tls_get_error functions. Python clients can use
the demonstrated method [6]. And while there is no assured way to do this
in Ruby or Go currently, patching Ruby's SSLSocket.sysread or Go's
crypto/tls/conn.go readRecord to use the information already available in
those source files might be possible.
You can inspect the TLS messages received from any server, including
the close_notify message, with the included Python script [6].
130 out of 208 servers listed on GUS [5] send the close_notify
message before disconnecting.
[1] https://tools.ietf.org/html/rfc8446
[2] gemini://gemi.dev/gemini-mailing-list/messages/000037.gmi
[3] gemini://gemi.dev/gemini-mailing-list/messages/002150.gmi
[4] gemini://gemi.dev/gemini-mailing-list/messages/002243.gmi
[5] gemini://gus.guru/known-hosts
[6] I dedicate this script to the public domain.
??? To query a different server, modify the second line.
import socket, ssl, time
server, port, path = 'gemini.circumlunar.space', 1965, '/'
start, out, cn = time.time(), [], False
def log(m): out.append('\n--- %f %s' % (time.time()-start, m))
def callback(connection, direction, v, c, m, data):
? p = [str(i)[str(i).find('.')+1:] for i in [direction, v, c, m]]
? log(''.join([i.ljust(j, ' ').lower()
????????????? for i, j in zip(p, [6, 8, 19, 20])]))
? if m == ssl._TLSAlertType.CLOSE_NOTIFY and direction == 'read':
??? global cn; cn = True
context = ssl.SSLContext(ssl.PROTOCOL_TLS)
context.verify_mode = ssl.CERT_NONE
context.check_hostname = False
context._msg_callback = callback
sock = socket.socket(socket.AF_INET)
sock.settimeout(5)
connection = context.wrap_socket(sock, server_hostname=server)
connection.connect((server, port))
log('SENDING GEMINI REQUEST TO SERVER')
connection.send(b'gemini://'+server.encode()+path.encode()+b'\r\n')
log('RECEIVING GEMINI RESPONSE FROM SERVER')
sameline = False
while True:
? try:
??? data = connection.recv().decode()
??? if len(data) < 1:
????? if cn: log('COMPLETE RESPONSE RECEIVED')
????? if not cn: log('COMPLETE RESPONSE MAY NOT HAVE BEEN RECEIVED')
????? break
??? if not sameline:
????? data = '\n' + data
??? out.append(data.rstrip().replace('\n', '\n'+' '*4))
??? sameline, data = (data[-1] != '\n'), ''
? except Exception as e:
??? log(str(e).upper())
connection.close()
print((''.join(out))[1:])
On Sat, 7 Nov 2020 19:20:21 -0600 Scot <gmi1 at scotdoyle.com> wrote: > Does anyone have any concerns about amending the spec to state that a TLS > close_notify message should be sent before closing the TCP connection? > While TLS guarantees the integrity of the data from the server, it does > not guarantee completeness until a close_notify is received by the client. > Interested and able clients could then determine that they received a > complete response. A close_notify alert "MUST" be sent prior to closing the write side of the connection in both TLS 1.2 and 1.3. I should think it's redundant to mention this in the Gemini spec since it's already required by the TLS specs, but the number of servers that don't implement TLS properly is alarmingly high, so maybe a reminder is necessary. -- Philip -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 488 bytes Desc: not available URL: <https://lists.orbitalfox.eu/archives/gemini/attachments/20201108/01dd d6da/attachment-0001.sig>
On Sat, Nov 7, 2020 at 8:19 PM Scot <gmi1 at scotdoyle.com> wrote: > > Does anyone have any concerns about amending the spec to state that a TLS > close_notify message should be sent before closing the TCP connection? > While TLS guarantees the integrity of the data from the server, it does > not guarantee completeness until a close_notify is received by the client. > Interested and able clients could then determine that they received a > complete response. > > The sending of close_notify is discussed in section 6.1 of RFC 8446 [1]. > > This approach was previously discussed on the list by Michael Lazar and > solderpunk [2] [3] and kooda [4]. > > <snip> > > You can inspect the TLS messages received from any server, including > the close_notify message, with the included Python script [6]. > > 130 out of 208 servers listed on GUS [5] send the close_notify > message before disconnecting. Thank you so much for this fantastic writeup! I agree with your conclusion that close_notify should be explicitly called out in the gemini spec, particularly in the transaction diagram under section 1.1. I don't fault server authors for not implementing this correctly. If anything, it just goes to show that TLS libraries are almost universally crappy and unintuitive. We can't expect newcomers to gemini, many of whom are writing their first server or trying out a new language, to know the inner workings of TLS. I took your code snippet and added to my gemini portal at https://portal.mozz.us Now anyone can view their TLS connection details, including if the `send_alert` message was sent, by clicking on the [view cert] link at the top of the page. Hopefully this helps out as a quick way to test connections. - mozz
2020/11/11 03:39, Michael Lazar: > I took your code snippet and added to my gemini portal at https://portal.mozz.us > Now anyone can view their TLS connection details, including if the `send_alert` > message was sent, by clicking on the [view cert] link at the top of the page. > Hopefully this helps out as a quick way to test connections. Thanks that's very helpful! R.
On 11/10/20 8:39 PM, Michael Lazar wrote: > On Sat, Nov 7, 2020 at 8:19 PM Scot <gmi1 at scotdoyle.com> wrote: > > Does anyone have any concerns about amending the spec to state that a TLS > close_notify message should be sent before closing the TCP connection? > While TLS guarantees the integrity of the data from the server, it does > not guarantee completeness until a close_notify is received by the client. > Interested and able clients could then determine that they received a > complete response. > > The sending of close_notify is discussed in section 6.1 of RFC 8446 [1]. > > This approach was previously discussed on the list by Michael Lazar and > solderpunk [2] [3] and kooda [4]. > > <snip> > > You can inspect the TLS messages received from any server, including > the close_notify message, with the included Python script [6]. > > 130 out of 208 servers listed on GUS [5] send the close_notify > message before disconnecting. > Thank you so much for this fantastic writeup! I agree with your conclusion > that close_notify should be explicitly called out in the gemini spec, > particularly in the transaction diagram under section 1.1. > > I don't fault server authors for not implementing this correctly. If anything, > it just goes to show that TLS libraries are almost universally crappy and > unintuitive. We can't expect newcomers to gemini, many of whom are writing > their first server or trying out a new language, to know the inner workings of > TLS. I agree. Even Python's TLS library does not send it by default when closing an SSLSocket connection. Maybe now that the TLS 1.3 spec allows unidirectional close_notify the library situation will slowly improve. > > I took your code snippet and added to my gemini portal at https://portal.mozz.us > Now anyone can view their TLS connection details, including if the `send_alert` > message was sent, by clicking on the [view cert] link at the top of the page. > Hopefully this helps out as a quick way to test connections. https://portal.mozz.us looks useful. Of the five servers I tested there our results match. > > - mozz
Michael Lazar <lazar.michael22 at gmail.com> writes:
> I took your code snippet and added to my gemini portal at
> https://portal.mozz.us Now anyone can view their TLS connection
> details, including if the `send_alert` message was sent, by clicking
> on the [view cert] link at the top of the page.
Thanks, this is particularly useful to me. Now the question remains as
to whether I can actually get the TLS library I'm using to do this.
--
Jason McBrayer | ?Strange is the night where black stars rise,
jmcbray at carcosa.net | and strange moons circle through the skies,
| but stranger still is lost Carcosa.?
| ? Robert W. Chambers,The King in Yellow
---