💾 Archived View for jsreed5.org › log › 2024 › 202402 › 20240201-browsing-gemini-with-kermit.gmi captured on 2024-07-08 at 23:50:26. Gemini links have been rewritten to link to archived content
⬅️ Previous capture (2024-02-05)
-=-=-=-=-=-=-
---
I first learned about Kermit a few months ago, from the indefatigable John Goerzen at complete.org.^ His introduction to Kermit covers the essentials quite well:
It is:
- A file transfer protocol for running over serial lines, Modems, or TCP/IP. The protocol is quite flexible, supporting everything from tiny embedded devices with 90-byte packets to streaming over ssh.
- A FTP- or SFTP-like system-agnostic protocol for looking at directories of files on remote systems, renaming files, deleting them, etc. with a standard process.
- Capable of operating under extremely challenging conditions, including 7-bit connections and connections that otherwise “eat” certain characters.
- A modem dialer.
- A fully-functional standalone TCP server.
- A system for inband communication over a single connection.
- On Windows, a terminal emulator. (On Linux, it runs within a terminal emulator already, so doesn’t need to provide that functionality).
- A thin wrapper around ssh providing new features.
- A scripting engine, which can be used to automate terminal interaction, file transfer, and more. There are some pretty neat kermit scripts out there - everything from renaming JPEGs based on embedded metadata to budgeting.
- It’s also a Telnet, SSH, FTP, and HTTP(S) client.
You can download kermit for dozens of platforms, many of which are decades out of date. They all interoperate. It is rather impressive. In fact, a new version of Kermit for TOPS-20 was released in June 2023! (TOPS-20 was active between 1976 and 1988).
Kermit has been continuously developed since 1981, first at Columbia University in New York City and since 2011 by Frank da Cruz independently. da Cruz is one of the original contributors to the project, meaning he has been involved with Kermit for a staggering 43 years!
I began to dig into Kermit after trying CKermit as an SSH wrapper. At first, I was only interested in its ability to facilitate file transfers without dropping the SSH connection. Soon I found myself wandering farther and farther into CKermit's extensive command list, deceptively encapsulated into a single `kermit` invocation at the command line.
What awaited me was an application paradigm that brought back memories of using MS DOS. Kermit's language, syntax and command structures are reminiscent of the time in which they were primarily developed: the late 1980s and early 1990s. They are jarringly different from any scripting of command language used today, and they are decidedly uglier than the elegant applications we have today. But something about their clunkiness beings me back to days gone by, when computing was still largely a frontier, and standards grew organically rather than being dictated from the top down.
One of Kermit's features that stood out to me is that it can make HTTPS connections. This implies SSL/TLS support, which is mandatory in Geminispace. I began to wonder: could I browse Gemini using CKermit? The answer is yes, by way of abusing Kermit's Telnet functionality.
In order for CKermit to access documents over Gemini, it must be compiled with SSL support enabled. Unfortunately, not all software repositories compile offer CKermit packages with this feature. The CKermit package available on Raspberry Pi OS had SSL/TLS enabled, but the packages on Fedora Linux and Termux did not. You can verify that your version of CKermit supports SSL/TLS by running at the prompt:
(~/) C-Kermit>check ssl ssl/tls available
The core steps to request a document over Gemini are rather straightforward. As an example, the following commands demonstrate how to request (shameless plug) a beginner-friendly solution guide I wrote for solving Rubik's Cube.
The first step is to establish a connection to the host we want to send a request to.
(~/) C-Kermit>set host jsreed5.org:1965 /ssl
DNS Lookup... Trying 3.133.216.238... (OK)
Warning: Server has a self-signed certificate
[0] Certificate Subject=
O=Rob's Capsule
CN=jsreed5.org
[0] Certificate Issuer=
O=Rob's Capsule
CN=jsreed5.org
Continue? (Y/N)
Kermit notices that the SSL certificate for my capsule is self-signed. As a security measure, it prompts us to either accept or reject the certificate.
Continue? (Y/N) y [SSL - OK] [TLS - TLS_AES_256_GCM_SHA384 TLSv1.3 Kx=any Au=any Enc=AESGCM(256) Mac=AEAD Compression: None Negotiations.. (OK)
CKermit is now connected to jsreed5.org on port 1965. We have fooled CKermit into thinking that we have connected via Telnet, but we have actually made a direct connection via SSL to the server on port 1965.
Next we open the connection so we can send a request.
(~/) C-Kermit>connect Connecting to host jsreed5.org:1965 Escape character: Ctrl-\ (ASCII 28, FS): enabled Type the escape character followed by C to get back, or followed by ? to see other options. ----------------------------------------------------
The true power of Kermit's Telnet functionality becomes apparent here. Below the line we can begin typing in any request to the server we want. The current Gemini protocol specification (at time of writing) states that a request must be a single CRLF-terminated line containing an absolute URL. CKermit's prompt allows us to send just such a request. In other tools such as ncat, the CR often needs to be included manually, but Kermit includes it for us automatically.
The snippet below is truncated after the first 6 lines of output.
----------------------------------------------------
gemini://jsreed5.org/twisty/nxnxn/3/layer-by-layer.gmi
20 text/gemini
## 3x3x3 Solution: Layer by Layer
---
### Preliminary Information
Something strange has happened: each line starts directly below where the previous line ends! This is because CKermit honors carriage returns, and the page we requested only uses line feeds. This is evident by the fact that the first line of the page (under the 20 response code) is left-aligned, and per the Gemini protocol specification the response code must be CRLF-terminated. We can set Kermit to format output as if it contained carriage returns, by changing a terminal setting.
(~/) C-Kermit>set terminal lf-display crlf
Returning to our requested page, we see the following at the end of the response:
Communications disconnect (Back at localhost) ---------------------------------------------------- (~/) C-Kermit>
Gemini is a one-request-one-response protocol, so the server immediately closed the connection when it finished sending the response. However, CKermit's connection is still open on the local side, If we issue `connect ` again, CKermit tries to reuse the previous connection, which is already closed on the remote side, and the connection fails.
(~/) C-Kermit>connect DNS Lookup... Trying 3.133.216.238... (OK) [SSL - OK] [TLS - TLS_AES_256_GCM_SHA384 TLSv1.3 Kx=any Au=any Enc=AESGCM(256) Mac=AEAD Compression: None Negotiations................................
Our abuse of Telnet is evident in Kermit's warning message. However, the issue is not necessarily that the listed commands are failing; the real issue is that the SSL negotiations never complete, because the connection on which negotiations would take place is closed server-side. The solution is to manually close and reopen the connection locally.
(~/) C-Kermit>close Closing connection (~/) C-Kermit>set host jsreed5.org:1965 /ssl DNS Lookup... Trying 3.133.216.238... (OK) [SSL - OK] [TLS - TLS_AES_256_GCM_SHA384 TLSv1.3 Kx=any Au=any Enc=AESGCM(256) Mac=AEAD Compression: None Negotiations.. (OK)
Note that we are not prompted to trust the server's certificate again. If little enough time passes between connections, CKermit retains the previous user-specified trust settings.
Perhaps most interestingly, Kermit can even make requests using client-side TLS authentication. That means Gemini client certificates work out of the box! If you have an RSA-encrypted certificate and key file, such as "rob-s-cert.pem" and "rob-s-key.pem", you can use them with the following commands:
set authorization tls rsa-cert-file rob-s-cert.pem set authorization tls rsa-key-file rob-s-key.pem
We can tell Kermit to use a different TLS certificate by setting other files, but in CKermit, there seems to be no way to unset these files (switch back to using no certificates) without fully exiting CKermit and restarting.
---
Kermit's true power comes from its scripts, which provide a protocol-agnostic way to automate all sorts of network operations--from server configurations over SSH to bulk file transfers and processing over FTP. The scripting language is Turing-complete and can run any command available on the host machine, not just its own command set. Kermit runs scripts using the `take` command, or we can pass in a script as an argument when invoking CKermit.
We can try naively putting the interactive commands we used into a script called "kermit-gemini.ksc" and running it. But the `connect` command we used earlier is strictly interactive; we need to enter our request manually when we invoke it. We use `lineout` instead to send the request programatically. We also add an `exit` statement to drop us directly back to our shell instead of the Kermit prompt.
$ cat kermit-gemini.ksc
set terminal lf-display crlf
set host jsreed5.org:1965 /ssl
lineout gemini://jsreed5.org/twisty/nxnxn/3/layer-by-layer.gmi
close
exit
$ kermit kermit-gemini.ksc
DNS Lookup... Trying 3.133.216.238... (OK)
Warning: Server has a self-signed certificate
[0] Certificate Subject=
O=Rob's Capsule
CN=jsreed5.org
[0] Certificate Issuer=
O=Rob's Capsule
CN=jsreed5.org
Continue? (Y/N) y
[SSL - OK]
[TLS - TLS_AES_256_GCM_SHA384 TLSv1.3 Kx=any Au=any Enc=AESGCM(256) Mac=AEAD
Compression: None
Negotiations.. (OK)
gemini://jsreed5.org/twisty/nxnxn/3/layer-by-layer.gmi
$
Our request was successful, but we didn't print the response. That's because Kermit doesn't automatically process the response from the server. To do that, we need to invoke the `input` command--and here's one place where Kermit betrays the old paradigm it was developed in.
The `input` command doesn't simply take in the entire response by default. The original intention of `input` was to parse a response and search for a specific string to ensure the response we expected was the response we received. To that end, `input` requires a timeout argument, which causes the command to report a failure if we cannot find the string we want. In order to receive the response itself, we need to tell `input` not to try matching anything. The "30" argument tells `input` to stop listening after 30 seconds.
input /nomatch 30
There is another issue with `input`: its echoing behavior may be unpredictable. In CKermit 10.0 beta 8, the input command only echoed the first 38 KB of the response when I requested the solution guide document. Requesting the index only gave me the first 1024 bytes. I'm not an expert on Kermit, but I believe this behavior is related to how CKermit buffers input data as it's being read. Luckily, Kermit stores the data in the input buffer to a built-in variable: "\v(input)". We can bypass the inconsistent behavior of input's echo and print the contents of this variable instead.
set input echo off echo \v(input)
However, a caveat exists with "\v(input)". Kermit does not expect responses over Telnet to be very large, so by default the input buffer is only 4096 bytes. When the buffer fills up, Kermit loops back to the beginning and overwrites its previous contents. The result is that much of the output gets overwritten and jumbled.
$ cat kermit-gemini.ksc
set input echo off
set host jsreed5.org:1965 /ssl
lineout gemini://jsreed5.org/twisty/nxnxn/3/layer-by-layer.gmi
input /nomatch 30
echo \v(input)
close
exit
$ kermit kermit-gemini.ksc
DNS Lookup... Trying 3.133.216.238... (OK) Warning: Server has a self-signed certificate
[0] Certificate Subject=
O=Rob's Capsule
CN=jsreed5.org
[0] Certificate Issuer=
O=Rob's Capsule
CN=jsreed5.org
Continue? (Y/N) y
[SSL - OK]
[TLS - TLS_AES_256_GCM_SHA384 TLSv1.3 Kx=any Au=any Enc=AESGCM(256) Mac=AEAD
Compression: None
Negotiations.. (OK)
--+R/B| R | R | R | R |\G\ +---+---+---+---+---+
| G | O | B |/B/| +---+---+---+G\R| | G | Y | Y | Y | B | R
+---+---+---+B/B| | R | R | R |\G\| +---+---+---+---+---+
| O | O | O |/B/ +---+---+---+G\G| L | R | Y | Y | Y | R |
+---+---+---+B/ \_W_\_W_\_W_\G\| +---+---+---+---+---+ | O | O | O |/ \_W_\_W_\_W_\G| | G | O | B |
+---+---+---+ D \_W_\_W_\_W_\| +---+---+---+ F F
\```
Apply (17) to solve the UBR corner. Rotate the entire cube in your hands until the solved corner is at UFL, then apply (18) to solve the remaining yellow corners.
In the second case, the cube will appear as below (UFR, DBL and U layer visible):
\```
____________ B +---+---+---+
U /_Y_/_Y_/_Y_/| +---+---+---+ | O | R | O |
/_Y_/_Y_/_Y_/G| | O | R | O |\ +---+---+---+---+---+
/ Y / Y / Y /B/| +---+---+---+B\ | B | Y | Y | Y | G |
+---+---+---+G/B| R | R | R | R |\G\ +---+---+---+---+---+
| R | O | R |/B/| +---+---+---+G\B| | G | Y | Y | Y | B | R
+---+---+---+B/B| | R | R | R |\G\| +---+---+---+---+---+
| O | O | O |/B/ +---+---+---+G\G| L | B | Y | Y | Y | G |
+---+---+---+B/ \_W_\_W_\_W_\G\| +---+---+---+---+---+
| O | O | O |/ \_W_\_W_\_W_\G| | R | O | R |
+---+---+---+ D \_W_\_W_\_W_\| +---+---+---+
F F
\```
Apply (17) to solve the UFR corner. Rotate the entire cube in your hands until the solved corner is at UFL, then apply (17) again to solve the remaining yellow corners.
At the point, all the yellow corners are solved, the yellow face is solved, the top equatorial layer is solved, all the corners are solved, and the entire cube is solved. Congratulations!
\```
____________ B
U /_Y_/_Y_/_Y_/| +---+---+---+
/_Y_/_Y_/_Y_/B| | R | R | R |\
/ Y / Y / Y /B/| +---+---+---+G\
+---+---+---+B/B| R | R | R | R |\G\
| O | O | O |/B/| +---+---+---+G\G|
+---+---+---+B/B| | R | R | R |\G\|
| O | O | O |/B/ +---+---+---+G\G| L
+---+---+---+B/ \_W_\_W_\_W_\G\|
| O | O | O |/ \_W_\_W_\_W_\G|
+---+---+---+ D \_W_\_W_\_W_\|
F
\```
An example solve using this method can be found below:
=> gemini://jsreed5.org/twisty/nxnxn/3/layer-by-layer-example.gmi 3x3x3 Layer-by-layer Example Solve
---
=> gemini://jsreed5.org/twisty/nxnxn/index.gmi Up One Level
=> gemini://jsreed5.org/index.gmi Home
[Last updated: 2023-04-20]
---+
| O | O | R |/B/| +---+---+---+G\G| | G | Y | Y | Y | B | R
+---+---+---+B/B| | R | R | R |\G\| +---+---+---+---+---+
| O | O | O |/B/ +---+---+---+G\G| L | G | Y | Y | Y | G |
+---+---+---+B/ \_W_\_W_\_W_\G\| +---+---+---+---+---+
| O | O | O |/ \_W_\_W_\_W_\G| | O | O | R |
+---+---+---+ D \_W_\_W_\_W_\| +---+---+---+
F F
\```
> (18) R' D' R U2 R' D R U' R' D' R U' R' D R
In the cases that no corners are solved, either two pairs of adjacent corners will need to be swapped, or two pairs of opposite corners will need to be swapped.
In the first case, rotate the entire cube in your hands until the two pairs of corners to be swapped are UFR-UBR and UFL-UBL (UFR, DBL and U layer visible):
all corners swapped, case 1 (, then re-solve)
\```
____________ B +---+---+---+
U /_Y_/_Y_/_Y_/| +---+---+---+ | G | R | B |
/_Y_/_Y_/_Y_/O| | B | R | G |\ +---+---+---+---+---+
/ Y / Y / Y /B/| +---+---+---+O\ | O | Y | Y | Y | O |
+---+---+-
$
We can apply two settings to fix this. The more critical fix is to increase our input buffer length; the following command sets it to 1 MB.
set input buffer-length 1048576
The second setting prevents the buffer from looping on itself and overwriting part of the response. Gemini responses can be any size, and the response header does not include information on how big the body is--we must be prepared for anything. We do this by including the /nowrap option when invoking `input`. /nowrap tells `input` to stop and fail if the buffer fills, in which case we can retry the request with a larger buffer.
input /nowrap
Our script now looks like this:
$ cat kermit-gemini.ksc set input echo off set input buffer-length 1048576 set host jsreed5.org:1965 /ssl lineout gemini://jsreed5.org/twisty/nxnxn/3/layer-by-layer.gmi input /nowrap /nomatch 30 echo \v(input) close exit
The response printed by this script finally looks how we expect it to! We do still see the status code returned by the server, though. As an optional final step, we can modify our echo statement to remove the response header.
$ cat kermit-gemini.ksc set input echo off set input buffer-length 1048576 set host jsreed5.org:1965 /ssl lineout gemini://jsreed5.org/twisty/nxnxn/3/layer-by-layer.gmi input /nowrap /nomatch 30 echo \fltrim(\v(input),\fbreak(\v(input),\10)\10) close exit
---
Kermit scripts can be run directly from the shell using a "kerbang" statement, analogous to a "shebang" in shell scripts. They can even take arguments from the shell if the kerbang statement includes a plus sign (+) as its only argument.
I've written a short Kermit script that takes a Gemini URL as its primary argument, followed optionally by cert and key files to make authenticated requests. By setting a flag in the script, the user can choose to either print a Gemini document to the terminal or save it as a file.
#!/usr/bin/kermit +
set input echo off
set telnet binary
set input buffer-length 1048576 # max document size 1 MB, edit for larger
set flag on # on: echo document, off: save to file
if ( defined \%1 ) {
if ( equal \%1 \? ) {
echo "Usage: \v(cmdfile) GEMINI_URL [RSA_CERT RSA_KEY]"
exit
}
}
else {
echo "Usage: \v(cmdfile) GEMINI_URL [RSA_CERT RSA_KEY]"
exit
}
if ( not available ssl ) {
echo "SSL not supported"
exit
}
if ( defined \%2 ) set authorization tls rsa-cert-file \%2
if ( defined \%3 ) set authorization tls rsa-key-file \%3
if ( not equal \fsubstring(\%1,1,9) gemini:// ) assign \%u gemini://\%1
else assign \%u \%1
assign \%h \fsubstring(\%u,10,\flength(\%u))
if ( > \findex(/,\%h) 0 ) assign \%h \fbreak(\%h,/)
else assign \%u \%u/
if ( = \findex(:,\%h) 0 ) assign \%h \%h:1965
set host \%h /ssl
if ( failure ) {
echo "Could not connect to URL"
close
exit
}
lineout \%u
input /nowrap /nomatch 30
close
assign \%s \fleft({\v(input)},1)
if ( equal \%s 2 ) {
if ( flag ) goto gecho
else goto gsave
}
else echo \v(input)
clear device-and-input
close
exit
:gecho
echo \flop(\v(input),\10)
clear device-and-input
close
exit
:gsave
assign \%n \flopx(\%u,/)
if ( equal flast(\%n,1) / ) assign \%n index.gmi
file open /write \%f \%n
file write \%f \flop(\v(input),\10)
file flush \%f
file close \%f
clear device-and-input
close
exit
---
---
[Last updated: 2024-02-02]