💾 Archived View for gemini.ctrl-c.club › ~tjp › c › sr-71 captured on 2023-12-28 at 16:04:03. Gemini links have been rewritten to link to archived content
⬅️ Previous capture (2023-11-04)
-=-=-=-=-=-=-
,
dMb,
,dMMMMb, ,,
,dMMMMMMMMMb, eeee8888"
,mMMm!!!!XXXXMMMMM"""
,d!!XXMMXX88888888W"
`MX88dMM8888WWWMMMMMMb,
'""MMMMMMMMMMMMMMMMb
MMMMMMMMMMMMMMMMMMb,
dMMMMMMMMMMMMMMMMMMMMb,,
_,dMMMMMMMMMMXXXX!!!!!!!!!!!!!!XXXXXMP
_,dMMXX!!!!!!!!!!!!!!!!!!XXXXX888888888WWC
_,dMMX!!!MMMM!!!!!!!!XXXXXX888888888888WWMMMMMb,
dMMX!!!!!MMM!XXXXXX88888888888888888WWMMMMMMMMMMMb
dMMXXXXXX8MMMM88888888888888888WWWMMMMMMMMMMMMMMMMMb ,d8
MMMMWW888888MMMMM8888888WWMMMMMMMMMMMMMMMMMMMMMMMMMMM,d88P'
YMMMMMWW888888WWMMMMMMMMMMP"""' `"YMMMMMMMMMMMXMMMMMP
`""YMMMMMMMMMMMMMP""' mMMMm!XXXXX8888888e,
,d!!XXMM888888888888WW
"MX88dMM888888WWWMMMMMMb
"""``'"YMMMMMMMYMMM
`"YMMMMM
`"YMP
The last airplane designed without the use of electronic calculators, the SR-71 was developed from concept to first flight in just over 2 years. It flew higher and faster than any jet of its time (holding the speed record to this day), and pioneered low-observability (stealth) characteristics.
This sr-71 project is a small web server, able to host your pages on the gemini, gopher, and finger protocols.
sr-71 is a single-package go project, so using the go toolchain installing is as simple as:
go install tildegit.org/tjp/sr-71@latest
Currently binaries are not made available for download, so the go command is a requirement.
sr-71 requires a configuration file, the path of which is provided as the only command-line argument.
sr-71 /path/to/sr71.conf
The format of the config file is designed to be straightforward, and human-readable and -writable. Any line-leading whitespace is always ignored, and after that is stripped off then any line beginning with a hash character (#) is treated as a comment and ignored.
Here's a simple configuration that will get you off the ground:
systemuser nobody
gemini {
host myhostname.com, gemini.myhostname.com
servertls key /path/to/tls/keyfile cert /path/to/tls/certfile
static /var/gemini at / with dirdefault index.gmi, dirlist
}
gopher {
host myhostname.com
static /var/gopher at / with dirdefault gophermap, dirlist, extendedgophermap
}
finger {
static ~/.finger
}
With the above configuration, sr-71 will:
The configuration file is built out of: servers, directives, and modifiers. Servers specify a protocol to be hosted and where to host it, and directives inside the server definition fill out the details. Directives can be provided inside of servers and also globally, and modifiers are applied to certain directives to further customize their behavior.
A server is structured as a block of configuration, starting with the protocol name and optionally the IP:port on which to host.
<protocol> [<IP>][:<port>] {
# directives...
}
# for example:
gopher 1.2.3.4:7070 {
# directives...
}
# or:
gemini {
# directives...
}
# or:
finger :7979 {
# directives...
}
The protocol must be supplied, but the IP and colon-prefixed port are both optional. By default "0.0.0.0" will be used as the IP (meaning it will serve on all available interfaces) and the port will be the default for the protocol.
Just for simplicity of the configuration parser, the opening brace ({) must be at the end of the same line as the protocol name, and the closing brace (}) must be on a line of its own.
Gopher servers are pretty straightforward. The default port is 70, and the only special thing you have to get right inside a "gopher" block is that the server must know *where* it is, so it needs a single "host" directive with exactly one hostname. It supports all other directives.
Finger is a little more constrained as a protocol than the other two here, because requests pretty much just point to a user name. So a "finger {...}" block must contain a single "static" or "cgi" directive, which must have a tilde (~) placeholder in the file system path and is lacking an "at <url path>" clause. See the sections below on the "static" and "cgi" directives for more details.
Finger's default port is 79.
Gemini's default port is 1965.
Because of the TLS requirement, a gemini server must contain a "servertls" directive.
sr-71 also supports virtualhosts in gemini, where multiple domains can be hosted on the same IP/port. This is done by having multiple "gemini {...}" blocks with the same IP and port (potentially defaults), but with "host" directives differentiating them. More on this in the section below on virtualhosting.
Directives really come in two flavors: global directives and server directives. In either case, a directive is always contained on a single line, which begins (after any leading whitespace) with the type of the directive. What follows the directive type depends on that type.
A config file may have one "systemuser" directive. In this case the type is followed by just a single word which is the user (or numeric user ID) the server should run as.
When there is a systemuser directive, sr-71 will drop privileges to the named user after doing a few things:
If sr-71 is not started as root but has a "systemuser" directive, it will fail at startup with an error message.
There may be a single "loglevel" directive, in which the word "loglevel" is followed by one of the levels: "debug", "info", "warn", or "error". Logs sent to stdout will then be filtered to only include the specified level and above.
Having no "loglevel" directive is the same as "loglevel debug", which allows everything through.
The global "auth" directive sets up an authentication mechanism which can then be used in server directives.
The first word following "auth" is a name which can later be used to reference the auth mechanism.
Next is a token specifying how the auth will be accomplished, which can be any of:
auth is_named hasclienttls auth is_tony clienttls 0284bcb38d7c98548df4a67587163276373ea8f9a8cc931a89f475557bd9f3a3 auth dev_team clienttls 95927842cde8d9bfe121602db7178f7d5d9b0d73ad6a815a6631f0af2d6c4ebf, f8f5c6e9e7f8e19852247648114d99e7ae682454d5f093afc695da58d9b2dabf auth user_private_gemini clienttlsfile ~/.private_gemini
The "host" directive tells a server what hostname(s) it is serving. It is followed by one or more comma-separated hostnames.
A gopher server *must* contain a single host directive with just one hostname - this is to enable it to generate local links on generated gopher menus.
In gemini servers, host directives control virtual hosting behavior.
"servertls" provides servers with the paths to their TLS server credentials. It is followed by two clauses: "key <path to key file>", and "cert <path to cert file>". Both clauses are required (if a single file contains both then use that path for both clauses).
Gemini servers must always host with TLS and so require a "servertls" directive. In gopher and finger, the presence of a "servertls" directive will cause them to host their content tls-encrypted.
The "static" directive causes the contents of a directory to be served directly. "static" directives only allow world-readable files to be read.
"static" is immediately followed by the file system path to host, then an "at" clause is usually required which provides the URL prefix to host (use "/" to host at the domain root), an optional "with" clause introduces modifiers, and finally an optional "auth" clause with an auth name defined elsewhere in a global "auth" directive.
The file system path and url prefix support user-custom path expansion as well.
It ends up looking like:
gemini {
#static <filesystem path to documents> [at <url path prefix>] [with <modifiers>] [auth <name>]
# for example:
static /var/gemini at / with dirdefault index.gmi
static /var/gemini/private at /private auth private_gemini
static /my/project/documentation at /docs with dirlist
}
Finger servers are a little special, in that the file system path *must* use a tilde (~) for user-custom path expansion, and it's the only context which does not support the "at" clause.
"cgi" directives are similar to "static" in that they define a filesystem directory to be hosted at a URL path. However "cgi" directives cause any world-readable and world-executable files to be *run* instead, then stdout to be served as the default type for the protocol. There is more detail on this in the section "Running CGIs".
CGI directives don't support any modifiers except "extendedgophermap", and even that only in gopher servers. Find more on the "extendedgophermap" modifier below in the section "Extended Gophermap Parsing".
Otherwise the structure is the same as "static":
gemini {
cgi /var/gemini/cgis at /cgi-bin
}
gopher {
cgi ~/public_gopher/cgi-bin at /~/cgi-bin with extendedgophermap
}
"git" directives point to a filesystem directory which contains git repositories and serves up git repo viewers at a given URL prefix. As another routing directive, it is structured similarly to the "static" and "cgi" directives.
gopher {
git /var/repos at /code
}
gemini {
git ~/code at /~/code
}
"git" is not supported in finger servers, but otherwise it builds appropriate views according to the protocol of the server it is under (gemtext on gemini, gopher menu on gopher).
The only supported modifier is "templates" - find more details on that in the section on "Git Viewing Templates".
The routing directives "static", "cgi", and "git" support modifiers in the "with" clause. These are comma-separated, and each modifier begins with the modifier type and then may be followed by type-specific additional information.
"dirdefault" is followed by a file name, and it customizes the behavior of requests for directory paths. If the path exists and "dirdefault" is given, it will look for the provided file name within the requested directory and serve that *as the directory itself*. Think "index.html" in web servers.
Allowed contexts: static directive (neither cgi nor git), gemini or gopher servers (no finger).
"dirlist" takes no additional parameter and causes a protocol-appropriate listing to be built for directory requests.
If both "dirdefault" and "dirlist" are in use then "dirdefault" will take precedence and the listing will only be built if the dirdefault filename doesn't exist.
Allowed contexts: static directive (neither cgi nor git), gemini or gopher servers (no finger).
"exec" allows static directives to also run CGIs. The major difference between "static...with exec" and "cgi" is that the former will also serve file contents for non-executable files, and can support more additional modifiers such as dirdefault and dirlist.
There is more detail in the section "Running CGIs" below.
Allowed contexts: static directive (neither cgi nor git), gemini/gopher/finger servers.
"cmd" overrides the CGI behavior of either "static...with exec" or "cgi" directives. It causes the named file to be run as the executable in place of the located executable file.
Importantly, in all other ways it will still run as the located file:
So it can, for instance, be a good opportunity for a system administrator to impose boundaries on user CGIs in a shared hosting environment. The "cmd" script can set a nice level, increment a semaphore potentially waiting for a slot, set system resource limitations, chroot, and finally "exec ./$(basename $SCRIPT_NAME)".
Allowed contexts: "static...with exec", cgi (no git), gemini/gopher/finger servers.
"extendedgophermap" enables lots of additional flexibility in writing the gopher menu format. The ideas are mostly borrowed from gophernicus, and sr-71's implementation is documented in more detail below in "Extended Gophermap Parsing".
Allowed contexts: static, cgi directives (no git), gopher servers (neither gemini nor finger).
The "autoatom" modifier customizes routing to recognize "<any other valid path>.atom", and parses any gemtext response, transforming it into atom XML according to the gemini companion specification.
"Subscribing to Gemini pages" gemini companion specification
Allowed contexts: static, cgi directives (no git), gemini servers (neither gopher nor finger).
The "titan" modifier takes an auth name (defined in a global "auth" directive) and enables the titan file upload protocol in a static route. Titan requests specifically will have to pass the named auth mechanism.
Allowed contexts: static (neither cgi nor git), gemini servers (neither gopher nor finger).
"templates" is a "git" directive modifier which allows custom templates (golang's text/template style) to control the rendering of the various pages in the git repo viewer.
The supported template names and their execution contexts are documented below in the section on "Git Viewing Templates".
Allowed contexts: git (neither static nor cgi), gemini or gopher servers (no finger).
The request handlers for "git" directives are built around rendering templates. There are default templates built-in to sr-71, but they can also be overridden with the "templates" modifier.
All templates are parsed and rendered with the golang standard library "text/template" package.
Documentation on the "text/template" format
The directory path given to "templates" should contain template files which define any of the supported template names. There are differences in the templates depending on the protocol in use though.
These are the types you will encounter as context objects in rendered templates.
The Repository object at ".Repo" has methods:
ObjectDescription is a representation of a blob or tree:
The routes defined by the gopher git router are:
The standard object for rendering gopher templates has 6 fields:
All templates use the above object as their rendering context except repo_root.gph (which doesn't have a repository selected). repo_root.gph is rendered with a list of the repository names found in the directory.
Gopher templates also have three additional functions defined:
The gemini git router defines these routes:
The standard object for gemini templates has:
This object is used for all templates except repo_root.gmi, which instead gets a list of the repository name strings.
sr-71 can run child processes in a CGI environment to respond to requests. It can do this as a result of a "cgi" directive, or "static...with exec". While parsing extended gophermap, it can also execute other files included with an "=" line (but even then, it will only execute the file if it is under "cgi" or "static...with exec"). Also, it only ever executes files which are both world-readable and world-executable.
When it runs a CGI process, sr-71 sets up a standard CGI environment for it, trying to follow RFC 3875 as closely as possible.
Standard in of the CGI process is set to the request body, if there is one (this is only the case for titan requests). Standard out of the CGI process is used as the response body, and the default format for the protocol is assumed (gemtext for gemini, gopher menu for gopher, text/no format for finger). "extendedgophermap" is usable in conjunection with "cgi" or "static...with exec" in gopher servers, in which case the standard output will be processed with the gophermap extensions.
The "cmd" modifier can be used on cgi and "static...with exec" directives, in which case the given world-executable file will be used in place of the resolved CGI program, however the environment will still be set up entirely as if the executable pointed at by the request is being run. This means the working directory is that of the located program (not the cmd override), and SCRIPT_NAME and PATH_INFO are set as if that program was being run. This means that a no-op cmd override could just "exec ./$(basename $SCRIPT_NAME)".
The cmd override can be a way to limit system resources, chroot, or otherwise impose a security sandbox around the CGI program.
Shared-hosting environments can benefit from utilizing /~username paths for the various users on a host. sr-71 has a compact solution for this situation. In routing directives (static, cgi, git) both the file system path and URL path may include the tilde (~) character. In the URL path it will match a "~username" request path segment and capture the user name. If the file system path begins with a ~ character it will be replaced with the requested user's home directory, otherwise it will simply be replaced by the username.
This captured username and file system path handling is usable in a few other contexts as well, such as "clienttlsfile" auth.
# The "user_private" auth lets users on the system define their own allowed users
auth user_private clienttlsfile /var/gemini/users/~/private/allowed_users
gemini {
# The "at /~" will route any paths beginning with "/~<username>/..." to /var/gemini/users/<username>/...
static /var/gemini/users/~ at /~ with dirdefault index.gmi, dirlist, autoatom
# Here we're overriding the "private" subdirectory to let users set their own allowed list of client TLS certs.
# Paths beginning with "/~<username>/private/..." will route to "/var/gemini/users/<username>/private/... but
# will ONLY be allowed through if their client TLS cert hash is in /var/gemini/users/<username>/private/allowed_users
static /var/gemini/users/~/private at /~/private with dirdefault index.gmi, dirlist auth user_private
}
gopher {
# Here in gopher we're using the users' $HOME directories instead (because the fs path *begins* with ~).
# Paths beginning with "/~<username>/..." will be routed to "<$HOME for username>/public_gopher/..."
static ~/public_gopher at /~ with dirdefault gophermap, dirlist, extendedgophermap
}
Gopher and Finger have no guarantee that a domain name will appear in a request. Therefore virtualhosting by domain doesn't make sense in these protocols. Multiple gopher or finger servers may still be defined in a configuration file, but they will have to appear on separate IPs and/or ports.
With Gemini, however, there can also be multiple servers defined on the same IP and port (perhaps defaults), which can be differentiated at request-time based on the requested domain and "host" directives in each server.
gemini {
# default 0.0.0.0:1965
servertls key /etc/ssl/mydomain.pem cert /etc/ssl/mydomain.pem
# only requests for "mydomain.com" will be served by the logic in this server
host mydomain.com
static /var/gemini at / with dirdefault index.gmi, dirlist, autoatom
}
gemini {
# ALSO default 0.0.0.0:1965
# "code.mydomain.com" does the git hosting defined here
host code.mydomain.com
git /var/repos at /
}
Currently a gemini server on a single IP:port can only use a single set of server TLS credentials, so the "servertls" directive is only required on one of the servers and will be used for all of them. That's why the second "gemini" server in the example above has no "servertls" directive. This is the only situation in which servertls is not required in a gemini server (it's already provided in another one with the same IP:port).
sr-71 issue to use SNI to enable separate TLS certs by domain
sr-71's gopher server supports extensions to the gophermap format to make it easier to write, and to provide flexibility to CGIs. The ideas here are mostly borrowed from the Gophernicus server. These aren't on by default but can be enabled with the "extendedgophermap" modifier for static and cgi directives.
documentation on Gophernicus's gophermap extensions
Any lines in extended gophermap which contain NO tab characters (normally exactly 3 tabs are required on all lines) will be converted to an info-message line (type 'i'), with the current document's selector, and the current server's hostname and port. The end result is that such lines are rendered as-is in gopher clients.