💾 Archived View for thrig.me › tech › rsync-clean-shell.gmi captured on 2023-07-22 at 17:39:24. Gemini links have been rewritten to link to archived content

-=-=-=-=-=-=-

... is your shell clean?

rsync over SSH may sometimes fail with a statement about the cleanliness of your shell, usually due to extraneous bytes leaking into the SSH session, perhaps from your shell rc printing some message. Generally this is a bad thing, especially if you actually do want to use rsync, or do not want to see the login spam each time you login.

Note that "rc" can mean various things. If someone speaks of "the shell rc" they probably mean files such as ~/.profile or /etc/ksh.kshrc or too many other such files sometimes read at startup. They could also mean "the shell, rc" which could mean the plan9 shell, or a clone thereof. "rc" here is short for "run command" which is pretty much what a "shell rc" or "the shell, rc" do with the contents of files. rc(8) could also describe an init system, traditionally comprised of shell scripts.

	$ pkglocate bin/rc | grep plan9
	plan9port-20210323:plan9/plan9port:/usr/local/plan9/bin/rc
	rc-1.7.4p1-editline:plan9/rc,editline:/usr/local/bin/rc
	rc-1.7.4p1-readline:plan9/rc,readline:/usr/local/bin/rc
	rc-1.7.4p1:plan9/rc:/usr/local/bin/rc

Anyways, the root cause of a connection being unclean according to rsync is that the shell itself has issued some message, or that a configuration file for the shell has caused some message to be printed. It may be good to show how to create both of these conditions.

A test account or a test virt could be good, unless you like locking yourself out of your system when you screw things up.

Bad Shell

Portability varies here; OpenBSD has a nologin(8) shell and other systems may have similar; otherwise, it is not difficult to write something that prints a message and then exits, which is pretty much what /usr/src/sbin/nologin/nologin.c on OpenBSD does.

    $ grep ueb /etc/passwd
    ueb:*:1001:1001:Web User:/home/ueb:/sbin/nologin
    $ rsync -a ueb@localhost:/var/empty/ .
    ueb@127.0.0.1's password:
    protocol version mismatch -- is your shell clean?
    (see the rsync manpage for an explanation)
    rsync error: protocol incompatibility (code 2) at compat.c(610) [Receiver=3.2.5]

This is due to nologin printing a message that rsync does not like:

    $ nologin
    This account is currently not available.
    $ echo $?
    1

Note that some vendors put files into their /var/empty directory. No, I do not know why they do this, given the purpose of having an empty directory. In hindsight, you may want the -n flag on rsync so that it does nothing, even on systems where the vendor has made /var/empty not empty. Also note that rsync does different things depending on whether the source directory ends with a / or not.

Bad Shell Configuration

This can be more difficult to reproduce, as a simple test shows no error.

    $ echo echo echo >> ~/.profile
    $ rsync -na localhost:/var/empty/ .
    $

There is a message printed by the shell at login, but not for one where a command is supplied.

    $ ssh localhost
    ...
    echo
    gear$ exit
    $ ssh localhost exit
    $

However, the error can happen:

    $ rsync -na ueb@localhost:/var/empty/ .
    ueb@127.0.0.1's password:
    protocol version mismatch -- is your shell clean?
    (see the rsync manpage for an explanation)
    rsync error: protocol incompatibility (code 2) at compat.c(610) [Receiver=3.2.5]

What I did here was to switch the shell for the ueb user to /usr/local/bin/zsh, and to print a message from the ~/.zshenv file, which is more likely to be run than a file like ~/.zprofile or ~/.zshrc is. This will vary by shell, and whether the shell is a login shell, or an interactive shell. Regardless, you probably do not want a ~/.zshenv or equivalent creating messages that then break rsync.

    $ grep ueb /etc/passwd
    ueb:*:1001:1001:Web User:/home/ueb:/usr/local/bin/zsh
    $ cat ~ueb/.zshenv
    print zshenv calling

SSH Banner

This does not break rsync; the message happens before rsync is involved with the stream. However, banner output may be captured by a PTY that ssh and rsync are run in, such as one created by expect(1) or similar.

    $ doas grep Banner /etc/ssh/sshd_config
    Banner /etc/noise
    $ cat /etc/noise
    a banner message
    $ rsync -na localhost:/var/empty/ . > x
    a banner message
    $ cat x
    $ rm x

SFTP and Tunnels

sftp does not involve a shell because it is implemented within SSH. sftp thus does not involve a shell and an exec as rsync or git need. However, rsync or git do not really work over SFTP. Other methods would be to forward a port via SSH and tunnel rsync or git over that, or to use wireguard to create a tunnel.

tags #rsync #sh