2019-11-09 Upgrading Debian from Stretch to Buster

I did it! I upgrade the server. No restart required. I was a bit intimidated by the extensive documentation. But I was fine.

the extensive documentation

First, I improved my backup. I currently use the following:

#!/bin/bash

# Using sudo rsync --archive to preserve ownership.
# Using --fake-super to avoid changes to groups and owners

echo Backing up Sibirocobombus
echo home directory
rsync --archive --fake-super --verbose --compress --delete --delete-excluded \
      --itemize-changes \
      --exclude '/home/alex/logs' \
      --exclude '/home/alex/alexschroeder.ch/share' \
      --exclude '/home/alex/planet/osr/' \
      --exclude '/home/alex/planet/indie/' \
      --exclude '.cpan/build' \
      --exclude '.cpan/sources' \
      --exclude '.cpanplus' \
      --exclude '.cpanm' \
      --exclude '.cache' \
      --exclude '.Trash' \
      --exclude '.local/share/Trash' \
      --exclude 'temp/' \
      --exclude 'pids/' \
      --exclude 'visitors.log' \
      --exclude 'referer/' \
      --exclude '.git/' \
      --rsh="ssh -p 882" \
      root@alexschroeder.ch:/home \
      /home/alex/Documents/Sibirocobombus

# https://www.debian.org/releases/buster/amd64/release-notes/ch-upgrading.en.html#data-backup
echo etc directory
rsync --archive --fake-super --verbose --compress --delete --delete-excluded \
      --itemize-changes \
      --rsh="ssh -p 882" \
      root@alexschroeder.ch:/etc \
      root@alexschroeder.ch:/var/lib/dpkg \
      root@alexschroeder.ch:/var/lib/apt/extended_states \
      root@alexschroeder.ch:/var/lib/dehydrated \
      root@alexschroeder.ch:/usr/lib/cgit/filters \
      /home/alex/Documents/Sibirocobombus

I also installed and ran `apt-forktracer`:

apt install apt-forktracer
apt-forktracer | sort > packages-not-from-debian

I also checked my system using `dpkg --audit`.

I ran everything inside `script -t 2>~/upgrade-busterstep.time -a ~/upgrade-busterstep.script`.

I was very afraid!

But eventually I just changed my `/etc/apt/sources.list` to the following:

deb http://deb.debian.org/debian buster main non-free contrib
deb http://security.debian.org/debian-security buster/updates main contrib non-free

# deb  http://ftp.debian.org/debian stretch main non-free contrib
# deb-src  http://ftp.debian.org/debian stretch main non-free contrib

# deb  http://ftp.debian.org/debian stretch-updates main non-free contrib
# deb-src  http://ftp.debian.org/debian stretch-updates main non-free contrib

# deb http://security.debian.org/ stretch/updates main non-free contrib
# deb-src http://security.debian.org/ stretch/updates main non-free contrib

I guess I should think about updates and security? Or perhaps they aren’t used anymore? I have no idea. I’ll look at this some other time.

Finally I ran the upgrade itself:

apt update
apt upgrade
apt dist-upgrade

I had two changes to system config files:

1. in `/etc/logrotate.conf` I changed `weekly` → `daily` (keeping 4 days worth of logs instead of 4 weeks)

2. in `/etc/ssh/sshd_config` I merged my changes with what upstream had provided

I am amazed to see that the server is still up. It didn’t even reboot. Let me sing the praise of Debian and all the things it includes and all the volunteers and maintainers and testers—thank you all. ❤️

I think I’m going to be fine. 😃

I noticed that I had to reinstall all my `pip3` stuff.

For the Mastodon bots:

pip3 install Mastodon.py cairosvg html2text

For Epicyon:

pip3 install requests commentjson beautifulsoup4 pycryptodome

My monitoring also broke down.

Monit stopped monitoring a bunch of stuff because the checksums failed.

Munin isn’t running at all...

Well, time to look at this tomorrow!

​#Debian ​#Administration

Comments

(Please contact me if you want to remove your comment.)

Cleaning up non-default config files... I installed `debsums` and use `debsums -ce` to find config files that I have changed.

Then I use a little script I got from StackExchange to find the diff.

from StackExchange

#!/bin/bash

# Usage: debdiffconf.sh FILE
# Produce on stdout diff of FILE against the first installed Debian package
# found that provides it.
# Returns the exit code of diff if everything worked, 3 or 4 otherwise.

command -v apt-get >/dev/null 2>&1 || {
  echo "apt-get not found, this is probably not a Debian system. Aborting." >&2;
  exit 4; }
command -v apt-file >/dev/null 2>&1 || {
  echo "Please install apt-file: sudo apt-get install apt-file. Aborting." >&2;
  exit 4; }

FILE=$(readlink -f "$1")
while read PACKAGE
do
  # verify from first installed package
  if dpkg-query -W --showformat='${Status}\n' | grep installed > /dev/null
  then
    DIR=$(mktemp -d)
    cd "$DIR"
    echo "Trying $PACKAGE..." >&2
    apt-get download "$PACKAGE" >&2
    # downloaded archive is the only file present...
    ARCHIVE=$(ls)
    mkdir contents
    # extract entire archive
    dpkg-deb -x "$ARCHIVE" contents/ >&2
    if [ -f "contents$FILE" ]
    then
      # package contained required file
      diff "contents$FILE" "$FILE"
      RET=$?
      # cleanup
      cd
      rm -Rf "$DIR"
      # exit entire script as this is the main shell
      # with the return code from diff
      exit $RET
    else
      # cleanup
      cd
      rm -Rf "$DIR"
    fi
  fi
done < <(apt-file -l search "$FILE")
# if we are here, it means we have found no suitable package
echo "Could not find original package for $FILE" >&2
exit 3

And if I think the changes I made could be reverted, I run `patch --reverse $FILE` on the file and paste the diff seen above to revert it.

– Alex Schroeder 2019-11-10 11:38 UTC

---

The tag(Munin) problem I had seems to due to changes I made to the Apache configuration. Apparently I created a special account with password to allow access to Munin from remote systems. This makes sense since I’m running Munin on the server...


--- apache24.conf	2019-11-10 12:47:00.533056622 +0100

  Alias /munin/static/ /var/cache/munin/www/static/

  <Directory /var/cache/munin/www>
!     Require local
      Options None
  </Directory>

  <Directory /usr/lib/munin/cgi>
!     Require local
      <IfModule mod_fcgid.c>
          SetHandler fcgid-script
      </IfModule>
--- 14,36 ----
  Alias /munin/static/ /var/cache/munin/www/static/

  <Directory /var/cache/munin/www>
!     Order allow,deny
!     Allow from all
      Options None
+     AuthUserFile /etc/munin/munin-htpasswd
+     AuthName "Munin"
+     AuthType Basic
+     require valid-user
  </Directory>

  <Directory /usr/lib/munin/cgi>
!     Order allow,deny
!     Allow from all
!     Options None
!     AuthUserFile /etc/munin/munin-htpasswd
!     AuthName "Munin"
!     AuthType Basic
!     require valid-user
      <IfModule mod_fcgid.c>
          SetHandler fcgid-script
      </IfModule>

– Alex Schroeder 2019-11-10 11:51 UTC