💾 Archived View for wilw.capsule.town › log › 2021-11-27-restic-backups.gmi captured on 2023-07-10 at 13:45:24. Gemini links have been rewritten to link to archived content
⬅️ Previous capture (2023-04-19)
-=-=-=-=-=-=-
A while ago I posted about [1] how I back-up my personal servers to Backblaze B2 [2]. That approach involved adding all of the files and directories into a single compressed archive and sending this up to an encrypted bucket on B2.
Whilst this does achieve my back-up goals (and I use Telegram to notify me each time it completes [3]), it felt _inelegant_. Every time the back-up executed, the entirety of the back-up file - several gigabytes - would be built and transferred.
In this post I will talk about my new approach, which uses a tool called Restic.
Restic [4] is an open-source cross-platform program to back-up files and directories. It works on the basis of snapshots and repositories [5].
A single repository can contain multiple "snapshots" representing various back-ups. Each snapshot is timestamped and can be filtered by the path of the back-up and the host that the back-up came from.
One of the most useful features of Restic is that each snapshot only stores the representation of what _changed_ since the last snapshot. This means that although the first back-up might take a little while, subsequent snapshots are much quicker.
Repositories can be local (i.e. on a filesystem accessible by the host), or remote. Restic supports a number of remote repository types [6] - from SFTP and RESTful interfaces through to S3-compatible backends and storage services operated by cloud providers.
Restic also supports Backblaze B2 out of the box, which is the repository type that I use.
In the rest of this post I will run through how this can be set-up and automated.
First, we need to install Restic. There are instructions for installation on a range of systems on the Restic website [7].
Next, in the Backblaze B2 console, create a new bucket. The bucket will need a unique name. Whilst there, create a new "App Key", with read/write access to the bucket. Make a note of the `keyID` and `applicationKey` once created.
You can close the B2 console. You now need a file that can be `source`d to tell Restic how to access your repository. Create a new file (e.g. in the home directory of the host you want to back-up), called `resticenv`, with these contents:
export B2_ACCOUNT_ID="<KEY_ID>" export B2_ACCOUNT_KEY="<APPLICATION_KEY>" export RESTIC_REPOSITORY="b2:<BUCKET_NAME>" export RESTIC_PASSWORD="<PASSWORD>"
You'll need to replace some of the parts of the file as described below:
Each Restic repository is secured with a _password_. This is used to encrypt the repository, so if this is lost then so is the associated data. I use my password manager to generate a strong password. It goes without saying that this `resticenv` file should be kept very safe.
Finally, we need to initialise the repository. To do so, first `source` the environment file, and then use the `init` command:
$ source ~/resticenv $ restic init
You're now ready to create back-ups.
Creating a new back-up is as easy as running the following (as long as your shell still has the `source`d environment as described above):
$ restic backup /path/to/directory
If the target directory is particularly large then the first back-up may take a while. However, subsequent back-ups will be much quicker.
Once done, you can run the following to view a list of all the snapshots held by the repository. You'll see your recently-created back-up listed:
$ restic snapshots ID Time Host Tags Paths ----------------------------------------------------------------------- 6b25863c 2021-11-27 08:39:44 myhost /path/to/directory ----------------------------------------------------------------------- 1 snapshot
The next time you run `restic backup` again, you'll notice new snapshots are created.
In some scenarios you may not want to keep back-ups around forever. For example, if you provide a service to users you may be in breach of your own Privacy Policy if you keep backed-up data of old deleted accounts.
Luckily it's straight forward to "forget" old snapshots. The command below tells Restic to hold onto the last 24 hours' worth of hourly snapshots and the last 7 days' worth of daily snapshots:
$ restic forget --prune --keep-hourly 24 --keep-daily 7
Back-ups are useless unless you can later recover them! Rustic makes this easy. You just need the `ID` of the back-up you want to recover (as provided by the `restic snapshots` command from earlier):
$ restic restore 6b25863c --target ~/restored
The `~/restored` directory should now contain the files included in the back-up at that snapshot's point in time.
Finally we can cover a way to automate back-ups. For this we can simply use `cron`. To begin, install a suitable cron service (if your system doesn't come with one). I use cronie [8].
Once ready, add an entry to your user's crontab by running `crontab -e` and adding the following line:
15 * * * * . /home/<USER>/resticenv; /usr/bin/restic backup -q /path/to/directory
The above job will run at quarter-past the hour for every hour the system is running. Note that cron requires that we include full paths to files. The `-q` flag suppresses output. You'll need to change parts of the line to suit your needs (run `which restic` to see where Restic is installed on your system).
I have much more confidence in a tool like Restic to handle my back-ups than a home-grown solution. It's great to have the peace of mind!