💾 Archived View for gemini.patatas.ca › posts › disk-usage-cron.gmi captured on 2024-05-10 at 10:52:12. Gemini links have been rewritten to link to archived content

-=-=-=-=-=-=-

2023-08-13

Reducing Mastodon's storage needs with an automated script

There's a nice example of a shell script for file cleanup here:

https://ricard.dev/improving-mastodons-disk-usage/

I ended up adjusting some of it, including adding a couple commands to log the time & disk usage, both at the beginning and at the end of the script. These commands may need adjusting for your system; to find your disk's filesystem type, enter the following command:

df -T

and replace 'ext4' in the following script with your disk's filesystem type if necessary. Here's my final script, which I saved as 'home/mastodon/purge-media.sh':

#!/bin/bash
# this script can be found at https://website.patatas.ca/pages/disk-usage-cron.html
# and is based on https://ricard.dev/improving-mastodons-disk-usage/

echo "before" >> /home/mastodon/purgelog.txt
/bin/date >> /home/mastodon/purgelog.txt
/bin/df -t ext4 >> /home/mastodon/purgelog.txt

#prune accounts
RAILS_ENV=production /home/mastodon/live/bin/tootctl accounts prune;

# Remove media attachments older than 4 days
RAILS_ENV=production /home/mastodon/live/bin/tootctl media remove --days 4;

# Remove remote statuses not interacted with
RAILS_ENV=production /home/mastodon/live/bin/tootctl statuses remove --days 14;

# Remove headers of remote accounts not followed by anyone on your server
RAILS_ENV=production /home/mastodon/live/bin/tootctl media remove --remove-headers --days 3;

# Remove link previews older than 14 days
RAILS_ENV=production /home/mastodon/live/bin/tootctl preview_cards remove --days 14;

# Remove files not linked to any post
RAILS_ENV=production /home/mastodon/live/bin/tootctl media remove-orphans;

echo "after" >> /home/mastodon/purgelog.txt
/bin/date >> /home/mastodon/purgelog.txt
/bin/df -t ext4 >> /home/mastodon/purgelog.txt

But as a novice admin, I had a lot of trouble getting this script to run properly with cron on Debian.

(If it works for you right off the bat, great! But if it doesn't, here's how I fixed it.)

The issue: when a shell script is run manually as a logged-in user, the system has already loaded the user environment. But the shell that cron runs in doesn't do this - you have to manually specify the environment variables. In my case, cron was trying to use an older version of Ruby - so I needed to figure out a way to let cron know the correct version and where to find it.

I tried a lot of different things, before finally finding some discussion of this command:

source ~/.bashrc

which loads the user's environment variables in a non-interactive, non-login shell, which is what cron runs from.

You may also need to specify the location of the bash shell to use:

SHELL=bin/bash

You'll probably want to log the output in order to help debug any issues, so add

>> /home/mastodon/purge.log 2>&1

to write it to a file.

Here's my crontab:

SHELL=/bin/bash

#cleanup runs every day at 03:45
45 3 * * * source ~/.bashrc && bash /home/mastodon/purge-media.sh >> /home/mastodon/purge.log 2>&1

Adjust as necessary, and let me know if this helped, or if you run into any issues. Find me on Mastodon and the web:

https://mstdn.patatas.ca/@smallpatatas/

https://thedabbler.patatas.ca/

or email me at smallpatatas (at) patatas.ca