💾 Archived View for senders.io › gemlog › 2021-04-17-capsule-log-retention.gmi captured on 2024-03-21 at 15:27:37. Gemini links have been rewritten to link to archived content
⬅️ Previous capture (2021-12-04)
-=-=-=-=-=-=-
I wrote previously that I had setup stats based on my servers access logs. I noted at the bottom that I was clearing my logs to keep somewhat compliant with user privacy.
I wrote a daily cronjob to remove any log lines older than 7 days from my access.log
#!/usr/bin/env bash
set -e
LOGFILE=$1
mindate=$(head -n1 $LOGFILE | cut -f1 | cut -d'T' -f1)
maxdate=$(date --date="-6 days" -u -Id)
echo "Deleting log lines from ${mindate} to ${maxdate}"
sed -i -E "/${mindate}/,/${maxdate}/d" $LOGFILE
echo "Cleared logs"
I had no idea you could do relative dates! I am not surprised bur it was nice to find out since it meant I could do this entirely in bash.
I essentially take two dates: the first date in the file, and the inclusive date I want to remove logs for. Then using the magic of sed, remove all the lines that match between those dates.
I do a lot of bash scripting for """productivity""". Sed is an amazing utility that I find allows me to do so much without needing to expand into other languages like awk or python.
The only "personal data" my access logs have are IPs. I am kicking around a good way to so I can still debug and trace requests, but for now keeping a week of logs should be compliant and hopefully personally reassuring.
Honestly, I find myself writing small script files like this (though this is basically just a sed command) to do day-to-day tasks. For instance my 'todo' app for work, to track asks, essentially just echos the args into a unique file in a directory. Then you can print the file contents to see whats open. Then using their UID to mark as done.
So I was happy to be able to quickly write this up to clear old logs. Do you do log rotation? Or just delete the old files?