💾 Archived View for going-flying.com › ~mernisse › 24.gmi captured on 2023-11-14 at 07:49:58. Gemini links have been rewritten to link to archived content
⬅️ Previous capture (2023-01-29)
-=-=-=-=-=-=-
In the previous three DNS posts I covered DNS from a fairly high level and then focused in on some key elements in BIND, the de-facto DNS server package. DNS is such an important part of inter-networking that it is bound to be a bit complex and given that complexity leads towards errors when humans are involved it is a pretty solid use-case for automation.
I mainly use Puppet to automate the state of my systems. I find it to be much more powerful than other tools and it allows me to specify a desired state instead of a set of tasks. It is also one of the most mature automation tools out there and is well supported on the platforms I use.
In general I automate three key pieces of my DNS infrastructure (other than making sure the required packages are installed and up-to-date, of course). 1) Zonefile generation 2) TLSA and DMARC records 3) DNSSEC key rotation and signing
Zonefile generation is easy, they are all generated from templates and concatenated together. This way I can specify records that may change in YAML using Hiera (Puppet's hierarchical key/value store), or resolve them via custom functions (as I do with the TLSA records for example) and have static blocks that are the same for all of my domains (like the NS and MX record sections).
My Puppet master runs various jobs that generate and rotate cryptographic keys. I wrote some custom plugins for LetsEncrypt certbot that lets Puppet manage the certificates and keys. Since they are available to Puppet I can generate the TLSA records that are used for DANE (a method to assure a client that the certificates they are getting from a TLS server are the correct ones). I provide the fingerprints used by the keys in the format the Apache (my webserver of choice) and BIND want using Facter (the Puppet host fact engine). The following Ruby code provides the custom facts to Facter.
# Generate the fingerprint of the SSL cert for things like
# Certificate pinning and TLSA records.
require 'fileutils'
require 'openssl'
Facter.add("ssl_pubkey_fingerprint") do
confine :kernel => "Linux"
setcode do
cert = "/etc/ssl/certs/ssl.ub3rgeek.net_cert.pem"
fingerprints = {}
if File.exist?(cert)
fd = File.read(cert)
s_cert = OpenSSL::X509::Certificate.new(fd)
p_key = s_cert.public_key().to_der()
dgst = OpenSSL::Digest::SHA256.new(p_key)
fingerprints['ssl.ub3rgeek.net'] = dgst.base64digest()
end
cert = "/etc/ssl/certs/going-flying.com_cert.pem"
if File.exist?(cert)
fd = File.read(cert)
s_cert = OpenSSL::X509::Certificate.new(fd)
p_key = s_cert.public_key().to_der()
dgst = OpenSSL::Digest::SHA256.new(p_key)
fingerprints['www.going-flying.com'] = dgst.base64digest()
end
fingerprints
end
end
Facter.add("ssl_pubkey_hexdigest") do
confine :kernel => "Linux"
setcode do
cert = "/etc/ssl/certs/ssl.ub3rgeek.net_cert.pem"
digests = {}
if File.exist?(cert)
fd = File.read(cert)
s_cert = OpenSSL::X509::Certificate.new(fd)
p_key = s_cert.public_key().to_der()
dgst = OpenSSL::Digest::SHA256.new(p_key)
digests['ssl.ub3rgeek.net'] = dgst.to_s()
end
cert = "/etc/ssl/certs/going-flying.com_cert.pem"
if File.exist?(cert)
fd = File.read(cert)
s_cert = OpenSSL::X509::Certificate.new(fd)
p_key = s_cert.public_key().to_der()
dgst = OpenSSL::Digest::SHA256.new(p_key)
digests['www.going-flying.com'] = dgst.to_s()
end
digests
end
end
Finally, the most complex automated task is DNSSEC. DNSSEC is different from DNS over TLS (or DNS over HTTPS). DNSSEC does not seek to provide any privacy, rather it provides a chain of trust back to the root of the global DNS system so your resolver can verify that the answer it is getting is in fact the correct answer. DNSSEC is a huge topic but in general it allows zone administrators to cryptographically sign the records in their zones and any zones that they delegate to, so in going-flying.com you can trace the signatures from going-flying, to com to '.' (the root of DNS) and know for sure that someone isn't spoofing the answer.
I have a script that is called by puppet any time the DNS configuration changes (which is at least monthly due to various cryptographic key rotations and things like LetsEncrypt).
#!/bin/sh
# dnssec-signzones (c) 2017-2018 Matthew J. Ernisse <matt@going-flying.com>
#
# Manage DNSSEC for zones, including rotating keys as needed.
set -e
PATH=$PATH:/sbin:/usr/sbin
# Paths to BIND files
KEYDIR="/etc/bind/keys"
STATEDIR="/var/tmp"
ZONEDIR="/var/cache/bind"
# Timeouts
DEACTIVATE_TIMEOUT="+3d"
DELETE_TIMEOUT="+5d"
PREPUBLISH_INTERVAL="+2d"
REKEY_AGE="30" # days
# delete any keys that are past their delete-by
delete_old_keys()
{
local dtime=0
local now=$(date +"%s")
for file in $(find ${KEYDIR} -name \*.key); do
if grep -q 'key-signing' "$file"; then
continue
fi
dtime=$(sed -ne 's/^; Delete:\s*[0-9]*\s*(\(.*\))/\1/p' \
"$file")
if [ -z "$dtime" ]; then
continue
fi
dtime=$(date -d "${dtime}" +"%s")
if [ "$dtime" -le "$now" ]; then
echo "Deleted expired key: $file"
rm -- "$file"
rm -- "$(echo "$file" | sed -e 's/\.key$/.private/')"
fi
done
}
find_domains()
{
for domain in $(ls -1 ${KEYDIR} | \
sed -ne 's/^K\([-\._a-z0-9]*\)\.\+.*/\1/p' | sort -u); do
echo $domain
done
}
# return the latest (currently used) DNSSEC keyfile
get_last_key()
{
if [ -z $1 ]; then
echo "usage: get_last_key domain"
return 127
fi
local domain fn m mtime=0
domain="$1"
for file in $(find ${KEYDIR} -name K${domain}\*.key); do
if grep -q 'key-signing' "$file"; then
continue
fi
keyage=$(sed -ne \
's/^; Activate:\s\{1,\}[0-9]\{1,\}\s\{1,\}(\(.*\))$/\1/p' \
"$file")
keyage=$(date -d "$keyage" +%s)
if [ -z "$keyage" ]; then
echo "get_last_key() failed to read activation time"
return 1
fi
if [ "$keyage" -gt "$mtime" ]; then
mtime="$keyage"
fn="$file"
fi
done
echo "$(basename $fn)"
}
need_new_key()
{
if [ -z $1 ]; then
echo "usage: need_new_key domain"
return 127
fi
local domain last_key keyage now
domain="$1"
now=$(date +%s)
last_key=$(get_last_key "$domain")
keyage=$(sed -ne \
's/^; Activate:\s\{1,\}[0-9]\{1,\}\s\{1,\}(\(.*\))$/\1/p' \
"$KEYDIR/$last_key")
keyage=$(date -d "$keyage" +%s)
if [ $(( $now - $keyage )) -ge $(( ${REKEY_AGE} * 86400 )) ]; then
echo "key ${last_key} needs rotating"
return 0
fi
return 1
}
rotate_keys()
{
if [ -z $1 ]; then
echo "usage: rotate_keys domain"
return 127
fi
domain="$1"
last_key=$(get_last_key "$domain")
# Logically this should go the other way, generate a new and then
# if successful expire the old but dnssec-keygen refuses to generate
# a successor key if there is no expire time on the current key..
echo "Setting expire times on ${last_key}"
if ! dnssec-settime -K "$KEYDIR" -I "$DEACTIVATE_TIMEOUT" \
-D "$DELETE_TIMEOUT" "$last_key"; then
exit 1
fi
echo "Generating new key..."
if ! dnssec-keygen -K "$KEYDIR" -S "$last_key" \
-i "$PREPUBLISH_INTERVAL" -q; then
exit 1
fi
}
sign_domain()
{
if [ -z $1 ]; then
echo "usage: sign_domain domain"
return 127
fi
domain="$1"
for view in "public" "private"; do
zonefile="${view}-${domain}.hosts"
if [ -f "$ZONEDIR/$zonefile" ]; then
sign_zone "$domain" "$zonefile"
fi
done
}
sign_zone()
{
if [ -z "$1" ] || [ -z "$2" ]; then
echo "usage: sign_zone domain zonefile"
return 127
fi
domain="$1"
zonefile="$2"
echo "signing $zonefile for $domain"
dnssec-signzone -S -K "$KEYDIR" -o "$domain" \
"$ZONEDIR/$zonefile"
}
# update the zone's serial -- this is no longer handled by puppet.
update_serial()
{
if [ -z $1 ]; then
echo "usage: update_serial domain"
return 127
fi
local new_serial old_rev old_rev_stripped old_serial=0
local zonefile
if [ -f "${STATEDIR}/${domain}.serial" ]; then
old_serial=$(< "${STATEDIR}/${domain}.serial")
old_rev=$(echo "$old_serial" | sed -ne \
's/^[0-9]\{,8\}\([0-9]\{,2\}\)$/\1/p')
old_serial=$(echo "$old_serial" | sed -ne \
's/^\([0-9]\{,8\}\).*/\1/p')
# Bash will think we're in octal if we feed it a number
# with a leading zero.
old_rev_stripped=$(echo "$old_rev" | sed -ne \
's/^0\([0-9]\)/\1/p')
if [ -n "$old_rev_stripped" ]; then
old_rev="$old_rev_stripped"
fi
fi
new_serial="$(date +%Y%m%d)"
if [ "$new_serial" -eq "$old_serial" ]; then
if [ $(( $old_rev + 1 )) -lt 10 ]; then
new_serial="${new_serial}0$(( $old_rev + 1 ))"
else
new_serial="${new_serial}$(( $old_rev + 1 ))"
fi
else
new_serial="${new_serial}00"
fi
echo "$new_serial" > "${STATEDIR}/${domain}.serial"
for view in "public" "private"; do
zonefile="${ZONEDIR}/${view}-${domain}.hosts"
sed -e "s/##SERIAL_NUMBER##/${new_serial}/" \
"${zonefile}.puppet" > "$zonefile"
done
}
for domain in $(find_domains); do
if need_new_key "$domain"; then
rotate_keys "$domain"
fi
delete_old_keys
update_serial "$domain"
sign_domain "$domain"
done
The script is complex because it needs to do several things but the for loop at the bottom tells the story of what is happening. The primary job of the script is to manage the DNSSEC keys, rotating them and expiring old ones, so since this script is run every time Puppet every time any of the DNS zonefiles are changed the first thing we do is to check to see if we actually need to rotate the keys. Once we do that we simply expire any old keys, update the serial numbers on the zones and then sign them. I can't imagine this script working verbatim for anyone else, it assumes a lot of things that are probably only true in my setup, but it should get anyone looking to automate DNSSEC with BIND well on their way.
DNS is one of the most important services provided on the Internet. It is used every time you type a name instead of an IP address. It is used for service discovery and configuration, e-mail SPAM detection, malware prevention and is the core method CDNs use to steer you to their closest edge location to optimize delivery of everything from JPEGs to critical software patches. Being comfortable with it gives a systems / network administrator tremendous control over network traffic and performance, it is an extremely powerful tool in the security administrator's portfolio, being able to short circuit command and control server connections and prevent malicious payloads from being downloaded, and any support people as trouble in DNS land is the root cause of so many end-user problems. I highly recommend _DNS and BIND_, by Cricket Liu and Paul Albitz, published by O'Reilly Media as further reading.
🚀 © MMXX-MMXXIII matt@going-flying.com