A hard DNS problem

While I'm in a testing mood [1], I came across this post:

The job was for a position involving the day-to-day operation of DNS (Domain Name Service) servers and firewalls involving DNS and client systems involving DHCP (Dynamic Host Configuration Protocol) and DNS systems, and was a senior role so the ideal candidate would probably be able to say a little more than "magic" as to how DNS works.
A hard DNS problem: the owner of the zonefile added a new record that you requested. The serial number of the zone did increment (and there is no funny invalid wrap-around of the serial number going on (bonus points for knowing that)). The new record is not visible. What went wrong? This is probably too hard a question for a job interview, though you might explain all this and then ask how they would go about debugging the problem.

“Magic [2]”

The first paragraph sets up the context, and at the end, presents a DNS problem. I worked with DNS before, and this doesn't seem that hard a question.

So, I've noticed an issue with a record I wanted added to a zone file, say a TXT (Text) RR (Resource Record) for foo.example.com that reads “I have a red pencil.” I'm also assuming I've done a check from an outside network and didn't see the record, and that looking up the SOA (Start Of Authority) RR (also from an outside network) showed the new serial number. My first step would be to query the authoritative name servers (typically two to four, could be more) and see if the record is there. If the record does show up, then it's a propagation issue, maybe related to caching or TTL (Time-To-Live) issues. If not, then my guess is that the owner of the zonefile messed up adding the record, possibly by adding a spurious “.” to the end of the domain name—the owner effectively added

foo.	IN	TXT	"I have a red pencil."

This might not even show up as an error in any log files, but this is as-if the record was never added. But if the record was indeed added correctly, as in:

foo	IN	TXT	"I have a red pencil."

Then the next place to look is at how the change was made—did the owner update the record in some editor that then failed to properly update DNS? That would be the next place to look, and the results from that should indicate where to look next.

Why yes, I have done my fair share of DNS troubleshooting … why do you ask?

Update on Tuesday, November 28^th, 2023

Some more information came back [3].

[1] /boston/2023/11/27.1

[2] gemini://thrig.me/blog/2023/11/26/magic.gmi

[3] /boston/2023/11/28.1

Gemini Mention this post

Contact the author