Smirk gave me a router to configure for a customer and asked that I save the current configuration before I make any changes. It's a Cisco [1] router, so saving the configuration is just copying it to a TFTP (Trivial File Transfer Protocol) server. Nothing terribly hard—I already have a TFTP server installed on my workstaion, and I can easily configure the spare network port on my workstation to match one of the ethernet ports on the router, string a cable (which I already have lying around) and voilà. Nothing to it.
I'm ready to kill. I'm thinking one of (or all of) the CentOS [2] maintainers, since I can't imagine the maintainers of logrotate [3], syslogd [4], xinetd [5] or tftpd (is that even maintained any more?) all botching it in a perfect storm of stupidity.
tftp is controlled and run by xinetd. But the Cisco router can't save the configuration file. It's not a networking issue since I'm logged into the Cisco router over the network. The configuration in xinetd to run tftpd looks good, but what the heck, I decide to restart xinetd and check the logs anyway. That's when I find that logrogate isn't. Or rather, it is, but it's failing to tell syslogd to reopen its log files. And the configuration files for logrotate look okay as far as I can tell.
But even worse, syslogd appears to be buffering the logs in memory, maybe updating the files every 30 seconds or so. And the syslogd configuration I have specifically states otherwise. It could be that xinetd is really slow, but I have a hard time believing that it takes xinetd a full twenty seconds to digest a 13 line configuration file (and there are about 30 such files—do the math) on a 2.6GHz (GigaHertz) dual-core box—the harddrives aren't that slow (heck, Firefox [6] comes up faster than that on this system).
Sigh.
I really hate it when the tools I expect to work don't. It's one thing to know going in that the tools don't work, or aren't there. I know I'm going to get sidetracked and can make better time estimates. It's making a time estimate expecting the tools I work with to work and then getting sidetracked debugging tool issues that pisses me off, because now I'm under a time crunch.
It's even more frustrating when the customer isn't even there to accept the router.
[3] http://linuxcommand.org/man_pages/logrotate8.html
[4] http://en.wikipedia.org/wiki/Syslog