More notes on logging

I mentioned yesterday about logging all mail related logs to a central server [1]. While we don't have a complicated email setup (unlike, say, Negiyo), we still have several email severs and we get enough tickets about slow or lost email that it's a pain having to slog through one or two servers piecing everything together. What I would like is, given a Message-ID (which is (supposed to be) a globally unique identifier for an email) or an email address, to make a query in one location and get something like:

>
```
message-id = <YzNCeWFXNW5RSE53Y21sdVoyUmxkeTVqYjIwPQo=@mx3oc.com>
from = gandalf@example.net
to = sean@example.com
[rhohan-isp.example.org] [gondor.example.net] Feb 10 22:46:56
[gondor.example.net] [spamfirewall.example.com] Feb 10 22:46:57
[spamfirewall.com] [compmailserv.example.com] Feb 10 22:47:02
[compmailserv.example.com] [workstation.example.com] Feb 10 22:47:06
[workstation.example.com] mbox of sean Feb 10 22:47:06
```

As an example, you see the Message-ID, who sent the email, who received it, and the five other lines can be read as: machine X sent email to machine Y at such-n-such a time,” with the last one showing local delivery of the email to a mailbox.

Anyway, that's what I would like to build. And I can almost do it. Sendmail [2] (which at The Company we use on our legacy systems), Postfix [3] (which we use for new servers) and Exim [4] (which we use on one server because it has a feature that's needed by a program that runs on that one server) all log a bunch of messages as email works through their respective systems. Each one uses an internal unique ID, but they at least log the Message-ID at some point, so I can map the respective MTA (Mail Transport Agent)s internal IDs to a globally unique ID.

The odd-man out though, is our spam firewall, which is used by a significant portion of our customers. But, given that our spam firewall is OpenSource™ I suppose I can modify the source code to emit a Message-ID, but the problem there is if (or when) we upgrade—I would have to patch the code again (or, convince the Powers That Be to accept the patch).

I would also like to convert as many software packages to log via syslog, and while most, like PostgreSQL [5] and even Apache [6], can be configured as such, there are a few holdouts (I'm looking at you, MySQL [7]) that can't.

[1] /boston/2010/02/09.1

[2] http://www.sendmail.org/

[3] http://www.postfix.org/

[4] http://www.exim.org/

[5] http://www.postgresql.org/

[6] http://httpd.apache.org/

[7] http://www.mysql.com/

Gemini Mention this post

Contact the author