So I've been learning more than I ever wanted to about the syslog protocol [1]. There's the non-spec that is RFC (Request For Comments)-3164 [2] that is optimistic in terms of the protocol. Then there's the cleaned-up spec that no one is using that is RFC-5424 [3] (which is quite nice, if a bit over-engineered).
RFC-3164 documents the use of UDP (User Datagram Protocol) as the transport protocol for the syslog protocol, reading that RFC one gets the impression that one should never actually use UDP as the transport mechanism, least some cracker intercept or change the messages, or worse yet—you lose some packets and get nailed in an Sarbanes—Oxley [4] audit (or even worse still—an ISO-9000 [5] audit—the horror! The horror!).
Well, you could try running the syslog protocol over TCP (Transmission Control Protocol), but even that isn't good enough for some people [6], claiming that you can still lose logging information under certain circumstances. No, for reliability you need to add a layer of framing over TCP and wrap the syslog protocol in XML (eXtensible Markup Language) and call it a day [7].
So far, the only syslog program I've found that even pays RFC-3195 lip service is rsyslogd [8], and even then, it's receive only and uses its own framing layer over TCP [9] for sending.
I personally haven't seen an issue with using UDP for the syslog protocol. Not only do I relay syslog messages to a centralized server (my desktop box at Chez Boca, so I can watch the stuff in real time) but copies are kept locally (just in case). Also, there have been times when a TCP version (yes, even if I was using RFC 3195 or the lighter RELP (Reliable Event Logging Protocol)) would have failed (at one point, our upstream provider upgraded a firewall that filtered out TCP traffic routed asymetrically and guess what? Our traffic was routed asymetrically; UDP traffic was unaffected and thus in that case, we were able to isolate the issue faster). Even the design of SNMP (Simple Network Management Protocol) centered around UDP simply because it was “fire and forget” and thus on a congested network, there was a greater chance of UDP traffic of making it out and accepted than TCP traffic (which requires an acknowledgment that might never make it back).
But in looking over these, I'm struck that a reliable syslog protocol doesn't use SCTP (Stream Control Transmission Protocol) [10], which has the reliability, ordering and (most importantly, congestion control) of TCP with the message-based semantics of UDP. Heck, for “reliability” SCTP has one feature that neither TCP nor UDP have: either peer can change the IP (Internet Protocol) address used for the session.
For now, I'll just stick with UDP.
[2] http://www.ietf.org/rfc/rfc3164.txt
[3] http://www.ietf.org/rfc/rfc5424.txt
[4] http://en.wikipedia.org/wiki/Sarbanes-Oxley_Act
[5] http://en.wikipedia.org/wiki/ISO_9000
[6] http://blog.gerhards.net/2008/04/on-unreliability-of-plain-tcp-syslog.html
[7] http://www.ietf.org/rfc/rfc3195.txt
[10] http://en.wikipedia.org/wiki/Stream_Control_Transmission_Protocol