I started seeing replies to an email a friend sent (he sent it to a bunch of friends, who started replying to all) way before I got the original email my friend sent. When I checked, it was as I feared, a large company (Adelphia [1]) had multiple machines for outoing mail, and each attempt was coming from a different IP (Internet Protocol) address, and coming too quickly to pass through the embargo timeout. For a while, I was actually afraid it would never make it through. When I did finally get it, some 9½ hours had passed from the first attempt:
>
```
Sep 10 08:06:55 brevard graylist: tuple: [68.168.78.202 , XXXXXXXX@adelphia.net , sean@conman.org]
Sep 10 08:58:00 brevard graylist: tuple: [68.168.78.187 , XXXXXXXX@adelphia.net , sean@conman.org]
Sep 10 09:53:08 brevard graylist: tuple: [68.168.78.178 , XXXXXXXX@adelphia.net , sean@conman.org]
Sep 10 09:53:35 brevard graylist: tuple: [68.168.78.178 , XXXXXXXX@adelphia.net , sean@conman.org]
Sep 10 09:53:59 brevard graylist: tuple: [68.168.78.178 , XXXXXXXX@adelphia.net , sean@conman.org]
Sep 10 09:54:17 brevard graylist: tuple: [68.168.78.178 , XXXXXXXX@adelphia.net , sean@conman.org]
Sep 10 09:54:30 brevard graylist: tuple: [68.168.78.178 , XXXXXXXX@adelphia.net , sean@conman.org]
Sep 10 09:54:38 brevard graylist: tuple: [68.168.78.178 , XXXXXXXX@adelphia.net , sean@conman.org]
Sep 10 10:49:24 brevard graylist: tuple: [68.168.78.205 , XXXXXXXX@adelphia.net , sean@conman.org]
Sep 10 11:50:29 brevard graylist: tuple: [68.168.78.211 , XXXXXXXX@adelphia.net , sean@conman.org]
Sep 10 13:01:35 brevard graylist: tuple: [68.168.78.175 , XXXXXXXX@adelphia.net , sean@conman.org]
Sep 10 14:06:15 brevard graylist: tuple: [68.168.78.181 , XXXXXXXX@adelphia.net , sean@conman.org]
Sep 10 14:06:20 brevard graylist: tuple: [68.168.78.181 , XXXXXXXX@adelphia.net , sean@conman.org]
Sep 10 14:06:29 brevard graylist: tuple: [68.168.78.181 , XXXXXXXX@adelphia.net , sean@conman.org]
Sep 10 14:06:52 brevard graylist: tuple: [68.168.78.181 , XXXXXXXX@adelphia.net , sean@conman.org]
Sep 10 14:07:14 brevard graylist: tuple: [68.168.78.181 , XXXXXXXX@adelphia.net , sean@conman.org]
Sep 10 14:07:34 brevard graylist: tuple: [68.168.78.181 , XXXXXXXX@adelphia.net , sean@conman.org]
Sep 10 14:08:07 brevard graylist: tuple: [68.168.78.181 , XXXXXXXX@adelphia.net , sean@conman.org]
Sep 10 14:08:24 brevard graylist: tuple: [68.168.78.181 , XXXXXXXX@adelphia.net , sean@conman.org]
Sep 10 14:08:33 brevard graylist: tuple: [68.168.78.181 , XXXXXXXX@adelphia.net , sean@conman.org]
Sep 10 14:08:41 brevard graylist: tuple: [68.168.78.181 , XXXXXXXX@adelphia.net , sean@conman.org]
Sep 10 15:12:39 brevard graylist: tuple: [68.168.78.44 , XXXXXXXX@adelphia.net , sean@conman.org]
Sep 10 16:17:17 brevard graylist: tuple: [68.168.78.196 , XXXXXXXX@adelphia.net , sean@conman.org]
Sep 10 16:17:23 brevard graylist: tuple: [68.168.78.196 , XXXXXXXX@adelphia.net , sean@conman.org]
Sep 10 16:17:45 brevard graylist: tuple: [68.168.78.196 , XXXXXXXX@adelphia.net , sean@conman.org]
Sep 10 16:17:53 brevard graylist: tuple: [68.168.78.196 , XXXXXXXX@adelphia.net , sean@conman.org]
Sep 10 16:17:59 brevard graylist: tuple: [68.168.78.196 , XXXXXXXX@adelphia.net , sean@conman.org]
Sep 10 16:18:06 brevard graylist: tuple: [68.168.78.196 , XXXXXXXX@adelphia.net , sean@conman.org]
Sep 10 16:18:51 brevard graylist: tuple: [68.168.78.196 , XXXXXXXX@adelphia.net , sean@conman.org]
Sep 10 17:20:50 brevard graylist: tuple: [68.168.78.178 , XXXXXXXX@adelphia.net , sean@conman.org]
```
It's this behavior that has us at The Office concerned about greylisting [2]; that delays of this magnitude will have our customers screaming at us. I've been keeping track of such emails, building up a list of IP addresses to immediately whitelist. P asked if the given IPs were listed as the MX (Mail eXchange) record, and if so, use that to whitelist the email. But when I checked, that wasn't the case for Adelphia. P then suggested I check the SPF (Sender Policy Framework) records.
Not a bad idea. The SPF record for Adelphia matched the IPs I was seeing. I then went on to check the SPF record for some of the other companies I was whitelisting, like AOL (America Online) [3] and BellSouth [4]. Sure enough, most (Yahoo [5] is the only exception so far) have SPF records. I may have to include an SPF check into the daemon, but I'd rather not immediately let through emails that pass the SPF check [6]. I'll have to think about how I want to do this.
Meanwhile, some stats from the currently running version (started sometime last week):
Table: Current Graylist statistics tuples 1,810 graylisted 20,775 whitelisted 42 graylist expired 18,965 whitelist expired 0
The row labeled “tuples” are all the tuples currently in memory (that haven't expired) and the row labeled “graylisted” have been the number of emails added to the graylist since the program started. It's been holding steady at about 1,800 tuples at any one time for the past few days (and this is just the emails being sent to my server—perhaps a dozen domains or so, but mostly to conman.org). So far, only 0.2% of all emails have been whitelisted, but that includes 18 spams. Not that bad considering prior to this I was getting something like 1,800 per day.
[2] http://projects.puremagic.com/greylisting/whitepaper.html
[6] http://www.computerworld.com/softwaretopics/software/groupware/story/0,10801,95617,00.html