2007-06-23 Gmail Fetch Problems

I use Gmail for my private emails. Google’s spam filter was the easiest to use (no setup required) and worked the best. You can also fetch mail from other accounts using POP3. This feature doesn’t really work for me. Every month or so I discover that my Emails are not being fetched. What is going on?

Gmail

Looking at Gmail → Settings → Accounts → View history:

||Sat, Jun 23, 2007 at 7:37 PM||200 mails fetched. 2357 mails remaining.|| ||:--:||:--:|| ||Sat, Jun 9, 2007 at 10:26 AM||No mails fetched.|| ||Sat, Jun 9, 2007 at 10:17 AM||2 mails fetched.|| ||Sat, Jun 9, 2007 at 10:12 AM||One mail fetched.|| ||Sat, Jun 9, 2007 at 10:07 AM||2 mails fetched.||

This is useless.

So I contacted ThomasWaldmann who runs thinkmo.de which hosts emacswiki.org, and asked for advice.

We had the following LDAP entry:

dn: mail=alex@emacswiki.org,ou=Mail,dc=thinkmo,dc=de
objectClass: mailAccount
objectClass: mail
mail: alex@emacswiki.org
gidNumber: 20008
uidNumber: 20008
mailDirectory: /srv/org.emacswiki/mail/alex/
homeDirectory: /srv/org.emacswiki/mail/alex/

We replaced it with the following:

dn: mail=alex@emacswiki.org,ou=Mail,dc=thinkmo,dc=de
objectClass: mailAlias
objectClass: mail
mail: alex@emacswiki.org
mailDrop: alex@gnu.org

I then got a 10MB tarball of `/srv/org.emacswiki/mail/alex/` including a directory call ’cur’ containing what looks to be one file per Email. What would be the easiest way to read them? I’d love to import them into ThunderBird and use my existing spam-filters on them.

Well, first things first. I knew that the subject line of these mails contained the string `***SPAM***` so I unpacked the tarball, changed into the directory with all the files and ran this little command:

for f in `grep -lF "***SPAM***" *`; do rm "$f"; done

`-l` prints only the file names that match and `-F` treats the pattern as a fixed string instead of a regular expression.

That reduced the number of candidates to 161. 😄

I tried using Emacs! I knew that Gnus could read and write Email in various formats. So here we go. Start Emacs. Run `M-x gnus`. Answer ’y’ to `Unable to open nntp:news, go offline? (y or n)` and answer ’y’ to `nntp (news) open error: ''. Continue? (y or n)` – I don’t care about news right now.

Emacs

Gnus

I must have some Gnus data lying around. My `*Group*` buffer listed two groups: `SPAM` and `BACKUP`. I have no idea. 😄 The last messages are from 2006-05-17. I guess I don’t need them anymore. Use ’c’ to “catch up”.

2006-05-17

Now, I need to create a group that uses my directory as a source... `gnus-group-make-directory-group` looks like a good candidate: “The user will be prompted for a directory. The contents of this directory will be used as a newsgroup. The directory should contain mail messages or news articles in files that have numeric names.”

Damn the numeric names! 😄

I get `Group nndir:/Users/alex/Documents/Mail/alex/cur contains no messages`.

I wrote a little script called `number-files`:

#!/bin/sh
NUM=1
for f in "$@"; do
    echo mv "$f" $NUM
    (( NUM++ ))
done

Calling `number-files *` was looking good, so I called `number-files * | sh`. I like double-checking my commands. (As Pierre noted in the comments, this breaks if the filenames contain spaces!)

I got 161 files named “1” to “161”. Still looking good!

Back to Gnus and ’G d’ to create that directory group.

Hah! **It worked!**

Now to export it to mbox format...

1. Use ’M P a’ to mark all articles with the process mark.

2. Use ’C-o’ to write it to a “Unix mail box file”. Too bad this required me to hit `RET` 161 times. Next time, I need a command without a confirmation question. Gah!

(*Update*: A helpful visitor suggested setting `gnus-prompt-before-saving` to `t` before using before using ’C-o’.)

Anyway, I used `~/mbox` as the output file, and everything is still looking good. A bit less than 4MB of spam to go.

So, where to I put this?

1. Find the Thunderbird directory. On my Mac, it is `/Users/alex/Library/Thunderbird/Profiles/0skwxq1c.default/Mail/Local Folders`.

2. Move the mbox file there. I also renamed it to “Mbox”. ;)

3. Start or restart Thunderbird.

4. Find your new folder in the “Local Folders” section.

5. Use “Apply Junk Filter to the Current Folder” from the “Extras” menu. (I’m translating from German to English, so your menu entry might have a slightly different name.)

Hah, done!

Except that now I have to read all the old mail and apologize for my Email problems. 😢

​#Mail ​#Software ​#Web ​#Spam ​#Emacs ​#Gnus ​#Thunderbird

Comments

(Please contact me if you want to remove your comment.)

“Calling number-files _ was looking good, so I called number-files | sh. I like double-checking my commands. ;)”_

Take care though, you break your otherwise correctly quoted script by piping it to sh and it will fail for filenames containing spaces

– PierreGaston 2007-06-24 05:43 UTC

---

Ah... Interesting. I hadn’t considered this. Luckily for me, no problem. 😄

– Alex Schroeder 2007-06-24 09:26 UTC

Alex Schroeder