Welcome Back Courier-IMAP

Send to Kindle

When Matt was maintaining this server, starting back in 2001, he installed Courier-IMAP for our mail service (both IMAP and POP). It worked extremely well for many years. At one point, IMAP folders started taking a long time to open. Once they were open, performance was excellent. I think this was due to not enough simultaneous open connections allowed from the same IP address.

I’ve been maintaining this machine for a while, but I didn’t bother to do anything with Courier-IMAP even though it had started annoying me. Over 18 months ago, I switched to a new server. I decided to build it from scratch, correcting some legacy problems along the way. One of them was the above-mentioned IMAP hangs (not always, but annoying nonetheless).

After some research, I chose Dovecot as the new POP and IMAP server. I was impressed with how easy it was to install and configure. While there were tons of options to choose from, all of them are controlled in one dovecot.conf file, with extremely clear descriptions of what each choice entails. Once installed, it worked perfectly, and I was very pleased with my decision.

This joy lasted for more than a year! Then, at some point (possibly after my server was physically moved from one data center to another), a few times a day (typically 1-3 times), Dovecot would think that the system clock had moved backwards in time (it never does, and no other program every complains about that). Dovecot sees this as a very bad event (understandably) and exits automatically.

After noticing this a few times, I installed a monitoring program called Monit. I am running a 5.0 beta version, but monit has been flawless in every respect, even in beta. Since I installed monit, every time that Dovecot would quit, monit would restart it within 30 seconds, and email me that it just restarted it. That’s how I know it’s a daily event, sometimes multiple times a day.

I’ve lived with this nonsense for way too long. Each time, I assumed that the next version of Dovecot would magically fix my problem, even though it started on a version that had worked perfectly for months before the problem started! As much as I like everything else about Dovecot, I finally gave up.

This weekend, I installed Courier-IMAP (a much newer version than the one we used to run). I made sure to allow more concurrent connections from the same IP address (both Lois and I always come across as coming from the same IP address). I had a few hassles with the configuration (even though there are way fewer things to choose than with Dovecot). After about an hour of messing around (probably all my own fault, over-thinking some of the choices), I got it working.

There was only one side-effect to the change. Under Dovecot, my IMAP folders were shown in the mail client as follows:

INBOX:

———>XXX

———>YYY

———>ZZZ

After switching to Courier-IMAP, the structure in the mail clients was:

INBOX:

———>INBOX:

——————–>XXX

——————–>YYY

——————–>ZZZ

We can all live with it, and no mail was lost. It’s a minor nuisance.

I tested on a separate port, and when POP and IMAP were working, I turned Dovecot off and restarted Courier-IMAP with the correct default ports.

I then wrote an email to all of my users (all four of them). 😉

However, when I went to send the email, the send failed with a SASL authentication error. Ugh. I have saslauthd on the system (it wasn’t running, because Dovecot was performing that service as well). I started it up, but even though I played around a bit, I couldn’t get Postfix to authenticate correctly through it.

After looking at the top of the dovecot.conf file, I saw that by changing one line (which protocols Dovecot should handle) to “none”, it would run in SASL authentication mode only. That worked. So, now I still run Dovecot for authentication (since I didn’t have to change anything), and Courier-IMAP for mail fetching.

So far, the system has been running for a little over 24 hours, with no exits on the part of Courier-IMAP or the Dovecot auth daemon. I also haven’t had any hangs on opening an IMAP folder. It’s still very early, but the Dovecot IMAP server would have died at least once by now (guaranteed), so it’s already a win.

Here’s hoping that this will be a permanent change…

Update: First, so far so good. No exits in 4+ days! More important, I just stubmbled across a post that gave me the answer to my nested mailbox problem above. Apparently, Dovecot repeats the .INBOX in front of each sub-folder in the Maildir folder. In other words, .INBOX.SPAM is the SPAM folder, directly under the main INBOX in Dovecot. Courier-IMAP expects it to just be .SPAM in the top-level Maildir folder, in order to be considered a direct sub-folder of the main .INBOX.

I moved the folder names, unsubscribed the old one and subscribed to the new ones, and my folder hierarchy is now sane again. Whew. 😉

Update: It turns out that the fault lies not with Dovecot, but with some bad code in the Linux Kernel for certain hardware configurations (obviously, mine included) that causes the system clock to jump a few times a day. There is a long thread about it on the Dovecot mailing list, which points to a very long thread on the Linux Kernel maintainers list. So, while the problem definitely affected me, it’s not Dovecot’s fault, which correctly notices the jump, and decides that it’s safer to exit than to guess. Perhaps a future kernel update (I just applied one today) will solve the problem. I don’t feel like hand-patching my kernel, or Dovecot…