Longish time no blog: not really getting enough hacking time at the moment 🙁
Recently we discovered a problem for IMAP users. IMAP clients can have multiple connections open at any one time, and some clients do that in practice to speed up certain operations: for example, Thunderbird’s filtering system runs more than one connection to move multiple messages at once. Which is great.
Our problem was twofold: primarily our indexing takes a little while, but the main issue is that while the indexing was happening, the mailbox was essentially frozen to other changes for various technical reasons. Which isn’t great.
It might be possible to lock and unlock mailboxes at clever points and do things in a certain order which would make everything ok, but that’s complex and error prone. So, the approach I’m looking at is converting our store from mbox format files to maildir specification file layout. This relieves us of a couple of troubles:
- we can push new content into the store and index it before we need to update the store meta-data, and it works out;
- we can do away with compacting mboxes, it’s unnecessary;
- removing and moving is probably a bit quicker;
- fewer times we need to lock the mailbox against external changes.
All in all, it’s looking like this new way is going to be a big win, and so far I’m removing more code than I’m adding (209 additions versus 234 removals), which is pretty good considering the featureset isn’t really changing.
