This is based on Miguel’s desire for an IMAP interface to his Evolution mail, but isn’t really focussed on that particularly problem: rather, the more general problem of where “collaboration brains” belongs. One of the things which I think seems to be a bit worrying about free software mail clients is that there is this continuing move to smarter and smarter MUAs.
Miguel is actually asking for something which is much closer to what KDE are doing: the Akonadi project, as I understand it, is basically almost a full-blown local groupware/collaboration server. It uses a private MySQL instance, and has everything “built in” to this middleware which can sit on top of existing mail / contacts / etc. stores.
This also came up in the discussion the other day about Thunderbird 3 too – in order to do the “great local search” thing that they want to do, they basically have to download an awful lot of information about mails and keep it around. And again, you end up with effectively this local cache of remote information, and a local “groupware” thing.
I really don’t like any of these approaches. For me, the brains belongs in one place: on the server. There are a whole tonne of good reasons for this – not least locality to the data and keeping the client simple – but it also seems to me that the main drivers for these fat clients are only feature requirements, not technical requirements (I absolve offline access of this – that’s a technical requirement).
The main problem with the fat client approach is that you get data loss, like Miguel suffers: if something gets entered into the client and doesn’t make it to the server, it doesn’t make it to other clients. If you use a desktop app and a web app, you have to suffer through not having your up-to-date address book, or not having your “junk” status flags, or different filing rules, etc. etc.
Of course, this isn’t really a client problem per se – it’s a server problem, because we don’t have the protocols and the ability right now to let clients do this “the right way”. People have tried using stuff like IMAP and it kind of works, but it’s a hack. But by working on the client side, it just exacerbates the server problem. I’m sure people who’ve moved from Evolution to Thunderbird, or who use an IMAP client which has one way of deleting items with another client which does things the other way, will know what I’m talking about.
It’s difficult to know what to do about this, though. Working on Bongo is one answer, because better servers will encourage more lightweight and featureful clients. Being able to access the same information from Evo/Tbird and from Bongo web UI is an absolutely crucial use-case. But it will take time, sadly…
I know it makes everything 10 times harder, but sometimes you need to use data when you are not connected to a network. at the very least i need to be able to read some sort of cache of the last time i connected, and to be able to search it. ideally i need to be able to modify the data, and have it resynced when i plug back in.
maybe one day there will be cheap network everywhere, but not yet
I totally agree with that – I didn’t make it very clear, but that’s what I meant by exempting offline caching from the “smarts” argument. Being able to access stuff without a network is obviously very useful, but I’m not sure you need the full-blown local groupware to achieve that – effectively what you’re doing then is trying to set up a distributed system and that starts getting very complicated indeed.
Hey Alex,
I understand where you’re coming from. In a way, I guess it depends on how you feel client-server technology should work. If you track current trends, all the functionality is heading towards the client and the server is just used for storage and information routing/processing. Case in point would be RIA-based applications using Adobe AIR, Flex, or Flash technologies, JavaFX, Web 2.0, etc…
Personally, I’m a standards guy. If you build iCal, CalDav, GroupDav, and IMAP support into the server, you’re allowing for the most compatibility with the broadest range of clients whether they be web-based or desktop application based. It’s up to the client makers to make sure their implementation of the standards is correct, if not, their product will suffer and users will use clients that do implement the standards correctly.
My suggestion would be concentrating on making the best standards based server that you can and don’t worry about client implementations. The Bongo web-client is really good, but there are many web clients that can interface with standards compliant servers that are really good too. My personal choice is for client-based access so I have the choice of Apple Mail/iCal or Microsoft Outlook or Mozilla Thunderbird/Sunbird – whichever I feel like using this week
Hey Alex, just got around to reading this blog post (about 2 months late, but that’s what happens when you have a huge mess of bookmarks that you are cleaning up finally
I agree that putting a mail server inside an MUA is totally broken. That’s actually one of the reasons I’m not a fan of Akonadi. Back when I used to work on Evolution, there was a push to make Evolution-Data-Server (the GNOME equivalent of Akonadi) to serve email as well and I’ve been consulted a number of times about my thoughts on this feature since then.
As you were hinting at, I think it’s important for mail servers to evolve. Unfortunately, things seem to have more or less halted with IMAP4rev1 which lacks a lot of needed functionality that users are demanding from their MUAs and so MUAs have started to implement these features by mirroring everything on the server locally.
Some of the needed functionality is implemented in IMAP servers via extensions, but a lot of times these extensions aren’t implemented in many servers (thus meaning the client has to implement a workaround anyway) or only partially implement the extension specification (it’s my experience that a lot of IMAP servers also only implement a subset of the IMAP specification, but that’s another story), which, again, generally means that the client has to implement the feature itself.
What needs to happen is a new IMAPv5 specification which pulls in all the more useful extensions to IMAPv4 and maybe adds some more forward-thinking features.
Or maybe a replacement to IMAP altogether.
One of the things you mentioned was different clients deleting mail differently. Unfortunately, IMAP does not force any sort of consistency here. As you know, deleting a message in IMAP is done by setting the \Deleted flag on a message. Some clients leave it at just setting the flag, while others copy the message to a “Trash” or “Deleted Items” (or whatever the client might call it) folder and thn mark the original with \Deleted. And still others might do the copy/delete and then expunge.
And then you often have MUAs that do client-side IMAP filtering because most IMAP servers aren’t also running a Sieve server – and, afaik, there’s no mail-filtering IMAP extension.
So yea, it’s sad to say that mail is not a solved problem – neither on the client nor server sides