Alex Hudson

Thoughts on Technology, Product, & Strategy

Category: bongo (Page 4 of 9)

Thunderbird & calendars

When the new “MailCo” (as it was at the time) was announced, the first thing they did was poll people on what changes they should be looking to make to Thunderbird as kick-ass as possible – you can see from the initial launch blog post that integrated calendaring was item #1 on a list of new features.

Now, only one year on, the word is that calendaring won’t be integrated or bundled at all – in fact, it gets worse, because the calendar developers have also announced that a number of full-time developers have been lost. Doesn’t say why, but I’m assuming you can blame the Global Downturnโ„ข.

I can’t express what a huge mistake I think this is for Thunderbird. One of the main reasons I got involved with Bongo was to provide a calendar server for my day job’s choice of mail client (Tbird), and when Jonny produced his CalDAV code things were really looking pretty sweet.

The problem for me comes down to basically one of maintenance. Extensions that you can install are great, but they have a tendency to bit-rot and suffer versioning issues. Even on Firefox you have issues with a version upgrade making an existing extension wobble or fall over; and with Tbird3 being developed at a more rapid pace, there is a real problem there.

The calendar devs can only concentrate on one thing, and it seems like that one thing has to be a plug-in. In my opinion, a non-working calendar in Tbird3 ought to be a release blocker, but that is now off the table. I have to say, if Evolution on w32 was a viable option, I think that’s where we’d be looking right now, but I’m going to reserve judgement until Tbird3 releases.

An argument against the “fat client”

This is based on Miguel’s desire for an IMAP interface to his Evolution mail, but isn’t really focussed on that particularly problem: rather, the more general problem of where “collaboration brains” belongs. One of the things which I think seems to be a bit worrying about free software mail clients is that there is this continuing move to smarter and smarter MUAs.

Miguel is actually asking for something which is much closer to what KDE are doing: the Akonadi project, as I understand it, is basically almost a full-blown local groupware/collaboration server. It uses a private MySQL instance, and has everything “built in” to this middleware which can sit on top of existing mail / contacts / etc. stores.

This also came up in the discussion the other day about Thunderbird 3 too – in order to do the “great local search” thing that they want to do, they basically have to download an awful lot of information about mails and keep it around. And again, you end up with effectively this local cache of remote information, and a local “groupware” thing.

I really don’t like any of these approaches. For me, the brains belongs in one place: on the server. There are a whole tonne of good reasons for this – not least locality to the data and keeping the client simple – but it also seems to me that the main drivers for these fat clients are only feature requirements, not technical requirements (I absolve offline access of this – that’s a technical requirement).

The main problem with the fat client approach is that you get data loss, like Miguel suffers: if something gets entered into the client and doesn’t make it to the server, it doesn’t make it to other clients. If you use a desktop app and a web app, you have to suffer through not having your up-to-date address book, or not having your “junk” status flags, or different filing rules, etc. etc.

Of course, this isn’t really a client problem per se – it’s a server problem, because we don’t have the protocols and the ability right now to let clients do this “the right way”. People have tried using stuff like IMAP and it kind of works, but it’s a hack. But by working on the client side, it just exacerbates the server problem. I’m sure people who’ve moved from Evolution to Thunderbird, or who use an IMAP client which has one way of deleting items with another client which does things the other way, will know what I’m talking about.

It’s difficult to know what to do about this, though. Working on Bongo is one answer, because better servers will encourage more lightweight and featureful clients. Being able to access the same information from Evo/Tbird and from Bongo web UI is an absolutely crucial use-case. But it will take time, sadly…

Thunderbird 3

Unlike Jono, whose experiences with Tbird 3 are worth a read, I’ve been a loyal Thunderbird user for a few years now – in fact, we’ve had it deployed at work relatively happily for a while now (I say relatively – the mail client is fine; lack of calendaring is a bit of an issue…). I also tried the Tbird 3 beta recently too, although I think I met with even less success.

For one thing, I was trying this out on the EeePC 901 I have, and for whatever reason, they really didn’t get on at all. I could log into my Bongo mail ok, but the mail box was entirely read-only: no marking read, no moving, no deleting. And the GUI problems just got worse from there – functions which should have been there were just completely missing. I suspect this is to do with either the distro or something I have in my environment; speaking to others, they had much more success, so I plan to try it again in the future.

I also see Tbird as a project with an enormous potential. However, the appearance of Tbird 3 really surprised me, and David Ascher’s blog post on the direction wasn’t something which filled me with a huge amount of confidence. I don’t get some of this at all; a great example is tabs. I just don’t see the value. I can see people lining up mail messages to respond to and that kind of thing, but that seems a really poor UI to my mind. Another example would be the complete Gmail-like effect in the screenshots in that blog: it’s basically Gmail offline, which is fine if that’s what you want, but.. eh.

As a server guy, the “improved search” stuff also worries me. It looks very much like client-side searching, which is fine for personal mailboxes. Once you have a few shared boxes and bigger corporate-level stuff, that starts to become a bad plan very quickly. But it seems to me that the business market is really where Tbird should be aiming: people at home are pretty happy with Gmail and stuff already. I don’t see why they’d want to access Gmail over IMAP.

It will be interesting to see how Tbird 3 turns out. Integrated calendar is absolutely crucial. Hopefully it can all be pulled off without becoming a less business-suitable product. Mozilla Messaging have the resources, but they seem to be trying to take on a lot with this first bite.

Brand new old blog.

You may have noticed that I’ve been very quiet on my blog over the past couple of months. This has been for a couple of reasons: primarily, I’ve been really busy, but also because I’ve been sort-of locked out of it while my laptop was out of commission. I was a Blosxom user, which is a lovely piece of software and very simple, and I chose it because it’s very easy to integrate into another site – however, that has now bitten the dust.

At some point, for some unknown reason, all my historic blog entries ended up getting dated to September. Blosxom takes the dates from the file stamps, and there’s no other information I seem to have available – so, basically, my blog history is now entirely buggered.

Anyway. I’ve used this as an excuse to move to WordPress 2.7 – the release that just came out. I’ve always been a bit against WordPress, mainly because it needs an SQL database, but also because it seemed so complex and difficult to work into my existing site. However, I did it earlier today, and it probably took literally an hour – the hard part was making the mod_rewrite rules actually work for the permalinks.

I found DeWitt Clinton’s blog post on moving from Blosxom, and with a bit of light editing to the script he provided, I managed to import all my old data – thanks, DeWitt ๐Ÿ™‚ All my old posts are back – minus the correct dates and permalinks, of course – which is nice.

Basically, it all seems the same from the front – albeit it I can now take pings, comments, and all that new lovely stuff. Plus, I can access my blog from whereever now, and hopefully that will mean I can blog more easily / more often. Let me know if you find anything broken!

Bongo at Expo

So, somewhat little advertised and somewhat exclusive, Linux Live Expo 2008 in London has been taking place. Friday was day two of three; for the first time it’s on a Saturday as well – and we’ll see how that goes tomorrow, I guess. But the first few days have been interesting.

Thursday started relatively slowly, although Bongo was sited next to the awesome OpenNMS guys – who travelled a lot further than I did to get there, and are exceedingly friendly. They have some interesting ideas about coloured shirts for contributors and stuff like that which I’m very keen on shamelessly ripping off, and Tarus is an incredible ambassador for their project (and the other guys are ok too ;).

Not long afterwards, I hooked up with the amazing Michael Meeks who was dishing up hip-hop hand-grabs like they were Hershey’s Kisses, and generally being extremely positive to all in the .Org village. Although Novell as a corporation have not seen fit to bless the Expo with their presence, it’s nice to see their more respected engineers about.

In general with Expo, things have been relatively quiet – and somewhat cold & dark, if truth be told – but we’ve made some interesting contacts with different people and projects, and it does seem like it was very much worth attending. Lance and I will be there on Saturday, and that will be extremely interesting too.

More thoughts on Thunderbird.

A while ago, I wrote some thoughts on Thunderbird in the aftermath of the announcement of the new MailCo and the resignation from Mozilla of two key developers. I was disappointed at the time, but looking back at my post, I don’t think it was particularly negative – if anything, I was pointing out how vital I still thought (and think) it is that Tbird be a successful project.

Having exchanged pleasantaries with David Ascher over the past couple of days, and read what he has to say about Tbird, I’m much less disappointed and much more hopeful. Tbird is important to Bongo for a variety of reasons, particularly having a good standards-compliant client for Windows users is crucial. Over time, I’d like to see Tbird become an exemplar client: it should be possible to have mail, events and contacts all integrated, and take full advantage of server-side functionality so that the sum of bongo + tbird is greater than the sum of the parts. Implementing server/client standards, like mail, is a chicken and egg problem. Having both the chicken and the egg drive each other forward (is that stretching the metaphor too far ? ๐Ÿ™‚ can only be a good thing.

It will be interesting to see the vision for Tbird develop over the coming months, and I hope that Bongo can play a role in that in some shape or form. At the very least, I’ve told David that I will keep a more active watch on Tbird, and hopefully Bongo can contribute much more than that.

As an aside, David is also collecting a list of interesting e-mail experiments – more grist to the mill, as it were. For what it’s worth, any links I come across like this I usually post on the Bongo mugshot group – I know I’m not the only member; other people should feel free to ping stuff on there too ๐Ÿ™‚

Finally, a note on M3 – Pat has done some great work tracking down various bugs, Jonny’s CalDAV work has been merged in, and the list of blocking bugs isn’t too bad. Another release will probably happen this weekend after we’ve knocked a few more off the list, and the full release should probably happen at the end of the month. It’s starting to take real shape now after the madness that has been getting rid of MDB, and that’s very pleasing to see!

M3 test release 1

By the time M3 will come out finally, it will have been five months since the last proper release. Time flies when you’re having fun! In all seriousness, that’s far too long, and we did bite off a bit more than we could easily chew. Ho hum.

The first test release for M3 is out there, though. I was going to call these release candidates, but I decided against it – there’s no way the first few will be released, so they’re not really RCs.

It would be great if people can test this a good amount. I have a reasonable idea of what most of the worst bugs are, and I really want us to get to the stage where people can seriously use Bongo for everyday stuff, so that’s mostly what I’m aimed at. Read the unfinished release notes, and then grab the tarball.

I was thinking about naming the people who have helped contribute to this release, but honestly, there’s way too many – which is great. Our last release was around 300 revisions ago, and so much has changed then, and there’s still another branch to merge into this release. I do think we need to have a much better list of contributors on the web site though, because I think people should get recognition for the input they put in.

I’ll allow myself to be distracted by M4 a little bit: I really want a quick M4 release this time around, so although we want to re-work our web UI, M4 might not get it – maybe just the new Flasher UI and get Hawkeye into almost final shape. Our mail stuff should be much closer to being done, and the command line tools also. Maybe we should move to time-based releases already, to focus the mind?

Diversity in the community

It’s not often that I’m reading through some article online and come across someone using my own words in print (as it were), but that happened this morning!

It is a very strange feeling. I was reading Free, Open and Eating its Young – the premise of the article is that many free software communities have significant proportions of essentially anti-social contributors. Both Pat and I have blogged about this before – something that is important to me is that the climate of the Bongo community be extremely welcoming.

The quote of mine that was used is part of a longer, better phrased, comment I left on Jono Bacon’s blog. It’s pretty representative of my views, though – I don’t feel particularly misquoted, which is good.

Out of Office Message: Alex is away

From Friday until late Monday night, I will be on retreat and entirely away from the internet. No, Bongo hasn’t scared me off, I will be back ๐Ÿ™‚ So, I figure that I’m going to try again my “pretty please” act to get people to look at some bugs: it worked great last time I tried it, and I can’t remember why I didn’t try again. Of course, many people won’t feel competent enough to look at some of these bugs, so there’s an extra-credit task at the end of this coming up in a further blog post ๐Ÿ™‚

  • Thunderbird and IMAP: we have a bug which a number of people have seen, where large amounts of e-mail can’t be fetched even though Bongo appears to be running correctly. I’ve tried tracing what happens and can’t figure it out; my guess is that the connection is being dropped early or something similar. Pat said turning off anti-spam checking on incoming e-mail seemed to solve/mitigate the problem for him; does anyone else have any workarounds they know of? Does someone want to go through doing some testing with various clients and see if they all trigger this bug? You’d need to setup a new account, fill it with mail somehow with the imap client offline, and then see if it happens. I think around 300 mails is enough to trigger it – at least, it does here. Does it happen if you remove the IMAP account on the client and re-create it? Anyone solving this bug will win a couple of beers at least.
  • Bug #10790 is about building against an external libical. Anyone familiar with the build system want to have a go at this? Shouldn’t be too hard ๐Ÿ™‚
  • Similarly, it’d be great if someone looked at the CLucene build test. At the moment, if you’re on x86_64, Bongo won’t let you compile against a system CLucene library because most of them are too broken for us to use. However, not all of them are. A better compile-time check would be to compile a small test program and see if CL_MAX_DIR is defined to be greater than twice CL_MAX_PATH – if so, we should fail then and only then.
  • I’d love someone to do a review of the various ISPs around – even just in one country, but preferably many – and look at what their SMTP requirements are. Some places do various things, like:
    1. preventing any kind of outgoing SMTP at all (Wanadoo/Orange; force people to use a mail relay);
    2. have an SMTP relay which requires SMTP AUTH (many);
    3. have an SMTP relay which requires TLS (?any);
    4. rewrite outgoing mail messages (e.g., Sky, altering From: to a Sky.com address – true evil).

    It would be great to hear if other ISPs have discovered/implemented other kinds of SMTP evil. Bongo – as it stands – won’t work in any of the above scenarios (1. requires manual config change, 2. I don’t think we can do, 3. we have buggy code for which is disabled, 4. we can’t solve anyway). I’d really like to know what features we need to implement in Bongo for it to be usable for a wide range of ISP services.

I’ll likely drop back in on Monday night, but I won’t otherwise have internet from tomorrow onwards, so don’t expect me to be around (except that my irc client will probably stay logged in, but away). See you all then!

Mozilla on the future of Thunderbird.

I sent this link around the Bongo people on Mugshot, and on IRC, but here it is again for the benefit of people watching our planet: the future of Thunderbird seems to be a bit cloudy, Mozilla Corp. seem to have realised that it doesn’t quite fit it.

Primarily, this disappoints me, because I always saw Tbird as a natural fit to the Bongo Project. We’re developing a kick-ass webmail, and that will appeal to a lot of people, but there are other people – like me – who simply don’t use webmail. The business I run uses Thunderbird, and (coupled with Lightning) we’re pretty happy with it, even though there are a lot of rough edges for a business. I had hopes for a Bongo add-on to Tbird which would automatically set it up for you, with your calendars and contacts and stuff, and work well with your server-side config. Ideally, you’d be able to edit your mail filing rules, get the server to mark mail as spam, and setup vacation response.

Plugging into Bongo would also tap Thunderbird into all the stuff we want to do: getting contacts from your social networks, being able to forward web links to your calendar to friends who use webmail, that kind of thing. It would open Thunderbird up.

It does seem, though, that the Thunderbird project is suffering a major identity crisis. Who would use Thunderbird? And, if it needs to stand on it’s own two feet, who’s going to pay for it? Personally, I refuse to cede the mail space to the likes of Gmail: yes, they’re very popular, but for heavens’ sake – email is a huge market, and there is a lot of money to be made there.

I worry that the discussion over the format of the corporation is really hiding the bigger question. It doesn’t matter whether it’s a non-profit or a business, or whether it’s related to Mozilla or not. The key question is, who is Thunderbird for? If it keeps trying to be an Outlook Express-level mailer which people use to access their ISP’s POP3 account or something, frankly, it has no future. That mail usage pattern is pretty simple, dying out, and people are not going to be willing to pay for it.

I further worry what this will do to an already-small Tbird community. How will the Lightning people respond to this? What is Penelope doing? Development has happened behind closed doors at the best of times, to an outsider like me, and I’ve never understood the roadmap. As a long-time Thunderbird user – basically since it was barely usable – I’d like to say it’s come on in leaps and bounds. But, it doesn’t feel like it has. And looking at the roadmap isn’t terribly inspiring: lots of talk about Gecko versions, not much talk about other stuff.

I commented on Mitchell’s blog that I think Thunderbird is too scared of competing with Outlook. Note, competing with doesn’t mean copying: lord knows, we don’t need another Outlook, but Tbird misses out on huge features simply because it won’t integrate with servers. E.g., if you want shared contacts on Tbird, you have to use something like Plaxo – which then ties you into a proprietary service. Tbird simply has to offer more integration: it cannot be it’s own little world on the desktop. Having your own anti-spam is great; not being able to integrate with the training on the server-side sucks. Ditto junk, conversation views, tags. Lightning needs to be a first-class component. Without offering these types of integrations, I don’t see how Tbird can generate revenue: if it needs to be a commercial product, and it sounds like it does, it needs to be something people will pay for.

Page 4 of 9