It's not often that I'm reading through some article online and come across someone using my own words in print (as it were), but that happened this morning!
It is a very strange feeling. I was reading Free, Open and Eating its Young - the premise of the article is that many free software communities have significant proportions of essentially anti-social contributors. Both Pat and I have blogged about this before - something that is important to me is that the climate of the Bongo community be extremely welcoming.
The quote of mine that was used is part of a longer, better phrased, comment I left on Jono Bacon's blog. It's pretty representative of my views, though - I don't feel particularly misquoted, which is good.
posted at: 10:23 | path: /bongo/community | permanent link to this entry
By the time M3 will come out finally, it will have been five months since the last proper release. Time flies when you're having fun! In all seriousness, that's far too long, and we did bite off a bit more than we could easily chew. Ho hum.
The first test release for M3 is out there, though. I was going to call these release candidates, but I decided against it - there's no way the first few will be released, so they're not really RCs.
It would be great if people can test this a good amount. I have a reasonable idea of what most of the worst bugs are, and I really want us to get to the stage where people can seriously use Bongo for everyday stuff, so that's mostly what I'm aimed at. Read the unfinished release notes, and then grab the tarball.
I was thinking about naming the people who have helped contribute to this release, but honestly, there's way too many - which is great. Our last release was around 300 revisions ago, and so much has changed then, and there's still another branch to merge into this release. I do think we need to have a much better list of contributors on the web site though, because I think people should get recognition for the input they put in.
I'll allow myself to be distracted by M4 a little bit: I really want a quick M4 release this time around, so although we want to re-work our web UI, M4 might not get it - maybe just the new Flasher UI and get Hawkeye into almost final shape. Our mail stuff should be much closer to being done, and the command line tools also. Maybe we should move to time-based releases already, to focus the mind?
posted at: 10:23 | path: /bongo/community | permanent link to this entry
My best wishes to everyone in the Bongo community for the year ahead. I'm sorry not to have been around much over the holiday; our family lost our Grandmother four days ago, and as you might imagine things have been a bit upside down since. One minute I was committing a patch for new mail processing, the next I was in hospital (almost literally), and things have been a bit strange since.
After that, Bongo hacking has actually come as a bit of a light relief, so it would be cool to hear if anyone has tried out the maildir'd store, and if things are looking good, we can think about releasing relatively soon I think.
posted at: 10:23 | path: /bongo/community | permanent link to this entry
We're probably going to see 0.3 this week. I'm planning on releasing 0.2.94 today, at which point we'll be back on SVN trunk, and then 0.3 sometime by the weekend. There are going to be a few known bugs not fixed, but it won't be in terrible state by any means.
I've just realised that there's going to be a lot of Bongo in the community over the next few months. We don't have an events page on the wiki - someone needs to fix that! - but just to run-down through them:
So, that's five meetings across three different continents in the next three months: no excuses! I'm expecting photos and/or videos from each of these events :)
posted at: 10:23 | path: /bongo/community | permanent link to this entry
I sent this link around the Bongo people on Mugshot, and on IRC, but here it is again for the benefit of people watching our planet: the future of Thunderbird seems to be a bit cloudy, Mozilla Corp. seem to have realised that it doesn't quite fit it.
Primarily, this disappoints me, because I always saw Tbird as a natural fit to the Bongo Project. We're developing a kick-ass webmail, and that will appeal to a lot of people, but there are other people - like me - who simply don't use webmail. The business I run uses Thunderbird, and (coupled with Lightning) we're pretty happy with it, even though there are a lot of rough edges for a business. I had hopes for a Bongo add-on to Tbird which would automatically set it up for you, with your calendars and contacts and stuff, and work well with your server-side config. Ideally, you'd be able to edit your mail filing rules, get the server to mark mail as spam, and setup vacation response.
Plugging into Bongo would also tap Thunderbird into all the stuff we want to do: getting contacts from your social networks, being able to forward web links to your calendar to friends who use webmail, that kind of thing. It would open Thunderbird up.
It does seem, though, that the Thunderbird project is suffering a major identity crisis. Who would use Thunderbird? And, if it needs to stand on it's own two feet, who's going to pay for it? Personally, I refuse to cede the mail space to the likes of Gmail: yes, they're very popular, but for heavens' sake - email is a huge market, and there is a lot of money to be made there.
I worry that the discussion over the format of the corporation is really hiding the bigger question. It doesn't matter whether it's a non-profit or a business, or whether it's related to Mozilla or not. The key question is, who is Thunderbird for? If it keeps trying to be an Outlook Express-level mailer which people use to access their ISP's POP3 account or something, frankly, it has no future. That mail usage pattern is pretty simple, dying out, and people are not going to be willing to pay for it.
I further worry what this will do to an already-small Tbird community. How will the Lightning people respond to this? What is Penelope doing? Development has happened behind closed doors at the best of times, to an outsider like me, and I've never understood the roadmap. As a long-time Thunderbird user - basically since it was barely usable - I'd like to say it's come on in leaps and bounds. But, it doesn't feel like it has. And looking at the roadmap isn't terribly inspiring: lots of talk about Gecko versions, not much talk about other stuff.
I commented on Mitchell's blog that I think Thunderbird is too scared of competing with Outlook. Note, competing with doesn't mean copying: lord knows, we don't need another Outlook, but Tbird misses out on huge features simply because it won't integrate with servers. E.g., if you want shared contacts on Tbird, you have to use something like Plaxo - which then ties you into a proprietary service. Tbird simply has to offer more integration: it cannot be it's own little world on the desktop. Having your own anti-spam is great; not being able to integrate with the training on the server-side sucks. Ditto junk, conversation views, tags. Lightning needs to be a first-class component. Without offering these types of integrations, I don't see how Tbird can generate revenue: if it needs to be a commercial product, and it sounds like it does, it needs to be something people will pay for.
posted at: 10:23 | path: /bongo/community | permanent link to this entry
A while ago, I wrote some thoughts on Thunderbird in the aftermath of the announcement of the new MailCo and the resignation from Mozilla of two key developers. I was disappointed at the time, but looking back at my post, I don't think it was particularly negative - if anything, I was pointing out how vital I still thought (and think) it is that Tbird be a successful project.
Having exchanged pleasantaries with David Ascher over the past couple of days, and read what he has to say about Tbird, I'm much less disappointed and much more hopeful. Tbird is important to Bongo for a variety of reasons, particularly having a good standards-compliant client for Windows users is crucial. Over time, I'd like to see Tbird become an exemplar client: it should be possible to have mail, events and contacts all integrated, and take full advantage of server-side functionality so that the sum of bongo + tbird is greater than the sum of the parts. Implementing server/client standards, like mail, is a chicken and egg problem. Having both the chicken and the egg drive each other forward (is that stretching the metaphor too far ? :) can only be a good thing.
It will be interesting to see the vision for Tbird develop over the coming months, and I hope that Bongo can play a role in that in some shape or form. At the very least, I've told David that I will keep a more active watch on Tbird, and hopefully Bongo can contribute much more than that.
As an aside, David is also collecting a list of interesting e-mail experiments - more grist to the mill, as it were. For what it's worth, any links I come across like this I usually post on the Bongo mugshot group - I know I'm not the only member; other people should feel free to ping stuff on there too :)
Finally, a note on M3 - Pat has done some great work tracking down various bugs, Jonny's CalDAV work has been merged in, and the list of blocking bugs isn't too bad. Another release will probably happen this weekend after we've knocked a few more off the list, and the full release should probably happen at the end of the month. It's starting to take real shape now after the madness that has been getting rid of MDB, and that's very pleasing to see!
posted at: 10:23 | path: /bongo/community | permanent link to this entry
From Friday until late Monday night, I will be on retreat and entirely away from the internet. No, Bongo hasn't scared me off, I will be back :) So, I figure that I'm going to try again my "pretty please" act to get people to look at some bugs: it worked great last time I tried it, and I can't remember why I didn't try again. Of course, many people won't feel competent enough to look at some of these bugs, so there's an extra-credit task at the end of this coming up in a further blog post :)
I'll likely drop back in on Monday night, but I won't otherwise have internet from tomorrow onwards, so don't expect me to be around (except that my irc client will probably stay logged in, but away). See you all then!
posted at: 10:23 | path: /bongo/community | permanent link to this entry
I was hoping we'd have an initial attempt at M3 by now - we're so close, honestly, but just not quite there yet. Annoying, even for me, but good things come to those who wait.
I'm acutely aware of one problem: our up-front decision with this project was to lay out a road to 1.0 that we thought was achievable, and basically just do it in milestone steps. Between M2 and M3, this hasn't really worked out in the way I would have wanted. From the outside, I assume that it looks like the project has slowed down a bit.
From the inside, I know that isn't the case, but that's not really proof :) So, pictures being worth a thousand words, I fired up my OpenOffice.org and fed it some data from subversion. It's a bit big to put inline, but click here to see what I'm talking about. Basically, it shows how the svn revision has changed over time: the steeper the line, the greater the activity.
There are a couple of things we can take away from this:
In the background, M3 is turning into something very different from what we initially planned. We didn't want to replace MDB, but ended up having to bite the bullet - I think we're much better placed for it. We're going to have working caldav support, pluggable authentication, much more flexible mail aliasing, and a host of other features.
I'm really looking forward to M3. I see this as the "last peak" almost: the road to 1.0 ought to be relatively downhill from here. We have a bit of a bump in terms of the plans for Dragonfly, but I don't feel those are quite the same kind of core change that our project has been through, and I hope that we're now going to be much more about filling out features, polishing, documenting and debugging, which is exactly what we need to be doing for 1.0.
posted at: 10:23 | path: /bongo/community | permanent link to this entry
For a long time, I've been wanted to do some serious research into the economics of free software: there is an awful lot of opinion about how you can and cannot make money out of free software, and not really much fact (or, at least, no-one has looked at the experiences of companies who've been in this industry for the last 10-15 years).
No-one has really gone through the history of all these businesses and tried to construct some picture of how businesses have done. There are examples of businesses that are pretty successful - Red Hat, Google, MySQL - are there are examples of those who haven't been successful. Progeny are the most recent example, and are why I'm thinking of this right now - I just read the article on Linux.com about their demise.
The case of Progeny is really quite interesting. It sounds like the advent of Canonical helped make up some people's minds - Murdock in particular seems to imply that he realised the game was up. Given that Canonical don't even plan to break even until 2008 (and I'd be somewhat surprised if they did by then), it's obvious it would be difficult to compete against someone so well funded and with such a similar plan.
I found a comment at the article a bit off:
"Earth to Galvin: in services, your "IP" is what your employees know, not the secrets of your widgets."
When two of your important employees have just walked out the door, your "IP" has just gone as well - to that extent, I agree with the comment. But I disagree with the rest of it - that kind of "IP" isn't something you can invest in, unless you're willing to invest in it on the basis that your investment could turn to nothing overnight.
I think in a way, that highlights a problem that free software companies face. If you don't have something which sets you apart, there isn't much reason for new customers to come to you, and if someone else starts doing what you do better, there isn't much reason for old customers to stay with you. From the point of view of the customer, they obviously like the idea of competition, but from the point of view of the company, it's not a great situation - in many ways, the free software market is very commoditised, and commodity markets are rather different to traditional IT.
My gut feeling is that free software is pretty difficult to fund per se: most of the success I see is free software almost as an industrial by-product, produced as an adjunct to some other process. But, it would be interesting to do an actual study.
posted at: 10:23 | path: /freesoftware | permanent link to this entry
So, this is the extra task I alluded to in my last blog post, but don't let that put you off!
I'd love to hear from people about how they use Bongo now, and how they want to use Bongo in the future. Especially, I'd like to hear from people who are watching this project but not participating more directly. So, to that end, I've kicked off a discussion on the forum: Where do you (want to) use Bongo?
There is no burning reason behind this, but I'd really like to have an idea of where people see the software being useful, and in what scenarios they'd think about deploying it. It's fine to say "I'm waiting for (X crucial feature)", and it's not likely to make me work on that crucial feature because they're different for everyone ;), but I'd really like to hear what kind of organisation you're in, or what tasks you'd want to use Bongo for, or why you're not happy with your current system.
Everyone can contribute to the forums, so no excuses! There are no right or wrong answers, and I imagine everyone will have a different reason. I've posted mine, so please post yours :)
posted at: 10:23 | path: /bongo/feedback | permanent link to this entry
Well, I'm sorry to announce that my prediction was about right: we didn't get into this year's Summer of Code. From what I've seen, this year was extremely competitive, and many good projects didn't get in.
It looks like only one project similar to us got in was OSAF/Chandler - they are a returning project, so that's not a huge surprise. They have some nice looking Javascript ideas which I don't think are very applicable to us, sadly, but who knows. They also have a natural language parsing idea which was something I wanted us to do, so I need to investigate what they have and what they're doing.. I hope they're doing it for international languages, but we'll see.
I'm also really happy to see that some of our sister projects at the Conservancy got in - Mercurial and OpenChange, as well as the Software Freedom Law Center itself. It's also nice to see that a number of projects aren't simply technical challenges, but software supporting people working on some of the toughest problems in the world: poverty, illness, education, that kind of thing.
Leslie offered feedback to projects, but unfortunately wasn't able to say too much to me: her response was roughly, "Another example of not enough space: good ideas, good docs, but we can't take everyone." In a way, I'm glad: I did everything I could to give Bongo the best chance, and one year our number will come up. Again, she asked that we apply again next year, and I have some ideas about how to improve our application even further: it will need more preparation, but I think we're almost there in terms of getting selected. We just need that final push to make us irresistable somehow :)
posted at: 10:23 | path: /bongo/community | permanent link to this entry
Recently, I became very annoyed at Ubuntu. The plan for the next version of Ubuntu, "feisty", is to include the binary graphics drivers that we all know and hate by default. Ubuntu has done a great job of creating the impression of a community-driven grass-roots distribution, which has left me feeling somewhat sold out and while it's not outside the wording of the Ubuntu licensing policy it feels like it's outside the spirit.
Now, in general, I don't have a huge issue with people needing firmware or binary drivers. There is a vast amount of hardware out there, specifically the current crop of video cards, which just won't plain work with the free drivers. It sucks, but that's life: I would prefer someone with an nVidia laptop to use a non-free driver and be able to use a free software OS, than for them not to be able to use the OS, particularly where the hardware cannot be replaced. (There are ways of changing this situation, but they mostly related to purchasing, I think).
But, there is a huge difference between giving someone enough information to make an informed decision about their system or automatically installing a non-free driver where you know the computer won't operate without it (very few cases), and this new policy of defaulting to the binary drivers. That's too much. Putting commercial application installers into the desktop was bad enough too, though.
Problem, though: where do I go? As I see it, I have these options:
Fedora. Similar level of polish to Ubuntu, better in some ways, worse in others. I strongly dislike rpm-based systems, having never had much good experience with them, but their "freeness" is pretty strong - modulo a few small pieces. This seems the obvious place to go.
Debian. Less polish than Ubuntu, but somewhat similar under the hood (less similar these days - different init, etc. - but still .deb based, which is good). Similar "freeness" to Fedora, they prefer not to have GNU documentation but that likely isn't an issue for me. More of a problem is the lack of up-to-date software in the stable distribution. I run it on my servers, so I guess I should try it on the desktop.
gNewSense. The new Freedom-enhanced Ubuntu. This would be the obvious choice, but it's very new and I'm not sure I trust it. It's not a current version of Ubuntu, which means there would be little reason for me to run this compared to Debian, and I'm not totally convinced that technically they know what they're doing (esp. since they're rebuilding large parts of the archive). Reports of key pieces of software (for me) not working :( I don't really trust the security situation, either.
OpenSUSE. Eh. Same reservations as Fedora; doesn't really seem to have the same commitment to free software. I could be wrong, though.
Other distros listed by GNU - two seem Fedora derivatives (with no removals), another is a Debian derivative. Doesn't seem to be any good reason to look at these.
That's all I could think of off the top of my head. None of them are necessarily hugely appealing alternatives, but I'm almost certainly not sticking with Ubuntu (even though I can remove/ignore all the crud they want to throw at me). I made a list the other day of all the stuff it does which annoys me:
messing with the stock GNOME desktop - messing with the default Nautilus setup is an obvious example, but it's not the only one;
sticking commercial app installers in the Add/Remove Programs: no thanks;
moving to an installer which you can't use to upgrade machines - with a six month release cycle, this is nuts;
Launchpad. Good grief that thing sucks - I'm sure if you use it on a daily basis it makes sense, but I'm apparently registered twice on that piece of shit and under both profiles it says "Alex Hudson does not use launchpad". Great. It then asks me to guess which e-mail address it has for me, in order to claim my identity. I'll just repeat that: it wants me to figure out which address it has for me, even though I've never used it. I mean, christ.
Advertising for commercial support and "the official book" built right into the desktop. Can I remove it? Hmm, no.
I guess maybe Ubuntu has like til Christmas before it gets blown off my laptop, and that's only because I'm busy.
posted at: 10:23 | path: /freesoftware | permanent link to this entry
So, someone asked me the first good question about the vision for Bongo. To paraphrase, why am I against using the word groupware?
The easy historical answer is that it was almost the rallying call for Hula, and the groupware bad essay by jwz expressed this quite neatly in some ways. But that's not a very good 'why' answer.
For me, the difference is probably quite slight in some respects, but it comes down to who we design features for. Do we design them for people, or do we design them for groups / teams / organisations? In my opinion, the answer has to be that you design for individual people.
A good example of a feature which I don't think is person-oriented is "Read receipts" on e-mails. For those who haven't seen it - and I can't believe there are many - the idea is that by setting a read receipt on an e-mail, the person you e-mail sends you a signal back when they've opened the mail. In some software, that signal goes back automatically, in others a dialog comes up, sometimes it's a setting that controls the behaviour.
From having seen how some businesses operate with regards to receipts, I can't think of another feature which induces complete communication avoidance in people: I know people who are actively afraid of opening their e-mail lest they accidentally read something and inadvertantly tell someone they've read it, implying that they're "actioning" the e-mail. Total loss of control over their time. (You can make similar criticisms of Blackberries too, I believe)
The basic issue with giving too much visibility into someone else's data is that you get that Big Brother mentality, which if it doesn't actually exist will certainly exist in the minds of the more paranoid members of an organisation. It's almost like having your boss stand over your shoulder watching you work.
Are there examples of features I think ought to be in Bongo, but might not appear in "groupware"? Possibly. It seems to me that the aim of groupware generally seems to be to suck people in online, whereas software which is about communication, time management, etc., is actually about getting people offline. Putting in work-flow which manages more of people's work in a groupware system seems to me to be totally the wrong answer, particularly where you have to have "group acceptance" for things to work properly: "Oh, you changed the meeting online to 3pm? We agreed it would be 4pm at the last meeting...". Bongo would never assume that everyone else is using Bongo in the way that Exchange divides people between "Users" and "Everyone else".
There's no really easy way of expressing this, and I don't think that there is a bright-line test for this either. In many ways, Bongo will be groupware, or be usable in the same way, and it will have most of the same features. But I think the focus has to be on the individual, and providing tools for an individual to use, not attempting to mold a team into a specific set of processes.
Of course, that could all still be "groupware". We're just redefining what groupware means :)
posted at: 10:23 | path: /bongo/community | permanent link to this entry
As releases go, 0.3 has been turning out pretty well. I meant to write about this sooner, but certainly for me after the release of 0.3.1 (which fixed some sadly quite obvious bugs), it's been going great guns - on my Xen machine (~128Mb RAM, P3) it's causing basically no load and has been running a couple of weeks without restart (which is rare, because any time I encounter a bug, I will try and fix it which necessitates a restart of Bongo). I have several thousand mails in my account over that time, and while there are a few oddities with mail parsing it's basically fine.
Spreading the good Bongo word at recent conferences in Australia and the US has been good to see - Alex Hixon at Linuxconf.au, and Stu Gott at SCALE - our hero Stu is pictured here with (I think) Ken VanDine to his right and Kevin Harris to his left (apologies if I have this wrong; I've never met any of these guys). The green shirts are the uniform of the Foresight Linux project, which is an rPath project in the same vein as the Bongo VM Stu was demonstrating - Stu is also an rPath guy in disguise, and Bongo was very lucky and grateful that rPath lent him to us :)
The road to the next release is going to be an interesting one. On the server side, we have few functions left to put in place: aside from reworking bongo-manager, the main tasks will be continuing i18n integration, bugfixing, documenting, and less interesting stuff like that. At some point, we also need to give the code something of a security review also, but our "exposure" is relatively limited for the most part due to the modular design.
The web UI is really the difficult piece of work left, but in terms of scale it's not the same kind of work that removing MDB was. It's more familiar territory, and more developers will be involved - so hopefully it won't be as fraught a process :)
posted at: 10:23 | path: /bongo/misc | permanent link to this entry
A couple of people now have asked how the store works. I've written a basic introduction to the features of the store, which is now available on the wiki - it's called Store Overview, somewhat unsurprisingly, and should give people a reasonable idea of the basic structure.
We do have a store protocol guide too - linked within the overview - and although it's both incomplete (the SEARCH command isn't documented) and incorrect (stuff like HELP isn't implemented, though that's not a huge disaster!), it's actually very good and only needs a small helping hand. (It would be a great job for a new bongo hacker ;)
posted at: 10:23 | path: /bongo/misc | permanent link to this entry
Since my last techie post on locks was appreciated and since I'm still crook, here comes another one. Again, the focus on making Thunderbird work better with Bongo was the main aim - after my locking changes, I found that we ran into a whole new set of problems: mostly, that the IMAP agent wasn't responding in a timely manner.
So, my next step was to address the progress reporting with a small new piece of code which sits in the various command loops inside the IMAP agent. Basically, as we spin around doing work, it checks to see when it last spoke to the IMAP agent, and if it was too long ago it responds again. I also found out that there was some code already in place which did similar, but it wasn't working because the socket wasn't being flushed: in non-techie terms, the responses were waiting on the server because the OS hadn't sent the data to the client. So, in those cases, the data is now sent, and things are happier.
The Tbird problems aren't totally done with yet - I do occasionally get errors still which I'm trying to track down - but it is much better. Certainly, copying large amounts of mail around is much better, and setting read/unread status works much more quickly. It's getting there as being actually usable long-term: I now have some tens of thousands of mails in my system, and it works, which is good news.
Going further with the "actually usable" thing, one thing I wanted to get some better information on was the performance of the Store in various places. From having read the code, I knew there were some places in the code which were obviously poorly performing: the algorithms in use are very naive, and there are much better ways of doing some things. But in order to know where to look, I decided I needed to instrument the Store a little bit. I've written a patch which I'll send along to -devel, but let me show you a little bit of output:
Command COPY took 0.726920 seconds.
- Out of band messages (0.000033)
- Starting document copy (0.000295)
- Files linked (0.033004)
- Processed doc (0.024436)
- Parse contents (0.602086)
- Index contents (0.049533)
- Commit transaction (0.016867)
- Finished processing (0.000047)
What happens is that every time a command is run, and during the command run, it pockets away little bits of data about the time performance so far. With enough bits of logging inside you get a reasonably granular idea of what's going on.
This example is of a particularly poor-performing COPY command. You may think you don't copy mail very often, but actually a lot of operations turn into COPY: deleting a mail is copying it to the Trash, filing a mail is copying it to the new folder, etc. (IMAP doesn't know about moving mail). This command took 0.7 seconds: most of the time it's quicker than that, but that's close to a mail per second. That's horrible performance.
If you'd have asked me before where all the time was going, I would have pointed the finger directly at CLucene. However, looking at the timings above, we can see it's not the main problem: far and away, accounting for 83% of the runtime, is actually the "Parse Contents" stage. The code at that point looks like this:
TCLog(client, "Processed doc");
errmsg = StoreProcessDocument(client, &info, collection, dstpath, 0);
if (errmsg) {
ccode = ConnWriteStr(client->conn, errmsg);
goto finish;
}
TCLog(client, "Parse contents");
(The log comes after the code it's measuring for technical reasons)
So, we can see that single StoreProcessDocument() call is actually the big deal. And what is that call doing? It's actually just filling in the SQLite database (but not writing to it - that happens later, and doesn't account for the speed). Re-generating data we already have is accounting for 4/5ths of the run-time cost. Ouch.
No, I don't have a patch which fixes COPY yet. But I'm thinking about it :)
posted at: 10:23 | path: /bongo/hacking | permanent link to this entry
There's a number of things I've been meaning to blog about recently. Thanks to Andrew, who has been blogging about stuff, people will have hopefully known about the web UI meeting we had recently. While relatively informal, it does feel like everyone wants to move in the same direction: a light-weight reliable UI framework pulling togther useful components to provide a compelling app. We're going to start with "Flasher", the UI for non-Bongo users, and I started story boarding some use cases. The key is simplicity as usual, but after a brief look around I realised something - no other free software package is doing this. A few doing something similar, but not this.
I was also interested to read Ten Ways to make OSS More Humane. There's a lot of stuff in there, but it's mostly stuff I agree with. In particular, I was quite happy to see that we do most of the "do" statements - I thought it was particularly apropos given the web UI discussions - and hopefully none of the "do not". We've had a few new faces in IRC recently, and they've all stuck around: I still think one of our primary strengths is that we're a really friendly community, and long may that continue.
Somewhat worried to read that David Bienvenu and Scott MacGregor - the driving forces behind Mozilla Thunderbird - are both leaving the Mozilla Foundation. After all the discussion about setting up a new "MailCo" to drive the product in the same way Firefox gets driven, I was hoping for great things. Now, with the very boiler-plate text on both their blogs and both wishing MailCo all the best, it seems they are not to be involved. I really hope I'm wrong, but I would assume that with MailCo on the table they wouldn't jump to some other Tbird vehicle - so it sounds like they'll both be reducing their involvement in the project. That's the last thing Tbird needs, and I wonder if that does spell the end.
posted at: 10:23 | path: /bongo/community | permanent link to this entry
Within this post, I mention the B-word (business) a few times. Don't let this put you off, please ;)
One thing I've noticed when meeting up with other free software users and developers is that a strikingly high proportion of them are involved in business in some way: either owner/managers of their own small business, or having a key role in one. Not most of them by any means, but a much higher proportion than any other group of people I meet (which perhaps says more about the company I keep - I don't know!). People who are involved with business will know that occasionally you do less day-to-day stuff and think more strategically - at least, I'm told that's what you're supposed to do - and the one I'm involved in is no exception. Part of this strategic thinking is revisiting the vision of the business(es).
So, I'm preparing for this meeting this evening, and I'm finding some of my tasks for this meeting pretty tough. "Blue sky thinking" is pretty hard, if you want to do it right: you can think up all sorts of ideas, but not matter how grand sounding they are, they can easily sound hollow and meaningless. As an example, Microsoft's vision statement is (or was, I can't find it on their site) "A computer in every home". Not, "To sell lots of copies of Windows" or "Provide powerful software to let users to more" - neither of those statements is as meaningful, emotional, or as motivational. And no matter what your opinion of Microsoft, they're coming pretty close to achieving their vision, at least for the western world.
There's plenty of other examples. You don't have to be a business, either - I'm told Tiger Woods has a vision statement. Again, not "A very good golfer", or "Win many tournaments", or even "Be the number one golfer in the world". His is (allegedly) "Be the best golfer ever". If that's not a vision, I don't know what is.
Getting back vaguely on topic. I'm finding these tasks tough, as I mentioned before. So, I'm thinking, maybe it would be easier to think about developing some for Bongo? It's easier in a way, because it's much more specific, so I got to thinking, and this is what I've come up with.
Vision statement: Be the recognised benchmark for personal collaborative software.
This is slightly wishy-washy, and could use improvement. What I mean by this is basically this: in the same way Word sets the benchmark for wordprocessing, and Apache does for web serving, Bongo should set the standard for collaboration, and be recognised by name for that. By "personal collaborative software" I'm kind of grasping around a little bit, because I don't want to say (and don't mean) "groupware". I mean personal tools which enable you to organise yourself and co-ordinate with others: power over your inbox, over your calendar, and your contacts. The word "personal" in there is important I think: we're centred around people.
Mission statement: The Project creates standards-based software for managing personal communications and organisational information, accessible online and from the desktop, aimed at small office / home office type users. Bongo is adaptable and versatile, while maintaining simplicity and usability, so people can use as many or as few of its features as they are interested in. The user experience is integrated, and doesn't offer simply the "lowest common denominator" or "most interoperable" functionality at the cost of user productivity. Bongo will continue to be completely free software.
This is more wordy, but is more about "this is how we will achieve our vision". Arguably, the scope of userbase I've offered above isn't enough to achieve the vision, but we can always revisit this again in the future :)
Culture and values: Bongo is the friendliest project on earth, and offers a uniquely accessible community of development. We aim for technical excellence and practicality while having fun developing the software.
We've actually said something a bit like this before: in the contributor agreement, I specifically wrote in from the beginning that friendliness was crucial. I'm not sure I needed to write it; I've never needed to remind people of it, and we have a community which has never suffered a flamewar, which is really great.
Now, none of the above is set in stone in any way - this is purely a personal exercise for me at the moment ;) But, I don't see why we can't adopt something like the above - I think it will tell people a lot more about what our project is trying to do. So, feel free to discuss this in the forums or on the mailing list, and if you know of any free software/open source projects who've already adopted a vision or mission statement, please let me know - I did search around and couldn't see anything.
posted at: 10:23 | path: /bongo/community | permanent link to this entry
It has taken a while for the paperwork and stuff to work through, and I'm sure people who followed the discussion on our mailing lists will have almost entirely forgotten about this by now - but today's big news is that Bongo is now officially a member of the Software Freedom Conservancy. This is strangely exciting - it's another step forward for our project which re-affirms our commitment to what we're trying to achieve.
Being part of the Conservancy will make it clearer to everyone that we're dedicated to a free software stack. Indeed, if we did anything non-free, we would probably get thrown out, since it would jepardise the tax status of the overall organisation.
We're also going to be under the same umbrella as some seriously cool free software projects: Samba are a well known member, but the related OpenChange project (attempting to implement the Microsoft MAPI mail APIs) is also there - and the Conservancy strongly encourages its member projects to interact and help each other.
In terms of what you can expect from the project and how it operates, basically nothing will change: it puts our project on some kind of official legal footing, being part of a US 501(c)(3) organisation, but the project will continue to operate as it has done. Pat Felt and I are the contacts with the SFC itself, but that is really an administrative thing.
It does mean that we can accept tax-deductible donations from Americans now. At some point, I will set up a donations link on the website, which will allow any one to send some money our way. I'm not under much illusion that we'll be getting much any time soon, but it will be interesting to see what happens. Having a little money about would actually help a great deal - for example, membership of standards organisations like CalConnect (who run a CalDav interoperability workshop each year) would cost Bongo a minimum of 00 annually. Realistically, if we wanted to send developers to a workshop, we'd actually be talking about double that or more. Getting sponsorship for that is well within the realms of possibility, and that's the kind of thing I see us spending money on.
If you have any other questions about our membership of the Conservancy, please drop me a line either privately or on the lists.
posted at: 10:23 | path: /bongo/community | permanent link to this entry
It has been a little while in the making, and the construction signs are still up, but we're almost there. After our community IRC meeting earlier this month, we decided that the uncertainty was a problem, and Bongo is the result.
I think this is working out well for a few reasons. We've seen more activity and participation this month than I think probably in the previous six or more beforehand. But also, we seem to have a strong idea of where we want to go.
So, what's the scoop?
As usual, the conversation is happening on the irc channel (#bongo on irc.oftc.net) and on the bongo-devel list (see the GNA! project above) - and you're welcome to join in!
posted at: 10:23 | path: /bongo/misc | permanent link to this entry
There has been a little bit of Bongo hacking. In an ideal world, Pat would get with the programme and get himself a blog. People should keep bugging him on IRC to start blogging, and he could be writing some of this stuff instead of me :)
First up, the big connio-on-GNUTLS patch has landed. Pat has worked extremely hard on this, and while I've helped out on some setup stuff, it's basically his work. This means that connio-using agents now use GNU TLS as their secure sockets and TLS implementation (pretty much, SMTP is the only renegade).
In practice, this won't make a huge amount of difference to the end user, but it does mean we're both GPL-compliant and secure again.
Like a machine, Pat is now looking at the SMTP agent to bring it into line. This is awesome work, and will hopefully mean that our SMTP implementation is much simpler than it was.
I'm still hopeful that our M1 release is more or less on schedule. To be fair, we're not working directly towards it at this point: there are a few things which ought to be part of later releases which are being helped along, but there are three main features we really need for M1:
It's a fair amount of work, but possibly sounds worse than it is. I'm hoping M1 will be a pretty useable release for developers, and I intend to migrate my Hula server to Bongo with it (technically, now we have GNU TLS, I could do that now). Later milestones will be a bit more straightforward.
posted at: 10:23 | path: /bongo/hacking | permanent link to this entry
Planet Debian has been a trove of green discussion today. Russell Coker and MJ both linked to a story about Spain getting most of its energy from wind, for the first time. 27% of their energy came from wind, 22% came from nucular power and 16% from coal power (where the other third came from, I'm not totally clear on - possibly imported?). Overall, in the last year, almost 10% of their energy has come from wind. I think that's pretty amazing, especially since it meant for that small time, almost 50% of their power was coming from relatively carbon-clean sources.
Spain is one of the few places I "boycott" for environmental reasons (along with Kenya and a few other places, I refuse to buy products which require large amounts of water to create - for example, salad crops), but it seems we could learn a lot from them. I do wonder, though, if the large amount of wind power is somehow related to the more relaxed planning process over there, as that's not something I would want to copy in many ways.
MJ also mentioned water temperature again, which has been a discussion on there recently. It's also something I've thought a lot about, because our house has amazing heat potential in the roof (we actually have a hell of a time cooling our house in any kind of warmth - it's easily comfortable at this time of year, and by April/May will start to get slightly uncomfortably warm). However, we don't store hot water at all - our system is powered by a combi boiler, which is virtually impossible to integrate into solar heating systems, and is too new to be replaced (we think the previous owners had it installed maybe two years before we moved in just over a year ago).
I was thinking about the temperature controlled valves that are now mandated in various places (I think the US, and also Australia?), but there are issues associated with them. The basic types of valve are simple pressure balancers, which work ok, but means that the water takes a while to warm up. If it takes 5 seconds for the water to come hot from cold, and the valve mixes 60:40, it will take up to 10 seconds for the water to come to temperature because it's mixing in much more cold water - that's almost double the waste. The posher thermostatic non-balancing types are much more expensive, require servicing, and have similar wastage issues. I think it's better to do it manually at the moment (though I wish there was a better way), especially if your hot water has to travel a reasonable distance.
Standby switches, mentioned by Adrian, are also something I looked into recently. In an ideal world, I would like to have remote control sockets: by allowing me to power off entire four-ways, we could keep some of the convenience (the sockets behind the TV are especially inaccessible, so it's difficult to turn things off properly), but save a good amount of energy (the sat box in particular doesn't seem to "turn off"). However, the stuff to do the job doesn't seem to be available.
I don't know whether it's because switching mains in general requires physical switches to cope with high currents (most equipment turns off the DC parts, leaving the power supply more or less on - just consuming a lot less power), or whether demand just isn't there. It seems to me, though, high time that equipment and mains sockets started talking to each other so that standby functionality can be delegated elsewhere.
posted at: 10:23 | path: /green | permanent link to this entry
A little while ago, we decided that we would put some system configuration into the store, and rely more or less on access controls to govern who could read/write that data, which is quite a nice flexible system. However, although I knew the ACL system existed, I didn't actually know anything about it, or even if it worked.
Like most things store related, the code is reasonably clear, and when you try it you find it basically works. I rarely leave the store code in Bongo feeling unimpressed, which I suppose is a good feeling!
So, I documented what I learned about the ACL system on the wiki, and that resulted in this patch for bongo-config (mostly). Before, without ACLs, even logging in as admin meant you couldn't read or write the Bongo configuration in the store - it just refused you access. Contrast with now:
= $ telnet localhost 689
Trying 127.0.0.1...
Connected to localhost (127.0.0.1).
Escape character is '^]'.
4242 NMAP <b7237b90laptop.alexhudson.com462e5b4d>
= AUTH USER admin ****
1000 127.0.0.1
= STORE _system
1000 OK
= COLLECTIONS
4240 Permission denied
= READ /config/manager
2001 nmap.document 654
...
= QUITEverything marked '= ' is something I typed in. So, you can see, we switch to the _system store, and try to look at the contents with a collections command - this fails, we don't have permission to do that. But we can read the bongo-manager configuration file.
Next stop, a basic but working Hawkeye :)
posted at: 10:23 | path: /bongo/hacking | permanent link to this entry
So, recently a lot of stuff has been going on in my life - displacing the lower status stuff like
hacking Bongo, painting the house, washing, shaving, that kind of thing. While I've tried to be
online to cajole inspire others, that doesn't really work - apparently my code talks better
than I do. Ho hum.
More bad news - it's looking increasingly likely that I'm not going to get to Ohio Lugfest in the States like I planned. The choice is coming down between "Ohio LugFest" and "New Kitchen", and somewhat surprisingly my better half is demanding the improved culinary facilities. I'm still going to hit the States this September, though, I just won't be able to stay quite as long - but I will try to make a point of seeing people while I'm over.
So, onto some good news. I don't think I mentioned this anywhere except IRC because it's not totally 100% golden yet, but Bongo will be at the UK Linux Expo 2008 which is happening this October (23rd-25th in London Olympia). We have a stand and I've been talking to some of the GNOME UK people about how we can help each other out. I've been to most of the UK Linux Expos, in London and elsewhere, and they're generally extremely good fun and a great opportunity to meet up and make things happen.
Slightly more good news. I've finally got my head around this store patch which has been evading me. I'm going to save the technical details for another post later this week, but basically it comes down to a realisation that I had the other day. A number of times working on this I've realised my approach was wrong (and almost unworkable, sometimes), so I've had a couple of false starts - this weekend I realised that my ideas were wrong, which is worse.
The problem is this: we have store commands, which previously would do some relatively complex things behind the scenes: query through the SQL database, run some stuff in the Lucene index, that kind of thing. Now most of the complexity is basically just hitting the database - virtually all of the time, what we're doing is translating store commands into some kind of SQL query. And sometimes - this is the harder bit - we also want to add some bespoke query parameters to our commands, to get back even more specific data. I wrote about the problem of parsing these bespoke queries in my last blog post.
What hit me, and seems totally obvious in hindsight, is that I was trying to translate store commands into SQL, these bespoke queries into SQL, and then kinda merge the two. It's hard, and basically stupid. What I actually want to do is turn every store command into a bespoke query.
This sounds a bit silly, but it actually makes sense. The stuff that we specify in the store commands are basically all properties of the data we want back - e.g., COLLECTIONS won't return documents, and is equivalent to a bespoke query with a "type=4096" parameter. And thanks to the wonder of boolean logic, if I turn our command into a query expression, I can AND it with the bespoke query, and it all joins up and works.
Initially, I worried that this could - in some circumstances - lead to sub-optimal expressions where we do things like duplicate columns, or express the same criteria twice, or something. Actually - I don't think we care. SQLite has a query optimiser like any other SQL db, and because it knows which columns have indexes and other useful information, can probably do a damned better job of optimising than my code ever could. So, I'm not going to worry about that - I'm just going to do it and see how it turns out.
posted at: 10:23 | path: /bongo/misc | permanent link to this entry
Actually Andrew, we didn't do a BOF last year: I did a talk (which was sadly moved at the last minute, and didn't get as many people as I hoped it would), and me and Seb P. had a chinwag, but we didn't have a proper BOF.
So I'm thinking this year we should definitely do a BOF. Whether or not we apply for a talking slot, I don't know - I think we have a lot more to talk about this year, particularly about community transition and perhaps release management :) - but I don't know to what extent it will be different to the other stuff they will have going on. I guess we could submit a paper and let them make that decision, though.
posted at: 10:23 | path: /bongo/misc | permanent link to this entry
We talked about it, and I did the form yesterday - we've applied as a mentoring organisation for this summer's Google of Code. I've never done an application to GSoC before, although I'm familiar with many of the other projects who've taken part. In hindsight, I wanted to say a lot more on our application form, but ho-hum. It would have been nice to find something like OpenSolaris' application before-hand, but I feel comfortable knowing that our application is completely our own :)
I'm not terribly hopeful about us getting in. As a project, we're pretty new (at least, I can understand how people will see us that way), so we have to earn the trust of people first. Also, there are a lot of excellent projects who apply every year and get turned down. Most of the projects that get in are "big names" that we all know and love. But, on the other hand, I looked down the 2006 project list, and I can hand on heart say that we're very different to every other participate (maybe Horde comes closest to us), and that we're offering something at least as exciting to students if not moreso (in my opinion!). So, who knows. I'm guessing we'll hear in the next couple of days.
The fact that our MediaWiki can't connect to the database at OSUOSL, and our website is therefore down right now, is also unlikely to enhance our chances :)
posted at: 10:23 | path: /bongo/misc | permanent link to this entry
So, Bongo didn't make it into Google's Summer of Code, but we did get some nice feedback from them. There's good news and bad news :)
The good news is that they thought our application was very well done, and it was basically because they don't have room for everyone that we didn't get in: it sounds like we would have been acceptable, it just wasn't our roll of the die. Unfortunate, but fair enough - I do suspect that Google have a theme or themes for each GSoC, and it might just be that we didn't fit well this year.
The "bad" news is, there isn't an awful lot they suggested we could do to improve. Two specific suggestions were made: that we add a list of required skills to each idea on the wiki, and that we classify them easy/medium/hard. Both of which are great suggestions, although I thought that our ideas were already better formed than other projects. But, I'm comparing us to some other projects which are probably "shoe-ins" (I get the sense many of the bigger projects know that they're getting in every year, and don't make the same effort).
They also said that they'd welcome an application from us next year, which I take as meaning that our application stood a good chance of getting through. We might be better prepared for students next year, anyway :)
posted at: 10:23 | path: /bongo/misc | permanent link to this entry
I recently wrote to bongo-devel about a problem in our store protocol: in the trunk version, some commands have a "query" parameter which is unspecified in our documentation. The reason it's unspecified is that it turns out that the parameter just gets passed through to CLucene, which as well as being something of a security issue also makes it more difficult to tell people how to use it.
In the experimental Store (coming soon to a server near you!) the CLucene index has been removed, and the query interface to it therefore doesn't exist any more. It's quite possible that CLucene will make a come-back, but the query interface is something I want to get rid of: for no better reason than it requires us to duplicate information from the SQL database to the Lucene index.
Because we need a parser for these queries which is simple, safe and isn't going to do nasty things with our memory allocator, I proposed a version of prefix notation / Polish notation, and have written some code which parses this into a tree structure stored in an array. The code is pretty straightforward, so I've documented the algorithm here - I'm sure it must be a known algorithm, but I've been unable to find any examples of anything similar.
In general, prefix notation isn't used as commonly as postfix notation. This is mainly because it's more efficient in terms of stack space to evaluate an expression in postfix notation, because the constants come before the operators and you can start calculating earlier. However, we're not evaluating an expression - we're going to turn it into SQL - so that isn't a benefit, and having the operators come first means we can find places to later store the constants simply.
The next step is to transform this parsed expression into something we can throw at SQLite. This is actually much easier said than done: for example, we have to be able to handle arbitrary properties on documents, which are stored in a separate table with a one-to-many relationship between the document and its properties.
The problem with that is that it's not that natural to say something like "give me rows from table A where there are rows in table B which relate to it and some of those rows contain X and Y". So, this may turn out to be something quite hard or even impossible to provide a general solution to if the logic is quite deeply nested - but ideally, I want all of that nastiness hidden from the client.
Some readers may be asking how the current Store does this. Effectively, it runs the query against the index with the properties it has at hand, and retrieves a list of Store guids of the documents which have those properties set. It then queries the SQL database for each individual guid to see if any of the properties there match the query also, and if so it outputs the match.
posted at: 10:23 | path: /bongo/hacking | permanent link to this entry
Unbeknownst to many, it seems, Microsoft are pushing a new referrals scheme for preinstalled Office 2007s. The "highlights":
You can see these new types of licences looking at you usual web store or something. You don't get an install media or anything; you just get a number to type into your PC.
From the point of view of the customer, there are some downsides: in particular, if you lose your original hard drive, it sounds like you're going to be quite screwed. You've lost your pre-install of Office, and presumably you're going to have to pay for a media from somewhere, or something, assuming you can even reinstall it. Of course, if you're the usual OEM customer, you've lost your entire operating install recovery partition too, so perhaps the loss of the Office suite seems minor in comparison.
From the point of view of free software, this is a worrying development. People often underestimate Microsoft, and the common refrain heard is "How can you compete with a free product?!". It seems Microsoft have found a solution: you give people financial incentives to spread it around as far as possible.
Not only does it mean that Office will be much easier to install - after all, OpenOffice.org is a 110Mb download, and even in this age of broadband that's still a fair amount of effort - but it means that OEMs will be doing post-sales work: they can still make money from customers who choose to "upgrade". That's going to be very difficult to compete with.
Is this bundling? Technically, I suspect it's not: it's just a very attractive offer. In reality though, it's extremely similar, and the effect is just the same. And a rose by any other name ...
posted at: 10:23 | path: /proprietary | permanent link to this entry
I noticed "boycott novell" made mention of Bongo today - they mistook our M3 release as the announcement of the project - but nice to see another mention in a new place. I'd like to comment slightly on the Yahoo! situation, though.
The boycott site is quite wrong when it talks of "pulling a Hula": Hula was never sold because it was competitive to Microsoft Exchange. Anyone who was involved with the project, or used Hula (it's still available in Ubuntu!), would know that's not the case. But, unintentionally, there is a point there: the fact is, Microsoft could/would be in a position of owning two groupware systems, which is similar to the position Novell found themselves in (actually, not as bad, because Novell originally had three ;).
If Microsoft do take over Yahoo! (and that doesn't seem certain to me, yet), they will be in the position of effectively having two products in the same basic market twice over - Exchange and Zimbra, and MSN Mail (or whatever it is - Hotmail?) and Yahoo! Mail. And for the most part, a business selling two products which do the same thing ends up doing not much more than confusing its customers.
My guess: First, Yahoo! Mail will get merged into the Microsoft version, somehow. I'm sure the UIs will be slowly aligned, and then people switched over without realising it.
Zimbra is a tougher call, though. I don't expect them to kill it. However, I also don't see them keeping it - if I was to guess, I would say Zimbra would get spun out into a standalone company. I don't think Microsoft is going to want to piss off a good number of customers, I don't think it will want to keep Zimbra, and I don't think those customers want Exchange. So the obvious answer to me is, get rid.
However, in that scenario, it may well end up a bit like Hula - the people involved in the "open source" side of the software may effectively find themselves in a whole new world they didn't expect themselves to be in. Whenever you have a software project that has a single corporate sponsor (if you will), you are always in that situation, because that type of sponsorship has to be continually justified.
In virtually all ways, this doesn't affect Bongo: if those people using Zimbra right now have to change (and I'm not at all convinced they'd have to), I'd say they are more likely to jump to Exchange than Bongo. But, I sure feel a lot of compassion for the situation: with two disparate communities (commercial users and open source users, in this case with slightly different products) there is always going to be one side which "loses out" somehow.
posted at: 10:23 | path: /bongo/community | permanent link to this entry
... today is World Water Day. There is a flashy 2007 website too, not sure why.
posted at: 10:23 | path: /green | permanent link to this entry
Sorry - another slightly technical post. This is yet again another visit to the subject of locking, which I briefly described the problems with a while ago, and then wrote some new code to help alleviate the problems.
One thing I didn't explain at all, looking back at those posts, is the different reasons for locking. We can simplify this down to two major reasons:
Those problems seem very similar, and indeed, our previous issues are mostly to do with the fact that those problems were conflated: the SQLite locking was used to ensure external consistency in the store. That's actually a really common - and often desirable - way of doing things. The way it works is you wrap all your SQL operations in an SQL transaction, and whether or not your operation (e.g., writing a file) succeeds stands or falls on whether or not the transaction succeeds. The database guarantees that when you use a transaction, you can make many alterations at once but no other database client gets an intermediate view of the partially-written data.
However, at this point, we run into an issue with SQLite. "Proper" SQL servers give you the ability to lock single tables, or even rows within tables, to help ensure that transactions succeed where-ever possible. SQLite doesn't give us that, it basically ensures consistency by locking the whole database. This makes the external consistency above suddenly very expensive to implement at the SQL level.
Now, surprisingly, there is also another set of locks within the store. At the moment, for example, collections are locked at a higher level when you read/write to them, because we also access other data - document contents on disk, the search index, etc. These locks are needed for the external consistency also, and almost make the SQL locks redundant.
So, my work on the store now is mainly to decouple the SQL-level locking from the provision of external consistency. We'll effectively only use the locks at that level specifically when we're reading/writing from the database to ensure that the data is written consistently when it matters (for example, allocating IDs) but not when it doesn't (for example, creating a document and then later adding the correct metadata to it). External consistency will then be provided by the higher-level locks which we already use.
posted at: 10:23 | path: /bongo/hacking | permanent link to this entry
Well, my informal little hackweek is coming to a close today, and it has been really good. The experimental store-sqlite branch is now well and truly merged onto trunk, and I've been running it the past couple of days. It's not managing my personal mail yet, but I will probably switch over this weekend: as well as being a lot more performant, a number of serious issues for IMAP users are now totally gone. You don't have to worry that too much mail is queued up for you any more, or that your mail filters will crap out because of all the re-processing happening on the server.
There is still plenty of work left to do in the store: currently, it's missing a few functions needed to support the web UI correctly, but I'm still planning on making a release soon. The release will likely be some kind of server-only release, for those people (like me) who are mainly running it for the SMTP/IMAP loveliness.
Going back to the dogfood; I've set up the new trunk on shell.bongo-project.org and it's running well. I've signed up to LKML, a couple of other mailing lists (I was already subscribed to OpenSUSE's discussion list, which must have been bouncing, so I've asked Andy Wafaa to pass on my profuse apologies to their ML admin: if it's any consolation to him, receiving backed up posts in batches of ~10 simultaneously really helped me debug a locking problem we had!), and at the moment I have a few hundred mails in there from running just over a day. I've also been testing with my mailbox backup, which is nearer 10 thousand e-mails, and there isn't even a slight different in performance.
I would love to see someone test Bongo against some of the mail test harnesses that are out there, so we have some good idea of how large a mailbox we can handle, how many simultaneous mails, etc. I'm really hoping that people will start using Bongo in progressively more serious situations because the widespread testing is the best way of finding a lot of our bugs.
posted at: 10:23 | path: /bongo/hacking | permanent link to this entry
First things, we just went past r800 yesterday - a commit I bagged myself, woo! Sadly no pretty graphs to show for it, but I think we're back to a decent pace of commits again.
The work on the experimental branch of the Store is a good way through. This branch of Bongo builds reasonably cleanly, and works somewhat - you can deliver mail into it, and get mail out of it via IMAP. It's at the point now where it needs some decent testing while the final pieces are put together.
Somewhat incredibly, the code size is on the way down again - it's already about 1000 lines smaller, and might be double that by the time it's done. Although this is extremely dammaging to our project valuation on Ohloh (which currently stands at some .5 million), it's beneficial for Bongo ;)
What's good is that there is a decent turn of speed already, especially when dealing with large amounts of mail, and things seem to be relatively stable for me copying large chunks around. And this is without doing any performance work at all: I know for sure there is plenty of really low hanging fruit on this; for example, we don't cache SQL statements in a parsed state ready for re-use, and we don't do simultaneous access to the store which should be quite easy now. I haven't timed anything, but Thunderbird feels a lot snappier.
So if you want to do some testing, please grab the /svn/bongo/experiments/store-sqlite branch of Bongo, and run your favourite software against it - trying using it with imap or pop3, or whatever, and see what happens.
posted at: 10:23 | path: /bongo/hacking | permanent link to this entry
I haven't been able to do much Bongo hacking the past couple of weeks - workload has been such that I resigned myself mostly to offering hopefully helpful remarks on IRC, working on the Google SoC application and a few little build fixes. It's been good to see that Bongo is building on a variety of OSes in the OpenSuse build service. It's not a great measure of quality, but it helps.
Pat: I've experienced similar community run-ins recently: both directly (discussions on #gnu about "free speech", and whether or not silencing/banning people is appropriate in any circumstance - no matter how rude they are!) and indirectly (witness Daniel Robbins resigning from Gentoo again - I think the point there about being too focussed on technical skills is excellent). Bongo IRC is constantly friendly, which I think is great, and I know that personally when I interact with a project's channel it definitely influences my views of that project greatly (#mono recently was very helpful for me, which leaves me feeling warm and fuzzy). I hope it's something that never changes about this project.
posted at: 10:23 | path: /bongo/misc | permanent link to this entry
This isn't a great looking URL, but hopefully it will continue to work into the future: this is a Bongo group on Mugshot. At the moment, it's just me - I wanted to try Mugshot out before I got into all the GNOME online desktop stuff that people are working on, but others are free to join in. The web swarm looks a particularly fun idea for sharing links we think are interesting.
In other news, the MDB removal hacking continues - at the moment, it's about 40% gone. Realistically, this means that there are only really a couple of big agents left before we're pretty much done with it. That will be a sweet day.
Also, in order to simplify things, I think Bongo M3 and M4 will be officially single-server only. Though this sounds like a change, it really isn't - Hula never worked properly multi-server, and Bongo has been no different up til now. The design isn't changing, but for now I don't want to worry about multi-server stuff. It just feels like trying to eat an elephant by swallowing it whole.
And, I've also posted my plan to remove connmgr. I have a plan on how we're going to replace some of that stuff, and maybe that will be M5 material too (I'm not really sure if it's 1.0 material per se, but we'll see). Speed-wise, it will blow it away :) But, it looks like M5 might become the big "networked Bongo" release....
posted at: 10:23 | path: /bongo/misc | permanent link to this entry
There's nothing which divides hackers so much as HTML as a format for sending mail. However, a recent discussion on another mailing list makes me think that there is hope for it.
I should state up-front that I'm very much in favour of HTML as a mail format. I outlined a couple of the technical reasons why in my posting on the subject, but it can be summarised mainly to:
For what it's worth, I don't think plain text is going away - in fact, there are many popular examples of it (witness SMS text messaging and, to some extent, instant messaging, Facebook wall posts, etc.). But it's also not the be-all and end-all of communication: the richer presentation and semantic of well-structured HTML isn't to be sniffed at.
Interestingly, I found a project online called Email Standards Project which wants to do for e-mail clients what web standards did for browsers. A lot of what they're saying makes sense, and what I like is that they have this quasi-profile of HTML for e-mail: a kind of "safe baseline".
Where I would differ from the ESP is their disregard for the plain text component. I think to work properly, at least right now, if you're sending HTML you also have to send a rendition in text which is pretty decent. I'm very interested in conversion to and from plain text: that's particularly important in scenarios where you have a mail agent with capabilities different to text terminals, for example, mobile phones.
But it's an interesting project nonetheless, and probably one we should be in contact with.
posted at: 10:23 | path: /bongo/misc | permanent link to this entry
Well, it was last weekend, but I meant to write about it before now. It was nice to have probably the biggest meet-up of Bongo Project contributors that has ever happened, and Project polo shirts are now in the hands of everyone. The weekend was mostly spent looking through the talk schedule (some of which was interesting, some of which was less so), wondering if Nat was going to say anything about Hula, and chatting about Bongo - we didn't go to too many talks, but the Selenium one was probably worth the weekend alone.
It felt like a smaller event this year, and I thought WUSU was a better venue - I like the Lighthouse a lot, but it's not really suitable for talks: the cinema is fine, but the atrium isn't good. I also really, really wish there was some kind of community lounge: the various stands are great, but I don't think they encourage interaction. Less formal tables with chairs/sofas, and the ability to form ad-hoc groups (which happened with the cafe's furniture anyway) would be really, really cool. But, all that said, I enjoyed the event and it was very worthwhile.
The Bongo BOF happened on the Sunday, quite late. Even though the BOF point is a bit out of the way, and there were other interesting things happening, we got some very interested people coming over, some of whom couldn't stay long but were very enthusiastic about what we're doing. I was hoping to do some multimedia stuff, which we didn't really get time to do, but it's perhaps for the best - it needs better planning, really.
Finally, the BOF photo - by this point, only one non-Bongo person was still there:
From the left, that's Jonny Lamb, Chris Lamb, myself, Lance Haig, a participant whose name I didn't catch (but I think is Lee), and Andrew Wafaa.
I'm hoping that we can all get back together again for LinuxWorld in London, happening on the 23rd-24th October this year. I'm thinking by this point we're going to be getting reasonably close to 1.0, which should mean it's a very exciting time, and Mr Hixon has also intimated that we might be able to persuade him to join us from Oz, which would be incredible.
posted at: 10:23 | path: /bongo/community | permanent link to this entry
In general, using shared libraries already available on a system is a good thing. Witness the difference compiling against the system CLucene, rather than having to build your own:
$ time make # with system CLucene, and Bongo r19 real 3m0.256s user 1m39.937s sys 0m29.177s $ time make # against the imported CLucene real 6m27.311s user 4m30.465s sys 0m47.084s
Basically, Lucene accounts for half of the entire build time. If you're lucky enough to have a clucene development package on your platform (on Fedora it's clucene-core-devel, on Debian it's libclucene-dev), you can ignore the CLucene we package and use the system one.
At some point, we'll probably drop CLucene, SQLite and Curl completely - you should be able to build Bongo against system versions of all those libraries. I'm hoping that the build system is now correctly detecting the presence of those libraries - but we do still have some problems on non-pkgconfig using systems. If we can make those systems work, we'll can get rid.
posted at: 10:23 | path: /bongo/hacking | permanent link to this entry
Probably most people will have noticed a little bit of a lull in Bongo hacking. I seem to have spent most of the last month taking care of a close relative with broken bones, which has meant me spending a lot of time away from home. At least, that's my excuse. We totally missed the M2 milestone, but hey-ho. I'm going to wait a couple of weeks before re-profiling, to see what rate progress gets made again.
However, I'm trying to hack again in earnest. Yesterday, I removed the dmc code from Bongo, which makes me immensely happy: it was horrible code, and it deserved to die. Basically we got rid of about 3000 lines of code, and all those 3000 lines did was produce a few numbers like "number of emails sent". The feature:code ratio was ghastly.
Today, I posted a patch to -devel which makes bongo-manager read part of its configuration out of the store. The change is pretty simple, on the face of it. Previously, bongo-manager had a piece of code like this:
/* FIXME: these shouldn't be hard-coded */
static const BongoAgentSpec AgentSpecs[] = {
{ "bongodmc", MSGSRV_AGENT_DMC, 0, FALSE },
{ "bongostore", MSGSRV_AGENT_STORE, 2, FALSE },
....
{ "bongopop3", MSGSRV_AGENT_POP, 3, FALSE },
{ "bongocalcmd", MSGSRV_AGENT_CALCMD, 3, FALSE },
};You can see from the comment that this is a problem. Why? Well, the only agents that bongo-manager knew about were the ones coded into it: if you wanted to add one, you had to recompile bongo-manager. Ouch.
This is now replaced with a document in the store, which looks a bit like:
{ agents: [
{ name: 'bongoqueue', pri: 3, enabled: true },
{ name: 'bongosmtp', pri: 3, enabled: true },
....
{ name: 'bongopop3', pri: 10, enabled: true },
{ name: 'bongocalcmd', pri: 7, enabled: true },
]
}You can see that the structures described are very similar. However, how this is implemented is now totally different. If you want to add a new agent, you can simply go into the store and add it to the JSON document. If you want to disable or enable an agent, you go into the store and change the document. It's very straightforward, and no recompilation is required.
I'm hoping that this is more or less how all agents will work: they will store their configuration in the store, rather than LDAP (as it is currently). This is much simpler a setup. It's much easier to get in there and change the configuration, and you don't have schemas and that kind of thing to worry about.
It's possible, though possibly unlikely :), that sometime in the next week or so we can almost completely get rid of MDB within Bongo. It seemed like a big deal when we made the decision, but it's looking increasingly easy as I realise how much infrastructure for all this we already have. Certainly, I predict that code using MDB in Bongo is going to start disappearing very soon.
What does this mean for end-users? Well, the first user-visible signs of this will be a couple of things. I'm hoping Bongo's antispam and antivirus agents will be the first to make use of this new infrastructure, so we'll be able to turn those back on very soon (we tried doing this with MDB a while ago, and it looked like a lot of work. I tried hacking on it. I didn't get half as far as I have with this new stuff....). Also, we'll be able to start working on Hawkeye, so that we have web-based administration again. And then we'll really start rocking :)
posted at: 10:23 | path: /bongo/hacking | permanent link to this entry
I wasn't going to mention Hula particularly, but since the recent announcement it seems the right time. I've said basically the same thing on bongo-devel too:
At the end of the day, we started Bongo because we didn't want to be reliant on the success (or not) of Hula. I think the inverse is true: the success of Hula doesn't depend on what happens with Bongo. Either or both projects can achieve that success, and that's what we really care about.
posted at: 10:23 | path: /bongo/misc | permanent link to this entry
Chris Lord has posted an essay entitled "Introducing Social Calendaring" which I think is an interesting read for Bongans. It took me a couple of reads of the paper to actually figure out what was being proposed, but that's probably because I skimmed it first a few times with my own preconceived idea of what was being proposed.
I would love to see some of these ideas implemented in Bongo, although I don't know to what extent some of it would work. While I like some of the proposals in the paper, it is a shame that there was no kind of implementation work, because there are a couple of things I really think deserve more analysis (this might not make too much sense until you read Chris' paper):
I'm hoping to read the paper in detail in a couple of weeks, and while it's not something which is relevant for Bongo 1.0 (sadly) I'll hopefully distil some of the salient points into the wiki, and perhaps look at some of these questions myself. I'm particularly thinking about the "thick client" type solution Chris talks about - where data storage is primarily local for offline access, etc. I'm not totally sold on that - for example, I think SMTP has a good balance of decentralised peer-to-peer design without suffering the availability problem, and that's one of the (few) things I like about iTIP.
posted at: 10:23 | path: /bongo/misc | permanent link to this entry
We all have New Year's resolutions in some form or another; I haven't really got mine straight in my head yet but there are a couple of things I'm committing to over the next month or two - one of which is to spend more time regularly hacking Bongo. I had hoped to get some hacking in over Christmas; family events overtook that and as a result I've spent virtually no time on Bongo at all. So, my immediate goals are:
Border suggested that one of us write up a kind of look back over our first year, as well as a look ahead - Pat did a little bit of that, but I do feel the need to take stock in more measured fashion. One goal I set which we totally haven't achieved is releasing often, and I think my main goal with Bongo this year is to do that: if I achieve nothing else, making steady, perhaps reliable, but recognisable progress I think is going to be very important to this project.
In other news, my Bongo hacking time today has been spent in a couple of ways. I've been getting my builds warmed up again - somehow they always bit-rot and there are things to fix - and I've been and renewed the project's domain name. Somehow there were only four days left on it, ho hum.
posted at: 10:23 | path: /bongo/community | permanent link to this entry
I had separate chats with people about MDB yesterday, originally about how we're going to fix a potential security problem with Bongo, but went into a wider-ranging discussion about MDB. For those who don't know, MDB is the LDAP-like API we use to store virtually all configuration.
There are a couple of issues with our code base right now:
It feels to me that we're on something of a sticky wicket with MDB (translations into colloquial English involve creeks and paddles, I'm given to understand).
There are a number of ways that we can think about solving the problems above in isolation, but the more I think about it, the more I think MDB is hamstringing us. The world involved in solving each individually also seems to be less than the work of a more complete solution. MDB was always something we were planning on getting rid of in the long term, but that was going to be a post-1.0 story.
I'm not going to put forward any potential solutions at this point, because this needs thinking about. In an ideal world, I would punt as much of this as possible: e.g., clustering isn't a feature I would be sad about dropping from 1.0. However, I also don't want to see us applying big band-aids: if we built Hawkeye over existing MDB, we're basically doing work we'd end up throwing away later anyway, or at least rewrite majorly.
This needs a lot of thought :)
posted at: 10:23 | path: /bongo/hacking | permanent link to this entry
First post after being "away" - my server was down for a little while too :/ No matter, I wasn't really in a position to blog anyway. Real life has interrupted for a bit, although I've managed a bit of hacking to distract myself.
So, we've had various troubles with IMAP. Somewhat hilariously, although I wrote a message to -devel about this, I did it from my trunk Bongo, which doesn't actually send e-mail since the smtpc code was committed (which I think is a configuration issue over here rather than a bug of Pat's). So, d'oh to that. I'll blog about this instead, then.
In one word - "Thunderbird". Tbird is a pretty smart IMAP client in many ways, and one of the smart things it does is use multiple IMAP connections to do many things at once: for example, doing things like mail filtering on your Inbox in the background while you're reading mails in another folder. The issue with doing many things at once is that you then enter the world of concurrency, which is relatively Hard.
To prevent concurrent tasks from corrupting each other, what I call "locks" are put in place. You can think of these a bit like signals in a railway shunting yard: you don't want to risk collision between trains, so you put signals on the track in such a way that trains obeying the signals will never meet. We have locks around access to store collections, access to the database, access to the text index, and access to internal data structures: all these locks are used to ensure access works consistently.
Following this metaphor, locks are great, but there are problems you want to avoid - one of which is "starvation". If a train waits for the green signal but only gets it rarely, it's effectively "starved of track" - similarly, a process waiting on a lock and not getting in there is starved of a resource - usually, some piece of data.
Starvation was effectively happening in our IMAP. When the IMAP agent is starved of access to a store object for too long, it returns an error to the IMAP client, and the IMAP client assumes that everything has failed and starts again. Sadly, with Tbird, starvation wasn't a rare occurance - it was effectively guaranteed. If Tbird was doing something like moving mail in one IMAP connection, that IMAP connection would work the Store quite hard and would effectively prevent the other IMAP connection from getting access unless by fluke of timing it was able to grab a lock in the short time the other process didn't hold it.
So, if we care about making sure that starvation doesn't occur, you need some system other than just basic signalling. You want a system to ensure some kind of basic fairness. Now, in computer science, you can write about "fairness" until the cows come home, but in this case it's probably good enough that a queue be put in place - that ensures that everyone gets a turn at some point.
It turned out that our existing locking system wasn't able to really do this. In fact, it was pretty complex, for another good reason: it has both "shared" and "exclusive" locks. The semaphore system works like signals in the train yard: only one process is allowed at a time. Let's say the code was written to a "look, but don't touch" rule: if the code isn't altering anything, you can have many of them looking at the same time without fear of corruption, which is good for performance. However, you can't do that with basic semaphores alone, you need to build that on top. In fact, you can look at the previous code and see how complex it is - a lot of that is machinery to allow shared locks, and mediate access when someone comes along and does want to touch - you have to get rid of all the look-ers first.
What I've written is along the lines of the original. There is a concept of shared and exclusive locks, and it attempts to mediate between those two. However, when the lock in unavailable, where the previous version waited a while and then gave up, this version puts the process in a queue and waits until access is possible. You might wonder why it needed a ground-up new version of locking - well, you don't need to be a coder to compare the original code to the new hotness, and you'll see mine is half the size, better commented and (in theory) more featureful - I tried bolting it on top of the old, it was buggy and awful.
How does it work? Well, it's relatively simple. Like the old system, there is a pool of locks, each of which is associated with a particular store resource that we want access to. Each lock is made up of two integers (readers, writer) and a queue. 'readers' is the number of processes who hold a shared lock, 'writer' is true if a process holds an exclusive lock. If 'writer' is false, you acquire and release a shared lock just by incrementing and decrementing 'readers'. If 'readers' is zero, 'writer' can be set to true to acquire a write lock.
What if those assumptions ('writer' is true or 'readers' is greater than zero) don't hold? That means someone else has the lock we want, and we're going to have to wait: we turn to the queue. We place a ticket in the queue, and wait for that ticket to be called. At the point the ticket is called, the lock is free and we can grab it for whatever purpose.
When does your ticket get called? Well, if you're a reader, you get called when either the writer holding the lock lets it go, or the reader in front of you in the queue got its shared lock. If you're a writer, it happens when all the readers have let go of their shared locks. Obviously, for the latter to happen, you have to ensure that readers don't skip in front of waiting writers: in fact, this turns out to be quite easy to code, because the logic is simple: when you come to acquire the lock (whether reader or writer), you look at the queue and if it's not empty, you join the back of it.
That's the idea of the code, anyway, and hopefully if you read through the code you'll find it follows that logic. It's already partly hooked into the store as of r680, and at least for me, I don't seem to get IMAP errors any more, which is great. I'm awaiting feedback from people like Pat who have huge mailboxes, though ;)
Like any locking system, it may well have serious bugs which cause the whole house of cards to fall down. Hopefully these will shake out pretty quickly: I'm using this revision on my system, so if there is a problem it should be obvious. Also, the interaction with the Store WATCH system is potentially dodgy at the moment. If this code works, there is a good amount of legacy code which can come away and again simplify things massively, which will be great - that was just a patch too far at this stage, though :)
posted at: 10:23 | path: /bongo/hacking | permanent link to this entry
I do love a good diagram, and I noticed that I have a number of Bongo-related diagrams stuck on my hard-drive. Most of them have seen action in a past blog post of mine, but my blog isn't really a Bongo development resource, and the ones which haven't yet been seen really ought to be a bit more widely available.
In general, though, programmers aren't great at drawing things and worse, updating diagrams in drawing programs is often tedious and time-consuming. So, I've committed some graphviz support into our build system which runs when you build the development documentation.
The first - and only - fruit of this endeavour is a UML-like diagram I've done to try to figure out what the Store's SQLite database structure should be. Behold (clicky for bigger):
I should at this point take a little bit of time to explain why I did this. I've decided that the SQLite usage in the store needs to be pretty much refactored, for a number of reasons. Amongst those is the fact that we store a binary blob within the tables which represents much of this data: some, like "mime_lines" in mail documents is completely hidden with this structure, other, "time_sent" appears in both this blob and as a column in a table.
So, job number one has basically been getting rid of this evil binary blob. This schema, then, isn't me really redesigning it per se - I'm simply pulling the structure out of this blob and putting it in tables, where it belongs.
I've also slightly denormalised the table a little. In our current database, event and mail attributes exist for all documents: and if the document isn't an event or mail, those fields are just empty. That's a bit of a waste, hence the one-to-one relations between "StoreObject" (which represents any object in a store - a mail item, a collection, etc.) and "MailDocument" et al.
The one big change I have made is the addition on the many-to-many "links" table. The purpose of this is simply to group together store objects. In our current store, we have a concept of "calendars" and "events", and events are linked into calendars rather than calendars being collections which hold events. Rather than having something specific to calendars, I've generalised this out. I'm not totally sure I like this as a way of representing calendars - I presume that it was done this way because collections can't be "typed" like a calendar document can - but it would also allow us to do stuff like linking conversations with events, so you could associate a meeting with the invitation conversation or something.
posted at: 10:23 | path: /bongo/hacking | permanent link to this entry
Last weekend I started looking at Hawkeye, prompted mostly by the need for people to be able to change the config for Bongo while we're putting the new config system in place.
After a bit of hacking last night, it's finally at the stage where it does something useful! You can run it under bongo-standalone (previously this wasn't possible), you can log in as a user, and you can enable or disable agents from startup.
Ok, it's not incredible functionality, but it's a start :) One thing which is slightly interesting is that we rely completely on the store for authentication now: so, I can log into the admin tool as any user. However, without permissions to read the various files (permissions which are enforced by the store itself), you can't change anything - you can't even see anything!
I'm hoping to have this committed by the end of the week. That will then open up the following tasks for work:
posted at: 10:23 | path: /bongo/hacking | permanent link to this entry
I mentioned this on IRC, but I think it's worth repeating again here: it's looking like I'm going to be in the States for most of the week ahead. Pretty short notice, but oh well.
I do want to get another 0.2.9x release out soon, because some good bugs have been squished and the caldav stuff is now integrated, but we'll see when. I'll likely be online over there anyway.
At some point, I'd also like to spark some discussion about how we do releases in the future. I've talked to various people about this informally, but the feature-based milestone idea that we started hasn't worked out amazingly well. Giving time-based releases a go seems a decent idea to see how we manage, and certainly from the point of view of the server side stuff releasing a development build every month or two doesn't seem a terrible idea. The web client UI doesn't fit into that scheme quite so well, but.. eh.
Thoughts on a postcard, please ;)
posted at: 10:23 | path: /bongo/misc | permanent link to this entry
Longish time no blog: not really getting enough hacking time at the moment :(
Recently we discovered a problem for IMAP users. IMAP clients can have multiple connections open at any one time, and some clients do that in practice to speed up certain operations: for example, Thunderbird's filtering system runs more than one connection to move multiple messages at once. Which is great.
Our problem was twofold: primarily our indexing takes a little while, but the main issue is that while the indexing was happening, the mailbox was essentially frozen to other changes for various technical reasons. Which isn't great.
It might be possible to lock and unlock mailboxes at clever points and do things in a certain order which would make everything ok, but that's complex and error prone. So, the approach I'm looking at is converting our store from mbox format files to maildir specification file layout. This relieves us of a couple of troubles:
All in all, it's looking like this new way is going to be a big win, and so far I'm removing more code than I'm adding (209 additions versus 234 removals), which is pretty good considering the featureset isn't really changing.
posted at: 10:23 | path: /bongo/hacking | permanent link to this entry
Sebastian has been working hard on logos and templates and things - we don't want Bongo to lose visual appeal. While others are working on code, Jonny, Dejan, Michel et. al. have been working on web content and it's all beginning to come together.
You'll have noticed the new theme being gradually worked in on the planet and the wiki, and the login page for Bongo has changed:
Michel is hoping to arrange something for the forum soon.
In hacking news, we're doing some work on the schemas which will make it easier to integrate new agents in the future, and we need to have a look at the SMTP agent. I'm also planning on reviewing the roadmap soon, and we're planning a Dragonfly meeting in the near future to work through what needs to be done there.
It's all go...
posted at: 10:23 | path: /bongo/hacking | permanent link to this entry
(What are we to call them? I'll stick with MoMsg for now I think)
Welcome to the planned MailCo, whose name has been revealed as Mozilla Messaging Inc.. This has been in the pipeline for a little while, and it doesn't seem to be quite ready yet, but it seems they're almost there.
I think this will be news to a lot of people - Bryan Clark tells us on his blog he only waved goodbye to RedHat yesterday - but it's best that things go forward pretty quickly in my opinion. If anything, MoMsg needs to get going yesterday: Thunderbird 2 has been out a long time now, and Firefox 3 is just around the corner. Getting Thunderbird 3 out of the door very quickly ought to be a priority.
The track record in this area isn't great. Other organisations have had seed funding, great ideas, engineers on board and haven't produced - I wouldn't want to see the next "Dreaming in Code" be about Mozilla - and I'm slightly surprised that for a commercial organisation, they only seem to have hackers on the payroll at this point. But, very early days yet: I'm very excited that they are looking to put calendaring features into Thunderbird directly (Lightning as a plugin is great, but it does feel like a bolt-on).
So, best of British to all those at MoMsg - I really look forward to seeing what you produce!
posted at: 10:23 | path: /bongo/community | permanent link to this entry
I haven't blogged about Bongo for ages, and this is something I've thought about writing a few times - but mostly didn't for fear of taking attention off the more practical stuff, like getting rid of MDB.
But, it seems timely to get some of this stuff out of my brain. First, go and watch Inbox Zero: it's an hour of your time, but it's worth it, and I think it's as seminal for Bongo as JWZ's Bad Groupware essay was for Hula.
I have one goal, one vision, for Bongo, and that's "Getting stuff done". Bongo isn't about technical goals: single copy message store, mail processing speed, that kind of stuff is the stuff JWZ warned us about. It's corporate box-ticking. That's not to say all of this stuff is unimportant, because obviously if Bongo doesn't fit into some situation you can't use it, and if you can't use it then it's pointless for you. But it's not the raison d'etre.
Bongo should be about having control over your e-mail. It's an e-mail tool, and managing e-mail itself isn't a worthwhile task, but managing the tasks, events, ideas, etc., that this e-mail represents is worthwhile. I want to get to stuff on time, I want to remember what I'm supposed to be doing, etc., etc. - it's about enabling me to be productive.
I should say now that this has very little to do with Bongo 1.0. For 1.0, we're going to be doing much of the standard stuff that people expect. I hope we can improve on things: I think our search is already very good, and will hopefully get better; I hope we can do some cool stuff with contacts management too.
There is a lot of stuff on the internet about getting things done. There's a great book by that title. Inbox Zero is one in the latest line of many "How do I organise my inbox?" type articles. Look at this stuff:
I could go on. There's tonnes of these things. But the key, for me, is that no e-mail (+cal +tasks, etc.) particularly supports this stuff. Sure, you can re-purpose features to support it, but it wasn't how people designed the app.
For example, for me: I w