Many interesting IRC discussions today, so I thought I’d write some of it down. This helps settle things in my mind’s eye if nothing else 🙂

Netmail traditionally allowed people to cluster things in quite interesting/bizarre ways, and you were able to set things up so that there was no single point of failure: the configuration directory could be replicated, as could the store (I believe), and obviously you could set up redundant agents, all over many servers.

We probably want to keep most of that design for Bongo, but in many ways Bongo isn’t capable of that kind of setup right now. So, I’m thinking for 1.0, we’ll limit the supported deployment strategies: mostly, to keep our sanity and ensure we’re not over-engineering this thing. The question of architecture also nicely fits into the current discussions about MDB.

I drew this diagram to outline my current thoughts:

In terms of how we limit deployment in Bongo 1.0, I think mainly that one store is enough for anyone 🙂 More specifically, multi-store design is really a bit beyond the 1.0 timeframe.

The arrows in the diagram above represent a vague combination of “is responsible for” and “talks to”. In particular, what we’re talking about is:

• bongo-manager still responsible for all agents, but we have a special bongo-manager which is basically the one responsible for the store;
• b-m talks to its child agents via a slightly more verbose messaging protocol, replacing the RPC parts of the management protocol;
• b-m tells the agents which store to connect to and how to auth to it; after that, they pull all their configuration from the store;
• when told to by their b-m, agents reload configuration from the store and/or write statistics to the store;
• the lead b-m is also responsible for passing messages to b-m instances on other servers.

In terms of MDB usage, we delegate user auth to a much simpler auth API. Configuration and preferences are stored as json documents in the store (the ACLs in particular being important from a security point of view), and we simply augment that with a standard API for reading/writing values.

As well as simplifying the usage of MDB, and the management interface, it means the security issue wrt. Dragonfly and Hawkeye is solved, we have a much better system for agent control and statistics collection (witness the Rules agent code versus, e.g., the SMTP agent), it’s something we can take in chunks, the schema update stuff is much less of a problem in the future, and the bongo-setup process would be much simpler. We can also dump LDAP, if we wanted.