We have scaled our primary database, MongoDB, on Engineyard over the last year. We started with a single mongod process running version 1.6.4, then went to 1.8.1 and now at the latest stable version. We’ve used master-slave, replica set, and also replica set + sharding. We’ve learned a lot about how to deploy MongoDB on Engineyard and wanted to share that with the startup community. I’ve even heard that Engineyard is working on productizing some of the configuration so you can launch a MongoDB just as easily as you use MySQL. While that’s still in the works, here are some specific tips from our experience that may save you time.
A good place to start is Engineyard’s best practices for MongoDB and custom Chef recipes. You should also familiarize yourself with the MongoDB documentation and perhaps even check out the Jira project that is viewable to the public. We’ve found the documentation for Mongo through and up-to-date. We deployed our first MongoDB environment using the example recipe. I suggest playing around with that for a while to get comfortable–e.g. mongo console, tutorials and such. Once you’re comfortable with a simple replica set, you can move on to more advanced recipes for sharding only if you need it. The transition isn’t too bad because a sharded environment is composed of individual replica sets.
Take Advantage of the Flexibility
We run multiple configurations in different environments because it’s expensive to shard every environment or even have replica sets. In your staging or developent environments, you probably only need a single MongoDB instance. The good news is that your application configuration can easily accomodate different these different configuration. For a sharded environment, you’re connecting to the cluster through “mongos”–this means your cluster looks like a single mongod process to your app. Having good recipes is the key to have everything work as they should. Develop some recipes for your app using the example, Engineyard, and other people (like us!) to customize the configuration to your needs. For example, we specify the version of MongoDB we use on every environment to have absolute control.
Know Your ORM and Drivers
When your recipes are good to go, you don’t need to change them much if at all over time. Getting your application working well with MongoDB can be a challenge. Recently, there have been big transitions in this world–Rails 2.x vs. 3.x, Mongoid 2.3.x vs. 2.2.x, BSON, BSON-ext mongo-ruby-driver 1.3.x vs 1.4.x vs 1.5.x There can be conflicts with Rails and Mongoid if you’re not careful about where you are on the Rails 3 divide. Drivers had issues going from 1.3.x to 1.5.x since 1.4.x was release and then quickly yanked from rubygems. On top of that, MongoDB was transitioning from 1.8.x to 2.0.x, which was a big change. The chaos will subside but knowing where all the pieces lie for you is crucial to avoiding problems. A configuration that worked for us was:
- Rails 3.1.x
- Mongoid 2.4.x
- mongo-ruby-driver, bson, bson-ext 1.5.x
- MongoDB 2.0.x
I recommend Engineyard and 10gen support to help with big deployment. We pay both for premium support but there are plenty of documentation resources and free support available if you’re strapped for cash. Of course, you can always find help on twitter along with @MongoQuestion as well as quora
Special thanks for Ines Sombra, MongoDB expert on the Engineyard Data Team, who has helped us tremendously over the last year!