DynamoDB and Me

May 8, 2013

I first became interested Amazon’s hosted NoSQL datastore, DynamoDB, after reading about Datomic. It’s interesting to consider using this hardened underlying datastore for the simplest possible operations and putting higher-level (and perhaps more dangerous complexity) in the application layer. Also, I’m a big fan of async io and writing web applications in Tornado. This is a problem with common database libraries. Accessing a database through HTTP means an async datastore is seemingly within grasp. It’s just web servers talking to web servers.

DynamoDB has some really interesting selling points. I’m on a small team, working on a new small project, but I’m very aware of the horrors of database failure which typically you can only afford to solve when you’re big. I don’t think I can really live that way. But, trying to setup and maintain a cluster of MySQL instances, with replication and backups just so I could sleep soundly at night wasn’t going to work. Also, every new project dreams about that moment when you’ll be panicking because you’re growing so fast that you’ll have to take downtime to upgrade your MySQL instances. AWS takes care of all of that since DynamoDB is really a hosted database “service”. The operational complexities are handled for you. All that’s required of the developer is to guess how much capacity you’ll need, and a big checkbook.

My current conclusions, after 8 months of use, are not too far off from my initial expectations:

  • DynamoDB is expensive for a small project. But you do get a lot for your money, just maybe more than you’d otherwise be willing to pay for right away.
  • Existing libraries and tooling (at least from the Python perspective) are seriously lacking. But this also provides a lot of opportunity for building cool tools to share with the world.
  • In many regards, using a NoSQL solution like DynamoDB is less flexible in the early parts of your application. You are forced to think about access patterns and what you want your data to do at the very beginning. This is good and bad. If your application doesn’t get big, you may have wasted a lot of time.

In summary, I’d say DynamoDB is a great tool for the right job. I probably wouldn’t advise jumping into it with a big mature application right away without getting your head around it on a smaller project. Unfortunately it might cost you a lot of time in a small project that will probably just slow you down in that critical prototyping phase. But, this is true of just about every new technology you might be considering introducing into a new project. Just don’t make the mistake of doing too many of these experiments at once.

Leave a Comment...