Sweet! Amazon finally took the wraps off of SimpleDB. They’ve been working on this for awhile, and as you can probably tell, it’s a natural fit with S3 and EC2. There’s a great write-up about it over on inside looking out.

This is nearly a perfect solution for some of our data-related scaling challenges, except for two issues:

• Physical proximity. Some of my datacenters aren’t close to Amazon’s, so the actual time to query SimpleDB is query time plus latency. This isn’t a problem if you’re doing all your queries from EC2, but we’re not there yet (we’d like to be, but a few pieces are missing. SimpleDB is one of those pieces, so we’re getting closer…). Amazon has promised me they’re workin the speed of light issue.
• Attribute size limits. We have some data fields that are longer than 1024 bytes (most aren’t and would work fine). We’ve thought about chunking the data up to get around this, which is a possibility, but it gets messy. Storing them in S3 is both overkill and probably too slow – if I need to get a few thousand photo captions *fast*, doing it through S3 isn’t optimal. If we could solve the latency problem I already mentioned, I’d be fine storing that specific data in some other store and working around it that way.

On the plus side, SimpleDB should be screaming fast, incredibly scalable, and almost all of our SQL queries would work with no changes other than syntax. Like many of you, I’m sure, we’re using much of our RDBMS as a fairly simple data store and aren’t using many advanced RDBMS capabilities. All of those queries could just use SimpleDB and then we could devote our DB iron to just the rare complex queries. We’re not alone – tons of web apps are gonna love this.

I’m thrilled to see the Amazon AWS stack continue to grow, and I’m shocked that they have as big of a lead as they do. I would have thought Microsoft / Google / Sun / whomever would have been out with some competition by now. It’s gonna happen – but I never would have guessed it would take this long.

Oh, and while I have your attention – SmugMug is now a fairly heavy user of EC2 and I have a write-up coming. So check back later if that’s of interest.

This entry was posted on Friday, December 14th, 2007 at 10:32 am and is filed under amazon, datacenter. You can follow any responses to this entry through the RSS 2.0 feed. Both comments and pings are currently closed.