home ¦ Archives ¦ Atom ¦ RSS

Amazon: S3

Just a few thoughts on Amazon's new web service S3, which is cheap, reliable, plentiful Internet based storage.

S3 is not a game changer.

To me it seems appropriate for boutique applications or proofs of concepts, but you wouldn't bet the ranch on it. I did a back of the envelope calculation on what 1 terabyte of data (a nice round size and the point at which this really matters) would cost on a monthly basis. Your first month, storage and transfer is roughly $360. Every month thereafter you're paying about $150. After about 5 months, you've forked over enough for one of LaCie's rackmount, network attached terabytes which costs $899. That 5 months is if you never transfer any of those stored bytes, so it's a lower bound on your cost. I know it's not quite apples to apples, since you'd really want to do RAID and possibly some clustering, but the gist is still the same: commercial, off the shelf, network attached storage is competitive with S3.

The upside of the LaCie is that you can park it as close as you want to your computation. If you're using S3 for anything more sophisticated than a dumping ground, and/or not planning to peer close to Amazon's routers, expect to do some serious cache engineering to deal with the Internet latency and congestion between you and S3. The LaCie also has standard network filesystem interfaces. To access your storage on S3 you've got REST and SOAP.

The upside of S3 is that they guarantee 99.99% availability and you don't have to deal with backups. And I am in no way pooh-poohing this cost savings.

So what you're paying for on an ongoing basis is the removal of a certain amount of engineering hassle and replacing it with others. I don't think you can build the whizzy, large scale, interactive web applications that folks are fantasizing about on top of S3, without a lot of smarts. At which point you might as well build your own cheap knockoff of S3, and own your own wheel.

But it would be a gas to build a tuple space frontend to S3 and run with it.

By the by, I wonder if Alexa Web Search has generated anything interesting.

© Brian M. Dennis. Built using Pelican. Theme by Giulio Fidente on github.