Amazon preannounces persistent storage for EC2

I don’t know if this preannouncement comes as a result of all of the Google App Engine publicity, but here it is: Amazon Web Services Blog: Storage Space, The Final Frontier. In a nutshell: AWS now lets you create a storage volume of 1GB to 1TB that can be mounted in one EC2 instance and will persist beyond the lifetime of an EC2 instance. As an added bonus, you can have automatic snapshots of your volume plunked into S3.

They say that this storage is a low-latency, high-throughput block device. So, you can run all kinds of traditional software on top of it.

This will change the competitive outlook a bit between AWS and GAE a bit, because it makes it easier for people to use all of the software pieces that they’re used to when they use AWS to manage the hardware infrastructure. This means that it’s easier to take your existing apps and skills and get them up on AWS. GAE has a fight ahead in terms of getting people to write their apps differently… but the benefit to doing so is that you no longer think of hardware infrastructure at all.

App Engine: I agree with Ian Bicking the Luddite

Isn’t quoting out of context great:

I hate computers.
[From Ian Bicking: App Engine: Commodity vs. Proprietary]

More seriously, though, in saying “I hate computers”, Ian is actually talking about the opposite of being a Luddite. He’s dreaming of a world in which much of computing just works in the background, so that we can spend our time doing more important and interesting things in the foreground.

I’m linking to Ian here because he’s said exactly what I have been thinking about App Engine: from a Python programming perspective the APIs are simple and clear. I can easily imagine a ZODB-based implementation of Google’s data store API. Just change your imports, and you can be off of Google’s infrastructure and on to your own.

Of course, for a great many people there won’t be any reason to be off of Google’s infrastructure. App Engine is just so darn easy. Amazon Web Services is impressive because it makes scalability affordable and available. App Engine interests me because, for its broad-but-still-limited set of use cases, it makes scalability a no brainer. “Build your app like this, and you never have to think of scaling” is a nice thought. I’ve been around enough to know that people using App Engine will still have to think of scaling some, but not nearly as much as with just about any other solution.

Back to the lock-in aspect, though. I still see App Engine as likely to be utterly unsuccessful with large businesses. That is, until a new Google Appliance comes out. I’ve been predicting such a beast since Google Docs was first introduced, and I think App Engine makes it all the more likely. I still believe that there will come a time when Google will sell boxes to big companies that those companies can toss into some racks on their networks and deploy App Engine apps locally, as well as run Google Docs on their private nets. Things will get even more interesting at that point.

You can bet that Amazon is studying App Engine closely and considering their own high-level service as I write this. From a developer’s perspective, this competition is going to be awesome.

Google App Engine, not really an AWS competitor?

It occurred to me just now that Google App Engine and Amazon Web Services are only barely in competition right now. If you want an infinite storage system like AWS S3 in App Engine, you need to code it yourself (ignoring the preview limits App Engine currently has). If you want to deploy apps as easily as you can with App Engine in AWS, you need a bunch of infrastructure that AWS does not provide.

I’m happy to see that App Engine’s datastore is transactional, unlike SimpleDB. I didn’t see anything in my skim of some docs about whether App Engine has eventual consistency or if you can immediately pull out data that you stuff in. My guess is that you can immediately pull out the data you shove in. This is a win over SimpleDB, in my opinion.

App Engine is just tons higher-level than AWS. Of course, you can host anything you want in AWS. But, by trading away a bunch of that flexibility, Google has made a service that allows people to build apps that scale well with a minimum of fuss.