Will Amazon's Latest Move Thrill or Kill the Cloud Storage Gateway Market?

The big question of the day is whether the AWS announcement of a cloud storage gateway (one of the worst kept secrets in tech) will validate or kill the existing gateway market. And that's a good question. For now it is a great validation that users are looking for a standards-based way to access the cloud for certain use cases. And we do see storage services adoption ramping up. In our 2012 spending intentions survey (coming soon, but I got a preview of the data that will be included), 51% of the respondents said they are using or plan to use Infrastructure as a Service (that is up from the 35% of respondents using or planning to use IaaS in last year's survey). Of the IaaS users, the biggest use case response is cloud storage (57%).

But an AWS new gateway won't kill the independent gateway market near term - this beta version is really basic and does not have much in terms of bells and whistles - it is much less feature-rich than the initial implementations from the existing independent gateway vendors (CTERA, Nasuni, Panzura, Riverbed, StorSimple, Twinstrata, and I probably missed some), and those vendors have not stood still. There is no deduplication (but there is compression) - depending on how compressible the data set is, without deduplication the AWS capacity and data transfer service charges could lead to some pretty big fees on the storage services side depending on the nature and use of the data. Uploading data (data transfer IN) is free, but retrieving data (data transfer out) costs $$, so a restore of any size or frequent restores could add up. This is a pure backup and DR play for now rather than a local cache for primary data in the cloud, but AWS is expected to expand functionality and use cases over time.

It is interesting to note that there is also no hardware node-not surprising, but interesting, and a possible challenge. The AWS gateway runs in a VMware virtual machine and uses local disk capacity. While some may consider this a good thing, experience has taught gateway providers like TwinStrata that sometimes you need to sell an appliance that can be dropped into the data center and dedicated to being the gateway - you often get better performance and have no worries about resource contention or workload management. Plus - there is the mentality that storage buyers want to touch and feel something - pure cloud is still a tough sell in the enterprise.

And I hate to use the term "vendor lock in" because at some level everyone is locked into someone, but the independent gateway providers do provide users an option of transparently migrating data to a different service provider if their current provider does something like change the pricing model or SLA. Just think about what happened when Iron Mountain decided to get out of the storage services business (not that I think this will happen at AWS, but pricing model tweaks are certainly a possibility). Nasuni went out and proactively migrated their customers under the covers to their new service provider of choice. By the time the IM news was public, Nasuni had done a ton of migration work. While not an "apples-to-apples" comparison, it is illustrative of what is possible when using an independent.

The initial target use cases for AWS are dead on though - of the current cloud storage users we surveyed in our spending intentions research, the biggest use cases for them are backup (a whopping 67% are using cloud storage for backup), followed by DR (58%) and Archive (58%). And that certainly reflects where some of the independent gateway vendors like Riverbed are finding a niche, so this will increase pressure on them.

Looking a little deeper - this is really a brilliant move by AWS, not because they have a gateway for cloud storage use, though. It is because these snapshots are stored as Elastic Block Storage (EBS) snapshots. This means they can be used to create EBS volumes and run against applications in EC2. Test and dev operations can now be run against production snapshots in EC2. And over time, users can just migrate applications into the cloud - after all, the data migration will already be done for them. Once you do test and dev and get comfortable, the leaps to running tier 2 applications, then tier 1, become smaller - just ask VMware. At the end of the day I don't think Amazon is really interested in being a gateway so some of your data can be stored in the cloud - I believe that they want all of your data, and compute, in the cloud. And it may take years, but this is the gateway to that.

You can read Terri's other blog entries at IT Depends.

Topics: Storage IT Infrastructure