Cloud-integrated Storage

Defining and Explaining a New Approach for SMB and Enterprise Storage

If you delight in being a part of-or being responsible for-a group of storage administrators who are forever getting deeper into the intricacies, complexities, and difficulties of complex storage environments, then this paper is probably of little interest. If, however, you view storage not only as a crucial tool that should be as flexible and economic as possible, but also as a vital asset that you should control, then you should probably keep reading.

Author(s): Mark Peters

Published: April 2, 2012

A New Storage Category

Everyone knows that the traditional storage models of recent decades need to change. This is because data growth, application demand, and complexity are all increasing, while data security requirements and limited budgets handcuff the ability of IT to meet these increased demands. The challenge is easy to understand, but answers have been limited.

Yet, a new approach is being seen that has the potential to address all these issues. While conceptually it is close to storage architectures that we already have, it also embodies key differences that contain dramatic operational and financial opportunities. Clarity of terminology matters-and that is the purpose of this paper.

There's often an awkwardness to the semantics of storage. On the one hand, we are keen to have new descriptors so we can put vendors and their offerings into neat boxes. On the other hand, if those descriptors are not adequately defined or if the nature of the offerings changes, then we can be left with something that is too broad or even misleading. "Cloud gateways"-perfectly good in and of themselves-have become a broad catch-all for links to the (invariably) public cloud. Yet there are now emerging variations in the storage implementation that compel us to define a new category. A gateway is just a point of access, allowing movement to and from the cloud; the new category defined in this paper includes such movement as one part of a more holistic approach.

This new category is called Cloud-integrated Storage, and the offerings within it are Cloud-integrated Storage Systems.

Introducing "Cloud-integrated Storage"

Because it has not been named before, Cloud-integrated Storage (CiS) has to date been variously-and inadequately-talked about as tiering, a gateway or as a hybrid cloud. Each of these descriptors could be argued to be a necessary-but-not-sufficient element for a comprehensive CiS offering. Numerous implementation options will arise as the CiS category develops and matures, but the essence of it is to integrate and control cloud storage as part of a user's regular/main IT operations, not to have it as "just" an external service.

Already, there are examples of traditional storage vendors using cloud storage as a tier for their storage array. Other vendors offer systems that can be used as on-premises storage together with an integrated connection to cloud storage. The configurations vary and will no doubt grow in flexibility and operational sophistication. Today, some vendors use cloud storage as the primary storage target, with the local appliance holding only cached data to reduce the latency associated with storing data offsite. Others have at least some primary data in the local appliance (whether held directly or pinned), with the cloud being used as an archival and disaster recovery (DR) target. There's also an emerging category of software that can leverage storage capacity and characteristics regardless of whether that storage is onsite or in the cloud, creating a stretch or geo-distributed cluster.

Whatever the precise implementation, the essence of the Cloud-integrated Storage approach is that-colloquially speaking-you are not just sending your data over the data center "fence" to the cloud; instead you are widening your reach to wrap your arms around the cloud and integrate it into your approach. It is an inclusive rather than an exclusive approach. And-as this paper will show-it is not only worthy of being its own category, but it also offers significant potential benefits to businesses ranging from SMBs to large enterprises.

So, why is CiS likely to be an important element in IT over the next few years? The answer is simple: IT is a combination of people, processes, and technology. For storage users who want to stay with legacy arrays or keep existing processes in place, using cloud as a storage tier behind what appears to be a conventional array gives them the extensibility and price points of cloud without radically altering processes or having to retrain staff. Alternatively, for users who need a more comprehensive DR strategy but can't afford a remote site, system, and the staff to manage it, using tools within a storage array and simply (if figuratively) turning some dials to mirror encrypted data to the cloud is low risk and affordable. Cloud-integrated Storage Systems are bridging technologies that tie the present to the future. They represent a familiar, safe way to wrap together the best of traditional and emerging (i.e., cloud) worlds, enjoying the economic attraction and flexibility of the latter while retaining the control, familiarity, and performance of the former. It is a beguiling proposition and worthy of being a standalone category.[1]

CiS: Definitions and Market Applicability


What CiS Is and Is Not

The quickest way to understand what CiS is not is to ask who has control of what goes where? If the last real control that a user has is when the data hits the wire to leave the building, then it is not a CiS solution. CiS systems may use a gateway, but they are more than that. As the name implies, the cloud elements of the solution are integrated into the whole, embraced as a pragmatic part of a flexible and economically attractive storage solution.

Again, it is worth stressing that many of the terms that get used in regular gateway/cloud storage discussions-such as migration, cloud economics, easy provisioning-can and do apply to a Cloud-integrated Storage System (CiSS) too. But a CiSS adds all-in-one control and management, where the advantages of cloud storage are just part of an orchestrated whole-storage approach ... everything from high performance, to archive, to data protection being managed as one organic system (meaning, of course, that good CiS implementations allow end-users to have direct access to anything anywhere in the physical system).

A typical CiS approach will have the primary and/or most active data staying onsite (as a tier or cache) with the cloud used-usually with the data encrypted of course-for less active data and as an easily scalable archive as well as for powerful remote data protection and business continuity capabilities.

To net it out in terms of "what" and "why" as far as CiS goes:

  • The WHAT is consolidation, ease, flexibility, and operational certainty (for VM sprawl, e-mail, files, etc.).
  • The WHY is ROI/TCO, plus improved (integrated!) resilience and recovery.

Market Applicability

While this paper explains and defines a new storage category, that category will only have any real chance at market relevance if the cloud on which it is based is itself becoming more popular and accepted. A few sample notes from various recent ESG research studies show that this is indeed happening:

  • Twenty percent of respondents list using cloud services as one of their most important overall IT priorities in 2012.
    • Seventy-four percent will increase their expenditure on cloud services in 2012.
    • One in five users say using cloud storage to add capacity without buying infrastructure will be one of their most significant investments over the next 12 to 18 months.
    • More than half of users already use or plan to use Infrastructure as a Service (IaaS), and of those already doing so, cloud storage is the top use case (noted by 57% of current IaaS users).
    • Significant IT budget is now actually being invested in the cloud; 7% of all IT expenditure in 2012 is tagged for "cloud." (While this might not seem large, IT staff is 31%, and all regular hardware is only 19%).
    • Cloud is seen as a useful tool to manage costs, which is itself the number-one business driver of IT spending decisions. Whereas in 2009, organizations were nearly three times as likely to cancel projects or reduce headcount in order to cut costs than they were to use cloud, by 2012, organizations were more likely to use the cloud to contain their costs.

Figure 1 shows graphically how cloud-in its various forms-is achieving a significant level of actual and planned adoption among IT users.[2] This is important for CiS to be a successful storage approach.

Figure 1. Public Cloud Adoption Trends

However, even as interest in, and expenditure on, cloud offerings has grown, there are plenty of users that have remained at best cautious, and sometimes completely opposed, to putting any of their vital data assets "out there" in the cloud. When IT professionals were asked in a recent ESG survey[3] why they thought public cloud computing services would have almost no impact on their organizations' IT strategies over the next five years, the top-three answers (multiple responses were accepted) were:

  1. "Data security/privacy concerns" (43%)
  2. "Feel like we would be giving up too much control" (32%)
  3. "Too much invested in current IT infrastructure and staff" (32%)

Clearly the use of cloud storage can fall foul of any or all of these concerns. Yet in order to be a viable storage approach, CiS must be able to address them. Encryption answers the first point. The fact that CiS doesn't demand any control to be surrendered (and may actually increase control as the cloud is added to the arsenal as an incremental, location-independent data protection and recovery resource) answers the second point. And, lastly, there is no immediate need for the existing infrastructure or staff to be changed-CiS simply allows for the cloud elements to be implemented and grown over time as the operational and financial needs of the business dictate.

Buying Considerations

There are existing vendors and stealth companies in the wings. And logic dictates that more will arrive as IT becomes more virtualized, flexible, and commoditized. As the CiS segment grows, what should interested users look for when considering their CiS options-whether for a whole company, or for particular applications or departments?

  • Basic needs: While it is true for any IT buying decision, the first thing to define is what needs and expectations do you have of the system? Do you need a motorbike, a sedan, a minivan, or a truck? What top speed and what seating capacity? Where is the cloud storage actually located (at an industry giant such as Amazon, Google, Dell, or HP, perhaps somewhere more specialized such as Rackspace or Nirvanix, or maybe at a specialist regional or vertical provider)? Is it "plug and play" (standards-based) and secure (encrypted)? Can you easily change the provider if desired to take advantage of a better deal elsewhere? Remember that you might not need high performance for all your data, but you do want it all online, available, and secure.
  • Advanced needs: There will likely be CiSS offerings suited for every organization, from SMBs to enterprises. What category are you in? What duty cycles, features, and functions will you need? At the enterprise end of the equation, for example, you will probably want to look for capabilities that you already need and use (or are considering) with more traditional storage infrastructure-easy provisioning, snapshots, replication, data reduction (such as compression and deduplication, which are inherently good but especially important to minimize CiS network traffic), a globally distributed file system (for collaboration), and high availability through non-disruptive upgrades, no single point of failure and hot-swap for everything (including controllers). Does the vendor have the right certifications with the right vendors (such as VMware and Microsoft)? Maybe you need a tracking tool for charge-back or client billing?
  • Credibility and proof: Of course, this is a new approach to storage, so no vendor has thousands of installations. Nonetheless, look for references where possible (many do exist), a willingness to offer a proof of concept, and other signs of maturity (such as extensive connectivity capabilities, strong interoperability with OS and hypervisor providers, and a comprehensive data management tool/GUI).
  • Overall: Whatever level and type of CiS you decide meets your needs, you will be converging multiple storage components that are currently separate. In all likelihood you'll be making your life a lot simpler with many management issues taken off the table. With good CiS implementations, you will not have to change your applications or operational procedures, yet you will gain enormous flexibility. (All by itself, this "on demand" element is a key motivation for CiS.) And you most likely will save a significant amount of money.

At the end of the day, CiS is an umbrella storage solution that will serve as a user's storage platform-so it should be viewed holistically in terms of providing all that you need and want from your storage. It is not just an add-on nicety; it is your storage, period. It may of course also be more flexible, more economic, and easier to manage, but all these will be for naught if your chosen CiS approach does not offer the full range of functionality you require. So be considerate and even pedantic in your evaluation.

CiS Is Compelling: Economics and Emotions

The operational basics of CiS are easy to grasp-everything (primary active storage, archive, backup, and DR) is managed in one well-utilized appliance, with flexibility and a user-suitable range of capabilities. This may be as simple as automatically backing up data to, for more advanced users, perhaps snapshots that can occur offsite and/or between multiple cloud vendor sites. The effectiveness and efficiency is abundantly obvious. But there are also two other "E" words that have a huge impact of the level of market success for any storage product. The first-economics-is no surprise. The second-emotions-may cause a raised eyebrow or two. It is less often discussed but is a key driver of what really gets bought.

Delivering Economic Value

Economics is the bedrock of the storage industry. All the factors that dominate storage discussions-capacity, performance, data management tools, and so on-are only relevant because storage isn't free. No one would put any data anywhere other than at the very top of the storage and memory hierarchy otherwise. But storage isn't free, so we have decisions to make.

CiS allows users to enjoy the economics of both consolidation and cloud storage without losing operational control. With a better use of all local storage resources (people, space, equipment, etc.) and the financial benefits of cloud storage, the impact of CiS on overall TCO can be dramatic. Just avoiding the costs of array-based remote-copy tools and maintaining a remote site (things that many users don't do simply because of the costs) can be very attractive.

And with data protection continuing to be a pain point (and with data growth rates being what they are, it only gets worse), having the ability to leverage cloud storage as a backup target or for archiving could have a significant cost-reduction impact relative to keeping data local and managing tape libraries or VTLs.

And the bottom line is that saving money matters in IT. ESG's latest spending intentions research asked users globally to identify the most important business initiatives impacting their IT spending decisions; the results, shown in Figure 2 and covering a four-year trend, demonstrate that-even as the economic situation has eased-cost reductions are the clear number-one motivation.

Figure 2. Business Initiatives That Will Impact IT Spending Decisions, Four-Year Trend

Delivering Emotional Value

It may seem strange to include "emotional value" in a discussion of IT tools. But the simple fact is that emotions are an important part of how people make decisions. Everyone is-for instance-risk averse to some degree; it is one reason why different people and organizations proceed at different speeds in adopting new ways of doing things. CiS offers a number of ways that will make users will feel more comfortable about beginning to use cloud storage:

  • In an IT world where growing expectations, data capacities, and rampant virtualization are making storage more challenging, there is a "comfort" in having the flexibility that CiS provides, based on skilled, specialist professionals at the cloud provider ...
  • ... and yet, CiS users do not have to relinquish ultimate control of their data; a CiSS is a markedly different proposition than "washing your hands of storage" and just passing ownership "out there."
  • CiS can be viewed as "the best of both worlds," inasmuch as both the operational and financial teams at user sites will be happy. As "cloud use" begins to appear in IT managers' MBOs alongside-or instead of-virtualization, CiS will be an attractive option for users: While most CiSS offerings are not intended for "tier one" applications, at least today (after all, there's still a network wire involved), the simple fact is that such data is a relatively small part of most organizations' overall storage needs, and most users could therefore find plenty of opportunities to try this new approach.

Overall, CiS doesn't require users to break all their emotional attachments. Very often, a big reason not to do something is "feelings," not facts; while CiS vendors will no doubt (and rightly) produce extensive verbiage about speed, capacity, functionality, and costs, it will be the reduced worry that is most compelling for many users.

The Bigger Truth

The summary here doesn't need to be long-and that in itself is a big clue that Cloud-integrated Storage is a compelling concept that looks set to catch a significant market share by integrating rather than simply adding cloud storage to users' onsite storage infrastructures. Basically, users can enjoy the benefits of the cloud (economy, data protection, and access to features and functions that they might not otherwise have or afford) while still retaining control of their data and operations. It can be thought of as your own storage that's just on a very long wire (or even multiple wires) from the controller!

The category has already attracted a number of players, each with their own interpretation of the basic segment theme and business advantages. Vendors from the cloud storage gateway category are natural candidates for the CiS category, and established storage behemoths are also interested. The number of vendors that are delivering CiS solutions is bound to grow, but holistic options are currently limited; StorSimple is a notable early innovator in this market, while other market entrants are focused in a variety of ways; some are primarily emphasizing gateway functions such as providing a local target for data protection and caching (examples are TwinStrata, Amazon Storage Gateway, Riverbed); some are emphasizing distributed access of files (such as Nasuni , Panzura); and some are focusing on the policy-based movement of files (one is F5).  EMC has EMC Atmos-based cloud storage integrated into its Celerra platform through a new cloud tiering appliance for files. However, most incumbent storage vendors are unlikely to proactively support cloud storage services that move data away from their storage systems. Big names, such as Google, Amazon, and Microsoft, also underpin the cloud storage component, which helps to give credibility and a sense of longevity to the segment.

As with all things in storage and IT, the key question for interested users should eventually be less about how this is all done (we will assume that everything from the vendors works as advertised) and instead should be far more about what is delivered and whether that meets their functional and financial needs. In other words, the point is for users to determine whether CiS is a good storage option for them. It has already attracted many users who value the combination of flexibility, control, data protection, and economic advantage that it can deliver.

However, the range of naming conventions has somewhat obscured the recognition of progress for this emerging segment. Hopefully, giving it a formal name-Cloud-integrated Storage (CiS)-will not only encourage more users to consider the options, but will also allow the vendors populating this segment to concentrate as much on "growing the pie" as on determining "what size slice of the pie" they get for their differentiated solutions.

CiS is a cloud-inclusive storage approach that offers enormous potential advantages for large swaths of user data, yet without requiring a large leap of faith, a huge check-book, or a total change in human nature.

[1] Elements of this paragraph have been adapted from an article by ESG Senior Analyst Terri McClure published in Storage Magazine.

[2] Source: ESG Research Report, 2012 IT Spending Intentions Survey, January 2012.

[3] Source: ESG Research Report, Cloud Computing Adoption Trends, May 2011.