ESG Validation

ESG Technical Review: End-to-end Data Protection for the Modern Data Center with Cobalt Iron Compass

Co-Author(s): Christophe Bertrand


Abstract

This ESG Technical Review documents hands-on testing and analysis of Cobalt Iron Compass (Compass). In the report, we examine how Compass uses automation and analytics to deliver a modern, single control pane, SaaS-based data protection experience.

The Challenges

Data protection is often the single largest application in a data center. For individuals tasked with managing data protection, the daily work can seem like a relentless set of tasks that include provisioning, creating new systems, application protection, event schedules, policy management, complex mapping of business requirements, applying software and security updates, constant management of backup configurations, operations monitoring, and data replication all while still meeting compliance requirements and SLAs.

As shown in Figure 1, ESG recently conducted a survey that asked organizations what they believe are the most important objectives for their digital transformation initiatives. More than half (55%) of the respondents identified becoming more operationally efficient as one of their top initiatives.1 This response demonstrates that organizations view data protection as a strategic objective and would like to transform it into a more intelligent and automated practice.

The Solution: Compass

Compass is a SaaS platform that plugs in and embeds various backup, storage, and cloud technologies to deliver enterprise backup in a software-as-a-service model. The Compass architecture is a new technical approach to data protection harnessing analytics and automation to drive down cost and complexity while delivering reliable, secure, immutable data and ransomware protection as well as valuable data insights to the business. Being plugged into the Compass platform, backup and storage technologies are completely managed, configured, maintained, monitored, and continually optimized with best practices allowing the power of these technologies to be experienced without the complexities.

Compass is designed to eliminate the majority of daily management tasks through analytics-optimized automation. As shown in Figure 2, at the core of the solution is Compass analytics and automation, which are always working to optimize the protection of data. This starts with very extensive data collection and monitoring of backup infrastructure and operations. Metadata is constantly collected from Compass Accelerators and exported to the Compass Analytics Engine where it is transformed into real-time intelligence, displayed in Commander, and leveraged to enhance overall automation.

Key solution elements of the Compass solution include:

Commander: an easy-to-use, efficient web interface providing a simple and intuitive user experience to monitor, manage, survey, and analyze all of the Compass-protected systems. RESTful APIs are integrated to connect with many elements.

Analytics Engine: a machine-learning-based engine that delivers data protection efficiency through metadata analysis. Using this metadata, Compass provides improved operations, proactive problem avoidance, and automated efficiencies through 22 worldwide cloud data center locations in a SaaS model.

Accelerator: a converged, integrated, and scalable enterprise-class data protection system, which lives where your data resides and is constantly monitored, maintained, and enhanced. Each Accelerator, whether physical or virtual, on-premises or in the cloud, can stand alone or work with a replicated Accelerator to further enhance data protection.

Agents and APIs: connect with and protect systems and applications. At the file system level, this includes Windows, Linux, Apple Mac OS, HP/UX, IBM AIX, IBM Linux zSeries, VTL, IBM i, VMware, Open VMS, and Oracle Solaris. At the application level, this includes MS SQL, DB2 Database, Exchange Mail, Domino Mail, Mongo Database, Oracle Database, SAP and SAP HANA Database, VMware Virtual Machines, and AWS snapshots.

ESG Validated

This ESG Technical Review documents how Compass operates as a SaaS model and the benefits of this approach. The focus of the paper is to demonstrate how Compass eliminates the daily management of data protection resources and orchestrates and automates administration processes and procedures.

Multi-cloud Orchestration-as-a-service

So, how does it work? Compass takes over the management of all data protection systems and eliminates the need for a customer to login to anything but Commander. Administrators securely access the Commander dashboard using two factor authentication and LDAP/Active Directory integration. It should be noted that even with full admin privileges to Commander, an administrator does not have access to Compass backup infrastructure or to the backup data, both of which are inaccessible. As shown in the upper left side of Figure 3, with Commander, the Organizations concept gives Compass customers the ability to manage all their sites across all public or private clouds or on-premises locations in a multi-tenant fashion. If you are assigned to the top-level organization, (depending on your role), you have access to the entire environment. If you are assigned to a sub-organization, such as the engineering department, you would not have access or visibility to parent organizational levels such as Corporate, or siblings at the same level as you, but you would have access to child levels below you. In the middle of Figure 3, we explore how system identifiers are used to control resource access and management. In this case, the identifier (cid0015-apollo-fil-00) for one of the systems being protected translates to Customer ID (CID0015), System Name (Apollo), Protection Class (File System), and Numeric ID (00). This string grants access rights, prevents unauthorized access to the resource, and creates a wall between and within organizations.

Next, as shown at the bottom of Figure 3, ESG navigated the Admin page within the Commander interface. Here we see three groupings (Provisioning, General, and Other) of management tiles. These tiles include Compass components and elements such as Users, Billing Codes, Traffic Policies, and Storage Classes. When viewing this page, the tiles that are visible are based on the user’s role and permissions. A top-level administrator can manage the entire environment including the full provisioning process. The red callout box in Figure 3 highlights the Protection Class tile. This is an important part of the Compass-as-a-service orchestration and automation. The protection classes distill all the provisioning steps and configuration, such as data protection servers, storage, and policies, into a streamlined management schema. Then, expanding the protection to new systems is as simple as adding them to a predefined protection class.

As shown in Figure 4, ESG next explored the process of adding a new system to the protection schema. Protection classes make this process quick and easy. As shown in the upper right side of Figure 4, we selected the yellow plus sign on the Systems page to launch a configuration wizard. Then we simply added a hostname, selected the platform type (in this case, Windows) and the protection class file system backup, and clicked CONTINUE to finish the process.

ESG used the manual process to add a system to the backup schema with just a few clicks. For bulk additions of systems, the Compass RESTful APIs can be leveraged. In addition, Cobalt Iron Support can work with larger clients to tie the process into their packaging delivery systems. It should be noted that when a system is added to a Compass environment, it inherits all the robust protection settings defined in the Policy Class it was added to. These settings include parameters such as backup schedules, retention policies, include/exclude lists, and client options like compression and deduplication that were predefined in the Policy Class to meet business requirements.

Next, as shown in Figure 5, ESG took a deeper dive into the platform support the Compass solution delivers. Here, as shown by the red callout box, workload support is broken into four categories that include Protection Type, System Platforms, Virtual Platforms, and Cloud Platforms.

Protection Type defines what backup operation will be performed, such as file system, databases, or application. The Systems Platforms category defines OS support such as Windows, Linux, IBM AIX, or Apple Mac OS. The Virtual Platforms tab covers support for different hypervisors including VMware vSphere, and the Cloud Platforms tab covers the more modern data protection workloads including AWS, Google, IBM Cloud, Azure, and Alibaba. Compass-as-a-service supports an extensive list of workloads from traditional on-premises infrastructures to native snapshot support for modern cloud workloads with the ability to augment cloud snapshot data protection with custom capabilities.

Finally, ESG explored how Compass manages data replication in today’s highly distributed data protection environments. Not only does Compass leverage deduplication and compression to efficiently move data between locations, but network traffic can be shaped to manage bandwidth during peak hours of operations. As shown in Figure 6, an administrator can set high, medium, and low thresholds for a traffic policy. In this example, high is 200 Mbps, medium is 100 Mbps, and low is 50 Mbps. Then, schedules can be created for different days of the week and different time periods. The admin can throttle down the bandwidth utilization during peak hours and increase it during slower times and off-peak hours. This process helps maintain control of the bandwidth and avoid causing any network problems for primary systems. Multiple traffic policies can be set and enabled on each Accelerator being managed either through global policies or manually on each system.

Why This Matters

IT environments are getting more complex due to higher volumes of data, greater numbers and variety of endpoints, and greater numbers and variety of applications. Protecting all of this without automated tools and intelligence has become a daunting task, which is amplified as an organization expands in volume as well as into multiple clouds and environments.

ESG confirmed that the Compass SaaS model with automation and central controls across all clouds provides several key benefits including the ability to constantly monitor, analyze, and manage all data protection resources including policies, storage, networking, and security. This allows for fast adoption of new technologies and the elimination of hundreds of regular tasks across the entire data protection environment.


Visibility and Intelligent Automation

Born in the cloud, the Compass architecture is designed to take full advantage of modern data protection technologies such as analytics, machine learning, and multi-tenancy. The Executive Dashboard report, as shown on the upper left side of Figure 7, provides a unified summary of an organization’s entire enterprise data protection environment. The dashboard includes reporting on systems by platform, protected system data growth over time, and backup jobs success and failures. Then, all operational management can be executed from the Commander including provisioning new systems, changing schedules, and adjusting management policies. The system provides in-depth, graphical reporting on all metrics needed to provide strong insight into data protection performance, capacity, and backup job status.

As seen in the lower right, Commander displays an overview of all activities being managed. On the bottom row, we see that there are 78 systems in the environment with ten organizations and a total of twelve system users. An organization can be a company and its divisions such as engineering and finance, or, if operating as a managed service provider, each organization could be a unique customer. Sub-organizations can also be established, and role-based access rules set to control who can view and modify resources. The top row shows data moved, jobs, and failures for the past 24 hours, with the same metrics repeated in the middle row for a seven-day period. Commander is fueled by the Compass Analytics Engine, which is continuously learning from the environment. The results of these analytics can be used to improve data protection outcomes for each organization. The Analytics Engine receives metadata, which is pushed by the exhaust manager on the Accelerators at each customer site. The data is analyzed, and appropriate updates are pulled back to the Accelerators for execution at the next scheduled backup. Because only metadata is transferred during this process, very little bandwidth is needed. In fact, in the event that connectivity is severed between the Accelerator and the Analytics Engine, the Accelerator will continue to run, protect data, and restore data accordingly.

As seen in the lower right, Commander displays an overview of all activities being managed. On the bottom row, we see that there are 78 systems in the environment with ten organizations and a total of twelve system users. An organization can be a company and its divisions such as engineering and finance, or, if operating as a managed service provider, each organization could be a unique customer. Sub-organizations can also be established, and role-based access rules set to control who can view and modify resources. The top row shows data moved, jobs, and failures for the past 24 hours, with the same metrics repeated in the middle row for a seven-day period. Commander is fueled by the Compass Analytics Engine, which is continuously learning from the environment. The results of these analytics can be used to improve data protection outcomes for each organization. The Analytics Engine receives metadata, which is pushed by the exhaust manager on the Accelerators at each customer site. The data is analyzed, and appropriate updates are pulled back to the Accelerators for execution at the next scheduled backup. Because only metadata is transferred during this process, very little bandwidth is needed. In fact, in the event that connectivity is severed between the Accelerator and the Analytics Engine, the Accelerator will continue to run, protect data, and restore data accordingly.

Why This Matters

Ransomware is on the rise and represents a serious threat to organizations of every size. According to the FBI, on average, more than 4,000 ransomware attacks have occurred daily since January 1, 2016.2 With Compass Cyber Shield, all backup data is protected in an immutable format, making stored data unchangeable, thus preventing ransomware from ever accessing, encrypting, or deleting backup data. Analytics and monitoring can alert on attacks, and if primary systems become encrypted by attackers, an organization can rely on its backups to quickly recover to the most recent clean state point-in-time copy.


Next, ESG explored how an administrator can drill deeper into a failure condition to determine the appropriate remediation steps to resolve a problem with a backup job. From the home screen dashboard view, we clicked on a failure, which brought us to the Events page. Here, all the backup job statuses are displayed, including those with any problems that may need to be addressed. As shown in the upper left side of Figure 8, the schedule, start time, and completion time is displayed for a suspect backup job on the protection client named quickbooksm Client 00. The middle of the figure shows more detail on the number of Files/Objects Protected, Bytes Protected, and Files/Objects Examined for this backup schedule.

As shown in the figure, we found that the daily incremental backup completed successfully, but some files were not processed. On further investigation, we found that a single QuickBooks file had changed during the backup processing and was skipped, but the rest of the backup job continued successfully. This level of detail allows a system user to quickly assess the level of failure severity and take appropriate action.

Finally, ESG reviewed the reporting capabilities of the solution. As shown at the top of Figure 9, there are three main tiles on this page including Reports, Schedules, and Archives. The Reports tile contains predefined reports that can be immediately run and reviewed or scheduled to run at any interval with the results file automatically sent to designated recipients by email. If reports are scheduled at future times and intervals, they appear in the Schedule section, and after reports are run, whether immediately or scheduled, the results are stored in the Archives section for future reference.

Compass provides more than 30 standard reports that were created based on customer feedback on important data protection visibility metrics. Report types include information on performance, daily activity, failures, and backup statistics as well as auditing information, governance, security, and access management. An administrator can control access rights to the reports and restrict who can actually run the different reports. Custom reports can also be created and managed in this reporting framework, and Cobalt Iron Support can help with their creation.

Why This Matters

In today’s fast-paced business world, a process as critical as data protection cannot be left to manual tasks that require human intervention when every second can cost the organization money and damage to reputation. An organization needs to have full insight and control over all protected systems on the network, strong data analytics to pinpoint areas of concern, and an automated process to recover faster from any type of system failure or data integrity issue, whether accidental or malicious.

ESG confirmed that the Compass visibility and intelligent automation capabilities help organizations address the challenges of quickly detecting, understanding, and responding to changes in their data protection environments. Compass’ process involves leveraging machine learning to identify normal operations and detect and alert on anomalies within the data protection ecosystem. Analytics are leveraged to continuously refine the settings and configurations and keep up with the evolving data protection landscape. Failures and exceptions are analyzed, and data loss is prevented with granular visibility into which applications and files were impacted. This is done by scanning and analyzing the entire environment to map changes in activity over time. The final step is comprehensive reporting to validate the success of the new settings and changes and keep the environment running smoothly.


The Bigger Truth

Digital transformation has fundamentally changed the IT landscape, driving up the diversity of applications, the number and types of devices to be managed, rapid growth in data volume, and hybrid architectures consisting of traditional and cloud-native applications running in on-premises data centers as well as in public and private clouds. These factors make ensuring that applications are properly protected more challenging than it has ever been. In addition to this increased complexity, IT organizations face challenges ensuring that they have the proper skillsets needed to manage these new environments.

The Cobalt Iron Compass solution dramatically increases operational efficiencies by changing hundreds of manual data protection tasks into automated processes and adding a new layer of intelligence and centralized control with reporting for all levels of stakeholders. With Compass, one single dashboard controls how data protection is managed across the whole data protection ecosystem. Compass is a multi-tenant SaaS model with its Analytics Engine driven from 22 global cloud data centers. From these locations, 44 countries are currently serviced. The Commander management application is browser-based and is accessible anywhere with internet connectivity. Accelerators are deployed and live where your data resides, on-premises or in any public or private cloud. Accelerators manage the backup and recovery process and also collect important information about the environment that is exported as metadata to the Analytics Engine. This constant exchange between the Analytics Engine and the Accelerators’ backup systems creates a process that ensures that backups run properly, and that the organization can recover quickly from any event, whether system-wide or related to a single file.

ESG verified that Compass can automate data protection practices and link disparate backup systems into one intelligent and centralized process while also enhancing data security by decreasing recovery time from any data loss issues or even malicious activity such as a ransomware attack. Automation not only eliminates daily tasks; it reduces manpower costs and creates a much faster recovery time. Leveraging machine learning, the Analytics Engine ingests metadata to identify what is normal behavior and alert on abnormalities. These abnormalities can include attempted ransomware attacks on a backup system. With support for hundreds of on-premises systems and multi-cloud, an organization has the flexibility to move, expand, or reduce resources without disruption, while still relying on Commander as the centralized control. Compass automates and simplifies the remediation of data backup and recovery issues, helping to make organizations more resilient, agile, and protected. If you are currently looking to automate data protection and have full centralized visibility and control, ESG believes Cobalt Iron Compass is worth serious consideration.



1. Source: ESG Master Survey Results, 2020 Technology Spending Intentions Survey, January 2020. All other ESG research references and charts in this technical review have been taken from this master survey results set, unless otherwise indicated.
2. Source: FBI Document, How to Protect Your Networks from Ransomware.
This ESG Technical Review was commissioned by Cobalt Iron and is distributed under license from ESG.
Topics: Data Protection