ESG Validation

ESG Technical Review: Optimizing the End-user Experience with Applications Using CloudGenix AppFabric SD-WAN


This ESG Technical Review documents hands-on testing of the CloudGenix AppFabric SD-WAN solution. Testing focused on how the solution helps organizations to optimize application performance over their virtual network connections, monitor that performance continuously with application- and network-focused metrics, and simplify and automate workflows when deploying, configuring, and managing multiple sites.

The Challenges

Recent ESG research uncovered that three of the top five most-cited reasons an organization would consider adopting an SD-WAN solution included simplified network management, increased automation capabilities, and centralized control and configuration (see Figure 1).1 ESG research also found that organizations considered whether or not the solution enabled application-level management of WAN traffic.2

End-users of on-premises, cloud, and SaaS apps demand high levels of app performance to perform their jobs, especially as business priorities constantly change. In traditional WAN architecture, routers continuously monitor network-level metrics, such as bandwidth utilization, packet loss, jitter, and latency, to achieve acceptable levels of app performance for specific app groups (e.g., voice calls vs. enterprise apps). However, these metrics cannot be tracked for individual apps (e.g., vs. Office365 email). While some SD-WAN solutions enable their customers to manage app performance, they vary widely in the metrics used and the granularity to which administrators can define app performance policies. Other solutions may only allow customers to define policies for a predetermined number of apps, decreasing their flexibility.

Organizations require an SD-WAN solution that will help define app performance acceptable to end-users by tracking app metrics that can easily be traced to business objectives, regardless of the number and type of apps used. In addition, the solution will simplify how app-level policies are created, applied, and modified consistently across multiple sites, while optimizing the use of available wide-area network (WAN) resources (e.g., MPLS, Internet, and wireless).

CloudGenix AppFabric SD-WAN

CloudGenix has designed its SD-WAN solution, named AppFabric, to focus on app-centric traffic forwarding. Instead of leveraging only network-level metrics to ensure app performance, the CloudGenix solution also leverages application-level metrics to enable organizations to construct policies for enterprise, cloud, and SaaS apps and deploy those policies once to all sites within the organization’s network: remote and branch offices (ROBO), data centers, and cloud service providers (CSPs). Policies reflect business rules and end-user expectations of app performance that determine the priority and network connections used to send packets during an app session. The solution continuously updates its knowledge of both network and application health and performance metrics to reroute traffic over alternative network paths if an app session is not adhering to defined policy.

Organizations deploy the CloudGenix solution either via physical devices or virtual instances called Instant-On Network (ION) elements. Using a SaaS-based management portal, organizations configure, deploy, and manage these elements in multiple sites. The solution generates app-level and network-level analytics using data collected at the endpoints of application sessions (e.g., between a ROBO and a data center). Organizations can gain near real-time and historical visibility of these analytics via the portal. Issues can be identified and resolved quickly, thus maintaining app performance as demanded by the business.

ESG Lab Tested

ESG Lab performed hands-on testing and evaluation of the CloudGenix SD-WAN solution using a test network located at the CloudGenix headquarters in San Jose, CA. The test bed simulated three ROBOs and two data centers geographically dispersed across the United States.

ESG Lab first examined how to create, edit, and deploy an app-centric policy. We started with the homepage of the CloudGenix Dashboard and clicked on Policies in the menu bar. We chose the policy associated with the Office365 version of Outlook (shown in Figure 2).

ESG Lab began by selecting the organization’s network to apply the Outlook policy. The administrator can deploy and manage policies for the same app on separate networks. We chose the default corporate network.

We proceeded to choose the app priority level, defined by the CloudGenix defaults of Platinum, Gold, Silver, and Bronze (in descending priority). Each level defines the priority with which an app’s traffic is transported relative to other apps. For example, an organization can denote as Platinum, while denoting Facebook Messenger as Bronze.

Next, we chose the network paths used to route Outlook traffic between sites. ESG Lab navigated to the Paths tab. Via dropdown menus, we chose three paths: via a direct link to the Internet, via an Internet VPN link to one physical site (e.g., data center), and via MPLS VPN link to a ROBO. All paths were assigned as “Active,” so that Outlook traffic would be sent over any path as long as app performance remained acceptable. The CloudGenix solution offers other network link options including direct link via Internet Cable and Metered 3G/4G/LTE Internet. Administrators have the option to define “Backup” paths in case all “Active” paths can no longer deliver acceptable app performance.

The CloudGenix solution can simplify how an organization defines and deploys app policies. Typically, an administrator would configure multiple routers individually, each with multiple network-level policies and IP addresses to accommodate every possible app used within an organization. Instead, an organization can create and deploy an app policy once across its sites. ESG Lab noted how this approach can save administrative time and resources, thus decreasing business downtime.

ESG Lab next examined how an organization defines an app-level metric for a single policy. From the portal’s home screen, we clicked on Policies then Apps and navigated to the policy named rtp, a real-time audio app, as seen in Figure 3. We focused on two parameters associated with app-centric metrics: the soft limit and the hard limits that define the organization’s thresholds of acceptable app performance.

As the CloudGenix solution collects data, it is compared to limits assigned to relevant app metrics. If the metric crosses the soft limit, the CloudGenix solution will continue to transmit packets for existing app sessions but will cease to initiate new sessions on the app’s defined active paths. End-users of only the existing app sessions experience degraded performance (e.g., email taking longer to download). If the metric crosses the hard limit, the CloudGenix solution will cease all sessions and move them to alternative paths.

For rtp, the relevant metric is the mean opinion score (MOS), a numerical ranking of the perceived quality of voice communications, expressed on a scale from one to five. ESG Lab set the soft and hard limits at four and three, respectively. If the MOS falls below four, the CloudGenix solution will cease new voice calls on the current active paths, and end-users of existing voice calls will experience degraded performance. However, once the MOS falls below three, all voice communications will seamlessly move to an available backup path.

With this control over app performance, organizations can leverage the soft and hard limits to reduce downtime and maintain productivity. The soft limit warns the CloudGenix solution that app performance may be degrading, yet does not move active app sessions, thus not disrupting business activities. For example, looking up data may take longer, but the user still has access. Remediating app performance does not have to result in switching app sessions onto backup paths (i.e., crossing the hard limit), as this can impact user experience, such as a user logging back into if the session hangs. Managing the limits also contributes to network resiliency, as the soft and hard limits will dictate how the underlying network reacts to degraded app performance without manual intervention.

An administrator can set soft and hard limits for internally developed apps as well as apps recognized by the CloudGenix solution. As illustrated in Figure 3, the administrator can set these limits for a variety of app types, including transactional, bulk, real-time audio, and real-time video. In our testing, the solution presented relevant metrics for each app type, such as failure percentage for transactional apps. We noted that custom app definition can gauge whether or not internal apps, especially those tied to business objectives, are performing as the organization demands.

Next, ESG Lab reviewed the app performance analytics. We navigated to the screen in Figure 4 by clicking on Activity from the home screen menu. While the portal provides summary views of all metrics across all apps, an administrator can focus on single apps when troubleshooting potential issues. We viewed metrics from the Chicago site associated with, specifically App Health and App Response Time. The App Health panel indicated the reachability of over the app’s allowable network paths from the Chicago site. In the App Response Time panel, we noted that while is a SaaS app, the CloudGenix solution can still gather data on the CSP side and calculate metrics such as server response time (SRT) and round-trip response time (RTT).

Administrators can leverage multiple metrics in real time to identify app performance issues and decrease resolution time. For example, we noted that the number of failures (noted as red in the Transaction Stats graph in Figure 4) slightly increased shorty after timestamp 10:30. An administrator can correlate that observation with the decreased reachability of over the Direct Internet path, as indicated under the App Health panel.

When end-users experience slow app performance, they commonly blame the network. ESG Lab noted that CloudGenix analytics can help identify root causes quickly. In the example, the administrator may see that server response time (SRT) and round trip time (RTT) are high, thus identifying the CSP as the likely cause of degraded performance.

ESG Lab also observed how an administrator can monitor performance across multiple apps simultaneously, rather than collecting performance data for individual apps from an organization’s data center and manually analyzing it. Decreasing the effort to track performance of multiple apps can further decrease IT administration time and costs.

ESG Lab then reviewed how the CloudGenix solution can simplify and automate IT workflows. We first reviewed the deployment of a physical element by navigating to the Devices list from the Map view. We saw a list of deployed sites, claimed devices, and unclaimed devices. CloudGenix requires an organization to “claim” a device via an authentication process so that no other customer uses it.

After claiming the device, ESG Lab assigned the device to a specific site by entering the location of a branch office or data center. (The solution also allows an organization to import a list of existing sites to facilitate this assignment.) Assuming that a local administrator has properly cabled the device at the site, we configured the network paths. The administrator can configure the device with Internet and/or Private WAN connections; we configured Internet connections. We clicked on Add Circuit and saw two menus to choose the type of network paths and its service provider, named Circuit Category and WAN, respectively (see Figure 5).

For the Chicago site, we defined an Internet DSL connection from Megapath as an Active path. We also edited the maximum downstream (“Link Down”) and upstream (“Link Up”) connection speeds. The administrator can set Bandwidth Configuration at either Manual or Automatic, which defines the available bandwidth on the network connection at any given time. The administrator can also monitor bandwidth capacity in real-time using CloudGenix’s proprietary machine-learning technology. This level of control and management of bandwidth improves capacity planning tasks for optimizing app performance.

While the CloudGenix solution simplifies the deployment of a single site, an organization may need to deploy multiple sites at once. ESG Lab observed how the solution can automate and scale deployments. Figure 6 shows the highlights of CloudGenix’s DevOps-based (or programmatic) site deployment.

We began by viewing how the CloudGenix automated configuration API works. The API code contains a template denoting site configuration details, such as physical device numbers, IP addresses, and network paths. This template contained details for deploying a physical device at Corvalis, OR. After initiating the code, the code created the site’s network connections (e.g., an AT&T MPLS WAN interface), claimed the device, and updated the device to the latest software version. When the process completed, we refreshed the network map and the Corvalis site appeared.

ESG Lab then clicked on the Corvalis site’s icon to reveal the configuration summary. We clicked on the device name UTSAC-3k1 (ion 3000) to navigate through the configuration details as shown in Figure 7. We saw that the code configured ports with IP and MPLS circuits, IP addresses, and static IP routes as specified in the template.

If an administrator wants to change one configuration detail without navigating to each site individually via the SaaS portal, this change can be done via the template. To illustrate, ESG Lab changed the static IP route address noted in Figure 7 from .59 to .60 (see Figure 8).

In the site template, we navigated to the static route 2.0 module, modified the destination prefix, and committed the code. We ran the template, and it deleted the old static route and created the new one. ESG Lab noted that the template only made the one change without recreating the configuration details of all sites. This lets organizations simplify and automate workflows, such as site deployment and change management, saving valuable time and resources. In addition, the organization decreases its overall downtime, as administrative workflows are expedited.

Why This Matters

As organizations evaluate SD-WAN solutions, managing app performance is emerging as a factor to consider. End-users are demanding a great user experience, especially when attempting to complete critical business tasks. In the end, however, organizations want their solutions to decrease network and administrative costs as well as time to value for customers.

The CloudGenix AppFabric SD-WAN solution enables an organization to manage app-level performance that will help end-users complete their tasks and achieve business objectives. The solution relies primarily on app metrics to decide automatically how app sessions are routed over available network paths between geographically dispersed sites. The organization can rely primarily on app-level metrics to monitor the network’s ability to deliver acceptable app performance, regardless of the number and type of apps employed by end-users. CloudGenix also enables its customers to simplify IT workflows via its SaaS-based management portal, or to automate these same workflows programmatically, thus decreasing time and resources spent on network administrative activities.

ESG Lab verified that the CloudGenix solution enables an organization to create and deploy app-level policies centrally via the SaaS-based management portal. We saw how an administrator can set thresholds for individual app-level metrics, which dictate how the network reroutes traffic once they are crossed. Our testing also validated that the analytics help in detecting performance-impacting issues within and across apps, decreasing resolution time. Finally, we observed how an organization can leverage a DevOps approach using code templates to simplify and/or automate tasks such as site deployment and configuration and change management.

The Bigger Truth

Beyond simplifying network configuration and management, app-level management and control is emerging as a key factor to consider when investing in SD-WAN. While other vendors may leverage network metrics only to resolve app performance issues, this approach can prove ineffective, especially as these metrics (e.g., packet loss or jitter) do not translate easily into business objectives. By focusing on how well applications perform, via metrics such as transaction time, especially those apps that directly affect the business’ bottom line, an organization can easily identify issues with the most potential negative impact and act faster to resolve them.

CloudGenix offers an SD-WAN solution in AppFabric that focuses on fulfilling app performance policies based on business rules and end-user expectations. While the CloudGenix solution does not completely remove all manual intervention, it can ease an administrator’s workload in many areas: defining and deploying metrics and policies governing multiple apps across multiple physical locations, tracking app and network analytics to identify and resolve issues that potentially degrade app performance, and simplifying and automating routing workflows (such as site configurations). By decreasing the overall workload, the organization can lower the time and resources spent on network-related administrative tasks.

ESG Lab validated that an organization can leverage the CloudGenix AppFabric SD-WAN solution to manage app performance across its network, along with the benefits of simplified site configuration, app and network-level analytics, and automated workflows. If your organization seeks an SD-WAN solution that takes a business-focused approach to managing app performance, as opposed to using network-level metrics, we suggest that you take a closer look at CloudGenix.

1. Source: ESG Master Survey Results, Trends in Network Modernization, November 2017.
2. ibid.
This ESG Technical Review was commissioned by CloudGenix and is distributed under license from ESG.
Topics: Networking Cloud Services & Orchestration