Validation

Co-Author(s): Tony Palmer

Introduction

This ESG Technical Validation explores the performance and efficiency benefits of Intel Data Center SSDs as a storage media foundation for all of the stages of the AI data pipeline.

Background

ESG research on AI and machine learning (ML) reveals that AI and ML are already widely adopted, and are marching toward ubiquity, with 50% of respondents confirming that they have already adopted AI/ML technology, and 50% expecting to use AI/ML within 12 months of the survey.¹ ESG also asked organizations which technology features are—or likely will be—most important in their consideration of the infrastructure solutions used to support AI/ML initiatives. As seen in Figure 1, storage is the most cited technology consideration for AI infrastructure.

With the rising demand for instantaneous, actionable insight, organizations have found that the adoption of data lakes is an effective approach for anchoring modern data platforms that look to serve multiple business units and processes that cross business unit domains. ESG asked organizations about data lake usage and 60% responded that they are either already using a data lake, are planning to implement one, or are evaluating the technology. When asked about their objectives for utilizing a data lake technology solution, the most cited response was to improve scalability, cited by 39% of responses.² Clearly, AI/ML is fast becoming a business-critical component of IT, and highly performant and scalable storage is a key to success.

Intel Data Center SSDs for the AI Data Storage Pipeline

Across the AI data pipeline, I/O requirements are unpredictable, widely variable, and extremely demanding. Data set sizes have a wide range; petabytes of raw ingested data are refined down to gigabytes of structured and semi-structured data for training and complete their journey as kilobyte-sized trained models. Workloads and access patterns vary significantly as well, and AI storage must satisfy the demanding requirements of high throughput and extremely low latency.

AI storage workload requirements start with ingest, where raw data is collected concurrently from various sources. Ingest is mostly sequential writes, periodically destaging to capacity storage. Ingested data is used in the training phase. Preparation is an iterative process that incorporates highly concurrent reads and writes, with random and sequential access patterns. Training needs ultra-low latency, high bandwidth random reads to maximize model accuracy while minimizing training time. Inference is where the trained model is used to make decisions. Storage requirements are complex here as well; low latency reads and writes speed both time to insight and real-time decision making.

Leveraging storage that is efficient, performant, available, and flexible enough to service performance and capacity requirements tiers of the AI data pipeline is a key to success when architecting a future-proof AI infrastructure. Typical HDDs and SSDs are well suited to meet the growing capacity requirement of AI workloads, but pairing Intel NAND SSDs with performance-optimized Intel Optane SSDs yields a tiered approach that’s ideally suited to address the AI business objectives across all of the phases of the AI data pipeline (see Table 1).

A comparison of the storage media specifications for a pair of prototypical Intel NAND SSD and Intel Optane SSD devices is summarized in Table 2. Intel Optane SSDs are 3.7 times faster for the write workloads during the ingest, preparation, and inference phases of the AI pipeline, and 3.6 times faster for training and inference read workloads.

Storage media endurance is an especially important consideration for emerging AI workloads that are ingesting massive real-time data sets. The number of full drive writes per day (DWPD) that a storage device can sustain before it “wears out” is a good way to compare the endurance of NAND and Optane SSDs. A 60 DWPD endurance rating of Intel Optane SSD (20x better than NAND SSD) is well suited for the write intensive nature of the ingest and preparation phases of the AI pipeline. The next section of this report explores how these low level device specifications translate into application level performance benefits for each stage of the AI pipeline.

ESG Technical Validation

ESG leveraged performance benchmark results that were documented in previously published and publicly available reports from ESG, Splunk, and the University of California San Diego with a goal of quantifying the performance benefits of Intel Data Center SSDs for each stage of the AI data pipeline. Instead of using AI and machine learning storage performance benchmark tools that weren’t generally available when this report was written, the I/O characteristics of each phase of the AI pipeline were cross referenced with existing benchmark results with a goal of quantifying the performance benefits of Intel Data Center SSDs.

Ingest

ESG first looked at AI ingest, where raw data is collected concurrently from various sources with a goal of accelerating the time that’s needed to collect data for insight and analysis. The I/O profile for AI ingest is typically 100% sequential writes.

This test compared the performance of Optane SSD to traditional NAND SSD for 32KB index writes during a real-world balanced Splunk Enterprise analytics workload with a mix of ingest, index, and search operations occurring in parallel.⁵ As shown in Figure 3, Intel Optane SSDs delivered more IOPS (22%) with significantly faster latency (59%).

A second ingest test compared the performance of Optane SSD and traditional NAND SSD for write caching during a 64KB sequential write workload with a 1.2TB working set for 75 minutes.⁶ As shown in Figure 4, Intel Optane SSDs sustained a high level of ingest workload throughput (2,500 MB/sec) during an aggressive cache destage to capacity storage, where NAND SSD performance dropped by 39% and stayed there for the remaining hour of the test.

Preparation

Because of the wide variety of sizes, formats, completeness, and accuracy of raw data, ingested data needs to be prepared for use in training. Data that is missing or incomplete should be enriched or ignored. Data inconsistencies such as decimals versus commas in numbered data sets must be standardized. Data with different attributes such as images for facial recognition must be normalized. Unstructured data requires tagging and annotation. Data may be combined from different sources. Finally, the data will need transformation to the target format such as TensorFlow. This is an iterative process of varying amounts of data that are read and written, both randomly and sequentially.

This iterative process drives a mixed workload with a high degree of concurrency. The read-write ratio will vary depending on the veracity of ingested data and the level of transformation required to achieve the target format. Worst case workloads can approach 50% writes.

This test compared the performance of Optane SSD to traditional NAND SSD for a 50% read, 50% update MongoDB workload.⁷ As shown in Figure 5, the Intel Optane SSD sustained 2.3x more operations per second than NAND SSD.

Training and Inference

Training is extremely resource intensive as it involves a repetitive set of steps that execute a sequence of mathematical functions on prepared data to identify the probability of obtaining the desired result. Results are then evaluated for accuracy and, if not acceptably high, the mathematical functions are modified and then executed again. Training is conducted with mostly random reads and some writes for checkpointing, so training benefits from storage that can sustain ultra-fast, high bandwidth random reads. Faster reads keep valuable training resources utilized and randomness helps improve model accuracy. Mitigating I/O wait time is critical at this phase.

This test compared the performance of Optane SSD to NAND SSD for training workloads by examining the search run times of a Splunk Enterprise analytics workload for search head storage media during a balanced workload test that was running a mix of ingest, index, and search operations in parallel.⁸ As seen in Figure 6, Intel Optane SSD completed the query in 0.2 seconds—6.5x faster—with 93% shorter I/O response times (0.05 milliseconds for Optane SSD versus 0.74 for NAND SSD).

This test also included performance testing of a typical commodity hard disk drive (HDD) for Splunk Enterprise search head storage media. Compared to a typical HDD, searches completed 3.9 times faster with Intel SSD and 25.5 times faster with Optane SSD.⁹

Next, ESG looked at inference, where the trained model is deployed to execute decisions. Inference can be deployed in the data center or—increasingly—on edge devices. Data movement involves reading the trained model from storage into inference, writing ingested data that is being evaluated into inference, and reading inferred results back into training to improve model accuracy.

This test compared the performance of Optane SSD to NAND SSD for inference workloads training workloads with a 4KB random I/O workload composed of 70% reads and 30% writes.¹⁰

As shown in Figure 7, the Intel Optane SSD was able to sustain 32% more I/O operations per second with 24% faster I/O response times compared to NAND SSD.

The Bigger Truth

ESG research reveals that organizations recognize that AI and ML can help them achieve multiple, strategic business objectives, including improved operational efficiency (60%), stronger cybersecurity (47%), better customer satisfaction (45%), and improved quality of products and services (44%). It should come as no surprise that 100% of organizations surveyed are either already leveraging AI/ML or are planning to in 2020.

In addition, enterprises are increasingly deploying data lakes or common data pipelines to reduce complexity, increase efficiency, and most importantly, derive more and better insights by building multiple instances of AI usages, data analytics, and reporting on these data lakes. This approach will challenge the performance storage layer, with the need to support high workload variability and high concurrency, while consistently delivering predictably low latency. As these organizations realize more value from AI, the amount of stored data will continue to increase at a quickening pace, stressing the ability of capacity storage to scale efficiently.

Intel SSDs are designed to offer the performance required for the I/O demands of AI/ML workloads using Intel Optane technology's consistently low latency to enable faster time to trained models, and high throughput, even through buffer destage, to improve ingest performance. Intel NAND-based SSDs can scale the capacity layer much more consistently and efficiently than HDDs, with better availability.

The performance test results and ESG analysis in this report illustrate how a combination of Intel Data Center SSD technologies can be used to meet the demanding storage requirements of all stages of the AI data pipeline. Intel Optane SSDs ingested data 59% faster than traditional NAND SSDs while avoiding a 39% performance impact of write cache destaging. During the AI preparation phase, more than twice the number of operations per second were achieved with Intel Optane SSDs (2.3x more). During the training phase, analytics search runtimes completed 3.9 times faster than a typical HDD with Intel SSDs and 25.5 times faster with Intel Optane SSDs. Workloads that simulate the access patterns and I/O characteristics of both training and inference proved that Intel Data Center SSDs provide higher performance, faster run times, and lower response times.

The results presented in this report were harvested from a variety of publicly available benchmarks from ESG, Splunk, and the University of California San Diego. The I/O characteristics of each phase of the AI pipeline were cross referenced with existing benchmarks with a goal of quantifying the performance benefits of Intel Data Center SSDs. While these discrete results demonstrate how Intel Data Center SSDs improve performance at each stage of the AI data pipeline, ESG looks forward to testing with a single storage solution due to the fact that reference architectures and emerging best practices are using a common storage infrastructure for all stages of AI data pipeline data processing.

If your organization is looking to build an efficient, scalable, future-proof AI infrastructure, leveraging storage that is efficient, performant, available, and flexible enough to service both performance and capacity tiers is a key to success. ESG believes that you should consider the advantages of Intel Data Center SSDs to optimize, store, and move larger, more complicated data sets through the AI pipeline and get to insight quicker.

Appendix

The performance benchmark configurations referenced in this report are summarized in Table 3.