Artificial Intelligence (AI) and Deep Learning (DL) are creating the toughest workloads in modern high performance computing (HPC) history. They pose exceptional challenges to compute, storage and network resources. Traditional storage technologies and protocols like NFS have difficulties in boosting storage performance to feed data-hungry GPUs, and invariably slow down AI applications and hinder business innovations.  New generation HPC & AI workflows calls for high performance storage systems which can provide high throughput, high IOPS (input/output operations) and low latency support.

The article reviews a few aspects related to the high performance storage system for AI applications.

In the last 10 to 20 years, the parallel file system has become dominant in the high performance storage systems.  Parallel File System is a type of distributed file system. The data set is striped and distributed to multiple storage devices. The system uses a global namespace to facilitate data access, which is carried out through simultaneous, coordinated input/output operations (IOPS) between clients and storage nodes.

Parallel File Systems often use dedicated metadata servers to store information about the data, such as the file name, location and owner, and the data is read/written to multiple storage devices using multiple I/O paths concurrently. The coordinated use of multiple I/O paths can provide a significant performance benefit, especially when streaming workloads that involve a large number of client servers.

Two of the most prominent examples of parallel file systems are IBM’s Spectrum Scale, and the open source Lustre file system.

Parallel file system changes the ways data is accessed from the compute servers. It helps the traditional HPC simulations like Computational Fluid Dynamics, Finite Element Analysis and Molecular Dynamics simulations. When it comes to the modern AI applications, it has to incorporate with a few other components in the compute and storage network so that it can achieve highest performance possible.

High Speed Network infrastructure used with the high performance storage will increase the data transfer throughput from the storage system to the computing cluster nodes. With the 100GE Ethernet infrastructure becoming more and more common, the network bandwidth will not be a bottleneck for the data accessing.

Another more common network technology is Infiniband. Infiniband is a computer-networking communication standard used in data interconnect both among and within computers. InfiniBand is also used as either a direct or switched interconnect between servers and storage systems. The network can deliverer non-blocking high performance and consistently low latency across all of its ports.

Traditionally, hard disk drives (HDD) is dominant in the storage system. With the evolvement of solid-state drives (SSD) and the improvement of its price/performance ratio, more and more SSD drive is used in the high performance storage system.

HDD is an electromechanical data storage device that uses magnetic storage to store and retrieve digital information using one or more rigid rapidly rotating disks (platters) coated with magnetic material. Mechanical drives are bad at random read/write, but are good at sequential read/write. Copying lots of smaller files is essentially random read/write and will be slower because of the extra seeking time required. For the deep learning workloads, the average I/O size compared to traditional HPC applications came down significantly. Not like the traditional HPC of which the data size is in megabyte or even gigabytes, AI workloads mostly deal with the data size in kilobytes. It’s no longer a question about the throughput of a system but rather what the random I/O capabilities of the media are. This is where something like SSD flash drive has obvious advantages over HDD hard disk.

Data transfer speed is not the only important performance aspect of a storage device.  In fact, the more important factor affecting the overall performance is how long it takes for a data transfer to begin, called latency. Moving data slowly from one system to another reduces performance, but the time it takes for the data to begin moving, when no useful work is being done, is a huge factor in the performance of data storage devices.

Modern high performance storage systems take the benefits from Remote Direct Memory Access (RDMA) – a protocol running on InfiniBand or 100GE Ethernet (RDMA over Converged Ethernet, aka, RoCE) to reduce the data transfer latency and improve the performance.

With RDMA, the application can perform data transfers without the involvement of the network software stack. Data is sent and received directly to the buffers without being copied between the network layers. RDMA allow applications to perform data transfers directly from user-space without kernel involvement, and without CPU consumption in the remote server.

In summary, to support AI applications, a high performance storage system should be a parallel file system equipped with all SSD drives and a high speed and low latency network with RDMA support.