Overview

<< Parallel File Systems | Overview | Disks and Interfaces >>

The term storage means different things to different people, so we will concentrate on defining and explaining several different HPC storage solutions. In HPC, your storage design is always driven by your applications mix and needs. Some codes may require local disk drives in each of your nodes for scratch, while other designs implement Terabytes (TB) or even Petabytes (PB) of high-speed network connected storage. Storage technologies cover a broad range of options and capabilities, each with its own strengths and disadvantages, and we'll attempt to explain the technology as well as some of the trade-offs in these pages. As always, contact your Aspen sales engineer for the latest information and to discuss your specific storage requirements.

 

It always starts with the disks!

 

Many HPC solutions may perform adequately with lower priced disk drive technologies such as SATA , while others may require SCSI , SAS , or even SSD implementations for faster access.

 

Every solution is a trade-off. SAS drives are quite fast and reliable, but they come in smaller sizes by default and and are more expensive. SATA drives have large capacities and good value, but they may be less reliable than the same solution implemented using SAS drives. SCSI is becoming less common, but drive sizes, performance, reliability, and prices closely match SAS drive charateristics. SSD is very fast, but expensive, generally smaller in capacity, and suffers from shorter life cycles. SSD technology is currently best used for metadata or data caching, and generally isn't deployed as the only disks in the entire storage solution.

 

Your storage solution will almost certainly include a RAID implementation.

 

Almost all performance storage solutions utilize RAID technology to ensure data integrity and increase data transfer speeds and reliability. Different RAID levels are used for different purposes. RAID 1 mirrors are used on service and critical nodes in a cluster to protect the operating system, configurations, and data kept there, ensuring continued operation of critical cluster services if a disk fails.

 

RAID 5 and RAID 6 are commonly used to maximize data space on cluster-wide data shares while providing acceptable reliability. With todays' larger drives, RAID 6 is becoming necessary to provide true reliability.  RAID 10 is often the fastest RAID implementation available, and exhibits extremely high reliability in most implementations, but half your raw disk space is used for mirroring. A trade-off must always be done between data reliability, speed, and cost in order to determine the best RAID configurations for your specific implementation.

 

You'll require file systems too.

 

Every storage system will be implemented with a file system, and to make it just a bit more confusing, there are many different file systems to choose from based on your implementation and requirements. Local node file systems may be EXT3 on /boot and / partitions, and data partitions may be provisioned with EXT4, XFS, or JFS in order to increase data access speed, then served out to the cluster via an NFS file system. Parallel file systems such as GFS , GPFS , Lustre , or Panasas parallel NFS may be deployed for higher data performance and to improve your solutions scaling.

 

The storage system architecture is important.

 

The best storage solution architecture for your specific needs will depend on aggregate data speed needs inside your cluster, the enterprise features your require, how reliable or redundant you need the solution to be, what your budget allows you to procure, and, as always, your applications mix. We'll give you some general guidelines in a bit, but first we have to discuss the different architectures that might be used to implement your storage inside your cluster.

 

There are three general classes of storage architectures;

  • Local Disk Configurations - are installed directly into your nodes, either in the compute nodes to house an operating system or local scratch space, or as larger RAID systems which are installed in head nodes or dedicated storage nodes to serve larger data spaces to the rest of the cluster. Local disk configurations are not sharable at the block level with other nodes. Instead, they are shared out using NFS or parallel file systems to the other nodes in the cluster.

    Compute node implementations can be single or multiple disks installed with or without RAID hardware or software implementations, depending on what local data access speed is needed on the nodes. Head node implementations are normally hardware RAID cards connected to disks that are installed directly into the chassis, or installed in JBOD units that use external SAS expander cables connected to the external port of the servers RAID card. This is a very cost effective storage implementation, but can have data access speed and management limitations.
  • NAS configurations - are stand-alone, customized hardware appliances that connect to the cluster or organizational network to share data space. They are normally customized for performance and reliability, and provide uniform administrative interfaces. Depending on the vendor, they serve either NFS or parallel file systems to your cluster. Clustered NAS systems exhibit very good scaling. Often adding another unit will increase data access speed linearly.

    NAS implementations generally provide customized management tools that make it easier to expand the system, diagnose faults, or tune performance parameters. Most NAS implementations are capable of multi-protocol file serving, and enable both your HPC solution and other organizational data clients (such as Windows workstations) to access the same data space using different protocols. They interoperate with industry standard back-up solutions well, and can have other reliability enhancements designed to ensure continuous service and high availability. All these features come at a cost, however; NAS solutions can be expensive.
  • SAN configurations - connect several or many server nodes to shared disks via Fibre Channel or another high speed network technology. A SAN configuration is used to share disk assets to multiple nodes at the physical device level and allows multiple servers to access the same data partition. This makes SANs ideal for use when deploying parallel file systems in an HPC solution. Allowing access to the same data partition by the servers allows server processing power and network data sharing speed to increase almost linearly up to the limits of the SAN interconnect or disk solution serving speed. SAN configurations are quite robust and standardized, with very good performance characteristics, but they can be quite expensive.

<< Parallel File Systems | Overview | Disks and Interfaces >>

Bookmark and Share