Flash Storage/NVMe

What is NVMe?

Non Volatile Memory Express (NVMe) is a scalable communications interface for Solid State Drives (SSDs) that’s designed to address the needs of high-end systems that utilize PCIe-based SSDs – unlocking throughput potential with reduced latency and increase Input/Output operations per second (IOPs).

While SSDs provide undisputed performance increases over Hard Drive Disks (HDDs), their true advantages are locked behind the limitations of the common interface technologies used to connect them. For instance:

  • SATA is limited to 6Gbps and 600 MB/s theoretical peak throughput
  • SAS is limited to 12Gbps and 2.4GB/s theoretical peak dual port throughput

Serial ATA (SATA) and Serial Attached SCSI (SAS) are bottlenecking performance of SSDs – but they are capable of far more with an NVMe interface.

aspen systems nvme storage compilation

Speak with one of our system engineers today

Benefits of NVMe

6x Throughput

Unlike SAS and SATA, which allow for one command queue capable of 254 and 32 commands respectively, NVMe enables 65,000 queues with 65,000 commands per queue, preventing bottlenecks associated with high volume I/O requests.

Measurement NVME SSD SATA3/SAS3 SSD NVMe Improvement
Bandwidth vs. SATA3 128K Sequential Read 2788.77 MB/s 470.60 MB/s 5.93x
128K Sequential Write 1838.98 MB/s 453.33 MB/s 4.06x
Bandwidth vs. SAS3 128K Sequential Read 2788.77 MB/s 980.34 MB/s 2.84x
128K Sequential Write 1838.98 MB/s 421.25 MB/s 4.37x

Performance Measurements

For the bandwidth test, Supermicro ran the IOMeter benchmarking tool on two systems model number SYS-1028U-TNR4T+ Ultra 1U servers, the first with standard SAS3 and SATA3 SSDs, and the second with NVMe SSDs. The upgraded configuration with NVMe delivered a maximum of 5.9x the performance of the server with standard SSDs.

7x Reduction in Latency

With a direct PCIe bus, NVMe sits directly between the device driver and the PCIe device, enabling the storage I/O capabilities of flash to be fed to the CPU much faster. This allows modern, multi-core processors to complete more useful work in less time.

Measurement NVME SSD SATA3/SAS3 SSD NVMe Improvement
Latency vs. SATA3 4K Sequential Read 15.9 µs 47.3 µs 2.97x
4K Sequential Write 17.6 µs 51.4 µs 2.92x
Latency vs. SAS3 4K Sequential Read 15.9 µs 114.1 µs 7.18x
4K Sequential Write 17.6 µs 109.853 µs 6.24x

Performance Measurements

For latency, the table shows the upgraded configuration with NVMe performed with up to 7.2x improved latency compared to the servers with standard SSDs.

intel nvme storage density

Storage Density

By breaking out of the 2.5″ mold from the 80’s (built around high density spinning drives), modern NVMe SSD form factors allow data centers to have more storage per unit of rack space (over 1PB per 1U chassis possible), while also reducing cooling costs.

Form Factors

NVMe comes in a number of form factors that allow for flash storage to be designed into a very small space, in some cases allowing up to a Petabyte of storage in a single 1U chassis.

M.2

Formerly called Next Generation Form Factor (NGFF), the M.2 specification replaced the mSATA standard for internally mounted computer expansion cards and associated connectors. Primarily used in small devices such as ultrabooks and tablets, the M.2 is still used in HPC today, primarily as boot or OS drives.

m2 kingston dc1000b data center series nvme

Speak with one of our system engineers today

samsung pm1733 nvme

U.2

Formerly known as SFF-8639, the U.2 is a 2.5″ SSD form factor that was developed with the intent to be usable with existing 2.5″ and 3.5″ enclosures, allowing newer PCIe drives to be mixed with legacy SAS and SATA drives in a hot swappable configuration. The U.2 connector is mechanically identical to SAS, but provides 4x PCIe lanes for greater bandwidth and lower latency..

EDSFF

The Enterprise & Data Center SSD Form Factor (EDSFF — Formerly called “Ruler”) is a high-capacity SSD technology from Intel, allowing for up to 1 Petabyte of data in a single 1U chassis server. EDSFF drives are hot-pluggable from the front-end. Intel’s new SSD D5-P4326 Series have been designed from the ground up to optimize capacity per drive and be switched out easily, to making scaling as seamless as possible.

intel edsff nvme