IT infrastructure budgets are not growing at the pace of data growth and are under tremendous pressure to maximize returns on infrastructure. Hence, enterprises have to rethink how data is captured, preserved, accessed, and transformed. Performance, economics, and endurance of data at scale are paramount. Non-volatile memory express (NVMe) has a significant influence on businesses and what they can do with data, particularly fast data for real-time analytics and other emerging technologies.
Over the years, CPU and DRAM speeds have increased, but mechanical drives and interfaces used to connect them are not that efficient. The NVM Express Work Group is a leading technology manufacturer who developed NVMe in 2009 to narrow the growing gap between fast DRAM and slow disk (FC, SAS, and SATA) storage.
What is NVMe?
NVMe (non-volatile memory express) is a new protocol for accessing high-speed storage that brings many advantages compared to legacy protocols. It is a host controller interface and storage protocol created to speed up the transfer of data between enterprise and client systems and SSDs over a computer's high-speed Peripheral Component Interconnect Express (PCIe) bus. NVMe drives are much faster and use the PCI-E bus, so they can make use of that speed when compared to SSDs and M2 SATA. PCIe, also known as PCI Express, is an I/O interconnect interface that was designed as a high-speed replacement for aging standards such as PCI and AGP. PCIe's features — scalability, latency, and performance — make it a compelling interface choice for storage applications and solid state storage.
Serial ATA (SATA™), with its third generation six Gb/s bandwidth, is currently the most widely used interface for hard disk drives (HDDs). The Advanced Host Controller Interface (AHCI), the technical standard that specifies the operation of SATA host bus adapters, was designed to connect the CPU/memory subsystem with much slower rotating media-based storage. This renders the AHCI inefficient for the much faster SSD, which behave more like DRAM than spinning media. AHCI, although compatible with SSDs, was designed for hard drives. It was developed to connect the CPU/memory subsystem with rotating storage media subsystem. The table below summarizes the differences between NVMe drive and AHCI device interfaces.
|
Key Parameters |
NVMe |
AHCI |
|
Maximum Queue Depth |
64K queues |
1 command queue |
|
64K commands per queue |
32 commands per queue |
|
|
Un-cacheable Register Accesses (2K cycles each) |
2 per command |
6 per non-queued command |
|
9 per queued command |
||
|
Efficiency for 4KB Commands |
Command parameters in one 64B fetch |
Command parameters require two serialized host DRAM fetches |
|
Parallelism and Multiple Threads |
No locking |
Requires synchronization lock to issue command |
|
MXI-X and Interrupt Steering |
2K MSI-X interrupts |
Single interrupt, no steering |
How do NVMe Commands Work?
As illustrated in the diagram below, the host writes I/O command queues to doorbell and doorbell gets registered (I/O Commands Ready Signal); the NVMe controller then picks the I/O command queues, executes them, and sends I/O completion queues followed by an interrupt to the host. The host records I/O completion queues and clears door register (I/O Commands Completion Signal). This translates into significantly lower overheads compared to SAS and SATA protocols.
SAS vs NVMe:
The current storage systems use SAS (serial-attached SCSI) links from their processors to communicate to flash disks in single queues. SAS links use legacy protocol SCSI, which was designed for spinning disks, and this has a serial bottleneck to communicate with flash because of its single queues with 254 and 32 entries, respectively. The SAS link creates performance bottlenecks for the back end of all-flash. The NVMe over PCIe protocol holds the potential to eliminate the SAS (SCSI) bottleneck. NVMe storage brings massive parallelism that supports multiple I/O queues, up to 64K, with each queue having 64K entries and lockless connections that can provide each CPU core with dedicated queue access to each SSD. NVMe storage is minimizing CPU overhead on data transfers and even provides the capability of changing their priority based on workload requirements.
Salient Features of NVMe Compared to Traditional Mechanical Disk Storages:
- NVMe technology reduces latency with higher IOPS
- Scalability for current and future performance
- Support for standard security protocols
- In random workloads, NVMe delivers 2X performance of SAS 12 Gb/s and 4-6X of SATA 6 Gb/s
- For sequential workloads, NVMe delivers 2X performance of SAS 12 Gb/s and 4X of SATA 6 Gb/s
NVMe Will Setback SAS Away:
With thousands of queues, NVMe enables compute and storage in massively parallel communication similar to high-performance processor architecture. NVMe’s huge throughput benefit over SAS is required to gain the advantages of tomorrow’s improvements in multi-core CPUs, superdense SSDs, new memory technologies, and high-speed intersects.
SCSI-less Server to Storage Connection: In addition to NVMe, NVMe/F (NVMe channeled over RDMA-enabled fast Ethernet, FC, Infiniband, or other fabrics) is another new protocol for dramatically faster, SCSI-less server-to-storage connections.
Better TCO with NVMe: NVMe can handle difficult application workloads with a smaller infrastructure footprint that delivers faster access and consumes less power, thus reducing total cost of ownership (TCO) for enterprises.
Data Reduction: Traditional storage has data reduction features that is also called ‘deduplication/compression.’ Deduplication, often called intelligent compression or single-instance storage, is a process that eliminates redundant copies of data and reduces storage overhead. Data deduplication techniques ensure that only one unique instance of data is retained on storage media. Redundant data blocks are replaced with a pointer to the unique data copy. It’s very difficult to use data reduction technology on traditional storage because of its speed limit, low latency, tedious mechanical spindle technologies, and poor IOPS. So, traditional storage array enterprise applications do not use those data reduction features which have a direct impact on per GB cost. With NVMe technology, these features can be used for enterprise applications because of its low latency and higher IOPS features. It gives control on data growth with a smaller footprint and lowers the per GB cost.
All major storage vendors have already started working on NVMe-based storage arrays because it is a next-generation storage technology. Today, we can see NVMe-based storage array that supports petabytes of data storage with its data reduction features from Dell EMC, IBM, HP, NetApp, and Pure Storage. Pure Storage recently announced NVMe-based storage with nondisruptive upgrade option from its traditional SSD-based storage technologies. Also, the Pure Storage array has the option of combining both SSD flash disk and NVMe modules.
PureStorage recently announced NVMe based storage with non-disruptive upgrade option.
