The NVMe interface, specifically PCIe 3.0 NVMe 1.3, has been the preferable SSD interface for virtually any use case for a while now. This interface has been used time and again as vendors continue to improve in performance with each new generation. PCIe has now entered into 4.0, bringing with it massive gains in sequential and random performance. The KIOXIA CD6 is the latest SSD to hit the market (and one of the first PCIe 4.0 for enterprise), focusing on consistent performance in demanding 24×7 environments. This makes it ideal for data-centers with mixed-use and read-intensive endurance needs. The KIOXIA CD6 SSDs are also SFF-TA-1001 conformant, which allow for connectivity with SAS, SATA and NVMe drives on the same backplane (as well as Tri-mode controllers). The KIOXIA line features capacities ranging from 800GB to 15.36TB and are available in several different security and endurance options. The CD6 also comes in 2.5-inch, 15mm Z-height form factors and is powered by KIOXIA’s proprietary controller, firmware and BiCS FLASH 96-layer 3D TLC memory. It uses a single-port design (i.e., there is one data path from the host to the SSD).
Performance of PCIe 4.0 has the potential to reach some pretty incredible numbers, and the KIOXIA CD6 certainly is no exception. KIOXIA quotes their new driveline to deliver sequential performance up to 6.2GB/s and 4.0GB/s in reads and writes, respectively, while random performance has the potential to hit up to a million IOPS in reads and 250,000 IOPS in writes. These are impressive numbers to say the least.
It also features a range of built-in reliability functionality, including 6th generation drive failure recovery, and double parity protection, and Power loss protection (PLP), end-to-end data correction for extra reliability. The KIOXIA CD6 is NVMe 1.4 compliant, which brings some important features to the drive. This includes persistent event logs, sanitizing improvements and namespace granularity and namespace write protection.
KIOXIA CD6 Performance
Testbed
Our new PCIe Gen4 Enterprise SSD reviews leverage a Lenovo ThinkSystem SR635 for application tests and synthetic benchmarks. The ThinkSystem SR635 is a well-equipped single-CPU AMD platform, offering CPU power well in excess of what’s needed to stress high-performance local storage. It is also the only platform in our lab (and one of the few on the market currently) with PCIe Gen4 U.2 bays. Synthetic tests don’t require a lot of CPU resources but still leverage the same Lenovo platform. In both cases, the intent is to showcase local storage in the best light possible that aligns with storage vendor maximum drive specs.
PCIe Gen4 Synthetic and Application Platform (Lenovo ThinkSystem SR635)
1 x AMD 7452 (2.35GHz x 32 Cores)
8 x 64GB DDR4-3200MHz ECC DRAM
CentOS 7.7 1908
ESXi 6.7u3
PCIe Gen3 Application Platform (Lenovo ThinkSystem SR850)
4 x Intel Platinum 8160 CPU (2.1GHz x 24 Cores)
16 x 32GB DDR4-2666Mhz ECC DRAM
2 x RAID 930-8i 12Gb/s RAID Cards
8 NVMe Bays
VMware ESXI 6.7u3
PCIe Gen3 Synthetic Platform (Dell PowerEdge R740xd)
2 x Intel Gold 6130 CPU (2.1GHz x 16 Cores)
4 x 16GB DDR4-2666MHz ECC DRAM
1x PERC 730 2GB 12Gb/s RAID Card
Add-in NVMe Adapter
Ubuntu-16.04.3-Desktop-amd64
Being the first set of reviews on a new platform, we’ve included past drive results, which are close but not 100% apples to apple comparisons since they were testing on an older platform. Our synthetic test differences won’t have much skew in results, but the application workloads working on the single CPU AMD platform vs Quad CPU Intel platform may to some degree. In our MySQL tests, one of the new Gen4 KIOXIA products did take the lead, but in SQL server latency was average. With only two Gen4 drives we’ve been able to publish around we don’t have a significant amount of comparable data, but it is something to take note of viewing these results. We’ve also ramped up our synthetic tests to take advantage of the faster SSDs, now showing test results with higher peak thread counts.
Testing Background and Comparables
The StorageReview Enterprise Test Lab provides a flexible architecture for conducting benchmarks of enterprise storage devices in an environment comparable to what administrators encounter in real deployments. The Enterprise Test Lab incorporates a variety of servers, networking, power conditioning, and other network infrastructure that allows our staff to establish real-world conditions to accurately gauge performance during our reviews.
We incorporate these details about the lab environment and protocols into reviews so that IT professionals and those responsible for storage acquisition can understand the conditions under which we have achieved the following results. None of our reviews are paid for or overseen by the manufacturer of equipment we are testing. Additional details about the StorageReview Enterprise Test Lab and an overview of its networking capabilities are available on those respective pages.
Application Workload Analysis
In order to understand the performance characteristics of enterprise storage devices, it is essential to model the infrastructure and the application workloads found in live production environments. Our benchmarks for the Kioxia CD6 are therefore the MySQL OLTP performance via SysBench and Microsoft SQL Server OLTP performance with a simulated TCP-C workload. For our application workloads, each drive will be running 4 identically configured VMs.
SQL Server Performance
Each SQL Server VM is configured with two vDisks: 100GB volume for boot and a 500GB volume for the database and log files. From a system-resource perspective, we configured each VM with 8 vCPUs, 64GB of DRAM and leveraged the LSI Logic SAS SCSI controller. While our Sysbench workloads tested previously saturated the platform in both storage I/O and capacity, the SQL test is looking for latency performance.
This test uses SQL Server 2014 running on Windows Server 2012 R2 guest VMs, and is stressed by Quest’s Benchmark Factory for Databases. StorageReview’s Microsoft SQL Server OLTP testing protocol employs the current draft of the Transaction Processing Performance Council’s Benchmark C (TPC-C), an online transaction-processing benchmark that simulates the activities found in complex application environments. The TPC-C benchmark comes closer than synthetic performance benchmarks to gauging the performance strengths and bottlenecks of storage infrastructure in database environments. Each instance of our SQL Server VM for this review uses a 333GB (1,500 scale) SQL Server database and measures the transactional performance and latency under a load of 15,000 virtual users.
SQL Server Testing Configuration (per VM)
Windows Server 2012 R2
Storage Footprint: 600GB allocated, 500GB used
SQL Server 2014
Database Size: 1,500 scale
Virtual Client Load: 15,000
RAM Buffer: 48GB
Test Length: 3 hours
2.5 hours preconditioning
30 minutes sample period
For our SQL Server transactional benchmark, the KIOXIA CD6 had a score of 12,633.7 TPS at 4VMs.
Average latency saw the KIOXIA CD6 have 5.5ms at 4VMs (the same as the CD6), putting it slightly behind the Huawei and Memblaze drives.
Sysbench Performance
The next application benchmark consists of a Percona MySQL OLTP database measured via SysBench. This test measures average TPS (Transactions Per Second), average latency, and average 99th percentile latency as well.
Each Sysbench VM is configured with three vDisks: one for boot (~92GB), one with the pre-built database (~447GB), and the third for the database under test (270GB). From a system-resource perspective, we configured each VM with 8 vCPUs, 60GB of DRAM and leveraged the LSI Logic SAS SCSI controller.
Sysbench Testing Configuration (per VM)
CentOS 6.3 64-bit
Percona XtraDB 5.5.30-rel30.1
Database Tables: 100
Database Size: 10,000,000
Database Threads: 32
RAM Buffer: 24GB
Test Length: 3 hours
2 hours preconditioning 32 threads
1 hour 32 threads
Moving on to the Sysbench transactional benchmark, the KIOXIA posted a leading score of 9,333 TPS at 4VMs.
In Sysbench average latency, we saw the CD6 with a latency of just 13.71ms at 4VMs, again leading the pack.
For our worst-case scenario latency (99th percentile) the CD6 hit 27.48ms at 4VMs, this time trailing the Huawei by a hair (27.22ms).
VDBench Workload Analysis
When it comes to benchmarking storage devices, application testing is best, and synthetic testing comes in second place. While not a perfect representation of actual workloads, synthetic tests do help to baseline storage devices with a repeatability factor that makes it easy to do apples-to-apples comparison between competing solutions. These workloads offer a range of different testing profiles ranging from “four corners” tests, common database transfer size tests, to trace captures from different VDI environments. All of these tests leverage the common vdBench workload generator, with a scripting engine to automate and capture results over a large compute testing cluster. This allows us to repeat the same workloads across a wide range of storage devices, including flash arrays and individual storage devices. Our testing process for these benchmarks fills the entire drive surface with data, then partitions a drive section equal to 25% of the drive capacity to simulate how the drive might respond to application workloads. This is different than full entropy tests which use 100% of the drive and takes them into steady state. As a result, these figures will reflect higher-sustained write speeds.
Profiles:
4K Random Read: 100% Read, 128 threads, 0-120% iorate
4K Random Write: 100% Write, 128 threads, 0-120% iorate
4K Random Read (high load): 100% Read, 512 threads, 0-120% iorate
4K Random Write (high load): 100% Write, 512 threads, 0-120% iorate
64K Sequential Read: 100% Read, 32 threads, 0-120% iorate
64K Sequential Write: 100% Write, 16 threads, 0-120% iorate
64K Sequential Read (high load): 100% Read, 64 threads, 0-120% iorate
64K Sequential Write (high load): 100% Write, 64 threads, 0-120% iorate
Synthetic Database: SQL and Oracle
VDI Full Clone and Linked Clone Traces
Comparables:
Memblaze PBlaze5 C926
Intel P4610
Huawei ES3000 v5
Toshiba PX04
Samsung PM1725a
Performance of PCIe 4.0 has the potential to reach some pretty incredible numbers, and the KIOXIA CD6 certainly is no exception. KIOXIA quotes their new driveline to deliver sequential performance up to 6.2GB/s and 4.0GB/s in reads and writes, respectively, while random performance has the potential to hit up to a million IOPS in reads and 250,000 IOPS in writes. These are impressive numbers to say the least.
It also features a range of built-in reliability functionality, including 6th generation drive failure recovery, and double parity protection, and Power loss protection (PLP), end-to-end data correction for extra reliability. The KIOXIA CD6 is NVMe 1.4 compliant, which brings some important features to the drive. This includes persistent event logs, sanitizing improvements and namespace granularity and namespace write protection.
KIOXIA CD6 Performance
Testbed
Our new PCIe Gen4 Enterprise SSD reviews leverage a Lenovo ThinkSystem SR635 for application tests and synthetic benchmarks. The ThinkSystem SR635 is a well-equipped single-CPU AMD platform, offering CPU power well in excess of what’s needed to stress high-performance local storage. It is also the only platform in our lab (and one of the few on the market currently) with PCIe Gen4 U.2 bays. Synthetic tests don’t require a lot of CPU resources but still leverage the same Lenovo platform. In both cases, the intent is to showcase local storage in the best light possible that aligns with storage vendor maximum drive specs.
PCIe Gen4 Synthetic and Application Platform (Lenovo ThinkSystem SR635)
1 x AMD 7452 (2.35GHz x 32 Cores)
8 x 64GB DDR4-3200MHz ECC DRAM
CentOS 7.7 1908
ESXi 6.7u3
PCIe Gen3 Application Platform (Lenovo ThinkSystem SR850)
4 x Intel Platinum 8160 CPU (2.1GHz x 24 Cores)
16 x 32GB DDR4-2666Mhz ECC DRAM
2 x RAID 930-8i 12Gb/s RAID Cards
8 NVMe Bays
VMware ESXI 6.7u3
PCIe Gen3 Synthetic Platform (Dell PowerEdge R740xd)
2 x Intel Gold 6130 CPU (2.1GHz x 16 Cores)
4 x 16GB DDR4-2666MHz ECC DRAM
1x PERC 730 2GB 12Gb/s RAID Card
Add-in NVMe Adapter
Ubuntu-16.04.3-Desktop-amd64
Being the first set of reviews on a new platform, we’ve included past drive results, which are close but not 100% apples to apple comparisons since they were testing on an older platform. Our synthetic test differences won’t have much skew in results, but the application workloads working on the single CPU AMD platform vs Quad CPU Intel platform may to some degree. In our MySQL tests, one of the new Gen4 KIOXIA products did take the lead, but in SQL server latency was average. With only two Gen4 drives we’ve been able to publish around we don’t have a significant amount of comparable data, but it is something to take note of viewing these results. We’ve also ramped up our synthetic tests to take advantage of the faster SSDs, now showing test results with higher peak thread counts.
Testing Background and Comparables
The StorageReview Enterprise Test Lab provides a flexible architecture for conducting benchmarks of enterprise storage devices in an environment comparable to what administrators encounter in real deployments. The Enterprise Test Lab incorporates a variety of servers, networking, power conditioning, and other network infrastructure that allows our staff to establish real-world conditions to accurately gauge performance during our reviews.
We incorporate these details about the lab environment and protocols into reviews so that IT professionals and those responsible for storage acquisition can understand the conditions under which we have achieved the following results. None of our reviews are paid for or overseen by the manufacturer of equipment we are testing. Additional details about the StorageReview Enterprise Test Lab and an overview of its networking capabilities are available on those respective pages.
Application Workload Analysis
In order to understand the performance characteristics of enterprise storage devices, it is essential to model the infrastructure and the application workloads found in live production environments. Our benchmarks for the Kioxia CD6 are therefore the MySQL OLTP performance via SysBench and Microsoft SQL Server OLTP performance with a simulated TCP-C workload. For our application workloads, each drive will be running 4 identically configured VMs.
SQL Server Performance
Each SQL Server VM is configured with two vDisks: 100GB volume for boot and a 500GB volume for the database and log files. From a system-resource perspective, we configured each VM with 8 vCPUs, 64GB of DRAM and leveraged the LSI Logic SAS SCSI controller. While our Sysbench workloads tested previously saturated the platform in both storage I/O and capacity, the SQL test is looking for latency performance.
This test uses SQL Server 2014 running on Windows Server 2012 R2 guest VMs, and is stressed by Quest’s Benchmark Factory for Databases. StorageReview’s Microsoft SQL Server OLTP testing protocol employs the current draft of the Transaction Processing Performance Council’s Benchmark C (TPC-C), an online transaction-processing benchmark that simulates the activities found in complex application environments. The TPC-C benchmark comes closer than synthetic performance benchmarks to gauging the performance strengths and bottlenecks of storage infrastructure in database environments. Each instance of our SQL Server VM for this review uses a 333GB (1,500 scale) SQL Server database and measures the transactional performance and latency under a load of 15,000 virtual users.
SQL Server Testing Configuration (per VM)
Windows Server 2012 R2
Storage Footprint: 600GB allocated, 500GB used
SQL Server 2014
Database Size: 1,500 scale
Virtual Client Load: 15,000
RAM Buffer: 48GB
Test Length: 3 hours
2.5 hours preconditioning
30 minutes sample period
For our SQL Server transactional benchmark, the KIOXIA CD6 had a score of 12,633.7 TPS at 4VMs.
Average latency saw the KIOXIA CD6 have 5.5ms at 4VMs (the same as the CD6), putting it slightly behind the Huawei and Memblaze drives.
Sysbench Performance
The next application benchmark consists of a Percona MySQL OLTP database measured via SysBench. This test measures average TPS (Transactions Per Second), average latency, and average 99th percentile latency as well.
Each Sysbench VM is configured with three vDisks: one for boot (~92GB), one with the pre-built database (~447GB), and the third for the database under test (270GB). From a system-resource perspective, we configured each VM with 8 vCPUs, 60GB of DRAM and leveraged the LSI Logic SAS SCSI controller.
Sysbench Testing Configuration (per VM)
CentOS 6.3 64-bit
Percona XtraDB 5.5.30-rel30.1
Database Tables: 100
Database Size: 10,000,000
Database Threads: 32
RAM Buffer: 24GB
Test Length: 3 hours
2 hours preconditioning 32 threads
1 hour 32 threads
Moving on to the Sysbench transactional benchmark, the KIOXIA posted a leading score of 9,333 TPS at 4VMs.
In Sysbench average latency, we saw the CD6 with a latency of just 13.71ms at 4VMs, again leading the pack.
For our worst-case scenario latency (99th percentile) the CD6 hit 27.48ms at 4VMs, this time trailing the Huawei by a hair (27.22ms).
VDBench Workload Analysis
When it comes to benchmarking storage devices, application testing is best, and synthetic testing comes in second place. While not a perfect representation of actual workloads, synthetic tests do help to baseline storage devices with a repeatability factor that makes it easy to do apples-to-apples comparison between competing solutions. These workloads offer a range of different testing profiles ranging from “four corners” tests, common database transfer size tests, to trace captures from different VDI environments. All of these tests leverage the common vdBench workload generator, with a scripting engine to automate and capture results over a large compute testing cluster. This allows us to repeat the same workloads across a wide range of storage devices, including flash arrays and individual storage devices. Our testing process for these benchmarks fills the entire drive surface with data, then partitions a drive section equal to 25% of the drive capacity to simulate how the drive might respond to application workloads. This is different than full entropy tests which use 100% of the drive and takes them into steady state. As a result, these figures will reflect higher-sustained write speeds.
Profiles:
4K Random Read: 100% Read, 128 threads, 0-120% iorate
4K Random Write: 100% Write, 128 threads, 0-120% iorate
4K Random Read (high load): 100% Read, 512 threads, 0-120% iorate
4K Random Write (high load): 100% Write, 512 threads, 0-120% iorate
64K Sequential Read: 100% Read, 32 threads, 0-120% iorate
64K Sequential Write: 100% Write, 16 threads, 0-120% iorate
64K Sequential Read (high load): 100% Read, 64 threads, 0-120% iorate
64K Sequential Write (high load): 100% Write, 64 threads, 0-120% iorate
Synthetic Database: SQL and Oracle
VDI Full Clone and Linked Clone Traces
Comparables:
Memblaze PBlaze5 C926
Intel P4610
Huawei ES3000 v5
Toshiba PX04
Samsung PM1725a
Comments
Post a Comment