The Enterpise TLC Storage Era Begins: Rounding Up 13 SSDs With Samsung, Intel, and Memblaze
by Billy Tallis on January 3, 2019 9:45 AM ESTQD1 Random Read Performance
Drive throughput with a queue depth of one is usually not advertised, but almost every latency or consistency metric reported on a spec sheet is measured at QD1 and usually for 4kB transfers. When the drive only has one command to work on at a time, there's nothing to get in the way of it offering its best-case access latency. Performance at such light loads is absolutely not what most of these drives are made for, but they have to make through the easy tests before we move on to the more realistic challenges.
*The colors on the graphs have no specific meaning; just to make it easier to read from the graph based on drive family
The Intel DC P4510 and Samsung 983 DCT offer identical QD1 random read performance, showing that Intel/Micron 3D NAND has caught up to Samsung after a first generation that was clearly slower (shown here by the Memblaze PBlaze5). The Samsung SATA drives are about 40% slower than their NVMe drives, and the Optane SSD is almost ten times faster than anything else.
Power Efficiency in kIOPS/W | Average Power in W |
QD1 isn't pushing the drives very hard, so they're not very far above idle power draw. That means the bigger, beefier drives draw more power but have little to show for it. The SATA drives all have higher efficiency scores than the NVMe drives, except that the Optane SSD provides more than 2.5x the performance per Watt at QD1.
All of these drives have pretty good consistency for QD1 random reads. The 8TB P4510 has a 99.99th percentile latency that's just over 2.5 times its average, and all the other flash-based SSDs are better than that. (The PBlaze5 is a bit slower than the P4510, but consistently). The Optane SSD actually has the biggest relative disparity between average latency and 99.99th percentile, but that hardly matters with its worst-case is as good or better than the average-case performance of flash memory.
The random read performance of most of these drives is optimized for 4kB block sizes: smaller block sizes perform roughly the same in terms of IOPS, and thus get much lower throughput. Throughput stays relatively low until block sizes exceed 64kB, after which the NVMe drives all start to deliver much higher throughput. This is most pronounced for the Samsung 983 DCT.
QD1 Random Write Performance
The clearest trend for 4kB random write performance at QD1 is that the NVMe drives are at least twice as fast as the SATA drives, but there's significant variation: Intel's NVMe drives have much faster QD1 random write performance than Samsung's, and the Memblaze drives are in between.
Power Efficiency in kIOPS/W | Average Power in W |
The SATA drives draw much less power than the NVMe drives during this random write test, so the efficiency scores end up in the same ballpark with no clear winner. The PBlaze5 is the clear loser, because its huge power-hungry controller is of no use here at low loads.
The Intel DC P4510 has some QoS issues, with 99th and 99.99th percentile random write latencies that are far higher than any of the other NVMe drives, despite the P4510 having one of the best average latency scores. Samsung's latest generation of SATA drives doesn't offer a significant improvement to average-case latency, but the tail latency has clearly improved.
As with random reads, we find that the random write performance of these drives always provides peak IOPS when dealing with 4kB transfers. The Memblaze PBlaze5 drives do extremely poorly with writes smaller than 4kB, but the rest of the drives handle tiny writes with roughly the same IOPS as a 4kB write. 8kB and larger random writes always yield fewer IOPS but usually significantly higher overall throughput.
QD1 Sequential Read Performance
At QD1, the SATA drives aren't quite saturating the host interface, but mainly because of the link idle time between the drive finishing one transfer and receiving the command to start the next. The PBlaze5 SSDs are only a bit faster than the SATA drives at QD1 despite the C900 being the drive with the most host bandwidth; reminding us how the first-generation IMFT 3D TLC could be quite slow at times. The Intel drives are a bit slower than the Samsung drives, coming in at or below 2GB/s while the 983 DCT U.2 hits 2.5GB and the M.2 is around 2.2GB/s.
Power Efficiency in MB/s/W | Average Power in W |
The latest Samsung SATA drives are comparable in efficiency to Intel's NVMe, while Samsung's own NVMe SSD is substantially more efficient. The Memblaze PBlaze5 is by far the least efficient, since it offers disappointing QD1 sequential read performance while drawing more power than all the other flash-based SSDs.
The Memblaze PBlaze5 doesn't seem to be any good at prefetching or caching when performing sequential reads: its throughput is very low for small to medium block sizes, and even at 128kB it is much slower than with 1MB transfers. The rest of the drives generally provide full sequential read throughput for transfer sizes starting around 64kB or 128kB.
QD1 Sequential Write Performance
At QD1, the Intel P4510 and Samsung 983 DCT are only slightly faster at sequential writes than the SATA drives. The Optane SSD and the Memblaze PBlaze5 C900 both perform very well, while the PBlaze5 D900 can't quite hit 1GB/s at QD1.
Power Efficiency in MB/s/W | Average Power in W |
Once again the SATA drives dominate the power efficiency rankings; Samsung's current-generation SATA drives draw less than half the power of the most efficient flash-based NVMe drive. The older Samsung PM863 shows that Samsung's SATA drives have improved significantly even though the performance is barely changed from earlier generations. Among NVMe drives, power consumption scales roughly as expected, with the Memblaze PBlaze5 C900 drawing over 22W to deliver 1.6GB/s sustained writes.
As with random writes, for sequential writes the Memblaze PBlaze5 doesn't like being handed writes of less than 4kB, and neither does the Intel Optane P4800X. The rest of the drives generally hit their steady-state sequential write speed starting with transfer sizes in the 8kB to 32kB range.
36 Comments
View All Comments
Greg100 - Thursday, January 3, 2019 - link
In Europe we have got:Samsung SSD 860 QVO 4TB for € 579 and
Samsung SSD 860 EVO 4TB for € 625
So I think it’s time for something bigger and faster…
Greg100 - Thursday, January 3, 2019 - link
especially, when we have many 15.36TB SAS SSDs from every major manufacturer:Western Digital Ultrastar DC SS530 15.36TB
Seagate Nytro 3330 15.36TB
Toshiba PM5-R 15.36TB
Samsung PM1643 15.36TB
with Samsung PM1643 even double capacity: 30.72TB
...but we can't use SAS SSD on consumer motherboard...
so I count on new U.2 drives.
phoenix_rizzen - Friday, January 4, 2019 - link
Well, you can use SAS drives with consumer-oriented motherboards, you just need to add an SAS HBA via one of the PCIe slots. :) An extra cost, for sure, but it's certainly doable. LSI 92xx-8i or similar aren't that expensive.Greg100 - Saturday, January 5, 2019 - link
Thank you for your advice. You're right, SAS SSDs can be connected to a consumer motherboard, but does it make any sense?SAS SSDs are only given as an example that there are even larger than 8TB SSDs available on the market. It's a pity that they are much more expensive for GB than Intel U.2 drives and much slower. The fastest of the above:
Western Digital Ultrastar DC SS530
15.36TB
2150MB/s (Read)
2120MB/s (Write)
We can get instead:
Intel DC P4510
8TB
3200MB/s (Read)
3000MB/s (Write)
£2,300.78
(Bleepbox on UK ebay; new, in stock)
so....
2x more expensive per GB than Samsung SSD 860 EVO 4TB
6x faster (sequentially) than Samsung SSD 860 EVO 4TB
Is it worth its price?
Is 6x faster CPU worth 2x higher price?
Greg100 - Saturday, January 5, 2019 - link
If someone has to use SAS drives, they pay more for lower performance, but they don't have to rebuild the entire infrastructure.For the consumer who wants:
-big
-fast
-cheap
I think that Intel DC P4510 is the best option
If someone can find
8TB SSD 3000MB/s+ read
cheaper than £2,300.78 let me know...
Billy Tallis - Thursday, January 3, 2019 - link
I don't think we'll see 8TB consumer-oriented drives in 2019, and if we do, they'll make even less sense than Samsung's premature introduction of the 4TB 850 EVO did. Going beyond 8TB will require a new generation of controllers for the consumer market, with support for more DRAM, and that may not be economical until 2TB and 4TB drives are as affordable as 1TB drives are now.Greg100 - Thursday, January 3, 2019 - link
Thank you for your opinion and information on compatibility.So... if I will not see something interesting on CES... I think it is time for Intel DC P4510 8TB (data) and maybe Intel Optane 905P (boot, software)
Greg100 - Thursday, January 3, 2019 - link
One more question:Do you know cause and solution to the issues with the 4TB Samsung drives, you describe here: https://www.anandtech.com/show/13633/the-samsung-8... ?
Billy Tallis - Thursday, January 3, 2019 - link
I haven't been able to pin down the cause or fix. One of the troublesome 4TB 860s did start behaving again, but I'm not sure what part of my messing with it accomplished that and I haven't been able to replicate it with the other two. And the 4TB enterprise Samsung SATA drives (860 DCT and 883 DCT) have been trouble-free but haven't been subjected to the full consumer test suite yet.Greg100 - Thursday, January 3, 2019 - link
Thanks.When you will be able to pin down the cause or fix PLEASE make short article about it or include it with the future 860 DCT and 883 DCT consumer test. I think, it can be very important, especially that they are the biggest client SSDs and with last week significant price drop in Europe they are more affordable.