Intel's new 3D XPoint non-volatile memory technology, which has been on the cards publically for the last couple of years, is finally hitting the market as the storage medium for Intel's new flagship enterprise storage platform. The Intel Optane SSD DC P4800X is a PCIe SSD using the standard NVMe protocol, but the use of 3D XPoint memory instead of NAND flash memory allows it to deliver great throughput and much lower access latency than any other NVMe SSD.

3D XPoint

The potential significance of 3D XPoint memory is immense. When it was first publicly announced by Intel and Micron in 2015, 3D XPoint memory was a fundamentally different storage technology from the flash memory that dominates the market. It is the first new truly mass market, high-density solid state storage medium to hit the market since NAND flash itself. It comes at a time where the NAND market is booming like never before, but also at a time when we know that there is a definite end of the line for NAND. The ongoing transition to 3D NAND flash is just a temporary postponement of the fundamental limitations of flash memory. Once NAND can no longer scale in density and cost-per-bit, it will fall to paradigm changes and next-generation memory technologies (one of which will be 3D XPoint) to continue to carry the industry forward. There are many other new memory technologies that may compete alongside flash memory and 3D XPoint in the coming years, but 3D XPoint is the one that's ready to go mainstream now.

In the near term, 3D XPoint is important because it offers a new set of performance tradeoffs entirely unlike NAND; tradeoffs that, for the right applications, can deliver performance far in excess of today's NAND products. By being able to read and write at the bit or word level - and not the 4K+ page level of NAND - 3D XPoint has the potential to deliver excellent performance across a wide range of workloads, but especially in minimally parallel workloads, which are common in the consumer and enterprise spaces.

The drawback here is that, due to various factors regarding time, production, and scope, 3D XPoint is more expensive than NAND. It also comes in as less dense, to aid in ease of production in this first stage, but this also adds to the cost. For now, due to scale and other factors, it won't be able to replicate the sheer capacity and cost effectiveness that has made NAND storage so popular in all market segments. Due to the scale, especially as a first-generation version of the technology, the first 3D XPoint products are being aimed at speciality and high-margin markets: enterprise performance, consumer caching, etc. Future products promised from Intel should add non-volatile DIMMs to the mix, and then later on, if everything goes to plan, a potential wholesale replacement of NAND flash (or at least a strong competitor).

The Intel Optane SSD DC P4800X

The new storage drive, and the focus of today's review, is the Intel Optane SSD DC P4800X. It uses a new NVMe controller Intel developed specifically for use with 3D XPoint memory. Where Intel's enterprise NVMe SSDs like the P3700 use a controller with 18 channels for interfacing to their flash memory, the Optane SSD's controller has only 7 channels. In order to achieve at least parity on peak performance, each of those channels has to provide much higher throughput than on a flash SSD, and it shows that each 3D XPoint memory die is delivering much higher performance than a die of flash memory.

The first capacity of the Optane SSD DC P4800X to ship and the model we've tested here offers a usable capacity of 375GB from a total of 28 3D XPoint memory dies (four per channel) for a raw capacity of 448GB. 3D XPoint memory has better endurance than NAND flash, but not enough to get away without wear levelling. The fine-grained accessibility of 3D XPoint memory gets rid of a lot of the wear leveling and write amplification headaches caused by flash pages and erase blocks being larger than the sector sizes exposed by the drives, but the drive still needs some spare area plus storage for error correction overhead, metadata for tracking the mapping between logical blocks and physical addresses, and potential replacement of bad sectors, similar to normal SSDs.

As with most NVMe SSDs, the Optane SSD DC P4800X supports a configurable sector size. Out of the box it emulates 512B sectors for the sake of compatibility, but using the NVMe FORMAT command it can be switched to emulate 4kB sectors. The larger sector size reduces the amount of metadata the SSD controller has to juggle, so it usually allows for slightly higher performance. The NVMe FORMAT command is also the mechanism for triggering a secure erase of the entire drive, and for flash SSDs the format usually consists of little more than issuing block erase commands to the whole drive. 3D XPoint memory does not have large multi-megabyte erase blocks, so a low-level format of the Optane SSD needs to directly write to the entire drive, which takes about as long as filling it sequentially. Thus, while a 2.4TB flash SSD can perform a low-level format in just over 13 seconds, the 375GB Optane SSD DC P4800X takes six minutes and 47 seconds. This is long enough that unsuspecting software tools or SSD reviewers will give up and assume that the drive has locked up.

Intel Optane SSD DC P4800X Specifications
Capacity 375 GB 750 GB 1.5 TB
Form Factor PCIe HHHL or 2.5" 15mm U.2
Interface PCIe 3.0 x4 NVMe
Controller Intel unnamed
Memory 128Gb 20nm Intel 3D XPoint
Typical Latency (R/W) <10µs
Random Read (4 kB) IOPS (QD16) 550,000 TBA TBA
Random Read 99.999% Latency (QD1) 60µs TBA TBA
Random Read 99.999% Latency (QD16) 150µs TBA TBA
Random Write (4 kB) IOPS (QD16) 500,000 TBA TBA
Random Write 99.999% Latency (QD1) 100µs TBA TBA
Random Write 99.999% Latency (QD16) 200µs TBA TBA
Mixed 70/30 (4kB) Random IOPS (QD16) 500,000 TBA TBA
Endurance 30 DWPD
Warranty 5 years (3 years during early limited release)
MSRP $1520 TBA TBA
Release Date HHHL March 19  Q2 2017 2H 2017
U.2 Q2 2017 2H 2017

So far, Intel has only started shipping the 375GB Optane SSD DC P4800X to select customers, and they have not released detailed specifications for the larger capacities that will ship later this year.

It is worth noting that the performance specifications for the P4800X, as provided in the product specification sheets, cover a different set of metrics than Intel usually reports for their enterprise SSDs. Sequential performance is not mentioned at all, but the product brief has quite a bit to say about latency: average latency for QD1 reads and writes, and 99.999th percentile latency for both reads and writes at QD1 and QD16. The fact that Intel is publishing a five-nines QoS metric at all suggests that they plan to set a new standard for performance consistency.

The throughput claims are also remarkable: half a million IOPS or more for reads, writes and a 70/30 read/write mix. There are already drives on the market that can deliver more than 550k random read IOPS, but those SSDs are far larger than 375GB and they require very high queue depths to hit 550k IOPS. There are even a few multi-TB drives that can beat 500k random write IOPS, but they can't sustain that performance indefinitely. The Optane SSD DC P4800X is promising an unprecedented level of storage performance both in absolute terms and relative to its capacity, so it is interesting to see where Intel is going to lay down its line in the sand.

The P4800X will not really occupy the same niche as the multi-TB monsters that offer comparable throughput. With limited capacity but the highest level of performance, this Optane SSD most closely fits the role of SLC NAND based SSDs. SLC has disappeared from the SSD market as virtually all customers preferred to sacrifice a little bit of performance to double their capacity by using MLC NAND. One of the last high-performance SLC SSDs was the Micron P320h, a PCIe SSD from 2012 that slightly pre-dated NVMe and used 34nm SLC NAND flash. Anyone still using a P320h for its consistent low latency performance will be very interested in the P4800X. Outside of that niche, the Optane SSD will obviously be desirable for its raw throughput, but the low capacity may be problematic for some use cases.

One of the unique and most notable performance advantages of the Optane SSD DC P4800X is that it does not require extremely high queue depths to reach full throughput. Enterprise customers have long had to design their systems around the fact that getting full performance out of the fastest PCIe SSDs requires loading them down with queue depths of 128 or higher, sometimes requiring applications to use dozens of threads for I/O. In the client space achieving such queue depths is outright impossible, and in the enterprise space it doesn't happen for free. The P4800X's high performance at low queue depths makes it a much easier drive to get great real-world performance out of.

Intel originally introduced 3D XPoint memory as having far higher write endurance than NAND flash—on the order of 1000x higher. The Optane SSD DC P4800X is rated for 30 drive writes per day (DWPD) for five years, and the current models shipping during this early limited availability period are only rated for three years, rather than the five years it expects the support for the full retail models. Intel says they're being extremely conservative with a new and unproven technology, and doing the math means that 30 DWPD doesn't provide any endurance advantage over the most highly over-provisioned flash-based enterprise SSDs. In terms of total petabytes written, the P4800X only has four-fifths the endurance of the SLC-based Micron P320h. Even allowing for Intel's original comparisons possibly having been relative to lower-endurance contemporary MLC or TLC flash, it seems like this first generation of 3D XPoint memory is not as durable as originally planned - the headline number of 30 DWPD is aimed at alleviating that issue, however for Intel to match its original intentions then the second and third generation parts will have to be a step up, and we look forward to testing them.

Pricing

The MSRP for the 375GB P4800X is $1520, though it will be quite some time before it can readily be ordered from major online retailers. At slightly more than $4/GB, the P4800X will be almost twice as expensive per GB as Intel's next most pricey SSD, the P3608 (which is really two drives in one plus a PCIe switch). Compared to Intel's fastest single SSD (the P3700), the P4800X will be more than three times as expensive per GB. In the broader SSD market, $4/GB is not completely unprecedented, but most companies selling drives in this price range don't even pretend to have a retail price.

This Review

For this review of the Intel Optane SSD DC P4800X, first, we are going to take a deeper dive into what 3D XPoint actually is. Then we go through our testing suite for enterprise drives, testing Intel's claims on performance.

It is worth noting that there is no such thing as a general-purpose enterprise SSD. Enterprise storage workloads are far more varied than client workloads and it is impossible to make general statements about whether random or sequential performance is more important, what kind of mix of reads and writes to expect, or what queue depth is apporpriate to test with. Real-world application benchmarks are difficult to construct and typically end up being far more narrowly applicable than we would hope. Our strategy for this review is to provide a very broad range of synthetic tests with the knowledge that not all results will be relevant to all use cases. Enterprise customers must know and understand their own workload. Since this is our first time testing anything with 3D XPoint memory, this review includes some new benchmarks that would probably not be applicable to a flash SSDs review, making for some interesting numbers.

3D XPoint Refresher
Comments Locked

117 Comments

View All Comments

  • Billy Tallis - Friday, April 21, 2017 - link

    I said the NVMe driver wasn't manually switched into polling mode; I left it with the default behavior which on 4.8 seems to be not polling unless the application requests. I'm certainly not seeing the 100% CPU usage that would be likely if it was polling.

    If I'd had more time, I would have experimented with the latest kernel versions and the various tricks to get even lower latency.
  • tuxRoller - Friday, April 21, 2017 - link

    I wasn't claiming that you disabled polling only that polling was disabled since it should be on be default for this device.
    Assuming you were looking at the sysfs interface, was the key that was set to 0 called io_poll or io_poll_delay? The later set to 0 enables hybrid polling, so the cpu wouldn't be pegged.
    Either way, you wouldn't need a new kernel, just to enable a feature the kernel has had since 4.4 for these low latency devices.
    Also, did you disable the pagecache (direct=1) in your fio commands? If you didn't, that would explain why aio was faster since it uses dio.
    Btw, it's not my intent to unnecessarily criticize you because i realize the tests were performed under constrained circumstances. I just would've appreciated some comment in the article about a critical feature for this hardware was not enabled in the kernel.
  • yankeeDDL - Friday, April 21, 2017 - link

    Optane was supposed to be 1000x faster, have 1000X endurance and be 10x denser than NAND (http://hothardware.com/ContentImages/NewsItem/4020...
    I realize this is the first product, but saying that it fell short of expectation is an understatement.
    It has lower endurance, lower density and it is measurably faster, but certainly nowhere close 1000X.
    Oh, did I mention it is 5-10X more expensive?

    I am quite disappointed, to be honest. It will get better, but @not ready@ is something that comes to ind reading the article.
  • Billy Tallis - Friday, April 21, 2017 - link

    3D XPoint memory was supposed to be 1000x faster than NAND, 1000x more durable than NAND, and 10x denser than DRAM. Those claims were about the 3D XPoint memory itself, not the Optane SSD built around that memory.
  • ddriver - Friday, April 21, 2017 - link

    It is probably as good as they said... if you compare it to the shittiest SD card from 10 years ago. Still technically NAND ;)
  • yankeeDDL - Monday, April 24, 2017 - link

    I disagree. I can agree that the speed may be limited by the drive, but even so, it falls short by a large factor. The durability and the density, however, are pretty much platform independent and they are not there by a very, very long shot. Intel itself demonstrated that it is only 2.4-3X faster (https://en.wikipedia.org/wiki/3D_XPoint).

    It clearly has a future, especially as the NAND is approaching the end of its scalability. Engineering wise is interesting, but today, it makes really little sense, while it should have been a slam dunk. I mean, who would have thought twice before buying a 500GB drive that maxes out the SATA for $20-30? But this one ... not so much.
  • zodiacfml - Friday, April 21, 2017 - link

    It will perform better in DIMM.
  • factual - Friday, April 21, 2017 - link

    I don't see xpoint replacing dram due to both latency and endurance not being up to par , but It's going to disrupt the ssd market and as the technology matures and prices come down, I can see xpoint revolutionizing the storage market as ssd did years ago.

    Competition is clearly worried since seems like paid trolls are trying to spread falsehoods and bs here and elsewhere on the web.
  • ddriver - Saturday, April 22, 2017 - link

    I just bet it will be highly disturbing to the SSD market LOL. With its inflated price, limited capacity and pretty much unnecessary advantages I can just see people lining up to buy that and leaving SSDs on the shelves.
  • factual - Saturday, April 22, 2017 - link

    You are either extremely ignorant or a paid troll !!! anyone who understands technology knows that new tech is always expensive. When SSDs came to the market, they were much more expensive and had a lot less capacity than HDDs but they closed the gap and disrupted the market. The same is bound to happen for Xpoint which performs better than NAND by orders of magnitude.

Log in

Don't have an account? Sign up now