LRDIMMs, RDIMMs, and Supermicro's Latest Twin
by Johan De Gelas on August 3, 2012 4:45 AM EST- Posted in
- IT Computing
- Intel
- Samsung
- Xeon
- Cloud Computing
- Supermicro
HyperCloud DIMMs
There is a fourth option: Netlist's HCDIMM. Netlist, a company that specializes in making VLP (Very Low Profile) memory, offers an alternative for LRDIMMs: HyperCloud DIMMs.
Instead of using a centralized buffer (LRDIMMs), HCDIMMs use a distributed buffer to reduce the electrical load on the memory channels. Combined with an HCDIMM register, four ranks are presented as two.
HCDIMM advantages
The advantage of HCDIMMs is that HCDIMMs run one speed bump faster than LRDIMMs. So while LRDIMMs have to throttle back to 1066MHz at 3 DPC, HCDIMMs run at 1333MHz. According to a Netlist sponsored report, HCDIMMs offer about 17% higher bandwidth, which sounds reasonable to us. Secondly, the distributed buffer architecture is the same architecture that DDR4 is converging on. However DDR4 will be running at a much lower 1.2V.
HCDIMM Disadvantages
The combination of a register and distributed buffers (and clock redundancy) comes with a serious drawback: power. Although we could not test HCDIMMs ourselves, most industry sources talk about 20% higher power than LRDIMMs, while LRDIMMs only consume more than RDIMMs with 1 DPC. In 3 DPC configurations, LRDIMMS consume about the same as RDIMMs.
Secondly, HCDIMMs are not a JEDEC standard. As a result the HCDIMM ecosystem—module, server and CPU vendors—is smaller. AMD and Intel do not officially validate HCDIMMs, so server vendors have do the complete validation effort themselves. IBM for example only offers them in one system (IBM system x3650 M4), and 16GB HCDIMMs are quite a bit more expensive than both 16GB LRDIMMs and RDIMMs. LRDIMMs are much more widespread; almost every server vendors supports them in a wide range of server models.
Lastly, HCDIMMs are only available from one module vendor, Netlist.
Nevertheless, HCDIMMs are viable but somewhat expensive alternative in the high capacity memory market if your server supports them. Netlist has been quite successful in convincing the server vendors: several models of HP, Supermicro, and Gigabyte support HCDIMMs. HP offers HCDIMMs in their most popular servers (DL380/DL360), although HCDIMMs can only be installed by HP.
Once Netlist gets the 32GB parts out in large volumes, the extra competition can probably drive prices down. Until then, HCDIMMs offer only a speed advantage over RDIMMs.
Overview
Let us structure all this info in a table.
DIMM Types, Speed, and Capacity Limitations | ||||||
DIMM type | UDIMM | RDIMM | RDIMM LV | LRDIMM | HCDIMM | |
Maximum speed at 2 DPC | 1333MHz | 1600MHz | 1333MHz | 1333MHz | 1600MHz | |
Maximum speed at 3 DPC | Not possible | 1333MHz | 1333MHz | 1066MHz | 1333MHz | |
Maximum capacity per CPU (Quad channel) | 64GB |
192GB (3 DPC) 256GB (2 DPC)* |
192GB (3 DPC) 256GB (2 DPC)* |
384GB | 192GB | |
Top speed at maximum capacity | 1066 |
1066 (DR) ** 800 (QR) ** |
1066 (DR) ** 800 (QR) ** |
1066 | 1333 | |
Voltage | 1.5V | 1.5V | 1.35 V | 1.35V/1.5V | 1.35V/1.5V | |
loaded, 3 DPC Power usage (at 1.5V) per DIMM | 4 W | 4.5 W | <= 4 W | 5-6 W | 8-9 W | |
Intel CPU support |
Xeon 5500 Xeon 5600 Xeon E5 |
Xeon 5500 Xeon 5600 Xeon E5 |
Xeon 5600 Xeon E5 Xeon E7 |
Xeon 5600 Xeon E5 Xeon E7 |
N/A | |
AMD CPU support |
Opteron 4000 Opteron 6100 Opteron 6200 |
Opteron 4000 Opteron 6100 Opteron 6200 |
Opteron 6100 Opteron 6200 |
Opteron 6200 | N/A | |
Server support | all servers | all servers | all servers | Dell, HP, IBM,Supermicro |
HP DL360G8 DL380G8 IBM x3650 M4 Supermicro |
* Quad Rank DIMMs require special BIOS support and validation and are not available on all servers.
** DR = Dual Rank, QR = Quad Rank.
1600MHz LRDIMMs are possible but not commercially available as far as we know. 32GB HCDIMMs are available in very small quantities. LRDIMMs are available from almost every server module manufacturer out there, while HCDIMMs are netlist modules only.
26 Comments
View All Comments
dgingeri - Friday, August 3, 2012 - link
"Most 2U servers are limited to 24 memory slots and as a result 384GB of RAM. With two nodes in a 2U server and 16 slots per node, you get cram up to 512GB of RDIMMs in one server. "It's not one server. It's actually 2 servers. just because they're in a 2U X 1/2 width form factor doesn't mean they're just one system. There are 2 systems there. Sure you can pack 512GB into 2U with 2 servers, but there are better ways.
1. Dell makes a PowerEdge R620, where you can pack 384GB into 1U, two of those gives you the same number of systems in the same space, with 50% more memory.
2. Dell also has their new R720, which is 2U and has a capacity of 768GB in a 2U form factor. Again, 50% more memory capacity in the same 2U. However, that's short 2 processor sockets.
2. Now, there's the new R820. 4 sockets, 1.5TB of memory, 7 slots, in 2U of space. It's a beast. I have one of these on the way from Dell for my test lab.
Working as an admin in a test lab, dealing with all brands of servers, my experiences with various brands gives me a rather unique insight. I have had very few problems with Dell server, despite having nearly 30% Dell servers. We've had 7 drives die (all Toshiba) and one faceplate LCD go out. Our HP boxes, at less than 10% of our lab, have had more failures. The IBMs, ahile also less than 10%, have had absolutely no hardware failures. Our Supermicros comprise about 25% of the lab, yet contribute >80% of the hardware problems, from motherboards that just quit recognizing memory to backplanes that quit recognizing drives. I'm not too happy with them.
JHBoricua - Monday, August 6, 2012 - link
Dgingeri,Sure, you can load each of those Rxxx Dell servers with boatloads of memory, but you fail to mention that it comes with a significant performance/penalty. The moment you put a third Dimm on a memory channel your memory speeds drops from 1600 (IF you started with 1600 memory to begin with) to 1066 or worse, 800. On a virtualization host, that makes a big difference.
Casper42 - Friday, August 10, 2012 - link
No one makes 32GB @ 1600 yet.So 512GB @ 2DPC would be 1333
And 768GB @ 3DPC would be 1066 or 800 like you mentioned.
384 using 16GB DIMMs would still be 3DPC and would drop from 1600 down to like 1066.
256GB @ 1600 @ 2DPC still seems to be the sweet spot.
BTW, why is the Dell R620 limited to 16GB DIMMs? The HP DL360p Gen8 is also 1U and supports 32GB LRDIMMs
ImSteevin - Friday, August 3, 2012 - link
MMhmmm yeahOh yeah ok
I know some of these words.
thenew3 - Friday, August 3, 2012 - link
The latest Dell R620's are 1U servers that can have two 8 core CPU's and 24 DIMM slots. Each slot can hold up to a 32GB DIMM giving total memory capacity of 768GB in a 1U space.We use these in our data centers for virtualization (we're 100% virtualized). Completely diskless (internal RAID 1 dual SD modules for ESXi)
Each machine has four 10gb NIC plus two 1gb NIC. All storage on iSCSI SAN's through 10gb backbone.
For most virtualization tasks, you really don't need the 2U R720, which has the same CPU/RAM options but gives you more drive bays and expansion slots.
shuntian8099 - Saturday, August 4, 2012 - link
We accept any form of payment.
http://commonprosperity.org/ )
S=h=0=x ==sh0es=4oUSD
J=0=r=d=a=n==sh0es=36USD
A=i=r==M=a=x=sh0es=39USD
C=0=a=c=h==b=a=g=36USD
S=u=n=g=l=a=s=s=15USD
simpletrading@hotmail.com
Finally (48 hours) time limit to buy.
LV Muffler $ 5.99
LV Bags $ 19.9
LV Wallet $ 6.55
Armani Glasses $ 5.99
LV Belt $ 6.9
simpletrading@hotmail.com
Buy addresses-
commonprosperity.org
╭⌒╮
∴★∵**☆.∴★∵**☆.∴★∵**☆.█████.::∴★∵**☆.∴★∵**☆.█田田█::
commonprosperity.org ╭⌒╮ ╭⌒╮
∴★∵**☆.∴★∵**☆.█田田█.∴★∵**☆.∴★∵**☆.█田█田█∴★∵**☆.∴★∵*
*☆.█田█田█.★∵**☆.∴★∵**☆.█████.*******************You
r satisfaction is our eternal pursuit!
ddr3memory - Sunday, August 5, 2012 - link
A few corrections - the 192GB for HCDIMMs is incorrect - it should also be 384GB.There is no data available that confirms a 20% higher power consumption for HCDIMMs over LRDIMMs. There is a suspicious lack of benchmarks available for LRDIMMs. It is possible that figure arises from a comparison of 1.5V HCDIMMs vs. 1.35V LRDIMMs (as were available at IBM/HP).
It is incorrect that LRDIMMs are somehow standard and HCDIMMs are non-standard.
In fact HCDIMMs are 100% compatible with DDR3 RDIMM JEDEC standard.
It is the LRDIMMs which are a new standard and are NOT compatible with DDR3 RDIMMs - you cannot use them together.
The 1600MHz HCDIMM mention is interesting - would be good to hear more on that.
ddr3memory - Sunday, August 5, 2012 - link
I have posted an article on the performance comparison of HyperCloud HCDIMMs (RDIMM-compatible) vs. LRDIMMs (RDIMM non-compatible).Cannot post link here it seems - search for the article on the ddr3memory.wordpress.com blog:
Awaiting 32GB HCDIMMs
ddr3memory - Monday, August 6, 2012 - link
VMware has had good things to say about HCDIMM (not a word from VMware about LRDIMMs though). Search on the net for the article entitled:Memory for VMware virtualization servers
ddr3memory - Monday, August 6, 2012 - link
The prices mentioned maybe off - I see IBM showing same retail prices for 16GB LRDIMMs/HCDIMMs and similar at the IBM resellers.These resellers show 16GB HCDIMMs selling at $431 at costcentral for example, $503 at glcomp and $424 at pcsuperstore.
Search the internet for this article:
What are IBM HCDIMMs and HP HDIMMs ?
It has the links for the IBM/HP retail prices as well as the reseller prices.