LRDIMMs, RDIMMs, and Supermicro's Latest Twin
by Johan De Gelas on August 3, 2012 4:45 AM EST- Posted in
- IT Computing
- Intel
- Samsung
- Xeon
- Cloud Computing
- Supermicro
Supermicro's 2U Twin
We already introduced Supermicro's Twin 2U server (6027TR-D71FRF) in our Xeon E5 review. It is basically a two node 2U server that offers the density of 1U servers without the disadvantages. Instead of four redundant PSUs you only need two, and instead of noisy, energy hogging and prone to break 40mm fans you get slower turning 80mm fans.
The two servers are held in place using screwless clips.
There was one big disadvantage: there were only four DIMM slots per CPU, which limits each node to 128GB of RAM (8 x 16GB). That is a bit on the low side for 16 cores and 32 threads and makes this server less suitable for virtualization loads.
Of course, this server was never meant to be a virtualization server as it is equipped with 56Gb/s TFDR InfiniBand interconnect technology, great for processing intensive cluster applications like some clustered HPC apps. Nevertheless, we were intrigued. Supermicro has recently released a new Twin, the 6027TR-D70RF+, which has 16 DIMM slots.
Most 2U servers are limited to 24 memory slots and as a result 384GB of RAM. With two nodes in a 2U server and 16 slots per node, you get cram up to 512GB of RDIMMs in one server. The Supermicro Twin node (6027TR-D70RF+) looks like an attractive alternative for the more common 1U and 2U servers:
- Much more (3, 2 full height) PCIe expansion slots than a 1U, almost as good as a traditional 2U
- Lower energy consumption as two (up to 95% efficient) PSUs are powering two nodes
- Much better and more efficient 80mm cooling fans than a 1U
- Density of a 1U
- 33% more DIMM slots than a 2U
That all sounds great for any cluster solution including a virtualization cluster, but there is more. If you use LRDIMMs, you can double your capacity. LRDIMMs at 1333MHz are available as quad rank 32GB DIMMs. But before we can introduce you to these DIMMs, we want to take a step back and look at all the RAM options that a typical server buyers has.
Facebook
Twitter
RSS
26 Comments
View All Comments
ddr3memory - Monday, August 6, 2012 - link
Your article is very interesting - and the first mainstream (and belated) examination of the LRDIMM (new standard - incompatible with RDIMMs) vs. HCDIMM (100% DDR3 RDIMM compatbile) choice for Romley.I have whittled down the use case for HCDIMMs/LRDIMMs and RDIMMs as follows:
The HCDIMM use case is at:
- 16GB at 3 DPC use
- 32GB (outperform both RDIMMs and LRDIMMs)
LRDIMMs are not viable at:
- 16GB (RDIMMs are better)
- 32GB (HCDIMMs are better)
RDIMMs are not viable at:
- 32GB (because they are 4-rank - trumped by LRDIMMs/HCDIMMs)
There is a reason the Netlist HCDIMMs were only released on the virtualization servers from IBM/HP - because at 16GB levels the only niche available for LRDIMM/HCDIMM vs. RDIMM is the 3 DPC space. This will expand considerably at 32GB to mainstream levels as soon as 32GB HCDIMMs are released (they are currently in qualification with IBM/HP and have not been announced yet - though maybe expected shortly).
I had created an infographic covering the memory choices - search the net for the article entitled:
Infographic - memory buying guide for Romley 2-socket servers
HCDIMMs are not available at SuperMicro (as they are for IBM/HP) - so I was surprised you even covered HCDIMMs (since the article is after all referring to the SuperMicro line of servers).
Casper42 - Friday, August 10, 2012 - link
BTW, Johan, I work for HP and asked some of the guys in ISS Technical Marketing why we don't send you our servers for eval like you get from SuperMicro and sometimes DellThey felt that you guys didn't do alot of Server Reviews, and that your readership wasn't generally the kind of folks that buy HP Servers.
So I am curious if you could spin up a poll or something in the future to prove them wrong.
If there is enough support I'm sure we can you some gear to play with.
I sometimes giggle when I see the stuff people on here get excited about in these reviews though. "Can you see the BIOS through IPMI?". Thats the kind of thing Compaq offered back with the RILOE II and have been integrated into the motherboard since iLO 1 which is like 4 or 5 years old at least.
iLO4 on the Gen8 line have taken that a step further and we now hook the Display system BEFORE POST starts so instead of an invalid memory config getting you a series of beeps, you now get a full blown screen either on local VGA or on the Remote Console that straight up tells you you have a memory mismatch and why. i have seen his demo'd with NO DIMMs even installed in the server and you still get Video and obvious status messages.
Casper42 - Friday, August 10, 2012 - link
Also you are about $2000 high on the HP SL unless I am missing something.I found these prices with QuickSpecs part numbers and Google, nothing magical inside HP.
Half of one of these:
http://www.provantage.com/hewlett-packard-hp-62923...
Includes 8 fans and 3 PS
2 of these
http://www.provantage.com/hewlett-packard-hp-65904...
2x2665 with 8GB
Comes to about $11,600
JohanAnandtech - Tuesday, August 14, 2012 - link
Hey Casper, contact me on my mail... thx!ad99 - Monday, April 1, 2013 - link
You say：a quad rank DIMM with 4Gb chips is a 32GB DIMM (4 Gbit x 8 x 4 ranks)，but I think 4 Gbit x 8 x 4 ranks make only 16GB,is that right?ad99 - Monday, April 1, 2013 - link
No,4 Gbit x 8 x 4 ranks should be 128GB，not 32GB or 16GB,is that right?