Testing SATA Express And Why We Need Faster SSDs
by Kristian Vättö on March 13, 2014 7:00 AM EST- Posted in
- Storage
- SSDs
- Asus
- SATA
- SATA Express
During the hard drive era, the Serial ATA International Organization (SATA-IO) had no problems keeping up with the bandwidth requirements. The performance increases that new hard drives provided were always quite moderate because ultimately the speed of the hard drive was limited by its platter density and spindle speed. Given that increasing the spindle speed wasn't really a viable option for mainstream drives due to power and noise issues, increasing the platter density was left as the only source of performance improvement. Increasing density is always a tough job and it's rare that we see any sudden breakthroughs, which is why density increases have only given us small speed bumps every once in a while. Even most of today's hard drives can't fully saturate the SATA 1.5Gbps link, so it's obvious that the SATA-IO didn't have much to worry about. However, that all changed when SSDs stepped into the game.
SSDs no longer relied on rotational media for storage but used NAND, a form of non-volatile storage, instead. With NAND the performance was no longer dictated by the laws of rotational physics because we were dealing with all solid-state storage, which introduced dramatically lower latencies and opened the door for much higher throughputs, putting pressure on SATA-IO to increase the interface bandwidth. To illustrate how fast NAND really is, let's do a little calculation.
It takes 115 microseconds to read 16KB (one page) from IMFT's 20nm 128Gbit NAND. That works out to be roughly 140MB/s of throughput per die. In a 256GB SSD you would have sixteen of these, which works out to over 2.2GB/s. That's about four times the maximum bandwidth of SATA 6Gbps. This is all theoretical of course—it's one thing to dump data into a register but transferring it over an interface requires more work. However, the NAND interfaces have also caught up in the last couple of years and we are now looking at up to 400MB/s per channel (both ONFI 3.x and Toggle-Mode 2.0). With most client platforms being 8-channel designs, the potential NAND-to-controller bandwidth is up to 3.2GB/s, meaning it's no longer a bottleneck.
Given the speed of NAND, it's not a surprise that the SATA interface quickly became a bottleneck. When Intel finally integrated SATA 6Gbps into its chipsets in early 2011, SandForce immediately came out with its SF-2000 series controllers and said, "Hey, we are already maxing out SATA 6Gbps; give us something faster!" The SATA-IO went back to the drawing board and realized that upping the SATA interface to 12Gbps would require several years of development and the cost of such rapid development would end up being very high. Another major issue was power; increasing the SATA protocol to 12Gbps would have meant a noticeable increase in power consumption, which is never good.
Therefore the SATA-IO had to look elsewhere in order to provide a fast yet cost efficient standard in a timely matter. Due to these restrictions, it was best to look at already existing interfaces, more specifically PCI Express, to speed up the time to the market as well as cut costs.
| Serial ATA | PCI Express | |||
| 2.0 | 3.0 | 2.0 | 3.0 | |
| Link Speed | 3Gbps | 6Gbps |
8Gbps (x2) 16Gbps (x4) |
16Gbps (x2) 32Gbps (x4) |
| Effective Data Rate | ~275MBps | ~560MBps |
~780MBps ~1560MBps |
~1560MBps ~3120MBps (?) |
PCI Express makes a ton of sense. It's already integrated into all major platforms and thanks to scalability it offers the room for future bandwidth increases when needed. In fact, PCIe is already widely used in the high-end enterprise SSD market because the SATA/SAS interface was never enough to satisfy the enterprise performance needs in the first place.
Even a PCIe 2.0 x2 link offers about a 40% increase in maximum throughput over SATA 6Gbps. Like most interfaces, PCIe 2.0 isn't 100% efficient and based on our internal tests the bandwidth efficiency is around 78-79%, so in the real world you should expect to get ~780MB/s out of a PCIe 2.0 x2 link, but remember that SATA 6Gbps isn't 100% either (around 515MB/s is the typical maximum we see). The currently available PCIe SSD controller designs are all 2.0 based but we should start to see some PCIe 3.0 drives next year. We don't have efficiency numbers for 3.0 yet but I would expect to see nearly twice the bandwidth of 2.0, making +1GB/s a norm.
But what exactly is SATA Express? Hop on to next page to read more!
Facebook
Twitter
RSS
131 Comments
View All Comments
Kristian Vättö - Tuesday, March 18, 2014 - link
Bear in mind that SATA-IO is not just some random organization that does standards for fun - it consists of all the players in the storage industry. The current board has members from Intel, Marvell, HP, Dell, SanDisk etc...BMNify - Thursday, March 20, 2014 - link
indeed, and yet its now clear these and the other design by committee organization's are no longer fit for purpose , producing far to little far to late....ARM IP =the generic current CoreLink CCN-508 that can deliver up to 1.6 terabits of sustained usable system bandwidth per second with a peak bandwidth of 2 terabits per second (256 GigaBytes/s) at processor speeds scaling all the way up to 32 processor cores total.
Intel IP QPI = Intel's Knights Landing Xeon Phi due in 2015 with its antiquated QPI interconnect and its expected ultra short-reach (USR) interconnection only up to 500MB/s data throughput seems a little/lot short on real data throughput by then...
Hrel - Monday, March 17, 2014 - link
Cost: Currently PCI-E SSD's are inexplicably expensive. If this is gonna be the same way it won't sell no matter how many PCI-E lanes Intel builds into it's chipset. My main concern with using the PCI-E bus is cost. Can someone explain WHY those cost so much more? Is it just the niche market or is there an actual legitimate reason for it? Like, PCI-E controllers are THAT much harder to create than SATA ones?I doubt that's the case very much. If it is then I guess prices will drop as that gets easier but for now they've priced themselves out of competition.
Why would I buy a 256GB SSD on PCI-E for $700 when I can buy a 256GB SSD on SATA for $120? That shit makes absolutely no sense. I could see like a 10-30% price premium, no more.
BMNify - Tuesday, March 18, 2014 - link
"Can someone explain WHY those cost so much more?"greed...
due mostly to not invented here is the reason we are not yet using a version of everspin's MRAM 240 pin, 64MByte DIMM with x72 configuration with ECC for instance http://www.everspin.com/image-library/Everspin_Spi...
it can be packaged for any of the above forms M2 etc too rathe than have motherboard vendors put extra ddr3 ram slots decicated to this ddr3 slot compatable everspin MRAM today with the needed extra ddr3 ram controllers included in any CPU/SoC....
rather than licence this existing (for 5 years ) commercial MRAM product and collaborate together to make and improve the yield and help them shrink it down to 45nm to get it below all of today's dram fastest speeds etc they all want an invented here product and will make the world markets wait for no good reason...
Kristian Vättö - Tuesday, March 18, 2014 - link
Because most PCIe SSDs (the Plextor M6e being an exception) are just two or four SATA SSDs sitting behind a SATA to PCIe bridge. There is added cost from the bridge chip other additional controller, although the main reason are the laws of economics. Retail PCIe SSDs are low volume because SATA is still the dominant interface and that increases production costs for the OEMs. Low order quantities are also more expensive for the retailers.In short, OEMs are just trying to milk enthusiasts with PCIe drives but ones we'll see PCIe entering the mainstream market, you'll no longer have to pay extra for them (e.g. SF3700 combines SATA and PCIe in a single chip, so PCIe isn't more expensive with it).
Ammohunt - Thursday, March 20, 2014 - link
Disappointed there wasn't a SAS offering compared 6GB SAS != 6G SATAjseauve - Thursday, March 20, 2014 - link
Awesome computerwestfault - Saturday, March 22, 2014 - link
"The SandForce, Marvell, and Samsung designs are all 2.0 but at least OCZ is working on a 3.0 controller that is scheduled for next year."When you say OCZ is developing on a PCIe 3.0 controller do you mean that they were working on one before they were purchased by Toshiba, or was this announced since they were acquired by Toshiba? I understand that Toshiba has kept the OCZ name, but is it certain that they have continued all R&D from before OCZ's bankruptcy?
dabotsonline - Monday, April 28, 2014 - link
Roll on SATAe with PCIe 4.0, let alone 3.0 next year!MRFS - Tuesday, January 20, 2015 - link
I've felt the same way about SATAe and PCIe SSDs --cludgy and expensive, respectively.
Given the roadmaps for PCIe 3.0 and 4.0, it makes sense to me, imho,
to "sync" SATA and SAS storage with 8G and 16G transmission clocks
and the 128b/130b "jumbo frame" now implemented in the PCIe 3.0 standard.
Ideally, end users will have a choice of clock speeds, perhaps with pre-sets:
6G, 8G, 12G and 16G.
In actual practice now, USB 3.1 uses a 10G clock and 128b/132b jumbo frame:
max headroom = 10G / 8.25 bits per byte = 1.212 GB/second.
132 bits / 16 bytes = 8.25 bits per byte, using the USB 3.1 jumbo frame
To save a lot of PCIe motherboards, which are designed for expansion,
PCIe 2.0 and 3.0 expansion slots can be populated with cards
which implement 8G clocks and 128b/130b jumbo frames.
That one evolutionary change should put pressure on SSD manufacturers
to offer SSDs with support for both features.
Why "SATA-IV" does not already sync with PCIe 3.0 is anybody's guess.
We tried to discuss this with the SATA-IO folks may moons ago,
but they were quite committed to their new SATAe connector. UGH!