Original Link: http://www.anandtech.com/show/7065/intel-ssd-dc-s3500-review-480gb-part-1
Intel SSD DC S3500 Review (480GB): Part 1by Anand Lal Shimpi on June 11, 2013 6:10 PM EST
Last year Intel introduced its first truly new SSD controller since 2008. Oh how times have changed since then. Intel's original SSD controller design was dual purpose, designed for both consumer and enterprise workloads. Launching first as the brains behind a mainstream Intel SSD, that original controller did a wonderful job of kicking off the SSD revolution that followed. Growing pains and a couple of false starts kept a true successor to Ephraim (Intel's first controller) from ever really surfacing over the next few years.
Last year, Ephraim got a true successor and it came in the form of a very high-end enterprise drive: the Intel SSD DC S3700. Equipped with tons of 25nm HET-MLC NAND, the S3700 officially broke the enterprise addiction to SLC for high endurance drives while raising the bar in all aspects of performance. In addition to the usual considerations however, Intel had a new focus with the S3700: performance consistency.
Due to the nature of NAND flash, there's a lot of background management/cleanup that happens in order to ensure endurance as well as high performance. It's these background management tasks that can dramatically impact performance. I love the cleaning your room analogy because it accurately describes the tradeoffs SSD controller makers have to deal with. Clean early and often and you'll do well. Put off cleaning until later and you'll enjoy tons of free time early but you'll quickly run into a problem. It's an oversimplification, but the latter is what most SSD controllers have done historically, and the former is what Intel always believed in. With the S3700, Intel took it to the next level and built the most consistent performing SSD I'd ever tested.
Performance consistency matters for a couple of reasons. The most obvious is an impact to user experience. Predictable latencies are what you want, otherwise your applications can encounter odd hiccups. In client drives, those hiccups appear as unexpected pauses during application usage. In the enterprise, the manifestation is similar except the user encounters the issue somewhere over the internet rather than locally. The other issue with inconsistent performance really creeps up in massive RAID arrays. With many drives in a RAID array, overall performance is determined by the slowest performing drive. Inconsistent performance, particularly with large downward swings, can result in substantial decrease in the performance of large RAID arrays. The motivation to build a consistently performing SSD is high, but so is the level of difficulty in building such a drive.
Intel had the luxury of being able to start over with the S3700's controller architecture. It moved to a flat indirection table (mapping between LBAs and NAND pages), which incurred a DRAM size penalty but ultimately made it possible to deliver much better performance consistency. The S3700 did amazingly well in our enterprise tests, and produced the most consistent IO consistency curves I'd ever seen. The only downside? Despite being much better priced than the Intel X25-E and SSD 710, the S3700 is still a very expensive drive. The move to a better architecture helped reduce the amount of spare area needed for good performance, which in turn reduced cost, but the S3700 still used Intel's most expensive, highest endurance MLC NAND available (25nm HET-MLC). With the largest versions capable of enduring nearly 15 petabytes of writes, the S3700 was really made for extremely write intensive workloads. The drive performs very well across the board, but if you don't have an extremely write intensive workload you'd be paying for much more than you needed.
We always knew that Intel would build a standard MLC version of the S3700, and today we have that drive: the Intel SSD DC S3500.
The Drives & Architecture
Architecturally, the S3500 looks a lot like the S3700. You get the same controller, similar firmware, AES-256 encryption and power loss protection. Where the two drives differ is in the type of NAND used. While the S3700 used 25nm HET-MLC (High Endurance Technology), the S3500 features high-grade 20nm MLC. As it's less focused on extremely write heavy applications, the S3500 also features less spare area than the S3700. You're still getting more than you would with a consumer drive, but far less than what we saw with its big brother. The result is lower endurance, lower sustained 4KB random write performance but a lower price as well.
|Enterprise SSD Comparison|
|Intel SSD DC S3700||Intel SSD DC S3500||Intel SSD 710||Intel X25-E||Intel SSD 320|
|Capacities||100 / 200 / 400 / 800GB||80 / 120 / 160 / 240 / 300 / 400 / 480 / 600 / 800GB||100 / 200 / 300GB||32 / 64GB||80 / 120 / 160 / 300 / 600GB|
|NAND||25nm HET MLC||20nm MLC||25nm HET MLC||50nm SLC||25nm MLC|
|Max Sequential Performance (Reads/Writes)||500 / 460 MBps||500 / 450 MBps||270 / 210 MBps||250 / 170 MBps||270 / 220 MBps|
|Max Random Performance (Reads/Writes)||76K / 36K||75K / 11.5K||38.5K / 2.7K IOPS||35K / 3.3K IOPS||39.5K / 600 IOPS|
|Endurance (Max Data Written)||1.83 - 14.6PB||45 - 450TB||500TB - 1.5PB||1 - 2PB||5 - 60TB|
|Power Safe Write Cache||Y||Y||Y||N||Y|
|Intel SSD DC S3x00 Endurance (Total Drive Writes)|
|S3700||-||1.825 PB||-||-||3.65 PB||-||-||7.3 PB||-||-||14.6 PB|
|S3500||45 TB||-||70 TB||100 TB||140 TB||-||170 TB||225 TB||275 TB||330 TB||450 TB|
Intel provided two MSRPs for the S3500: $115 for an 80GB drive and $979 for an 800GB drive. With a range of $1.22 to $1.43 per GB, the S3500 is clearly more expensive than consumer drives but it hardly feels like it's priced as an enterprise solution.
|Intel SSD Overprovisioning Comparison|
|Advertised Capacity||Total NAND on-board||User Acessible Capacity||MSRP|
|Intel SSD 710||200GB||320GB||186GB||$800|
|Intel SSD DC S3700||200GB||264GB||186GB||$470|
|Intel SSD DC S3500||240GB||264GB||223GB||~$320|
Like the S3700, the S3500 is available in both 1.8" and 2.5" form factors. The 1.8" version is limited to 80GB, 240GB, 400GB and 800GB capacities, while the 2.5" version is available in all of the capacities. Also like its bigger brother, the S3500 supports both 5V and 12V operation. Power consumption is a bit lower than on the S3700, but idle power is still too high for notebook use at 600mW. Intel really needs a consumer optimized version of this controller for use in the client space.
|Intel SSD DC S3x00 Power Consumption (5V, Max)|
My S3500 sample showed up shortly before I left for Computex, which unfortunately left me without much time to go through and do a thorough job of evaluating the drive. Thankfully I had enough time to get some of the basics done, so what I'm presenting here is the first part of our look at the S3500. We're also continuing work on building some of our own flagship enterprise SSD benchmarks in Johan's mini datacenter, so I'm hoping to be able to run some of those workloads on the S3500 in the not too distant future.
In our Intel SSD DC S3700 review I introduced a new method of characterizing performance: looking at the latency of individual operations over time. The S3700 promised a level of performance consistency that was unmatched in the industry, and as a result needed some additional testing to show that. The reason we don't have consistent IO latency with SSDs is because inevitably all controllers have to do some amount of defragmentation or garbage collection in order to continue operating at high speeds. When and how an SSD decides to run its defrag and cleanup routines directly impacts the user experience. Frequent (borderline aggressive) cleanup generally results in more stable performance, while delaying that can result in higher peak performance at the expense of much lower worst case performance. The graphs below tell us a lot about the architecture of these SSDs and how they handle internal defragmentation.
To generate the data below I took a freshly secure erased SSD and filled it with sequential data. This ensures that all user accessible LBAs have data associated with them. Next I kicked off a 4KB random write workload across all LBAs at a queue depth of 32 using incompressible data. I ran the test for just over half an hour, no where near what we run our steady state tests for but enough to give me a good look at drive behavior once all spare area filled up.
I recorded instantaneous IOPS every second for the duration of the test. I then plotted IOPS vs. time and generated the scatter plots below. Each set of graphs features the same scale. The first two sets use a log scale for easy comparison, while the last set of graphs uses a linear scale that tops out at 50K IOPS for better visualization of differences between drives.
The high level testing methodology remains unchanged from our S3700 review. Unlike in previous reviews however, I did vary the percentage of the drive that I filled/tested depending on the amount of spare area I was trying to simulate. The buttons are labeled with the advertised user capacity had the SSD vendor decided to use that specific amount of spare area. If you want to replicate this on your own all you need to do is create a partition smaller than the total capacity of the drive and leave the remaining space unused to simulate a larger amount of spare area. The partitioning step isn't absolutely necessary in every case but it's an easy way to make sure you never exceed your allocated spare area. It's a good idea to do this from the start (e.g. secure erase, partition, then install Windows), but if you are working backwards you can always create the spare area partition, format it to TRIM it, then delete the partition. Finally, this method of creating spare area works on the drives we've tested here but not all controllers may behave the same way.
The first set of graphs shows the performance data over the entire 2000 second test period. In these charts you'll notice an early period of very high performance followed by a sharp dropoff. What you're seeing in that case is the drive allocating new blocks from its spare area, then eventually using up all free blocks and having to perform a read-modify-write for all subsequent writes (write amplification goes up, performance goes down).
The second set of graphs zooms in to the beginning of steady state operation for the drive (t=1400s). The third set also looks at the beginning of steady state operation but on a linear performance scale. Click the buttons below each graph to switch source data.
|Intel SSD DC S3500 480GB||Intel SSD DC S3700 200GB||Seagate 600 Pro 400GB|
While it's not quite as pretty of a curve as what we saw with the S3700, the two drives are clearly related. Intel's SSD DC S3500 delivers incredibly predictable performance. The biggest take away is that Intel is able to still deliver good performance consistency even with much less NAND spare area than on the S3700. Architecture enables what we see here, not just spare area.
|Intel SSD DC S3500 480GB||Intel SSD DC S3700 200GB||Seagate 600 Pro 400GB|
Remember this predictable little pattern? The periodic dips in performance are Intel's defrag/GC routines operating at regular (and frequent) intervals. You'll also notice the slight upward trend here; the S3500 is looking to improve its performance over time, even under heavy load, without sacrificing consistency.
|Intel SSD DC S3500 480GB||Intel SSD DC S3700 200GB||Seagate 600 Pro 400GB|
This zoomed in view really gives us great perspective as to what's going on. I included comparable data from other drives in the S3700 review, but the size/scales of those charts made inclusion here not possible in short form.
Random & Sequential IO Performance
Similar to our other Enterprise Iometer tests, queue depths are much higher in our Iometer benchmarks. To measure sequential performance I ran a 1 minute long 128KB sequential test over the entire span of the drive at a queue depth of 32. For random performance, I ran a 3 minute long 4KB random read test over the entire span of the drive at a queue depth of 32. Random write performance was measured at steady state (QD32), which varies depending on capacity and other factors. The results reported are in average MB/s over the entire test length.
Random read performance is just excellent. The S3500 gives up nothing compared to the S3700 and ends up being one of the fastest enterprise drives we've tested.
Random write performance takes a big hit compared to the S3700. At 47MB/s, the S3500 isn't bad but it's not class leading like the S3700.
Sequential read speed is limited by 6Gbps SATA, and the S3500 has no issues hitting that limit given the number of NAND die inside the 480GB model. It's not until you get down to the 160GB version that performance will drop below 500MB/s.
Sequential write performance is substantially better on the S3500, making this drive even better for large sequential caching applications.
In our first high level look, Intel's SSD DC S3500 looks to be everything we loved about the S3700 but at a more affordable price point for enterprise customers who don't need insane amounts of write endurance. With SSDs in the enterprise there's this tendency to over estimate endurance needs. I was guilty of it in the design of our SSD requirements for serving AnandTech. Part of the problem is there aren't tons of good software applications to quietly monitor/report/analyze enterprise workload behavior. I suspect that for the vast majority of use cases however, the S3500 is likely more than enough. Even though the AnandTech database servers (content, stats tracking and forums) are fairly write intensive, the S3500 is actually the right target for us - the S3700 would deliver far more endurance than we'd ever use.
Other than write endurance, the only other thing you give up is random write performance. Intel's specs list the S3500 at roughly 1/3 of the sustained 4KB random write performance of the S3700, which I saw in our numbers as well. Given the lower price point however, most customers are likely comparing performance to an array of hard drives. On an individual level, a good high-end HDD will provide somewhere around 1 - 2MB/s in 4KB random write performance. The S3500 by comparison is good for about 40MB/s. Intel's own data shows that 12 S3500s will deliver roughly the same random IO as 500 15K RPM hard drives. Based on the data I've seen, that comparison is pretty accurate.
All of the other S3700 benefits remain. Performance consistency is excellent, which makes the S3500 ideal for use in many-drive RAID arrays. Intel's enterprise drives have typically done very well in terms of reliability as well (and I haven't heard any complaints about the S3700), making the S3500 a safe bet. My only real complaint about this drive is the idle power rating is too high for notebook use, because otherwise I'd even suggest looking at the S3500 for consumer use as well.
All in all I've been pleased with Intel's work in the enterprise SSD space. Most interesting to me is just how aggressive Intel has been in terms of enterprise SSD pricing. The S3500 shows up at well under $1.50/GB, and consumer drives aren't that far off in terms of pricing. Intel typically doesn't push this aggressively for lower prices with its enterprise products, so when it happens I'm very happy.