Samsung Explains The Firmware Bug Causing The Failures of SSD 840/840 Pro
by Kristian Vättö on December 7, 2012 1:05 PM EST- Posted in
- Storage
- SSDs
- Samsung
- Samsung SSD 840
We've been covering the issues surrounding Samsung's SSD 840/840 Pro lately. The issue was first discovered when Anand's pre-production review sample died during testing and we also noted that in our initial review. Samsung quickly sent us another drive but it also failed after a couple of days of testing. My SSD 840 managed over a month but ironically enough, it died right after I had completed endurance testing.
Earlier Samsung told us that all review samples including our three shipped with a pre-production firmware that had a bug in it causing the failures (retail units were shipped with a newer firmware without the bug). At the time we didn't know what exactly was wrong in the firmware, but now we do. When the drive was issued a secure erase command, it would clear all table mapping information at the Address Translation Layer (ATL) but not at the Host Interface Layer (HIL). The data in both layers needs to be up-to-date for the drive operate properly, so when a write request came in, the controller wasn't able to map the data correctly, which caused the firmware to hang. An SSD obviously can't operate without a functioning firmware so from a user's standpoint, it looked like the drive had completely died even though only its firmware was broken.
All our three failures support this explanation. Our first 840 Pro sample died during a 128KB sequential write pass that we use to pre-condition our drives for enterprise tests, but the drive was secure erased just before beginning to fill the drive. The second 840 Pro died during power consumption testing but again it was secure erased right before starting the test. The regular 840 actually died when I tried to secure erase it. The secure erase command resulted in an error so I power cycled the drive but it was no longer detected by the system after reconnecting it.
| Comparison of Samsung SSD Firmware Versions | ||
| Pre-Production | Retail | |
| Samsung SSD 840 Pro | DXM02B0Q | DXM03B0Q |
| Samsung SSD 840 | DXT05B0Q | DXT06B0Q |
The good news is that all retail units have shipped with a newer firmware, only reviewers and others who have access to pre-production units were affected by this bug.
For users considering the SSD 840/840 Pro, this should be reassuring news. The 840 Pro is still the fastest SATA 6Gbps SSD we have tested and it's definitely one of the top choices where performance is concerned today. The TLC NAND based SSD 840 is more mainstream focused but from what I have seen, it seems to be fairly competitively priced. The SSD 830 spoiled many with low prices but that was only to clear stocks. If you can still find a bargain SSD 830, don't hesitate to pull the trigger as those won't be available much longer, but we're more comfortable recommending the 840/840 Pro now.
While long-term reliability is still unknown, if history is any indication the 840/840 Pro are in good company as the 830 was a solid drive. Our third 840 Pro with the new fixed firmware has been going strong for weeks now and we have even recreated the scenarios that killed the earlier samples. We are also waiting for more samples from Samsung to test all capacities of SSD 840 and 840 Pro, so stay tuned!
Facebook
Twitter
RSS
56 Comments
View All Comments
longtom - Sunday, December 9, 2012 - link
Both of these drives did not work straight out of the packaging. Yes, they only do not work when in a RAID. When out of RAID they work fine. However, when in any form of RAID configuration on the AMD SB950 Southbridge (990FX, a popular chip right now) the drive is INVISIBLE to both BIOS and Windows, even after all drivers are installed.I have confirmed that this is the fault of the drive, as several other SSDs that I have work fine in RAID, thus ruling out any board, BIOS, or chipset problem.
Are you suggesting that somehow a secure erase will make the drive work, or are you wondering if I have tried a secure erase to see if it shuts down the drive, as it has with previous models? In case of the latter, I have not attempted a secure erase at all.
I can understand a mistake in development, so therefore I will be contacting Samsung to see if I can get either confirmation that they know about the issue and are working on a fix, a replacement drive of a different model, or my money back. How they answer will determine whether they are up to snuff as a reliable company or not. I will post results back when I get them.
frief - Sunday, December 9, 2012 - link
Hi longtom,> In case of the latter, I have not attempted a secure erase at all.
Yes, the latter. It's good if these can be kept separate issues.
Kristian Vättö - Sunday, December 9, 2012 - link
Definitely contact Samsung then. Validation is always limited to certain setups, so it's possible that not so common setups face a bug (unfortunately this usually means AMD). Especially RAID is probably not tested on every possible setup that exists given that the regular 840 is mainly meant for mainstream users (who don't even know what RAID is). This happens a lot with laptops because the OEMs often customize BIOS which can cause issues.If you can't get any proper help through the regular customer service, drop me an email (kristian[AT]anandtech[DOT]com) and I'll forward it to the right people.
Kristian Vättö - Monday, December 10, 2012 - link
longtom,Can you email me at kristian@anandtech.com? I contacted Samsung and they are looking into the issue but would like your contact information for possible further questions about your issue/setup.
longtom - Saturday, December 22, 2012 - link
Here is the end of the story:After nearly a week on the phone with Samsung technical support, I learned that their entire tech support department is actually outsourced to another company. They are kept in the dark about a lot of issues, and are also not given the power to make important decisions about returns. I was told that turn-around time for an issue like mine could be several days to weeks before permission is granted.
If a customer has a ten to thirty day return window on a product, this is unacceptable.
On the very first day, I was told to return the drives to the retailer where I got them, and even after I explained that I had a "replace only" purchase, they still balked about taking care of me. I felt that it was their issue, and trying to be a good customer, I VERY GENEROUSLY asked if I could downgrade my drives to some known working models, such as the 830 for NO MONEY ACROSS THE TABLE. After all, their product did not do what they claimed it would do. I was instead tasked with 5 days worth of troubleshooting before they would consider writing a letter to their parent company in Korea to "ask" for permission to return the items. In those five days, THEY TOLD ME THAT THEY NEVER GOT AROUND TO SENDING THAT EMAIL.
On evening three, I learned ON MY OWN that Samsung had VERY QUIETLY rolled out a firmware update. On day four of negotiating with their outsourced tech support department, I asked about the firmware update. I was told that it was only for issues with the Trim. This was also not true. I will go into that more in a minute.
I was asked to try to flash the firmware, anyway. The flash from my motherboard failed, and Samsung tech support responded by asking me to find another computer to try it on. Luckily, I have several around the house. I tried the firmware update on my wife's computer, and it would not work. I then went out to my workshop, where I have some older motherboards. I had to scrounge together a power supply, memory, and a hard drive, installed windows and Magician, and it worked.
As an aside, part of the problem was that the Samsung Magician software is not functioning as it should. It gives three options to attempt to flash your firmware. The first is within Windows, using Magician its self, and the other two use a bootable usb flash drive, and a bootable CDROM option. The thumb drive and CDROM both use FreeDos as an operating environment to run their software. The problem with the USB drive option was that Samsung coded the FreeDos boot procedures wrong, and it tried to load its kernel from the wrong drive letter that it had assigned itself. The CDROM, on the other hand, told me that there was no firmware found on the disk which Magician burned. That left me with only the Windows option.
These are grave errors on the part of Samsung. One problem I can understand, but every single option firmware option in the Magician software is broken in some way. On the upside, I've been informed that Samsung has plans to re-design Magician from the ground up. This is a positive note. At least they're working hard to fix known issues.
For right now, though, their maintenance software is junk.
Back to the lying. Even though I was told that the firmware fixes were small TRIM adjustments, both of my drives exhibited the same problems before the firmware update, refused to update on the same motherboards, and then exhibited the same behavior AFTER the updates.
They lied to me, and to their own customer service people. It WAS INDEED a fix. I did not like being misled, and I will not be purchasing any more Samsung products in the future. I will vote with my wallet, and try to support a company that has customer satisfaction as the number one concern.
Ah, the joys of a free market. I can take my business elsewhere, thereby encouraging healthy competition.
There is a gentleman out there at Samsung tech support named Jose. Jose bent over backwards to try to make me happy, and he needs a commendation for his excellence in service, but he was being stonewalled by Samsung at every pass. THANK YOU, Jose. I can not tell you enough how much I appreciated your hard work. You need a raise and possibly a promotion. Merry Christmas, Jose. I wish the best for you and for your family.
Basically, even though Samsung tells them nothing, the guys in their tech support company are finding fixes for themselves through trial and error (and dissatisfied customers). They're trying to satisfy their contract from Samsung and do a good job anyway. That is a sad situation, right there, Samsung. FIX IT.
Because if it, they lost a customer.
TemjinGold - Sunday, December 9, 2012 - link
Well good for you. Would you like a medal?Azethoth - Monday, December 10, 2012 - link
Yes, he needs everyone to send him a tiny medal. He then melts it down and makes money on the recycled metal. The metal is then used to make more medals.EJ257 - Sunday, December 9, 2012 - link
Although I'm surprise Samsung didn't find this in their internal testing and fix the bug before sending out your review samples. Maybe they already found it and were working on the fix when they sent out your drives. Anyway good to know it won't affect production units.Brahmzy - Sunday, December 9, 2012 - link
No data lost as I don't put anything critical on my R0 arrays, but I lost the whole thing.My ASUS board would not boot to bios with both drives connected - the boot device LED would light up solid and Id have s black screen. If I unplugged one, it would boot up fine, and of course the OROM would show my I had a failed array. Took me a while to figure this out.
I then booted off another W7 SSD I have for maintenance and both drives appeared healthy in RST when I plugged them both back in, while not running as a system disk. The array verified fine. As soon as I would reboot and the bios would try to read the array (which it does even before posting) I'd get a black screen. Nothing. Unplug either of the two drives and it would post.
After dicking around for an hour trying everything, I destroyed the array (1 disk at a time) boot off the other SSD, secure erased both Pros and rebuilt the array. It boots fine again and I installed a base OS without issues. I'm not happy - I have never seen this behavior before in 30+ SSD RAID 0 setups.
It's like the array's metadata got corrupted or something. I do not trust these drives right now and am a step away from returning them.
A couple points:
*drives ran fine without a hitch for a 1.5 weeks in RAID0
*both drives have been secure erased probably 6-8 times each
*drives never actually died. They were recoverable with a secure erase. The bios refused to post with them in a R0 array.
*both drives are on the latest firmware
*this sucks because I've got a total of 4 of these drives on my systems and am not feeling real confident right now
-Anandtech, please send this info to Samsung if you have a fast channel to their SSA tech group, thanks.
Brahmzy - Sunday, December 9, 2012 - link
So I know weird things happen and I was going to chalk this up as a fluke because I absolutely despise when people spread/feed FUD on these kinda of things. However, the poster above me with the AMD setup is what has me worried. His drives are fine until they are put into an array. That is similar to what I experienced, the difference is mine we're fine for almost 2 weeks and are running in another array, again, right now. For how long? Nobody knows.