I come not to praise RAID-5...
August 3, 2012
...but not really to bury it, either.
Edit Just as warning, I have ignored many aspects of RAID-level choices here, and concentrated on that of the likelihood of catastrophic failure. There are lots of reasons to not select RAID-5 in a new array, but a comprehensive review of all aspects of the technology is way beyond the scope of a single blog entry. Take this entry for what it is: an explanation of a particular phenomenon of failure, and what makes it more likely or not. Don't base your decision solely on this article, but let it be what it was meant, as a piece of the puzzle.
RAID-5 is misunderstood by too many people.
It's cyclical, to be sure. We started off with no one knowing what RAID was, then we went to everyone knowing what RAID was (or at least, being familiar enough with it to mistake it for being a backup). RAID-5 was sweet, because it had a parity stripe, and if you had a bunch of drives shoved together, it could tolerate the loss of a drive. How awesome was that?
Then Robin Harris went and wrote Why RAID-5 Stops Working in 2009. It was an excellent article, and it was exactly right for the purposes that Robin meant it. It's been a good way to point out to people that maybe they shouldn't have RAID-5 configurations with 40TB worth of the cheapest SATA drives known to man.
But people started equating RAID-5 with being broken or bad. Or always wrong. And it isn't. You can't just take the shortcut and say that a technology like that is absolutely improper because it breaks down under certain circumstances. So I'm going to try to give you my perspective on this by recycling something I put together for an individual who had questions about when to use RAID-5 and when not to. I hope this helps clarify things.
Starting with the basics...hard disk drives that are not flash based operate by having small "heads" move back and forth across spinning platters (usually made of ceramic these days, but there are still metal ones floating around). Think of it like a record player arm where the needle is the head.
The head can read or write, depending on its instructions. Each hard drive has multiple platters, with one head per platter. Good so far?
OK, so the drive writes data and reads data...but it's reading incredibly small magnetic data from the platters. You know how sometimes the magnetic cards in your wallet get erased from being around magnets? Those bits are HUGE compared to the ones and zeros on a hard drive platter.
The bits in the drive are not stored in your wallet, of course; they're inside a metal case which is inside of your computer...but occasionally strange things will happen, like maybe something bumped the disk while the head was reading or writing, and it creates a gouge on the surface of the disk (remember, the platters are spinning at 5,400RPM at least!), or maybe dust got into the drive and is blocking the sector, or maybe even cosmic rays have flipped a bit on the platter (really!).
Whatever it is, something stops the head from being able to read the data on the disk. This is called an Unrecoverable Read Error (or URE for short).
The likelihood of encountering a URE is dependent on things like the build quality of the head and the speed, but mostly on the size of the sectors on the disk (remember how your magnetic stripe has big bits? those are harder to overwrite).
Manufacturers often figure out how likely a URE is for a particular drive, and they usually put it in the documentation for that drive. You can find it if you go to Pricewatch and pick a hard drive at random, then look up the product manual for that part number. Here's an example of the Western Digital Green Line, or maybe you'd prefer a Seagate Barracuda.
Anyway, get the product manual, and look in the specs for something like "Non-recoverable read errors per bits read", or "Non-recoverable read errors". What this gives you is the likelihood of your encountering a URE.
Both of those examples have "1 per 1014 bits read". If we use the handy-dandy Google calculator, you can see that it really means one URE per 11.3 terabytes.
So, putting all of this together...if you have a 3TB drive and you fill it to capacity, you could probably read from it over 3 times completely before you'd encounter a URE, and lose the data that was held by that particular sector.
That's kind of unsettling, right? 3 full reads on a 3TB drive, then a high likelihood of going kaput on the 4th read-through?
Fortunately, we use RAID levels which can protect us.
Lets start with RAID-1, which is mirrored disks. We are reading along happily when suddenly we get a URE on one of the drives! It would be sad but, on the other drive, we have an exact copy of the data as it was originally written! The RAID controller reads the data from the other drive, re-writes it on the drive that had the URE, and we continue merrily along.
Suppose we lose a RAID-1 drive. That leaves us with a good copy, but assuming the array was full, when we put a replacement drive into the array, we've got to re-read all of the data on the good drive. Keeping in mind that we're likely to experience a failure after 11.3TB, what is the statistical probability that we'll experience a URE reading 3TB? Around 1 in 4.
Now, lets move on to RAID-5. You need at least 3 drives in a RAID-5, because unlike the exact copy of RAID-1, RAID-5 has a parity, so that any individual pieces of data can be lost, and instead of recovering the data by copying it, it's recalculated by examining the remaining data bits.
So when we encounter a URE during normal RAID operations, the array calculates what the missing data was, the data is re-written so we'll have it next time, and the array carries on business as usual. But when a drive dies, we have to replace it, and that's when things get hairy.
In order to rebuild the array, the new drive needs to be populated, and in order to do that, the entire contents of the remaining drives need to be read, in order to calculate the parity information. Assuming we have a RAID-5 array that has 3 3-TB disks, we're now reading 6 terabytes of information. What is the statistical likelihood of encountering a URE?
1 in 2. A coin-flip.
Have a RAID-5 array with 4 3-TB disks? That's 1 in 1, almost certainly a failure. You can see how quickly this goes downhill.
Now, a lot of people see this, freak out, and say "oh my god, I'm never using RAID-5 again! RAID-5 is the devil! It's EVIL!", but remember what is driving the numbers...it's the URE rate.
Check out this Hitachi Ultrastar. It's URE rate is 1:1016 which is a kind-of-amazing 1.1 petabytes...so the odds of your UltraStar-based RAID-5 array dying during a rebuild because of a URE is...very low.
So you can't vilify RAID-5 across the board. It's very much a matter of what you're using it for, what the quality of the drives involved are, and what the capacity of the array is (or really, how much data is stored on the array).
Does that help you understand? Have questions, comments, or suggestions? Please leave them below, thanks!
Posted in Storage, System Administration
Email me
content rss
August 3rd, 2012 at 10:18 am
Well, sure. The (newer) rule of thumb is that RAID5 should not be used with dense storage. (IE. 1-2TB+) With smaller disks, it is still cool. RAID6 helps offset that but with the quantity of disks being used; you might as well go 1+0.
If you care about your data and performance, with the lower cost of storage these days, you might as well use 1+0 in most cases.
A nice write up though, a quality introduction.
August 3rd, 2012 at 11:49 am
Great intro to how RAID5 works, but nothing about the performance penalty on writes to the disks? This is the principle reason that people "hate" raid5. I know I've struggled with phb's over the years trying to explain the difference between raid5 raid1+0 and why raid5 is not going to be used on my database server!
August 3rd, 2012 at 12:11 pm
That didn't really solve the problems with RAID 5. Yes, you can spend more money to make it safe (ish) but in doing so you've made it no longer cheap - which was its only saving grace to begin with. So you still aren't as safe as RAID 10 nor are you as performant. But you end up costing as much.
Just because you can make RAID 5 safe enough to not be crazy, can you make it make sense in any particular deployment? Maybe in a very niche case where you are trying to squeeze a low performance, high capacity system into an already existing chassis and RAID 10 just won't fit? That would be very rare and that would imply some really high capacities that would rule out RAID 5 because of resilver times.
August 3rd, 2012 at 12:17 pm
Tina nails it. RAID 5 is hated because of the combination of reliability, performance and unavailability during recovery.
We've always had ways to make RAID 5 reliable. SAS drives have had URE rates low enough to make them just fine.
The article that you were discussing was based on the assumption that you are using RAID 5 to cut corners - to lower your budget. Going with expensive drives ruins that assumption and brings all of the other issues like performance and cost back into play so focusing on URE failures during resilver isn't really the factor at hand.
So I don't see where you've address the issues.
August 3rd, 2012 at 12:17 pm
It should also be worth noting that content context matters when choosing RAID, and for some things, the file system sitting atop the RAID array is going to compensate for those single bit errors. Are you storing a massive image archive, or content for a home media server? A few errant bits across many terabytes aren't going to hurt you too much.
Also, 1 bit in 11 Terabytes is still pretty phenomenal, though admittedly the Ultrastar's number is better. Unless you are talking scientific data, where absolute bit-precision is paramount*, a single bit flipped isn't going to make much difference, especially if the file systems checksums catch it. To get multiple bits flipped, the odds change, to get multiple bits flipped in sequence, the odds change even more.
So while I agree with Warner, than the benefits of RAID 5 fall off when using larger disks, its still better than JBOD, again, depending on the context of the data being stored.
* I still argue that an absolute level of precision is impossible and you are still going to require parity data somewhere in the chain to maintain integrity.
August 3rd, 2012 at 12:22 pm
You should check out SpiceWorks. RAID 5 is discussed there more than anywhere else, I would imagine, and everything here has been debunked thoroughly for actual business use. The ability to use low URE drives was always assumed, just not found to apply in the real world.
Here are some articles addressing this that you might want to check out:
http://www.smbitjournal.com/2010/02/raid-revisited/
http://www.smbitjournal.com/2012/05/when-no-redundancy-is-more-reliable/
http://www.smbitjournal.com/2012/07/hot-spare-or-a-hot-mess/
August 3rd, 2012 at 12:25 pm
"Check out this Hitachi Ultrastar. It's URE rate is 1:1016 which is a kind-of-amazing 1.1 petabytes...so the odds of your UltraStar-based RAID-5 array dying during a rebuild because of a URE is...very low."
But what would the resilver time be? Months? How long can you be without normal performance and quite possibly without availability on an array that large? And that is a long, long time to go without losing a second drive. Fear of losing a second drive isn't a factor until you start coming up with ways to make mammoth RAID 5 arrays then issues come back that really don't exist elsewhere.
August 3rd, 2012 at 12:51 pm
@Nick: The issue with UREs in RAID 5 (but not in RAID 1 or 10) is that if a URE is encountered during a resilver operation that the entire array is lost, not just one bit. It is the resilvering of the parity that causes this failure.
So what sounds like "a bit flip" is actually "catastrophic array failure" if hit during a resilver. And a resilver has to read every bit on every disk in the array, sometimes more than once. So even a moderately small array is going to read many TB of data during a resilver. Lots of opportunity for the array to fail completely.
August 3rd, 2012 at 12:53 pm
With RAID 1 and RAID 10 (mirrored systems, not parity) a URE during resilver simply means that one bit is bad. No big deal (hopefully it wasn't in your database.) No different than hitting a URE on a regular drive on your desktop. You probably won't notice.
It is the parity of RAID 5/6 that puts the array at incredible risk of UREs.
August 3rd, 2012 at 1:08 pm
Hi everyone,
to reply to people in general:
No, I didn't address the performance penalties of RAID-5 (and no, I didn't mention RAID-6). I was concentrating on the chances of catastrophic failure during rebuild.
There are performance reasons why you would not prefer RAID-5 (or RAID-6 for that matter), but all other things being equal, using RAID-5 is almost always a choice of economics, not technical feasibility.
If you have a certain amount of money to spend, and a certain capacity you have to hit, then RAID-10 is maybe not a viable choice for you. RAID-5 might be, or it might not be. But if you are going to spend money on RAID-5 at all, then you should be aware of when it will die on you.
Alternatively, lots of big arrays with many shelves of disks still use RAID-5 as their go-to RAID level. I've read many, many kneejerk responses from people who say, "RAID 5? I hope you don't like your data!" because they have heard RAID-5 will lose your data, but they didn't understand the underlying reasons, and that's what I was trying to communicate in this blog post.
I did not want to discuss the economics of it, because everyone's budget is completely dissimilar. I work in a place where the guy setting the budget remarked to me during an interview, "It was only ten grand, and I wasn't going to let something small like that get in our way". Contrast that to the previous company I worked for, where $10k might have been two years worth of IT capex. And the previous company had higher uptime requirements than the one I'm working for now!
Everyone is different, and each person knows more about their situation than anyone who doesn't work in it, so if I can help people understand the universal basics, such as "why RAID-5 loses your data, and when", maybe they'll be able to make better decisions about their infrastructure.
August 3rd, 2012 at 1:16 pm
The problem is is that RAID 5 is "corner cutting" system for budgetary reasons. So ignoring the cost factor makes it not meaningful to real world system admins. The fact that RAID 5 was dangerous only existed within the context of the cost of the system.
You could have summed it all up by saying "RAID 5 remains technically viable as long as it no longer needs to be considered within the context of its cost."
But you could apply that to almost anything.
No enterprise storage vendor sells RAID 5 today. Even RAID 6 is effectively gone. What storage vendor are you using?
People's dissimilar budgets isn't an issue. If I have $5 or $5m, it doesn't change that in order to make RAID 5 safe enough (but never safer) we have to waste money. Wasting money just to make a technical point.
The cost of the drives necessary for RAID 5 offset any advantage you get from using it but don't address the other issues.
Yes, there are many knee jerk reactions to RAID 5, just as there used to be knee jerk reactions for it in the other direction previously. But I feel that you are telling people that RAID 5 is safe because you can, in theory, eliminate one particular problem and then act like its other issues don't exist. People looking for an excuse to use RAID 5 will read this, think that you've address all the issues and put their businesses at risk unnecessarily.
The issues with RAID 5 are far too complex and numerous to address in such a way. It is the combination of factors that eliminate RAID 5 from consideration from anyone that is not a storage expert as the factors are just too complex for anyone needing to read an article of this type.
August 3rd, 2012 at 1:24 pm
The fundamental problem with RAID 5 from a budget standpoint is that...
If you are cutting corners on your RAID you will not be looking at good drives. If you had the budget to spend, you can balance out more cost effectively in more reliable ways.
Continuing to use RAID 5 for existing deployments can be justified. We need not vilify every attempt to utilize what already exists as rip and replace can be very costly. But in continuous discussion on this topic, every day for two years in the SpiceWorks community with ardent supporters of parity RAID not yet has anyone come up with a "new" deployment scenario where RAID 5 could be justified. And each day as drives get larger, the likelihood of the real world, niche scenario where it might make sense dwindles.
Is there a niche case where experts could justify it? Yes. Is there a niche case that could afford those specialists vs. overbuying for safety? I'm apt to say no.
August 3rd, 2012 at 1:38 pm
>I feel that you are telling people that RAID 5 is safe because you can, in theory, eliminate one particular problem and then act like its other issues don't exist.
I wasn't addressing the other issues, but they certainly exist. As you say in your next paragraph, it's a complex topic. A comprehensive blog post wouldn't be a blog post - it would be a novella.
Be that as it may, though, the reality is that a *lot* of people out there (most, actually) don't have the time to become storage experts (and I'm not one either), but the reality is that they have to deal with RAID-5 for legacy reasons, and if you have an old system running RAID-5 and you go to set up a new array, you might automatically select RAID-5 because, hey, that's what you've already got. And that's a bad decision, but people need to know WHY it's a bad decision.
I'm never a fan of encouraging people to make uneducated choices. I've added a disclaimer at the top of this entry to let people know that this isn't an all-inclusive article, but I stand by it as written.
August 3rd, 2012 at 1:56 pm
That's a tough one because if they are to educate themselves on all of the issues they would become a storage expert :) If they don't do that, then they are forced into a position of a decision that they don't understand. The big risk is in explaining only one facet because without understanding all of the factors those who were avoiding RAID 5 for reasons they did not understand might now justify it again, for reasons they don't understand.
So the issue isn't solved. My argument is that if someone is going to implement RAID and not understand all of the factors, then they must use RAID 10 or take on incredible risk because they don't understand, necessarily, when RAID 5 will be safe or, at the very least, safe for them.
Any argument for RAID 10 can safely be stated as "unless you know exactly why to do otherwise, stick with RAID 10." Any argument for RAID 5 requires a complete dissertation because there are so many dangers and so many of them are so complex.
Basically if you err on the side of RAID 10 even when wrong, it's not a bad choice. If you err on the side of RAID 5, when it is wrong it is often disastrous.
August 3rd, 2012 at 2:13 pm
Mirroring is always a decision of economics. If you have the extra money, then mirror. But if you don't, then what do you do?
Lots of people don't have the funds, and they've got to make a decision. You can't force someone into being a storage expert, but you can help give them information.
Write a blog entry that takes all of the things into account, and I'll link to it.
August 3rd, 2012 at 3:01 pm
Okay Matt, you've sufficiently scared me.
My home fileserver is likely going to be converted to RaidZ2.
Thanks for that.
August 3rd, 2012 at 3:05 pm
Scott: See, success! ;-)
August 3rd, 2012 at 3:10 pm
LOL, well that's a start :) For those unaware, RAIDZ2 is a version of RAID 6 with some extra niceties implemented in the ZFS filesystem.
August 3rd, 2012 at 3:14 pm
My point about mirroring, though, is that it is not a question of economics, at least not when compared to RAID 5 (compared to RAID 6, I'll grant.) The reason being that the cost of making RAID 5 safe enough to use also causes it to become more expensive that a mirror based system while still being slower, less reliable and with the risks of rebuild performance.
If RAID 5 systems were all free and mirrored systems all were not, then yes, RAID 5 can be overbuilt enough to make it very useable. But the cost of that overbuilding defeats the use cases where it would make sense.
In theory, someday, super low URE drives will exist at a very low price point. That would bring RAID 5 back towards an acceptable place. But this is not expected. Drive size growth has always outpaced drive reliability improvements.
August 3rd, 2012 at 3:18 pm
URE rates on SSDs are amazingly low too, if the mfg numbers are to be believed. I'm not saying it's an awesome idea, but you could totally make a RAID 5 set out of SSDs. ;-)
August 3rd, 2012 at 3:22 pm
Yes, RAID 5 on SSD can make sense. SSDs tend to be very small as well so they are years beyond spindles on the curve for URE / size rations.
But SSD users tend to have large budgets too :)
August 3rd, 2012 at 4:24 pm
Don't believe everything you hear about SSD. I've seen ~3 of them fail in the last 6-9 months in my office. About 30 desktops each with an 80-120GB ssd. Some of the failures were ~3 year old devices, at least one failure was less than 6 months old.
August 3rd, 2012 at 4:27 pm
Trever: I've heard that pretty much everyone has had trouble with the controllers (since that's kind of where the magic is. The underlying flash is all made by just a couple of companies).
Of course, when your drive dies, it doesn't matter that it's the flash or the controller, you're still rebuilding.
August 3rd, 2012 at 5:11 pm
Assuming that you need to replace one of the disks in a RAID 5 array, if a URE is encountered on one of the remaining disks during the rebuild, does this result in an immediate fail of the rebuild? This seems like a huge consequence from a tiny issue. I may be misunderstanding a few concepts here so apologies for anything ridiculous in the following...
If a URE occurs on a regular drive that isn't part of a RAID array, I assume the only consequences relate to the inability to read the file which is stored (in part) on the unreadable bit. If that file happened to be a photo, for example, that's just one photo lost - the rest of the drive is still usable. But if a URE occurs on a disk in a RAID array you essentially lose everything when it becomes necessary to rebuild the array? Does the RAID controller not have any mechanism to deal with this? If a bit can only be a 0 or a 1, I'd prefer the controller have a guess and maybe throw up a warning than to give up entirely. I appreciate that's a very simplistic view and a wrong bit can have serious impact, but I was hoping that the RAID controller might have a few tricks up its sleeve?
Excellent article though.
August 3rd, 2012 at 5:15 pm
@Andy Yes, a URE failure encountered during a RAID 5 rebuild means total lose of the entire array even though no additional drive has failed. As this is orders of magnitude more likely than losing a second drive, this is why we point out that counting the number of drives that RAID protects you against failing is misguided. UREs are the big risk, not second drive failures.
I've never heard of any array that can survive a URE failure if using parity. If you don't want the risk, just avoid parity RAID. RAID 1/10 don't have that problem.
August 3rd, 2012 at 5:17 pm
If you check my "Hot Spares or a Hot Mess" link above, Andy, there is a good breakdown (I think) of how RAID 5 handles URE failures - in teh context of why a hot spare actually makes things worse as it automates your disaster before you have time to manually stop it.
August 4th, 2012 at 8:22 pm
The RAID-5 haters out there conveniently overlook the nasty failure modes of RAID-10.
First, with 3 TB drives, you still have a 1/4 chance of a URE and blown array during rebuild. It's basically the same risk as a small RAID-5 set.
Secondly, the odds of double drive failure taking you down are only reduced by 1/N, where N is the array width. You are not protected against double disk failure to the degree that you are with RAID-6.
Finally, large controller and OS caches ameliorate most of the performance problems of parity RAID.
NetApp
August 4th, 2012 at 8:31 pm
Got cut off there. NetApp and other array manufacturers do advocate double-parity RAID for many (most?) use cases, as the caches absorb the performance hit and make the extra safety worthwhile. The newer purpose-built all-flash array vendors use double-parity (and even inline dedupe!) almost exclusively.
Parity RAID is poised for a serious comeback as solid-state storage takes over. We all use "parity RAID" for our server RAM already; ECC and parity RAID are the essentially the same thing.
August 5th, 2012 at 8:47 am
@Ryan No one has overlooked RAID 10 failure modes. It has been discussed both in the comments here and in the links provided in the comments and in the description. URE has no impact on mirror rebuilds. The risk is the parity failing on rebuild. The URE risk to array rebuild is unique to parity RAID (RAID 5/6.) Mirrored RAID will remirror happily with or without encountering a URE.
And even if URE was a risk with RAID 10, which it is not, you never have a RAID set as small as a capacity equivalent RAID 5. The closest you would ever get is 67%. And that would be rare as RAID 5 is never recommended with so few spindles because of the performance hit.
August 5th, 2012 at 8:59 am
Talking about double disk failure is really a trivial thing. Yes, RAID 6 sacrifices array reliability and performance in order to make the marketing note that it can "always" survive multiple disk failures. Anyone talking about this and not pointing out how pointless this is is just providing misdirection.
There are three ways that stating that is misleading:
1) While RAID 6 always has double parity for every bit, it suffers from the URE risk just like RAID 5. So if using high URE SATA drives, for example, the chances that RAID 6 will be unable to survive even a single disk failure starts to become a very real possibility. It is certainly safer than RAID 5, but safer than unsafe isn't necessarily safe. Remember, RAID 1 and RAID 10 don't have the parity URE risk so this is of incredible significance.
2) RAID 6 takes a long time to resilver as it must resilver the entire array and, if encountering just one URE, it has to do it twice. So, like RAID 5, it carries the "data not lost but availability or performance loss" risk over a potentially long period of time. The longer the resilver the greater the risk of a second drive (or third) failing. A RAID 6 resilver could potentially take days or weeks - a very long time for a system to be working as hard as it can while being completely fragile and at risk. RAID 1/10 don't do this and only make a direct copy of a single drive in the array so no matter how large the array gets the rebuild time is the time to make a single drive copy. So the risk of another drive failing is very small in comparison.
3) While RAID 6 always can lose two drives (assuming nothing else goes wrong), RAID 10 can survive up to half of its drives being lost. In a four disk array RAID 6 sounds better. In an eight or sixteen drive array it is pretty obvious to see that RAID 10 is way safer even from this unlikely scenario. Both because it is less likely to lose drives in the first place (it burdens them less than RAID 6), because it doesn't have extended periods of fragility and because it is more likely than RAID 6 is a moderately sized array to be able to lose more drives.
Talking about multiple drive loss out of context is what you do when you want to hide why parity RAID is dangerous. You get people to focus on the mechanism of redundancy rather than the actual reliability of the array. Did you read the links provided? I'm just stating what has already been published.
Think of having two straw houses or one brick house and wanting to survive a windstorm? Which is more reliable? We all know the answer. Parity RAID is the straw houses of the storage world. Redundancy without reliability. Just misdirection.
August 5th, 2012 at 9:05 am
I don't foresee parity making a comeback with SSDs for one reasons - latency. Even when working perfectly parity RAID (5/6) increases latency and this is the place where SSDs really shine. So the parity overhead in latency terms would be really significant compared to what it is on spinners. Unless someone overcomes this, which I see as unlikely at an affordable price, I don't see this catching on at any significant level.
RAID 5 is all about being cheap. It was never about reliability or performance, even back in the day when it made sense with tiny arrays and expensive drives, it was always about cutting corners to save some dough. It will be quite some time before SSDs are likewise being purchased to be cheap. Until they are, people won't likely buy expensive SSDs but then cut corners on the RAID to go with them.
Not, at least, until SSDs are chosen for capacity rather than for performance.
August 6th, 2012 at 10:18 am
@Scott Alan Miller The only double-drive failure I've enountered in 17 years was two mirrored members of a RAID-10 array. On a circa-2001 Dell server, one drive failed, followed by another a few minutes later. This was very likely a firmware bug, as the drives came up clean when sent for RMA, but it still blew the whole array and we had to restore from backups.
I cannot see how RAID-10 could possibly be immune to URE during rebuild: the data has to come from somewhere. If you just report the block as bad but don't fail the whole array, that's fine. But there is no reason the same cannot be done with parity RAID schemes. This is, in fact, what NetApp and ZFS do with RAID-DP and RAID-Z2 respectively.
I must again point to NetApp and other HDD array manufacturers that use dual-parity RAID as default. NetApp in particular has defaulted to RAID-DP for years. I have not heard NetApp customers screaming about the unreliability of their arrays. And NetApp customers are not typically "penny pinchers".
As for the latency of dual-parity schemes with SSDs, that is a non-isuse on modern hardware. The parity calculation itself takes nanoseconds, and the additional seeks on SSDs take microseconds. As with mechanical disk arrays, the raw write is often stored in NVRAM and the physical writes are done asynchronously. Again, I point to the newer all-flash arrays such as PureStorage, Nimbus, etc. which use parity for reliability rather than mirroring. They don't seem to have any latency issues.
August 6th, 2012 at 10:24 am
Mirrored RAID (1/10) is immune to URE (by this the RAID is immune, not the filesystem underneath) because it does not do parity. It is parity that is affected by URE, not RAID in general. Because it is a mirror it is able to copy the data, as is, from the mirrored drive, URE or not. Parity RAID doesn't have this capability - it must reconstruct the data.
Think of it from a different perspective. Imagine you have a folder with 100 files in it (no RAID, just think about a folder on a file system.) Now if you have a URE hit you, it is one single block so only one of those 100 files will be impacted. 99 files won't know that anything bad happened.
No imagine if you had zipped all of those files together into a single zip archive. Now if there was a URE the single zip file would be corrupt and all 100 files inside of it are gone because it cannot be uncompressed to pull them back out.
Parity RAID is like this. Your data doesn't exist on its own - it has to be restored computationally after a drive is lost. Mirrored RAID has no process like this - it never "computes" data based on other data.
August 6th, 2012 at 10:27 am
RAIDZ2, for example, does not do what you said it does by default. If it encounters dual failures, it fails. That is a RAID 6 system, so it has protection against the first level failure. But it has no magic to work around the limitations of parity RAID.
Parity is dangerous because the data is in-flight during the encountering of the URE. A parity RAID system is not stable during its resilver process. Mirrored RAID is.
There are many articles on this. Mirrored RAID's safety comes from its lack of a destructive process. At no time does mirrored RAID put its own data in danger while performing an algorithm. Parity RAID must do this to resilver.
August 6th, 2012 at 10:36 am
NetApp uses enterprise drives and RAID 6 (well, a combination of RAID 4 and 6 that they call DP) so they mostly avoid the issue by spending their way out of it. However, as has been discussed ad nauseum in the SpiceWorks community and elsewhere, no matter how many clients complain about a SAN or NAS vendor's systems, how exactly would you expect to hear about that? Big businesses do not publish failure rates nor do the vendors. Using a lack of information to determine that no problem exists is misguided.
NetApp makes good stuff but I for one have had them fail under load when a Linux box easily handled the same load. But that isn't published anywhere and NetApp just ignored the issue. It's a different issue, but you probably haven't heard about it. And that's my point. No lab has hundreds of all these things from different vendors just to see which are reliable and which are not.
Same goes for any RAID array. There is no shop on the planet collecting the stats that you need. Any shop with those resources already knows not to run RAID 5 and isn't going to run gobs of them just to prove to you that they will lose money.
August 6th, 2012 at 10:38 am
Here are some numbers on the RAID levels and their write penalties in IOPS.
http://www.yellow-bricks.com/2009/12/23/iops/
Yes, you can cache your way out of many problems today. But only so many, it depends on your needs, and, again, shops willing and able to spend so much on other things are hardly going to be the same shops trying to cut corners and run RAID 5.
The issue with RAID 5 is a cultural one, primarily. Shops cut corners and run risky, consumer SATA then they cut corners again and run risky RAID 5. Put the two together and everything blows up.
August 6th, 2012 at 10:48 am
Here is a more modern recap of the article above. I think that it does a good job of talking about how RAID-DP works due to the combination of not actually being RAID 6 exactly (it is dual parity like RAID 6 but not dual parity RAID 5 like 6 is but more like dual parity RAID 4... which has no number, but probably should. But RAID 7 is generally accepted to likely to apply to things with triple parity RAID 5 such as RAIDZ3.)
http://theithollow.com/2012/03/21/understanding-raid-penalty/
It is RAID-DP's software nature combined with the filesystem all in one that allows the WAFL + RAID-DP system overcome unnecessary write penalties.
August 6th, 2012 at 11:25 am
@Scott Alan Miller: RAID-Z2 absolutely protects against the failure I mentioned by default. Any combination of URE or disk failures on two devices does not result in data loss. Same with NetApp WAFL.
As for your secnario where a RAID-10 re-mirrors bad data when a URE is encountered during rebuild, I suspect you're wrong. I'll bet that most controllers in the field fail a whole disk when a URE is encountered, no matter the RAID level in use.
Even if they didn't fail the whole disk, you are in the same situation as with parity RAID, asumming the array returns "bad block" instead of failing a whole disk and taking the array offline in both cases.
Think about it this way: if I get a URE on a parity array, and don't have enough parity to reconstruct, I can always assume "URE block has all zeros" and continue, but still return an error up the stack for that block. But I do NOT need to fail the whole disk, or a whole array. This results in the same situation as you suggest with "mirroring a URE" in a RAID-10 rebuild: one bad block reported to the OS, which may or may not be critical depending on the filesystem and data itself.
There is nothing at all "destructive" about parity RAID schemes, and in fact the "real data" is stored in the clear on N stripes, only the parity data is calculated.
August 6th, 2012 at 11:52 am
Nothing destructive? Have you never seen a parity array kill itself out of confusion? In the real world parity arrays do destructive things all the time.
August 6th, 2012 at 11:56 am
Your belief that RAID 10 will fail is based on the belief that the array needs to return "bad block" but it does not. A mirroring operation doesn't need to do anything at the entire array level like parity RAID does. Someone could sabotage the array but they don't need to do so. That would be crazy.
It is a common myth that RAIDZ (from ZFS) somehow avoids all parity issues when all it actually does is overcome the write hole problem. I've never seen it seriously suggested that they've discovered a workaround to the URE issue with parity. Please provide documentation if possible.
August 7th, 2012 at 12:43 am
@Trever - I've had an SSD fail 11 days after purchase. Thankfully it was an early gen Sandforce drive - I now use an Intel 320 SSD and it's been trouble-free.
Thanks for an informative article.
August 7th, 2012 at 10:23 am
@Scott Alan Miller: in RAID 10, if one disk fails, and a URE is encountered from its mirror during a rebuild, the controller has NO CHOICE but to return a bad block up the stack the next time the URE sector is read, or fail the drive with the URE and therefore fail the entire array. The data simply doesn't exist anymore in a usable form, and known-corrupt data cannot be returned to the application.
As for how RAIDZ(n) handles UREs during recovery, I am claiming that it does not fail a whole disk and therefore the whole array. It does not magically recover data if there is not enough parity to recover it. Documentation link.
August 13th, 2012 at 4:19 am
[...] Some rethinks about today’s RAID 5 http://www.standalone-sysadmin.com/blog/2012/08/i-come-not-to-praise-raid-5/ [...]
August 17th, 2012 at 1:08 am
So, what I understand is, I can either have a 3TB drive with 1/1E14 URE, or a 1/20 sized 5x priced 1/1E16 URE one?
Meaning, a 100 times more reliable, but a 100 times more expensive-per-GB drive.
Sweet.
So you could go with the cheap ones, and RAID 1 the hell out of them up to whatever reliability, overall capacity reduction and price point you wish, all the way up to a 100 drive RAID 1, where they would meet the price, reliablity and capacity point of 20 of the Enterprise-grade ones (that would be 5 RAID5 4-drive sets).
Of course you might consider a 100-drive RAID 1 somewhat of an overkill, and just stick to something like 10-drive RAID 1. That would be a 1/1E15 URE, with half the cost of a RAID 5 of Enterprise-grade disks. Seems like a good compromise.
On the other hand... URE refers to 1 bit errors. What is the probability of THE SAME bit presenting errors on more than one drive?
I dare say much MUCH lower than 1/1E16. So just a 3 drive RAID 1 (with error notification), or a 4 drive RAID 4 (without error notification, taking a vote for the most popular bit), should be more than enough to boost reliability sky high, while keeping costs barely close to a single Enterprise grade drive.
August 17th, 2012 at 10:06 am
Jarfil, you are absolutely spot on. The problem with RAID 5 is that to make it safe enough to use you have to make it so expensive that you could have had a faster, more reliable... AND CHEAPER... RAID 10 array instead. People are implementing RAID 5 to prove a point that it "can" be done forgetting that the reason not to is that it "shouldn't" be done.
I wrote an article specifically addressing this backwards thinking (feeling that RAID 5 is "good enough" when there is no winning factor)...
http://www.smbitjournal.com/2012/08/nearly-as-good-is-not-better/
August 22nd, 2012 at 5:25 pm
Does the media scanning/consistency checking that many raid controllers perform reduce the risk of experiencing a URE during a rebuild?
August 22nd, 2012 at 5:27 pm
itpro4: From what I understand, controllers that constantly scrub have a lower chance of corrupted data, but I don't have any numbers to back that up.
August 22nd, 2012 at 5:28 pm
@itpro4 Yes, it absolutely reduces it. However, all of the calculations are done ASSUMING that that is already included. So think of it that if you don't have that feature that the URE risks are higher than stated. We assume best case scenario when doing these to make sure there is no wiggle room for stating the dangers.
We also assume variable stripes and no write hole, which are advanced options and not widely available. So if you aren't running ZFS, for example, your chances of hitting problems just get worse and worse.
August 22nd, 2012 at 5:33 pm
Maybe several other questions to ask is - Is a URE caused by a damaged/worn sector on the disk? Is there other factors that would cause a URE?
August 22nd, 2012 at 5:35 pm
I'm sure that that encourages it but a URE can happen in any sector at any time. I assume some sectors are more likely to be hit than others. But overall, it is just a generic failure rate.
August 22nd, 2012 at 5:43 pm
you guys are quick to respond, thanks. Scott, you said that bad media can encourage it. It seems that you believe there is other factors at play that can cause a URE? I'm just curious as to what the root cause of these URE's actually is?
August 22nd, 2012 at 6:11 pm
URE is simply causes by the disk failing in a sector. Presumably increases wear and tear increases failure rate. Not really sure what you are asking. We are talking about mechanical failure here, so "use" normally contributes to failure.
August 22nd, 2012 at 6:36 pm
That's exactly what I'm wondering, thanks. So the error rate that manufactures specify seems to be their calculated rate at which the disk wears (in bits) based off the amount of data read.
I did some more researching and found this interesting article of some empirical testing or Drive Error rates:
http://arxiv.org/ftp/cs/papers/0701/0701166.pdf
August 22nd, 2012 at 6:43 pm
The fail rate is measured in reads, but not caused by reads.
The article is interesting but is dealing with UREs that make it through for corruption, remember that we only care about the UREs actually on the platter here because we are talking about a RAID array rebuild. They talk about OS masking... which has no effect at the platter level. So their info, while out of date, is useful and interesting, it does not shed very much light here.
August 22nd, 2012 at 7:24 pm
I agree, it's old. I'm just a big fan of empirical testing vs the theoretical :).
To skip back a few posts. May I ask how you know that the the media scrubbing is assumed when error ratings are calculated? It occurred to me that desktop drives almost never have any kind of media scrubbing occurring, which would either mean they don't assume this for desktop drive ratings and they do for enterprise drives. However, It seems to make sense the test would be standardized across all drives, which would leave them not assuming media scrubbing for enterprise drives.
August 22nd, 2012 at 7:54 pm
@itpro4 URE fail rates are calculated from reads. So our fail rates for resilvers are calculated from that rate. If you didn't, the resilver fail rate would be far higher because bad UREs would be lingering. We are only considering the UREs that occur DURING the rebuild... that is, only the new ones.
The "tests" aren't what we are talking about but the actual URE fail rates. If you do tests you will find that RAID 5, in the real world, is even more likely to fail because few systems can eliminate all UREs from before the resilver process. So actual fails could be quite bit higher.
Even systems that scrub (like XRAID or ZFS) can't do so perfectly. So the URE rate is always conservative. Rail rates will always be higher than the estimate here.
August 22nd, 2012 at 7:57 pm
This is all great information, thanks for taking the time to respond to all these questions, Scott. I think the last one I have is, if a URE occurs while all drives in the array are functioning, will the entire array fail?
August 22nd, 2012 at 8:04 pm
If a URE hits a single drive (no RAID) what you get is bit rot... a single bit that is bad and this will corrupt a single file (if that bit occurs where there is file data... it does nothing if it happens in unused space.)
In a RAID array (not RAID 0, obviously) the array has redundant data for any single bit and so can repair the URE transparently. Technically RAID doesn't "have" to do that by definition, but no respectable RAID doesn't do that. So UREs pose no threat to a functioning array.
August 25th, 2012 at 3:15 pm
This is what I do not understand. I have been working with RAID 5 arrays in mainframe environments since they came out in the mid 90's and have worked on and supported the mainframe storage for probably over 100 fortune 500 companies. Back in the 80's and early 90's when we did not have raid, storage was a nightmare. I never knew if I would get to sleep through the night because disk drives were always failing. Since the mid 90's, I have never lost a single nights sleep due to a hard drive failure. I find out the next day, the ce was alerted by the disk subsystem to replace a drive and everything keeps running along just fine. We may see a performance hit for a few hours, but I have yet to loose all the data in an array due to a single drive failure. I have heard of it happening, but it has never happened to me. Yes the arrays are normally never larger than 4 drives in a single array and the subsystems are continually scrubbing the disks in the back ground and we have dynamic spares that are automatically copied to in the event of a drive showing signs of problems. But it seems to be trouble free and just taken for granted we will never have the system go down due to a disk drive failure. Why such a drastic difference?
August 25th, 2012 at 3:22 pm
For all the reasons stated in all the articles...
1) Because historic drive sizes were small and the issues with RAID 5 did not significantly manifest themselves until the modern era of large drives.
2) RAID 5 today is about cutting corners on cost (it always was but much more dramatically today) so lower end SATA drives are finding their way into large, business arrays with reckless abandon.
3) Even people using no RAID at all have stories of people who've "gotten lucky". There will always be those people.
So the issue is that times have changed. Any experience of arrays prior to the last few years is irrelevant because those arrays were small and likely SCSI. This is a modern problem not a historic one.
Here is a video talking about the history of RAID arrays in practical use from just three days ago:
http://www.youtube.com/watch?v=VxlRkPB3MZk&list=UUu4Nr6fVsCj-sD5zuOG14zw&index=1&feature=plcp
August 25th, 2012 at 3:26 pm
If you ask someone from the desktop world, on average, you will talk to people who've never had their desktop drive die. They would tell you that RAID is a waste of money. Likewise, people who've not experienced RAID 5 dying tell you the same thing.
In all cases, it is about learning the risk factors, the chances of those things and the cost of mitigation (plus other factors like performance.) Just because you are at unnecessary risk doesn't state that you WILL fail, that's not what risk means. It means you've been more likely to fail, but a massive margin, than if you were using RAID 10 or RAID 6 even.
If you ask your mother if driving down the road at 100mph without a seatbelt is risky, she will slap you upside the head because she understands risk. And if you drive down the road like a maniac without a seatbelt on just to show that you "can" survive, you've not demonstrated that it is safe.
Same with Russian roulette. 5 out of 6 players believe that Russian roulette is completely safe.
August 25th, 2012 at 4:30 pm
I don't think I have just been lucky (ok, maybe a little). It sounds to me more like the manufacturers in x86 world are just building crap and throwing it out there. Thanks for the video link. You make many good points.
August 25th, 2012 at 4:35 pm
RAID 5 issues are pretty universal. CPU architecture doesn't really impact storage, but rather than the manufactures, I think that you'll find that companies that spend big bucks on Itanium, Power and Sparc don't cut corners on storage. But companies buying cheap x86 white boxes are going to skimp on storage too. It's not on the manufacturers end (normally) but on the consumer end where the decisions are often made.
The one big hold out, Dell, has finally changed their tune and RAID 5 has been eliminated there too.
August 25th, 2012 at 9:52 pm
While you are right about cheap drives leading to trouble. I suspect a really big issue is that these failure prone raid 5 controllers are not doing the background data scrubbing that an enterprise class subsystem does. So if your controller never checks the drives and just keeps chugging along until a complete failure of one of the drives occurs, then I bet the odds are pretty high one (or more) of the remaining drives needed to rebuild the data are marginal and actually should have been replaced long ago but were never identified. Which means you now have a double failure leading to complete loss.
August 25th, 2012 at 10:02 pm
No, everyone jumps to that conclusion but background scrubbing is ASSUMED in the calculations. Any lack of scrubbing makes the situation even worse.
There is no concept of a "bad RAID 5 controller." The issue is RAID 5 itself. There is no getting around the fact that there is a fundamental parity fragility in the RAID 5 system conceptually.
All of the dangers around RAID 5 assume 100% scrubbing with no existing UREs at the time of failure, no write hole problem, no controller failure, etc. Check the math, we assume 100% pristine scenario before starting. If we had existing UREs the failure rates shoot through the roof.
August 27th, 2012 at 12:49 pm
I was reading the article that was posted above titled, "Why RAID-5 Stops Working in 2009" ( http://www.zdnet.com/blog/storage/why-raid-5-stops-working-in-2009/162 ). In that article the following comment is made:
"So now what? The obvious answer, and the one that storage marketers have begun trumpeting, is RAID 6, which protects your data against 2 failures. Which is all well and good, until you consider this: as drives increase in size, any drive failure will always be accompanied by a read error. So RAID 6 will give you no more protection than RAID 5 does now, but you'll pay more anyway for extra disk capacity and slower write performance."
Isn't the whole idea that RAID6 will indeed protect you from a URE that occurs during a rebuild? If there is still parity during the rebuild, which is true in RAID6, that protects against a URE. It's only after the 2nd drive failure that you would not be protected against a URE.
August 27th, 2012 at 2:47 pm
http://www.zdnet.com/blog/storage/why-raid-6-stops-working-in-2019/805
RAID 6 significantly hedges against URE resilver errors in RAID 5. It does not, however, protect against any of the other issues with RAID 5 (lack of availability during rebuilds, horrifically long rebuild times, overall performance issues, etc.)
RAID 6 remains a viable solution for nearly all array sizes today but its days are numbered. But even so, with reliability far below that of RAID 10, higher costs than RAID 5 (one more drive needed), lower performance and potential lack of availability issues, RAID 6 is really ideal only for extreme cost cutting scenarios and specialty storage cases like archiving. It's not a "never" technology like RAID 5, but it is not the fix for the problem.
The right answer, for most systems today, is RAID 10. RAID 10 works today and it scales for tomorrow.
This is all in the video linked above.
February 8th, 2013 at 3:42 pm
"In theory, someday, super low URE drives will exist at a very low price point."
Unlikely, only in my dreams...the industry will not permit it as the profit are to high on enterprise drives; theoretically possible but the industry has no incentive to do so.
What is needed more robust/recoverable error checking built into the raid controllers, as in a separate coprocessor dedicated to error checking, and firmware allowing recovery from a normal scenario which would fail an array.
As is a consistency check verifies the area where data resides in, but not the remainder of the disk..this is old technology, basically useless in today's large arrays. The newer raid adapters have scrubbing algorithms which check the entire array surface, as in patrol reads, on a regular basis, a great improvement but error recovery still needs to more robust. I find it ridiculous an array possibly containing multiple millions of files can be failed due to a relative few media errors, as a result of firmware/hardware design thinking based on technology of the late 80's.
March 9th, 2013 at 4:46 pm
The specs I'm looking at for consumer drives all say < 1 bit in 1E+14. That's saying the drive shouldn't have a 1 bit loss in 12.5TB (11.4TiB). It's not saying what you will encounter. And no ceiling is placed on the specification.
Also, what exactly is a URE? If it when reading a bad sector that can't be corrected by disk firmware ECC, resulting in an ATA ERR UNC message, there's no such thing as 1 bit of loss. That's 4096 bits of loss for conventional disks with 512 byte sectors, and 32768 bits of loss for AF disks. In which case the URE spec interpretation can't conflate bits with sectors or you end up with completely incorrect math.
One 512 byte sector of data loss every 12.5TB is an error rate of 1 bit in 2.4^10 bits. That's a nearly 5 order magnitude difference from what's published.
If a URE includes SDC, then it makes the incidence of SDC astronomical compared to lost sectors.
March 9th, 2013 at 5:11 pm
Chris: Those stats are in bits read, so the URE will happen roughly once in every 12.5TBish (the use of "less than" in those stats is too abstract for me to like. I just assume it's "roughly equal to" for the purposes of fault tolerance).
A URE is when the drive goes to read a sector and fails a certain number of times, eventually giving up. The number of times it retries the read is determined by the firmware. Consumer drives will retry a great deal more times than an enterprise drive - that's because an error like this is blocking. Enterprise drives know that they're likely part of a RAID set and will fail more quickly, relying on the redundancy in RAID to provide the data. A consumer drive will continue to retry for a long time, into "minutes", trying to reread the magnetic track.
Also, you can't be certain that there's an entire sector of data missing. In a SAS drive, the sectors are indeed 512 bytes, with 8 bytes of CRC data, so that the actual sector on disk is 520 bytes. A URE could occur if the data on disk doesn't match the stored CRC and the data can't be recovered mathematically, or maybe if the drive was vibrating for long enough that the head couldn't be aligned to the track for a time longer than the timeout (in an earthquake or construction scenario).
The end result is that the way I've interpreted the URE is basically, the disk can't get the data. If the data can be read from elsewhere in the RAID group, then it will be. If it can't, then you have a problem. If you're in a degraded RAID situation where there's no more redundancy, then you will have a problem, so be in that state as infrequently as possible.
March 9th, 2013 at 6:37 pm
1. The < sign is not abstract, by definition it mean not equal, so you don't get to just redefine it as roughly equal. So I refuse your assumption for the purposes of anything, including fault tolerance.
2. As you describe a URE, that is a UNC read error. When that kind of error occurs, zero bytes are transmitted from the drive to the controller. That is a fact, I can be totally certain of it. If data is transmitted, that means the firmware ECC did not detect any error, or it detected error and thought it corrected it: but in either case error is transmitted. That's typically under the category of SDC because the data has been corrupted, corruption not detected, transmitted, without an error message.
Let's not even bring SAS 520 byte disks using PI, that's a totally different scenario. And let's not even bring RAID into the picture. To understand anything, the very basics must be agreed upon.
I think it's more likely that the specs are not a per disk spec, but rather a significantly larger population of disks. Clearly drives die and all data on all sectors is functionally unrecoverable. I don't know what the population is, but if you take 100,000 drives of the same model and give it a (reasonable) 4% failure rate, and throw in some bad sectors for drives that don't actually die, you get a URE that's better than 1E+14 but not as good as 1E+15.
It's easily proven totally wrong to assume that a 3TB drive read 4 times *will* encounter a bit of error. I've done this test on several 3TB drives, reading them 12x without a single disk reported error or a failed checksum. So I think the spec is a statistic for a larger population, and is a means of comparing classes of drives, not to have an idea of when a particular drive is going to give up null or bad data.
March 15th, 2013 at 11:58 am
Chris: Agreed, these are statistics for failures, not guarantees of failures. But you have to assume that they're accurate over a given number of drives, at least to the manufacturer's knowledge or ability to determine.
And the < sign means less than. If the odds were 1E+15 (113TB), they would have said that. But they didn't, they said < 1E+14 (aka more than 11TB).
There's obviously an order of magnitude of difference between those numbers. Where does it fall in there? You don't know, and neither do I. So when I'm planning for failures, I take the most conservative numbers available. It's less than 1E+14, but maybe it averages out to 1E+14-1k? Who knows. So Person A assumes 1E+14, and Person B assumes 1E+15, both will have failures at around the same rate, but Peron B will get a nasty surprise when it probably happens sooner than they expected.
April 30th, 2013 at 2:19 am
And this is why I use ZFS, though some of the above evangelical's would make a southern preacher blush.
RAID 5 is perfectly fine when kept in small disk sets, typically no larger then five. Also the failure rates are no where near what's being touted, people are off by two orders of magnitude or more. The BER published by manufacturers are worst case scenario with the disk working a harsh desert or freezing environment not a relatively clean data center. That number is also calculated prior to the QA process at the manufacturer removing failed
Now for what is really going to shock everyone and prevent you from sleeping. No amount or combination of RAID protects you from data loss, period end of story. This is because the primary reason for failure is not wear & tear but faulty manufacturing. Drives are either good or bad with the manufacturer trying to catch the bad ones before shipping. Bad drives then to happen in batch's, you don't get one bad drive but dozens if not hundreds out of a batch. When we order storage we tend to order large drives from a single manufacturer which then sends us disks from what's on hand, typically all with the same manufacture date, often from the same batch. Thus if one drive in a RAID is going to fail then there is a high chance that several others from the same batch will also fail. That RAID10 has no greater reliability advantage then the RAID5 but at a significantly higher cost.
Performance is based on the number of reading disks and the quality of the FS involved. A four disk RAID10 set will have the same read performance as a five disk RAID5 set while having half the write performance and half the capacity. Also if we're talking a SAN environment then the storage processor does it's own form of error checking and integrity validation.
In all cases the true source of corruption are file systems that are not self-repairing. It's just plain crazy to store data on a file system that doesn't do it's own integrity checking. I recommend using a ZFS or similar file system for any serious data storage. ZFS itself maintains layers upon layers of CRC's and is constantly doing integrity validation and using a copy on write methodology.
Finally no amount of RAID or storage subsystem is a replacement of data replication / backup.