PowerEdge: How to fix Double Faults and Punctures in RAID Arrays

Table of content

1. Fixing double faults and RAID punctures
2. Data Errors and Double Faults
3. Punctures: What Are They and How Are They Caused?
4. Preventing Problems Before They Happen and Solving Punctures After They Occur
5. How To videos for creating/ deleting an array or importing/ exporting a foreign configuration

Fixing double faults and RAID punctures

1. Discard preserved Cache (if it exists)
2. Clear foreign configurations (if any)
3. Delete the array
4. Check for any failed drives
5. Reseat any failed drives
6. Clear any foreign configuration again
7. Replace all failed drives including predictive failed drives
8. Update the firmware (Controller, backplane (BP), drives) if needed
9. Create the array
10. Perform a Full Initialization (not a Fast Initialization)
11. At this stage, the array should be ready to be used

Data Errors and Double Faults

RAID arrays are not immune to data errors. RAID controller and hard drive firmware contain functionality to detect and correct many types of data errors before they are written to an array/drive.

• Data errors can be caused by physical bad blocks, such as a "Head Crash" or degradation of the platter's ability to magnetically store bits in a specific location.
• A bad block, also known as a bad Logical Block Address (LBA), can also be caused by logical data errors, such as a "bit flip" or incorrect data being written to a drive.
• Bad LBAs are commonly reported as the Sense Code 3/11/0.
• Dell hardware-based RAID controllers offer features such as Patrol Read and Check Consistency to correct many data error scenarios.

Perform regular Check Consistency operations will correct for single faults, whether a physical bad block or a logical error of the data.

Check Consistency will also mitigate the risk of a double fault condition in the event of additional errors.

Figure 1 Multiple Single Faults in a RAID 5 array - Optimal Array

Figure 2 Double Fault with a Failed Drive (Data in Stripes 1 and 2 is lost) - Degraded Array.

Figure 3 Punctured Stripes (Data in Stripes 1 and 2 is lost due to double fault condition)  - Optimal array.

Back to table to content

Punctures: What Are They and How Are They Caused?

A puncture is a feature of Dell's PERC controllers designed to allow the controller to restore the redundancy of the array despite the loss of data caused by a double fault condition.

• A puncture is also known as "rebuild with errors."
• A puncture can occur in one of two situations: a double fault already exists, or a double fault does not exist.
• A puncture can occur in three locations: a blank space, a non-critical data space, or a data space that is accessed.
• Any condition that causes data to be inaccessible in the same stripe on more than one drive is a double fault
• Double faults cause the loss of all data within the impacted stripe
• All punctures are double faults but all double faults are NOT punctures

Back to table to content

Preventing Problems Before They Happen and Solving Punctures After They Occur

Proactive maintenance can correct existing errors and prevent some errors from occurring.

• Update drivers and firmware on controllers, hard drives, backplanes, and other devices.
• Perform routine Check Consistency operations.
• Review logs for indications of problems.

• Perform a routine  Check Consistency (the array must be optimal)
• Determine if hardware problems exist
• Check the controller log
• Perform hardware diagnostics
• Contact Dell Technical Support as needed

Back to table to content

How To videos for creating/ deleting an array or importing/ exporting a foreign configuration

How to Create or Delete a Virtual Disk in iDRAC 9