Dell ObjectScale 1.3 Administration Guide

Data protection with ObjectScale Erasure Coding schemes

ObjectScale uses various Erasure Coding schemes for data protection. Erasure coding (EC) is a method of data protection in which data is broken into fragments, expanded and encoded with redundant data pieces. The data pieces are stored across different locations or storage media.

The goal of erasure coding is to enable data that becomes corrupted at some point in the disk storage process to be reconstructed by using information about the data that is stored elsewhere in ObjectScale. Erasure code schemes are often used instead of traditional RAID because of their ability to reduce the time, overhead required to reconstruct data, and greater data resiliency, depending on the EC scheme used.

During the object store creation process using ObjectScale Portal, the available EC schemes that are presented within the New Object Store wizard is based on the number of Kubernetes nodes, either physical servers or worker nodes, in the cluster. ObjectScale uses the Kubernetes anti-affinity rules to ensure that the storage server (SS) instances are properly placed across the nodes in the cluster. The New Object Store wizard ensures that the number of SS instances for the new object store is not below the minimum for the selected EC scheme.

ObjectScale implements Reed Solomon error correction using these schemes:

Upon a Kubernetes node permanent failure, the copies of lost data segments are recreated using remaining data and coding segments. During temporary Kubernetes node failure, data services continue with data and coding segments that are being used to recreate data when needed.

ObjectScale minimum disk requirements vary based on object store EC requirements. When an object store is created, the total raw capacity and EC scheme are specified. Administrators choose the topology based on input to provide optimal protection and SS size. The number and size of SS instances in an object store represent the persistent storage capacity allocated for raw user data. SS instances attach to Kubernetes persistent volumes (PVs) on disks using Kubernetes persistent volume claims (PVCs). ObjectScale writes data for best protection considering number of volumes on disk, disks per SS, and SS instances across the cluster.