Dell VxRail Architecture Overview

Site to site

Where a total /IO profile requires 100,000 IOPS, where 70 percent are write, and 30 percent are read in a stretched configuration. The write I/O is what is sized against for intersite bandwidth requirements.

With stretched clusters, read traffic is serviced by default by the site that the VM resides on. This concept is called Read Locality. The required bandwidth between two data sites (B) is equal to Write bandwidth (Wb) * data multiplier (md) * resynchronization multiplier (mr):

B = Wb * md * mr

The data multiplier consists of overhead for VMware vSAN metadata traffic and miscellaneous related operations. VMware recommends a data multiplier of 1.4. The resynchronization multiplier is included to account for resynchronizing events. Allocate bandwidth capacity on top of required bandwidth capacity for resynchronization events. Provide an additional 25 percent for resynchronization traffic.

For example, a workload of 10,000 writes per second to a workload on VMware vSAN with a 4 KB size write requires 40 MB/s or 320 Mbps bandwidth.

B = 320 Mbps * 1.4 * 1.25 = 560 Mbps

Including the VMware vSAN network requirements, the required bandwidth would be 560 Mbps.

Witness to site

Witnesses maintain component metadata. Witnesses do not maintain VM data. Data is stored on VMware vSAN in the form of objects that consist of one or more components such as: VM home, swap, disks, and snapshots. Objects can be split into more than one component when the size is >255 GB and/or a Number of Stripes (stripe width) policy is applied.

The number of objects/components for a given VM is multiplied when a Number of Failures to Tolerate (FTT) policy is applied for data protection and availability.

The required bandwidth between the witness and each site is equal to ~1138 B x Number of Components / 5 s:

1138 B x NumComp / 5 s

The 1138 B value comes from operations that occur when the preferred site goes offline, and the secondary site takes ownership of all the components. When the primary site goes offline, the secondary site becomes the leader. The witness sends updates to the new leader, followed by the new leader replying to the witness as ownership is updated. The 1138 B requirement for each component comes from a combination of a payload from the witness to the backup agent, followed by metadata indicating that the preferred site has failed.

In the event of a preferred site failure, the link must be large enough to allow for the cluster ownership and ownership of all the components to change within five seconds.

For example, with a VM consisting of:

• 3 objects: VM namespace, .vmdk (under 255 GB), vmSwap
• Failure to Tolerate of 1 (FTT=1)
• Stripe Width of 1

Approximately 100 VMs with the above configuration require the witness to contain 600 components:

(100 VMs * 3 components/VM * 2 (FTT+1) * 1 (Stripe Width))

To satisfy witness bandwidth requirements for 600 components on VMware vSAN, use the following calculation to convert Bytes (B) to Bits (b):

B = 1138 B * 8 * 600 / 5 s = 1,092,480 Bits per second = 1.09 Mbps

VMware recommends adding a 10 percent safety margin and round up:

B + 10 percent = 1.09 Mbps + 109 Kb/s = 1.199 Mbps

With the 10 percent buffer included, for every 6,000 components, 1.2 Mbps is appropriate.