The rebalance throttling policy determines the priority of rebalance I/Os versus application I/Os when accessing SDS devices.
Rebalance throttling sets the rebalance priority policy for a storage pool. The policy determines the priority between the rebalance I/O and the application I/O when accessing SDS devices. Please note that application I/Os are continuously served. Rebalance, unlike rebuild, does not impact the reliability of the system, and therefore reducing its impact is not risky.
NOTE:Rebalance throttling affects the performance of the system, and should be used only by advanced users.
The following possible priority policies may be applied:
No Limit: No limit on rebalance I/Os. Any rebalance I/O is submitted to the device immediately, without further queuing. Please note that rebalance I/Os are relatively large and hence setting this policy will speed up the rebalance, but will have the maximal effect on the application I/O.
Limit Concurrent I/O: Limit the number of concurrent rebalance I/Os per SDS device. The rebalance I/Os are limited to a predefined number of concurrent I/Os. Once the limit is reached, the next incoming rebalance I/O waits until the completion of a currently executed rebalance I/O. For example, setting the value to "1" will guarantee that the device will only have one rebalance I/O at any given moment, which will ensure that the application I/Os only wait for 1 rebalance I/O in the worst case.
Favor Application I/O: Limit rebalance in both bandwidth and concurrent I/Os. The rebalance I/Os are limited both in bandwidth and in the amount of concurrent I/Os. As long as the number of concurrent rebalance I/Os, and the bandwidth they consume, do not exceed the predefined limits, rebalance I/Os will be served. Once either limiter is reached, the rebalance I/Os wait until such time that the limits are not met again. This imposes a bandwidth limit on top of the Limit Concurrent I/Os option.
Dynamic Bandwidth Throttling: This policy is similar to Favor Application I/O, but extends the interval in which application I/Os are considered to be flowing by defining a minimal quiet period. This quiet period is defined as a certain interval in which no application I/Os occurred. Note that the limits on the rebalance bandwidth and concurrent I/Os are still imposed.
Default Values:
The default policy for rebalance: Favor Application I/O
Rebalance concurrent I/O Limit: 1 concurrent I/O per SDS device