ECS: Appliance: ECS Guidance on 500 error rate response in ECS

When performing normal Create, Read, Update, Delete (CRUD) activities against the ECS object storage system there are occasions when 500 errors may be experienced.

The official definition of a 500 error is:

The server encountered an unexpected condition which prevented it from fulfilling the request.

Dell advises a 500 failure rate of 0.1% falls within acceptable parameters. Any value greater than this requires investigation and an SR can be opened with our support team to further investigate.

Retry mechanism from customer application
When an application encounters this specific error, it should issue a retry of the request. In terms of guidance of performing a retry, we recommend using a backoff algorithm for better flow control. The idea behind the backoff is to use progressively longer waits between retries for consecutive error responses. For example, let 2 seconds elapse before the first retry. Then 3 seconds before the second retry and 4 seconds before the third retry, and so on. Another example would be to look at an exponential backoff of one second elapse for first retry, four seconds for second, 16 seconds for third retry and so on. Ultimately it is the application development team should decide on retry mechanism rate. However, if the request has not succeeded after a minute, the problem might be a hard limit and not the request rate. For example, may have reached the maximum number of pipelines allowed. Set the maximum number of retries to stop around one minute.