Retry behavior

Retry behavior includes settings regarding how the SDKs attempt to recover from failures resulting from requests made to AWS services.

Choosing between standard and adaptive retry modes

We recommend you use the standard retry mode unless you are certain that your usage is better suited for adaptive.

If you choose to use adaptive mode, your application must construct clients that are designed around each resource that might be throttled. A resource, in this case, is finer-tuned than just thinking of each AWS service. AWS services can have additional dimensions that they use to throttle requests. Let's use the Amazon DynamoDB service as an example. DynamoDB uses AWS Region plus the table being accessed to throttle requests. This means that one table that your code is accessing might be throttled more than others. If your code used the same client to access all the tables, and requests to one of those tables is throttled, then adaptive retry mode will reduce the request rate for all tables. Your code should be designed to have one client per Region-and-table pair. If you experience unexpected latency when using adaptive mode, see the specific AWS documentation guide for the service you are using.

Retry mode implementation details

The AWS SDKs make use of token buckets to decide whether a request should be retried and (in the case of the adaptive retry mode) how quickly requests should be sent. Two token buckets are used by the SDK: a retry token bucket and a request rate token bucket.

Following is the high-level pseudocode for both the standard and adaptive retry modes:

MakeSDKRequest() {
  attempts = 0
  loop {
    GetSendToken()
    response = SendHTTPRequest()
    RequestBookkeeping(response)
    if not Retryable(response)
      return response
    attempts += 1
    if attempts >= MAX_ATTEMPTS:
      return response
    if not HasRetryQuota(response)
      return response
    delay = ExponentialBackoff(attempts)
    sleep(delay)
  }
}

Following are more details about the components used in the pseudocode:

GetSendToken:

This step is only used in adaptive retry mode. This step acquires a token from the request rate token bucket. If a token is not available, it will wait for one to become available. Your SDK might have configuration options available to fail the request instead of wait. Tokens in the bucket are refilled at a rate that is determined dynamically, based on the number of throttling responses received by the client.

SendHTTPRequest:

This step sends the request to AWS. Most AWS SDKs use an HTTP library that uses connection pools to reuse an existing connection when making an HTTP request. Generally, connections are reused if a request failed due to throttling errors but not if a request fails due to a transient error.

RequestBookkeeping:

Tokens are added to the token bucket if the request is successful. For adaptive retry mode only, the fill rate of the request rate token bucket is updated based on the type of response received.

Retryable:

This step determines whether a response can be retried based on the following:

Transient errors (HTTP status codes 400, 408, 500, 502, 503, and 504) and throttling errors (HTTP status codes 400, 403, 429, 502, 503, and 509) can all potentially be retried. SDK retry behavior is determined in combination with error codes or other data from the service.

MAX_ATTEMPTS:

The default number of maximum attempts is set by the retry_mode setting, unless overridden by the max_attempts setting.

HasRetryQuota

This step acquires a token from the retry token bucket. If the retry token bucket is empty, the request will not be retried.

ExponentialBackoff

For an error that can be retried, the retry delay is calculated using truncated exponential backoff. The SDKs use truncated binary exponential backoff with jitter. The following algorithm shows how the amount of time to sleep, in seconds, is defined for a response for request i:

seconds_to_sleep_i = min(b*r^i, MAX_BACKOFF)

In the preceding algorithm, the following values apply:

b = random number within the range of: 0 <= b <= 1

r = 2

MAX_BACKOFF = 20 seconds for most SDKs. See your specific SDK guide or source code for confirmation.

Compatibility with AWS SDKs

The following SDKs support the features and settings described in this topic. Any partial exceptions are noted. Any JVM system property settings are supported by the AWS SDK for Java and the AWS SDK for Kotlin only.