Boto3 Retry Config: Why AWS Scripts Fail Silently

Q: What's the default boto3 retry mode?

`legacy`. It retries a narrower set of errors than standard mode and caps at 5 attempts by default. You have to opt into `standard` or `adaptive` explicitly.

Q: Does standard mode retry throttling errors automatically?

Yes — `ThrottlingException`, `TooManyRequestsException`, `SlowDown`, and several others are retried with backoff automatically. A manual `time.sleep()` retry loop around a boto3 call is usually redundant once you're on standard mode.

Q: How do I check if a boto3 call actually retried?

Read `response["ResponseMetadata"]["RetryAttempts"]` after a successful call, or enable DEBUG logging on `botocore.retries.standard` (standard/adaptive) or `botocore.retryhandler` (legacy). **Quick Summary:** - boto3's default retry mode is `legacy`, not `standard` — you have to opt in explicitly - `Config` objects use `mode` and `total_max_attempts`; the AWS config file uses `retry_mode` and `max_attempts` — mixing them up fails silently - Standard mode already retries throttling errors with backoff; a hand-rolled retry loop on top of it is usually redundant - Catch `botocore.exceptions.ClientError` and read `error.response["Error"]["Code"]` — most services don't have a dedicated exception class for every error - Check `response["ResponseMetadata"]["RetryAttempts"]` to confirm your retry config is actually doing something Check whichever script is running your scheduled AWS jobs right now — if it's still on the legacy default, fixing your boto3 retry config is four lines of `Config` with `mode: "standard"`, not a third-party wrapper.

boto3 retry mode comparison showing legacy, standard, and adaptive configuration

Boto3’s default retry mode is legacy, not standard — and legacy mode only retries a narrow list of errors: ConnectionError, Throttling, ThrottlingException, and ProvisionedThroughputExceededException. If your script fails on a transient RequestTimeout or SlowDown error that standard mode would have retried automatically, that’s why. You’re on the older, narrower default until you explicitly ask for something else.

boto3 Retry Modes: Legacy, Standard, and Adaptive

There are three modes, and they are not equally capable.

Legacy (the default) retries 5 attempts total against a short error list: connection errors and a handful of throttling exception names. It uses exponential backoff with a base factor of 2.

Standard retries 3 attempts by default, adds circuit-breaking for sustained service outages, and covers a longer error list — RequestTimeout, TooManyRequestsException, TransactionInProgressException, RequestLimitExceeded, BandwidthLimitExceeded, SlowDown, plus the HTTP 500/502/503/504 status codes for nondescriptive transient errors. Backoff is capped at 20 seconds.

Adaptive builds on standard mode and adds client-side rate limiting through a token bucket that adjusts your call rate based on how the service is actually responding. AWS marks this one experimental — expect its behavior to change between botocore versions.

Configuring boto3 Retry Behavior: The Syntax That Trips People Up

Here’s the part the docs bury: the field names are different depending on where you configure retries, and mixing them up fails silently instead of raising an error.

In code, with a Config object, use mode and total_max_attempts:

from botocore.config import Config

import boto3



config = Config(

    retries={

        "total_max_attempts": 10,

        "mode": "standard",

    }

)



client = boto3.client("ec2", config=config)

In ~/.aws/config, the same settings use different key names — retry_mode and max_attempts:

[myConfigProfile]

region = us-east-1

max_attempts = 10

retry_mode = standard

Copy a Config object’s keys into the AWS config file (or the reverse) and boto3 won’t error — it just ignores the keys it doesn’t recognize and falls back to defaults. total_max_attempts also counts the initial request, so total_max_attempts: 3 means one call plus two retries, not three retries on top of the original.

Environment variable, if you’d rather not touch either file:

export AWS_MAX_ATTEMPTS=10

export AWS_RETRY_MODE=standard

This environment variable approach is the same pattern covered in our guide to setting up multiple AWS profiles for the CLI — a boto3 retry config set per-profile in ~/.aws/config follows the same profile-scoping rules.

Catching ClientError the Right Way

Every service error in boto3 — throttling, permission denial, a bad parameter — surfaces as botocore.exceptions.ClientError, not a service-specific exception class in most cases. The error code and message live inside the response dictionary, not as separate exception attributes:

import botocore.exceptions



try:

    client.describe_stream(StreamName="myDataStream")

except botocore.exceptions.ClientError as error:

    error_code = error.response["Error"]["Code"]

    error_message = error.response["Error"]["Message"]

    request_id = error.response["ResponseMetadata"]["RequestId"]

    http_status = error.response["ResponseMetadata"]["HTTPStatusCode"]

⚠️ Note: standard retry mode already backs off and retries throttling errors for you automatically. If you’ve hand-rolled a time.sleep() retry loop around a boto3 call, switching to standard mode and deleting that loop is usually the better fix — not adding a second retry layer on top of the one you just configured. If that script is provisioning infrastructure rather than just calling an API, the same Config object works whether you’re on plain boto3 or using Pulumi with Python, since Pulumi’s AWS provider sits on top of boto3 underneath.

If you genuinely need to branch logic on a specific error (not just retry it), check the code string directly:

except botocore.exceptions.ClientError as error:

    if error.response["Error"]["Code"] == "LimitExceededException":

        logger.warning("API call limit exceeded; backing off and retrying")

    else:

        raise

You can also catch a named exception off the client itself — client.exceptions.LimitExceededException — but parsing the code string through ClientError is the more reliable approach across services, since not every error has a dedicated exception class generated for it. The official boto3 error-handling guide covers the full exception hierarchy if you need to catch something more specific than a generic ClientError.

Checking Whether a Retry Actually Happened

ResponseMetadata tells you how many retries a successful call actually took, which is the fastest way to confirm your retry config is doing anything at all:

response = client.some_operation()

print(response["ResponseMetadata"]["RetryAttempts"])

For deeper visibility, enable DEBUG logging. Legacy mode logs through botocore.retryhandler; standard and adaptive log through botocore.retries.standard — different logger names, so a log filter tuned for one mode goes silent if you switch to the other without updating it.

Why Third-Party Retry Wrappers Exist at All

If boto3’s built-in retry modes were sufficient out of the box, packages like botobackoff and awsretry wouldn’t exist on PyPI and GitHub. They do, because plenty of scripts were written against the legacy default and nobody revisited the retry config after AWS added standard mode. The fix usually isn’t a third-party wrapper — it’s four lines of Config. The official boto3 retries guide is worth reading end to end once if you’ve never configured a boto3 retry policy explicitly; it covers the circuit-breaker behavior in standard mode in more depth than fits here. If this script runs against more than one AWS account, switching cloud accounts and profiles fast covers the credential-scoping side of the same setup.

Frequently Asked Questions

Q: What’s the default boto3 retry mode?
A: legacy. It retries a narrower set of errors than standard mode and caps at 5 attempts by default. You have to opt into standard or adaptive explicitly.

Q: Does standard mode retry throttling errors automatically?
A: Yes — ThrottlingException, TooManyRequestsException, SlowDown, and several others are retried with backoff automatically. A manual time.sleep() retry loop around a boto3 call is usually redundant once you’re on standard mode.

Q: What’s the difference between total_max_attempts in Config and max_attempts in the AWS config file?
A: Same concept, different key name depending on where you set it — total_max_attempts in a Config object, max_attempts in ~/.aws/config. Both count the initial request as attempt 1.

Q: Should I use adaptive mode in production?
A: AWS marks it experimental, so treat it as subject to change between botocore releases. Standard mode is the stable, predictable choice for most production scripts.

Q: How do I check if a boto3 call actually retried?
A: Read response["ResponseMetadata"]["RetryAttempts"] after a successful call, or enable DEBUG logging on botocore.retries.standard (standard/adaptive) or botocore.retryhandler (legacy).

Quick Summary:
– boto3’s default retry mode is legacy, not standard — you have to opt in explicitly
– Config objects use mode and total_max_attempts; the AWS config file uses retry_mode and max_attempts — mixing them up fails silently
– Standard mode already retries throttling errors with backoff; a hand-rolled retry loop on top of it is usually redundant
– Catch botocore.exceptions.ClientError and read error.response["Error"]["Code"] — most services don’t have a dedicated exception class for every error
– Check response["ResponseMetadata"]["RetryAttempts"] to confirm your retry config is actually doing something

Check whichever script is running your scheduled AWS jobs right now — if it’s still on the legacy default, fixing your boto3 retry config is four lines of Config with mode: "standard", not a third-party wrapper.

DevToolHub