boto.md

boto

Resource vs Client

Low-level clients.

Both clients and resources have waiters.

Region

A region can be specified when creating a client. If none is specified the lookup order is:

1. AWS_DEFAULT_REGION environment variable
2. ~/.aws/config file, for the profile specified (or default profile)
3. Raise a NoRegionError

NB: See AWS Region for more info on which clients use which env vars. aws-cli and boto use AWS_DEFAULT_REGION

Use STS credentials created by the AWS CLI

The following will use STS credentials created by the AWS CLI, if they exist.

# Create boto3 client from session
client = boto3.Session(botocore_session=session).client('ec2')


def get_caching_session(profile_name=None):
    """Construct botocore session using cached STS creds if any

    Stolen from: https://github.com/mixja/boto3-session-cache
    """
    logger.info("Reading AWS credentials")

    try:
        sess = botocore.session.get_session()
    except botocore.exceptions.PartialCredentialsError:
        logger.error("Credentials are not complete. "
                     "Maybe use --profile or set AWS_PROFILE")
        raise

    if profile_name:
        sess.set_config_variable("profile", profile_name)

    # read cached STS creds
    cli_cache = os.path.join(os.path.expanduser("~"), ".aws/cli/cache")

    sess.get_component("credential_provider").get_provider(
        "assume-role"
    ).cache = credentials.JSONFileCache(cli_cache)

    return sess

See also Boto3 Docs / Developer guide / Credentials

Config vs credentials

Curious about the differences between ~/.aws/credentials and ~/.aws/config?🧵

Logging

Be careful enabling the debug loglevel on botocore as botocore.endpoint will log requests and botocore.parsers will log the response which may contain sensitive info.

To log retries and connections:

logging.getLogger("botocore.retryhandler").setLevel(logging.DEBUG)
logging.getLogger("urllib3.connectionpool").setLevel(logging.DEBUG)