# Using the AWS SDK for Java 2.x
This chapter shows you how to use the AWS SDK for Java 2.x effectively. Learn to create service clients, make requests, handle responses, and manage errors. The chapter covers synchronous and asynchronous programming, paginated results, waiters for resource monitoring, and performance optimization.
You'll also find best practices for client reuse, troubleshooting guidance, Lambda startup optimization, HTTP/2 support, and DNS configuration.
**Contents**
+ [
# Making AWS service requests using the AWS SDK for Java 2.x
](work-witih-clients.md)
+ [
## Using service clients to make requests
](work-witih-clients.md#using-service-client)
+ [
### Create a service client
](work-witih-clients.md#work-with-clients-create)
+ [
### Default client configuration
](work-witih-clients.md#using-default-client)
+ [
### Configure service clients
](work-witih-clients.md#using-configure-service-clients)
+ [
### Close the service client
](work-witih-clients.md#using-closing-client)
+ [
## Make requests
](work-witih-clients.md#using-making-requests)
+ [
### Use requests to override client configuration
](work-witih-clients.md#using-override-client-config-request)
+ [
## Handle responses
](work-witih-clients.md#using-handling-responses)
+ [
# Programming asynchronously using the AWS SDK for Java 2.x
](asynchronous.md)
+ [
## Use asynchronous client APIs
](asynchronous.md#basics-async-non-streaming)
+ [
## Handle streaming in asynchronous methods
](asynchronous.md#basics-async-streaming)
+ [
## Configure advanced asynchronous options
](asynchronous.md#advanced-operations)
+ [
# Best practices for using the AWS SDK for Java 2.x
](best-practices.md)
+ [
## Prevent hanging requests by configuring API timeouts
](best-practices.md#bestpractice5)
+ [
## Improve performance by reusing service clients
](best-practices.md#bestpractice1)
+ [
## Prevent resource leaks by closing unused service clients
](best-practices.md#bestpractice-close-client)
+ [
## Prevent connection pool exhaustion by closing input streams
](best-practices.md#bestpractice2)
+ [
## Optimize HTTP performance for your application workload
](best-practices.md#bestpractice3)
+ [
## Improve SSL performance with OpenSSL for async clients
](best-practices.md#bestpractice4)
+ [
## Monitor application performance with SDK metrics
](best-practices.md#bestpractice6)
+ [
# Handling errors in the AWS SDK for Java 2.x
](handling-exceptions.md)
+ [
## Why unchecked exceptions?
](handling-exceptions.md#why-unchecked-exceptions)
+ [
## AwsServiceException (and subclasses)
](handling-exceptions.md#sdkserviceexception-and-subclasses)
+ [
## SdkClientException
](handling-exceptions.md#sdkclientexception)
+ [
## Exceptions and retry behavior
](handling-exceptions.md#retried-exceptions)
+ [
# Using paginated results in the AWS SDK for Java 2.x
](pagination.md)
+ [
## Synchronous pagination
](pagination.md#synchronous-pagination)
+ [
### Iterate over pages
](pagination.md#iterate-pages)
+ [
### Iterate over objects
](pagination.md#iterate-objects)
+ [
#### Use a stream
](pagination.md#use-a-stream)
+ [
#### Use a for-each loop
](pagination.md#for-loop)
+ [
### Manual pagination
](pagination.md#manual-pagination)
+ [
## Asynchronous pagination
](pagination.md#asynchronous-pagination)
+ [
### Iterate over pages of table names
](pagination.md#iterate-pages-async)
+ [
#### Use a `Subscriber`
](pagination.md#use-a-subscriber)
+ [
#### Use a `Consumer`
](pagination.md#id1pagination)
+ [
### Iterate over table names
](pagination.md#iterate-objects-async)
+ [
#### Use a `Subscriber`
](pagination.md#id2)
+ [
#### Use a `Consumer`
](pagination.md#for-loop-async)
+ [
### Use third-party library
](pagination.md#use-third-party-library)
+ [
# Using waiters in the AWS SDK for Java 2.x
](waiters.md)
+ [
## Prerequisites
](waiters.md#prerequisiteswaiters)
+ [
## Using waiters
](waiters.md#id1waiters)
+ [
### Synchronous programming
](waiters.md#synchronous-programming)
+ [
### Asynchronous programming
](waiters.md#asynchronous-programming)
+ [
## Configure waiters
](waiters.md#configuring-waiters)
+ [
### Configure a waiter
](waiters.md#configure-a-waiter)
+ [
### Override configuration for a specific request
](waiters.md#override-configuration-for-a-specific-request)
+ [
## Code examples
](waiters.md#code-examples)
+ [
# Troubleshooting FAQs
](troubleshooting.md)
+ [
## How do I fix "`java.net.SocketException`: Connection reset" or "server failed to complete the response" error?
](troubleshooting.md#faq-socketexception)
+ [
## How do I fix "connection timeout"?
](troubleshooting.md#faq-connection-timeout)
+ [
## How do I fix "`java.net.SocketTimeoutException`: Read timed out"?
](troubleshooting.md#faq-socket-timeout)
+ [
## How do I fix "Unable to execute HTTP request: Timeout waiting for connection from pool" error?
](troubleshooting.md#faq-pool-timeout)
+ [
## How do I fix a `NoClassDefFoundError`, `NoSuchMethodError` or `NoSuchFieldError`?
](troubleshooting.md#faq-classpath-errors)
+ [
## How do I fix a "`SignatureDoesNotMatch`" error or "The request signature we calculated does not match the signature you provided" error?
](troubleshooting.md#faq-signature-does-not-match-error)
+ [
## How do I fix "`java.lang.IllegalStateException`: Connection pool shut down" error?
](troubleshooting.md#faq-connection-pool-shutdown-exception)
+ [
## How do I fix "Unable to load credentials from any of the providers in the chain AwsCredentialsProviderChain"?
](troubleshooting.md#faq-credentials-provider-chain)
+ [
### Common causes and solutions
](troubleshooting.md#faq-cred-provider-chain-common-causes-and-solutions)
+ [
#### Review your credential configuration
](troubleshooting.md#faq-cred-provider-chain-check-config)
+ [
##### For Amazon EC2 instances
](troubleshooting.md#faq-cred-check-ec2)
+ [
##### For container environments
](troubleshooting.md#faq-cred-check-container-env)
+ [
##### For local development
](troubleshooting.md#faq-cred-check-local-dev)
+ [
##### For web identity federation
](troubleshooting.md#faq-cred-check-web-id-federation)
+ [
#### Network or proxy connectivity issues
](troubleshooting.md#faq-credentials-provider-chain-network-issues)
+ [
# Reduce SDK startup time for AWS Lambda
](lambda-optimize-starttime.md)
+ [
## Use an AWS CRT-based HTTP client
](lambda-optimize-starttime.md#lambda-quick-url)
+ [
## Remove unused HTTP client dependencies
](lambda-optimize-starttime.md#lambda-quick-remove-deps)
+ [
## Configure service clients to shortcut lookups
](lambda-optimize-starttime.md#lambda-quick-clients)
+ [
## Initialize the SDK client outside of the Lambda function handler
](lambda-optimize-starttime.md#lambda-quick-initialize)
+ [
## Minimize dependency injection
](lambda-optimize-starttime.md#lambda-quick-di)
+ [
## Use a Maven Archetype targeting AWS Lambda
](lambda-optimize-starttime.md#lambda-quick-maven)
+ [
## Consider Lambda SnapStart for Java
](lambda-optimize-starttime.md#lambda-quick-snapstart)
+ [
## Version 2.x changes that affect startup time
](lambda-optimize-starttime.md#example-client-configuration)
+ [
## Additional resources
](lambda-optimize-starttime.md#lambda-quick-resources)
+ [
# Implement `ContentStreamProvider` in the AWS SDK for Java 2.x
](content-stream-provider.md)
+ [
## Use `mark()` and `reset()`
](content-stream-provider.md#csp-impl-mark-reset)
+ [
## Use buffering if `mark()` and `reset()` are not available
](content-stream-provider.md#csp-impl-unsupported-streams)
+ [
## Create new streams
](content-stream-provider.md#csp-impl-new-stream)
+ [
# Set the JVM TTL for DNS name lookups
](jvm-ttl-dns.md)
+ [
## How to set the JVM TTL
](jvm-ttl-dns.md#how-to-set-the-jvm-ttl)
+ [
### Option 1: Set it programmatically in your application
](jvm-ttl-dns.md#set-ttl-programmatically)
+ [
### Option 2: Set it in the java.security file
](jvm-ttl-dns.md#set-ttl-java-security-file)
+ [
### Option 3: Use the JDK system properties fallback (command-line)
](jvm-ttl-dns.md#set-ttl-system-property)
+ [
# Work with HTTP/2 in the AWS SDK for Java
](http2.md)
# Making AWS service requests using the AWS SDK for Java 2.x
## Using service clients to make requests
After completing the steps in [Setting up the SDK](setup.md) and understanding how to [configure service clients](configuring-service-clients.md), you are ready to make requests to AWS services such as Amazon S3, Amazon DynamoDB, AWS Identity and Access Management, Amazon EC2, and more.
### Create a service client
To make a request to an AWS service, you must first instantiate a service client for that service by using the static factory method, `builder()`. The `builder()` method returns a `builder` object that allows you to customize the service client. The fluent setter methods return the `builder` object, so that you can chain the method calls for convenience and for more readable code. After you configure the properties you want, call the `build()` method to create the client.
As an example, the following code snippet instantiates an `Ec2Client` object as a service client for Amazon EC2.
```
Region region = Region.US_WEST_2;
Ec2Client ec2Client = Ec2Client.builder()
.region(region)
.build();
```
**Note**
Service clients in the SDK are thread-safe. For best performance, treat them as long-lived objects. Each client has its own connection pool resource that is released when the client is garbage collected.
A service client object is immutable, so you must create a new client for each service to which you make requests, or if you want to use a different configuration for making requests to the same service.
Specifying the `Region` in the service client builder is not required for all AWS services; however, it is a best practice to set the Region for the API calls you make in your applications. See [AWS region selection](region-selection.md) for more information.
### Default client configuration
The client builders have another factory method named `create()`. This method creates a service client with the default configuration. It uses the [default provider chain](credentials-chain.md) to load credentials and the [default AWS Region provider chain](region-selection.md#automatically-determine-the-aws-region-from-the-environment). If credentials or the Region can’t be determined from the environment that the application is running in, the call to `create` fails. See [Using credentials](credentials.md) and [Region selection](region-selection.md) for more information about how the SDK determines the credentials and Region to use.
For example, the following code snippet instantiates a `DynamoDbClient` object as a service client for Amazon DynamoDB:
```
DynamoDbClient dynamoDbClient = DynamoDbClient.create();
```
### Configure service clients
For details about how to configure service clients, see [Client configuration externally](configuring-service-clients-ext.md) and [Client configuration in code](configuring-service-clients-code.md).
### Close the service client
As a best practice, you should use a service clients for multiple API service calls during the life of an application. However, if you need a service client for a one-time use or no longer need the service client, close it.
Call the `close()` method when the service client is no longer needed to free up resources.
```
ec2Client.close();
```
If you need a service client for one-time use, you can instantiate the service client as a resource in a `try`-with-resources statement. Service clients implement the `[Autoclosable](https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/lang/AutoCloseable.html)` interface, so the JDK automatically calls the `close()` method at the end of the statement.
The following example demonstrates how to use a service client for a one-off call. The `StsClient` that calls the AWS Security Token Service is closed after it returns the account ID.
```
import software.amazon.awssdk.services.sts.StsClient;
String getAccountID() {
try (StsClient stsClient = StsClient.create()) {
return stsClient.getCallerIdentity().account();
}
}
```
## Make requests
Use the service client to make requests to the corresponding AWS service.
For example, this code snippet shows how to create a `RunInstancesRequest` object to create a new Amazon EC2 instance:
```
// Create the request by using the fluid setter methods of the request builder.
RunInstancesRequest runInstancesRequest = RunInstancesRequest.builder()
.imageId(amiId)
.instanceType(InstanceType.T1_MICRO)
.maxCount(1)
.minCount(1)
.build();
// Use the configured request with the service client.
RunInstancesResponse response = ec2Client.runInstances(runInstancesRequest);
```
Rather than create a request and pass in the instance, the SDK provides a fluent API that you can use to create a request. With the fluent API you can use a Java lambda expressions to create the request 'in-line'.
The following example rewrites the previous example by using the version of the `runInstances` [method that uses a builder](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/ec2/Ec2Client.html#runInstances(java.util.function.Consumer)) to create the request.
```
// Create the request by using a lambda expression.
RunInstancesResponse response = ec2.runInstances(r -> r
.imageId(amiId)
.instanceType(InstanceType.T1_MICRO)
.maxCount(1)
.minCount(1));
```
### Use requests to override client configuration
Although a service client is immutable, you can override many of its settings at the request level. When you build a request, you can provide an [AwsRequestOverrideConfiguration](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/awscore/AwsRequestOverrideConfiguration.html) instance to provide the overridden settings. Some of the methods you can use to override client settings are:
+ `apiCallAttemptTimeout`
+ `apiCallTimeout`
+ `credentialProvider`
+ `compressionConfiguration`
+ `putHeader`
For an example of overriding a client setting with a request, assume that you have the following S3 client that uses default settings.
```
S3Client s3Client = S3Client.create();
```
You want to download a large file and want to be sure the request doesn't timeout before the download finishes. To accomplish this, increase the timeout values for only a single `GetObject` request as shown in the following code.
------
#### [ Standard API ]
```
AwsRequestOverrideConfiguration overrideConfiguration = AwsRequestOverrideConfiguration.builder()
.apiCallTimeout(Duration.ofSeconds(100L))
.apiCallAttemptTimeout(Duration.ofSeconds(25L))
.build();
GetObjectRequest request = GetObjectRequest.builder()
.bucket("amzn-s3-demo-bucket")
.key("demo-key")
.overrideConfiguration(overrideConfiguration)
.build();
s3Client.getObject(request, myPath);
```
------
#### [ Fluent API ]
```
s3Client.getObject(b -> b
.bucket("amzn-s3-demo-bucket")
.key("demo-key")
.overrideConfiguration(c -> c
.apiCallTimeout(Duration.ofSeconds(100L))
.apiCallAttemptTimeout(Duration.ofSeconds(25L))),
myPath);
```
------
## Handle responses
The SDK returns a response object for most service operations. Your code can process the information in the response object according to your needs.
For example, the following code snippet prints out the first instance id returned with the [https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/ec2/model/RunInstancesResponse.html](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/ec2/model/RunInstancesResponse.html) object from the previous request.
```
RunInstancesResponse runInstancesResponse = ec2Client.runInstances(runInstancesRequest);
System.out.println(runInstancesResponse.instances().get(0).instanceId());
```
Not all operations return a response object with service-specific data, however. In these situations, you can query the HTTP response status to learn if the operation was successful.
For example, the code in the following snippet checks the HTTP response to see if the [https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/sesv2/SesV2Client.html#deleteContactList(software.amazon.awssdk.services.sesv2.model.DeleteContactListRequest)](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/sesv2/SesV2Client.html#deleteContactList(software.amazon.awssdk.services.sesv2.model.DeleteContactListRequest)) operation of Amazon Simple Email Service was successful.
```
SesV2Client sesv2Client = SesV2Client.create();
DeleteContactListRequest request = DeleteContactListRequest.builder()
.contactListName("ExampleContactListName")
.build();
DeleteContactListResponse response = sesv2Client.deleteContactList(request);
if (response.sdkHttpResponse().isSuccessful()) {
System.out.println("Contact list deleted successfully");
} else {
System.out.println("Failed to delete contact list. Status code: " + response.sdkHttpResponse().statusCode());
}
```
# Programming asynchronously using the AWS SDK for Java 2.x
The AWS SDK for Java 2.x features asynchronous clients with non-blocking I/O support that implement high concurrency across a few threads. However, total non-blocking I/O is not guaranteed. Asynchronous client may perform blocking calls in some cases such as credential retrieval, request signing using [AWS Signature Version 4 (SigV4)](https://docs.aws.amazon.com/AmazonS3/latest/API/sig-v4-authenticating-requests.html), or endpoint discovery.
Synchronous methods block your thread’s execution until the client receives a response from the service. Asynchronous methods return immediately, giving control back to the calling thread without waiting for a response.
Because an asynchronous method returns before a response is available, you need a way to get the response when it’s ready. The methods for asynchronous client in 2.x of the AWS SDK for Java return *CompletableFuture objects* that allow you to access the response when it’s ready.
## Use asynchronous client APIs
The signatures of asynchronous client methods are the same their synchronous counterpart, but the asynchronous methods return a [CompletableFuture](https://docs.oracle.com/javase/8/docs/api/index.html?java/util/concurrent/CompletableFuture.html) object that contains the results of the asynchronous operation *in the future*. If an error is thrown while the SDK's asynchronous method executes, the error is thrown as `CompletionException`.
One approach you can use to get the result is to chain a `whenComplete()` method onto the `CompletableFuture` returned by the SDK method call. The `whenComplete()` method receives the result or a Throwable object of type `CompletionException` depending on how the asynchronous call completed. You provide an action to `whenComplete()` to process or check the results before it is returned to the calling code.
If you want to return something other than the object returned by the SDK method, use the `handle()` method instead. The `handle()` method takes the same parameters as `whenComplete()`, but you can process the result and return an object.
To wait for the asynchronous chain to complete and retrieve the completion results, you can call the `join()` method. If the `Throwable` object was not handled in the chain, the `join()` method throws an unchecked `CompletionException` that wraps the original exception. You access the original exception with `CompletionException#getCause()`. You can also call the `CompletableFuture#get()` method to get the completion results. The `get()` method, however, can throw checked exceptions.
The following example shows two variations of how you can work with the `listTables()` method of the DynamoDB asynchronous client. The action passed to `whenComplete()` simply logs a successful response, whereas the `handle()` version extracts the list of table names and returns the list. In both cases if an error is generated in the asynchronous chain, the error is rethrown so the client code has a chance to handle it.
**Imports**
```
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.dynamodb.DynamoDbAsyncClient;
import software.amazon.awssdk.services.dynamodb.model.ListTablesRequest;
import software.amazon.awssdk.services.dynamodb.model.ListTablesResponse;
import java.util.List;
import java.util.concurrent.CompletableFuture;
```
**Code**
------
#### [ whenComplete() variation ]
```
public class DynamoDbAsyncListTables {
public static void main(String[] args) {
Region region = Region.US_EAST_1;
DynamoDbAsyncClient dynamoDbAsyncClient = DynamoDbAsyncClient.builder().region(region).build();
try {
ListTablesResponse listTablesResponse = listTablesWhenComplete(dynamoDbAsyncClient).join(); // The join() method may throw a CompletionException.
if (listTablesResponse.hasTableNames()){
System.out.println("Table exist in this region: " + region.id());
}
} catch (RuntimeException e) {
// Handle as needed. Here we simply print out the class names.
System.out.println(e.getClass()); // Prints 'class java.util.concurrent.CompletionException'.
System.out.println(e.getCause().getClass()); // Prints 'class software.amazon.awssdk.services.dynamodb.model.DynamoDbException'.
}
}
public static CompletableFuture listTablesWhenComplete(DynamoDbAsyncClient client) {
return client.listTables(ListTablesRequest.builder().build())
.whenComplete((listTablesResponse, throwable) -> {
if (listTablesResponse != null) { // Consume the response.
System.out.println("The SDK's listTables method completed successfully.");
} else {
RuntimeException cause = (RuntimeException) throwable.getCause(); // If an error was thrown during the SDK's listTables method it is wrapped in a CompletionException.
// The SDK throws only RuntimeExceptions, so this is a safe cast.
System.out.println(cause.getMessage()); // Log error here, but rethrow so the calling code can handle as needed.
throw cause;
}
});
}
```
------
#### [ handle() variation ]
```
public class DynamoDbAsyncListTables {
public static void main(String[] args) {
Region region = Region.US_EAST_1;
DynamoDbAsyncClient dynamoDbAsyncClient = DynamoDbAsyncClient.builder().region(region).build();
try {
List tableNames = listTablesHandle(dynamoDbAsyncClient).join(); // The join() method may throw a CompletionException.
tableNames.forEach(System.out::println);
} catch (RuntimeException e) {
// Handle as needed. Here we simply print out the class names.
System.out.println(e.getClass()); // Prints 'class java.util.concurrent.CompletionException'.
System.out.println(e.getCause().getClass()); // Prints 'class software.amazon.awssdk.services.dynamodb.model.DynamoDbException'.
}
}
public static CompletableFuture> listTablesHandle(DynamoDbAsyncClient client) {
return client.listTables(ListTablesRequest.builder().build())
.handle((listTablesResponse, throwable) -> {
if (listTablesResponse != null) {
return listTablesResponse.tableNames(); // Return the list of table names.
} else {
RuntimeException cause = (RuntimeException) throwable.getCause(); // If an error was thrown during the SDK's listTables method it is wrapped in a CompletionException.
// The SDK throws only RuntimeExceptions, so this is a safe cast.
System.out.println(cause.getMessage()); // Log error here, but rethrow so the calling code can handle as needed.
throw cause;
}
});
}
}
```
------
## Handle streaming in asynchronous methods
For asynchronous methods with streaming content, you must provide an [AsyncRequestBody](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/core/async/AsyncRequestBody.html) to provide the content incrementally, or an [AsyncResponseTransformer](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/core/async/AsyncResponseTransformer.html) to receive and process the response.
The following example uploads a file to Amazon S3 asynchronously by using the asynchronous form of the `PutObject` operation.
**Imports**
```
import software.amazon.awssdk.core.async.AsyncRequestBody;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.s3.S3AsyncClient;
import software.amazon.awssdk.services.s3.model.PutObjectRequest;
import software.amazon.awssdk.services.s3.model.PutObjectResponse;
import java.nio.file.Paths;
import java.util.concurrent.CompletableFuture;
```
**Code**
```
/**
* To run this AWS code example, ensure that you have setup your development environment, including your AWS credentials.
*
* For information, see this documentation topic:
*
* https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/get-started.html
*/
public class S3AsyncOps {
public static void main(String[] args) {
final String USAGE = "\n" +
"Usage:\n" +
" S3AsyncOps \n\n" +
"Where:\n" +
" bucketName - the name of the Amazon S3 bucket (for example, bucket1). \n\n" +
" key - the name of the object (for example, book.pdf). \n" +
" path - the local path to the file (for example, C:/AWS/book.pdf). \n" ;
if (args.length != 3) {
System.out.println(USAGE);
System.exit(1);
}
String bucketName = args[0];
String key = args[1];
String path = args[2];
Region region = Region.US_WEST_2;
S3AsyncClient client = S3AsyncClient.builder()
.region(region)
.build();
PutObjectRequest objectRequest = PutObjectRequest.builder()
.bucket(bucketName)
.key(key)
.build();
// Put the object into the bucket
CompletableFuture future = client.putObject(objectRequest,
AsyncRequestBody.fromFile(Paths.get(path))
);
future.whenComplete((resp, err) -> {
try {
if (resp != null) {
System.out.println("Object uploaded. Details: " + resp);
} else {
// Handle error
err.printStackTrace();
}
} finally {
// Only close the client when you are completely done with it
client.close();
}
});
future.join();
}
}
```
The following example gets a file from Amazon S3 by using the asynchronous form of the `GetObject` operation.
**Imports**
```
import software.amazon.awssdk.core.async.AsyncResponseTransformer;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.s3.S3AsyncClient;
import software.amazon.awssdk.services.s3.model.GetObjectRequest;
import software.amazon.awssdk.services.s3.model.GetObjectResponse;
import java.nio.file.Paths;
import java.util.concurrent.CompletableFuture;
```
**Code**
```
/**
* To run this AWS code example, ensure that you have setup your development environment, including your AWS credentials.
*
* For information, see this documentation topic:
*
* https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/get-started.html
*/
public class S3AsyncStreamOps {
public static void main(String[] args) {
final String USAGE = "\n" +
"Usage:\n" +
" S3AsyncStreamOps \n\n" +
"Where:\n" +
" bucketName - the name of the Amazon S3 bucket (for example, bucket1). \n\n" +
" objectKey - the name of the object (for example, book.pdf). \n" +
" path - the local path to the file (for example, C:/AWS/book.pdf). \n" ;
if (args.length != 3) {
System.out.println(USAGE);
System.exit(1);
}
String bucketName = args[0];
String objectKey = args[1];
String path = args[2];
Region region = Region.US_WEST_2;
S3AsyncClient client = S3AsyncClient.builder()
.region(region)
.build();
GetObjectRequest objectRequest = GetObjectRequest.builder()
.bucket(bucketName)
.key(objectKey)
.build();
CompletableFuture futureGet = client.getObject(objectRequest,
AsyncResponseTransformer.toFile(Paths.get(path)));
futureGet.whenComplete((resp, err) -> {
try {
if (resp != null) {
System.out.println("Object downloaded. Details: "+resp);
} else {
err.printStackTrace();
}
} finally {
// Only close the client when you are completely done with it
client.close();
}
});
futureGet.join();
}
}
```
## Configure advanced asynchronous options
The AWS SDK for Java 2.x uses [Netty](https://netty.io), an asynchronous event-driven network application framework, to handle I/O threads. The AWS SDK for Java 2.x creates an `ExecutorService` behind Netty, to complete the futures returned from the HTTP client request through to the Netty client. This abstraction reduces the risk of an application breaking the async process if developers choose to stop or sleep threads. By default, each asynchronous client creates a threadpool based on the number of processors and manages the tasks in a queue within the `ExecutorService`.
You can specify a specific JDK implementation of `ExecutorService` when you build your asynchronous client. The following snippet create an `ExecutorService` with a fixed number of threads.
**Code**
```
S3AsyncClient clientThread = S3AsyncClient.builder()
.asyncConfiguration(
b -> b.advancedOption(SdkAdvancedAsyncClientOption
.FUTURE_COMPLETION_EXECUTOR,
Executors.newFixedThreadPool(10)
)
)
.build();
```
To optimize performance, you can manage your own thread pool executor, and include it when you configure your client.
```
ThreadPoolExecutor executor = new ThreadPoolExecutor(50, 50,
10, TimeUnit.SECONDS,
new LinkedBlockingQueue<>(),
new ThreadFactoryBuilder()
.threadNamePrefix("sdk-async-response").build());
// Allow idle core threads to time out
executor.allowCoreThreadTimeOut(true);
S3AsyncClient clientThread = S3AsyncClient.builder()
.asyncConfiguration(
b -> b.advancedOption(SdkAdvancedAsyncClientOption
.FUTURE_COMPLETION_EXECUTOR,
executor
)
)
.build();
```
# Best practices for using the AWS SDK for Java 2.x
## Prevent hanging requests by configuring API timeouts
The SDK provides [default values](https://github.com/aws/aws-sdk-java-v2/blob/a0c8a0af1fa572b16b5bd78f310594d642324156/http-client-spi/src/main/java/software/amazon/awssdk/http/SdkHttpConfigurationOption.java#L134) for some timeout options, such as connection timeout and socket timeouts, but not for API call timeouts or individual API call attempt timeouts. It is a good practice to set timeouts for both the individual attempts and the entire request. This will ensure your application fails fast in an optimal way when there are transient issues that could cause request attempts to take longer to complete or fatal network issues.
You can configure timeouts for all requests made by a service clients using `[ClientOverrideConfiguration\$1apiCallAttemptTimeout](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/core/client/config/ClientOverrideConfiguration.html#apiCallAttemptTimeout())` and `[ClientOverrideConfiguration\$1apiCallTimeout](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/core/client/config/ClientOverrideConfiguration.html#apiCallTimeout())`.
The following example shows the configuration of an Amazon S3 client with custom timeout values.
```
S3Client.builder()
.overrideConfiguration(
b -> b.apiCallTimeout(Duration.ofSeconds())
.apiCallAttemptTimeout(Duration.ofMillis()))
.build();
```
**`apiCallAttemptTimeout`**
This setting sets the amount of time for a single HTTP attempt, after which the API call can be retried.
**`apiCallTimeout`**
The value for this property configures the amount of time for the entire execution, including all retry attempts.
As an alternative to setting these timeout values on the service client, you can use [https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/core/RequestOverrideConfiguration.html#apiCallTimeout()](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/core/RequestOverrideConfiguration.html#apiCallTimeout()) and `[RequestOverrideConfiguration\$1apiCallAttemptTimeout()](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/core/RequestOverrideConfiguration.html#apiCallAttemptTimeout())` to configure a single request .
The following example configures a single `listBuckets` request with custom timeout values.
```
s3Client.listBuckets(lbr -> lbr.overrideConfiguration(
b -> b.apiCallTimeout(Duration.ofSeconds())
.apiCallAttemptTimeout(Duration.ofMillis())));
```
When you use these properties together, you set a hard limit on the total time spent on all attempts across retries. You also set an individual HTTP request to fail fast on a slow request.
## Improve performance by reusing service clients
Each [service client](work-witih-clients.md) maintains its own HTTP connection pool. A connection that already exists in the pool can be reused by a new request to cut down the time to establish a new connection. We recommend sharing a single instance of the client to avoid the overhead of having too many connection pools that aren't used effectively. All service clients are thread safe.
If you don't want to share a client instance, call `close()` on the instance to release the resources when the client is not needed.
## Prevent resource leaks by closing unused service clients
Close a [service client](work-witih-clients.md) to release resources, such as threads, if it is no longer needed. If you don't want to share a client instance, call `close()` on the instance to release the resources when the client is not needed.
## Prevent connection pool exhaustion by closing input streams
For streaming operations such as `[S3Client\$1getObject](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/s3/S3Client.html#getObject(java.util.function.Consumer,java.nio.file.Path))`, if you are working with `[ResponseInputStream](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/core/ResponseInputStream.html)` directly, we recommend that you do the following:
+ Read all the data from the input stream as soon as possible.
+ Close the input stream as soon as possible.
We make these recommendations because the input stream is a direct stream of data from the HTTP connection and the underlying HTTP connection can't be reused until all data from the stream has been read and the stream is closed. If these rules are not followed, the client can run out of resources by allocating too many open, but unused, HTTP connections.
## Optimize HTTP performance for your application workload
The SDK provides a set of [default http configurations](https://github.com/aws/aws-sdk-java-v2/blob/master/http-client-spi/src/main/java/software/amazon/awssdk/http/SdkHttpConfigurationOption.java) that apply to general use cases. We recommend that customers tune HTTP configurations for their applications based on their use cases.
As a good starting point, the SDK offers a [smart configuration defaults](http-configuration.md#http-config-smart-defaults) feature. This feature is available starting with version 2.17.102. You choose a mode depending on your use case, which provides sensible configuration values.
## Improve SSL performance with OpenSSL for async clients
By default, the SDK's [https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/http/nio/netty/NettyNioAsyncHttpClient.html](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/http/nio/netty/NettyNioAsyncHttpClient.html) uses the JDK's default SSL implementation as the `SslProvider`. Our testing found that OpenSSL performs better than JDK's default implementation. The Netty community also [recommends using OpenSSL](https://netty.io/wiki/requirements-for-4.x.html#tls-with-openssl).
To use OpenSSL, add `netty-tcnative` to your dependencies. For configuration details, see the [Netty project documentation](https://netty.io/wiki/forked-tomcat-native.html).
After you have `netty-tcnative` configured for your project, the `NettyNioAsyncHttpClient` instance will automatically select OpenSSL. Alternatively, you can set the `SslProvider` explicitly using the `NettyNioAsyncHttpClient` builder as shown in the following snippet.
```
NettyNioAsyncHttpClient.builder()
.sslProvider(SslProvider.OPENSSL)
.build();
```
## Monitor application performance with SDK metrics
The SDK for Java can [collect metrics](metrics.md) for the service clients in your application. You can use these metrics to identify performance issues, review overall usage trends, review service client exceptions returned, or to dig in to understand a particular issue.
We recommend that you collect metrics, then analyze the Amazon CloudWatch Logs, in order to gain a deeper understanding of your application's performance.
# Handling errors in the AWS SDK for Java 2.x
Understanding how and when the AWS SDK for Java 2.x throws exceptions is important to building high-quality applications using the SDK. The following sections describe the different cases of exceptions that are thrown by the SDK and how to handle them appropriately.
## Why unchecked exceptions?
The AWS SDK for Java uses runtime (or unchecked) exceptions instead of checked exceptions for these reasons:
+ To allow developers fine-grained control over the errors they want to handle without forcing them to handle exceptional cases they aren’t concerned about (and making their code overly verbose)
+ To prevent scalability issues inherent with checked exceptions in large applications
In general, checked exceptions work well on small scales, but can become troublesome as applications grow and become more complex.
## AwsServiceException (and subclasses)
[AwsServiceException](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/awscore/exception/AwsServiceException.html) is the most common exception that you’ll experience when using the AWS SDK for Java. `AwsServiceException` is a subclass of the more general [SdkServiceException](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/core/exception/SdkServiceException.html). `AwsServiceException`s represent an error response from an AWS service. For example, if you try to terminate an Amazon EC2 instance that doesn’t exist, Amazon EC2 will return an error response and all the details of that error response will be included in the `AwsServiceException` that’s thrown.
When you encounter an `AwsServiceException`, you know that your request was successfully sent to the AWS service but couldn’t be successfully processed. This can be because of errors in the request’s parameters or because of issues on the service side.
`AwsServiceException` provides you with information such as:
+ Returned HTTP status code
+ Returned AWS error code
+ Detailed error message from the service in the [AwsErrorDetails](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/awscore/exception/AwsErrorDetails.html) class
+ AWS request ID for the failed request
In most cases, a service-specific subclass of `AwsServiceException` is thrown to allow developers fine-grained control over handling error cases through catch blocks. The Java SDK API reference for [AwsServiceException](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/awscore/exception/AwsServiceException.html) displays the large number of `AwsServiceException` subclasses. Use the subclass links to drill down to see the granular exceptions thrown by a service.
For example, the following links to the SDK API reference show the exception hierarchies for a few common AWS services. The list of subclasses shown on each pages shows the specific exceptions that your code can catch.
+ [Amazon S3](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/s3/model/S3Exception.html)
+ [DynamoDB](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/dynamodb/model/DynamoDbException.html)
+ [Amazon SQS](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/sqs/model/SqsException.html)
To learn more about an exception, inspect the `errorCode` on the [AwsErrorDetails](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/awscore/exception/AwsErrorDetails.html) object. You can use the `errorCode` value to look up information in the service guide API. For example if an `S3Exception` is caught and the `AwsErrorDetails#errorCode()` value is `InvalidRequest`, use the [list of error codes](https://docs.aws.amazon.com/AmazonS3/latest/API/ErrorResponses.html#ErrorCodeList) in the Amazon S3 API Reference to see more details.
## SdkClientException
[SdkClientException](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/core/exception/SdkClientException.html) indicates that a problem occurred inside the Java client code, either while trying to send a request to AWS or while trying to parse a response from AWS. An `SdkClientException` is generally more severe than an `SdkServiceException`, and indicates a major problem that is preventing the client from making service calls to AWS services. For example, the AWS SDK for Java throws an `SdkClientException` if no network connection is available when you try to call an operation on one of the clients.
## Exceptions and retry behavior
The SDK for Java retries requests for several [client-side exceptions](https://github.com/aws/aws-sdk-java-v2/blob/13985e0668a9a0b12ad331644e3c4fd1385c2cd7/core/sdk-core/src/main/java/software/amazon/awssdk/core/internal/retry/SdkDefaultRetrySetting.java#L79C41-L79C41) and for [HTTP status codes](https://github.com/aws/aws-sdk-java-v2/blob/13985e0668a9a0b12ad331644e3c4fd1385c2cd7/core/sdk-core/src/main/java/software/amazon/awssdk/core/internal/retry/SdkDefaultRetrySetting.java#L72C31-L72C31) that it receives from AWS service responses. These errors are handled as part of the legacy `RetryMode` that service clients use by default. The Java API reference for `[RetryMode](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/core/retry/RetryMode.html)` describes the various ways that you can configure the mode.
To customize the exceptions and HTTP status codes that trigger automatic retries, configure your service client with a `[RetryPolicy](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/core/retry/RetryPolicy.html)` that adds `[RetryOnExceptionsCondition](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/core/retry/conditions/RetryOnExceptionsCondition.html)` and `[RetryOnStatusCodeCondition](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/awscore/retry/conditions/RetryOnErrorCodeCondition.html)` instances.
# Using paginated results in the AWS SDK for Java 2.x
Many AWS operations return paginated results when the response object is too large to return in a single response. In the AWS SDK for Java 1.0, the response contains a token you use to retrieve the next page of results. In contrast, the AWS SDK for Java 2.x has autopagination methods that make multiple service calls to get the next page of results for you automatically. You only have to write code that processes the results. Autopagination is available for both synchronous and asynchronous clients.
**Note**
These code snippets assume that you understand [the basics of using the SDK](using.md), and have configured your environment with [single sign-on access](get-started-auth.md#setup-credentials).
## Synchronous pagination
The following examples demonstrate synchronous pagination methods to list objects in an Amazon S3 bucket.
### Iterate over pages
The first example demonstrates the use of a `listRes` paginator object, a [https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/s3/paginators/ListObjectsV2Iterable.html](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/s3/paginators/ListObjectsV2Iterable.html) instance, to iterate through all the response pages with the `stream` method. The code streams over the response pages, converts the response stream to a stream of `[S3Object](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/s3/model/S3Object.html)` content, and then processes the content of the Amazon S3 object.
The following imports apply to all examples in this synchronous pagination section.
#### Imports
```
import java.io.IOException;
import java.nio.ByteBuffer;
import java.util.Random;
import software.amazon.awssdk.core.waiters.WaiterResponse;
import software.amazon.awssdk.regions.Region;
import software.amazon.awssdk.services.s3.S3Client;
import software.amazon.awssdk.services.s3.paginators.ListObjectsV2Iterable;
import software.amazon.awssdk.core.sync.RequestBody;
import software.amazon.awssdk.services.s3.model.S3Exception;
import software.amazon.awssdk.services.s3.model.PutObjectRequest;
import software.amazon.awssdk.services.s3.model.ListObjectsV2Request;
import software.amazon.awssdk.services.s3.model.ListObjectsV2Response;
import software.amazon.awssdk.services.s3.model.S3Object;
import software.amazon.awssdk.services.s3.model.GetObjectRequest;
import software.amazon.awssdk.services.s3.model.DeleteObjectRequest;
import software.amazon.awssdk.services.s3.model.DeleteBucketRequest;
import software.amazon.awssdk.services.s3.model.CreateMultipartUploadRequest;
import software.amazon.awssdk.services.s3.model.CreateMultipartUploadResponse;
import software.amazon.awssdk.services.s3.model.CompletedMultipartUpload;
import software.amazon.awssdk.services.s3.model.CreateBucketRequest;
import software.amazon.awssdk.services.s3.model.CompletedPart;
import software.amazon.awssdk.services.s3.model.CreateBucketConfiguration;
import software.amazon.awssdk.services.s3.model.UploadPartRequest;
import software.amazon.awssdk.services.s3.model.CompleteMultipartUploadRequest;
import software.amazon.awssdk.services.s3.waiters.S3Waiter;
import software.amazon.awssdk.services.s3.model.HeadBucketRequest;
import software.amazon.awssdk.services.s3.model.HeadBucketResponse;
```
```
ListObjectsV2Request listReq = ListObjectsV2Request.builder()
.bucket(bucketName)
.maxKeys(1)
.build();
ListObjectsV2Iterable listRes = s3.listObjectsV2Paginator(listReq);
// Process response pages
listRes.stream()
.flatMap(r -> r.contents().stream())
.forEach(content -> System.out
.println(" Key: " + content.key() + " size = " + content.size()));
```
See the [complete example](https://github.com/awsdocs/aws-doc-sdk-examples/blob/f807d60010caf3d14fe4cd0801b842fb8e9511ca/javav2/example_code/s3/src/main/java/com/example/s3/S3ObjectOperations.java#L112) on GitHub.
### Iterate over objects
The following examples show ways to iterate over the objects returned in the response instead of the pages of the response. The `contents` method of `ListObjectsV2Iterable` class returns an [https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/core/pagination/sync/SdkIterable.html](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/core/pagination/sync/SdkIterable.html) that provides several methods to process the underlying content elements.
#### Use a stream
The following snippet uses the `stream` method on the response content to iterate over the paginated item collection.
```
// Helper method to work with paginated collection of items directly.
listRes.contents().stream()
.forEach(content -> System.out
.println(" Key: " + content.key() + " size = " + content.size()));
```
See the [complete example](https://github.com/awsdocs/aws-doc-sdk-examples/blob/f807d60010caf3d14fe4cd0801b842fb8e9511ca/javav2/example_code/s3/src/main/java/com/example/s3/S3ObjectOperations.java#L127) on GitHub.
#### Use a for-each loop
Since `SdkIterable` extends the `Iterable` interface, you can process the contents like any `Iterable`. The following snippet uses standard `for-each` loop to iterate through the contents of the response.
```
for (S3Object content : listRes.contents()) {
System.out.println(" Key: " + content.key() + " size = " + content.size());
}
```
See the [complete example](https://github.com/awsdocs/aws-doc-sdk-examples/blob/f807d60010caf3d14fe4cd0801b842fb8e9511ca/javav2/example_code/s3/src/main/java/com/example/s3/S3ObjectOperations.java#L133) on GitHub.
### Manual pagination
If your use case requires it, manual pagination is still available. Use the next token in the response object for the subsequent requests. The following example uses a `while` loop.
```
ListObjectsV2Request listObjectsReqManual = ListObjectsV2Request.builder()
.bucket(bucketName)
.maxKeys(1)
.build();
boolean done = false;
while (!done) {
ListObjectsV2Response listObjResponse = s3.listObjectsV2(listObjectsReqManual);
for (S3Object content : listObjResponse.contents()) {
System.out.println(content.key());
}
if (listObjResponse.nextContinuationToken() == null) {
done = true;
}
listObjectsReqManual = listObjectsReqManual.toBuilder()
.continuationToken(listObjResponse.nextContinuationToken())
.build();
}
```
See the [complete example](https://github.com/awsdocs/aws-doc-sdk-examples/blob/f807d60010caf3d14fe4cd0801b842fb8e9511ca/javav2/example_code/s3/src/main/java/com/example/s3/S3ObjectOperations.java#L90) on GitHub.
## Asynchronous pagination
The following examples demonstrate asynchronous pagination methods to list DynamoDB tables.
### Iterate over pages of table names
The following two examples use an asynchronous DynamoDB client that call the `listTablesPaginator` method with a request to get a [https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/dynamodb/paginators/ListTablesPublisher.html](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/services/dynamodb/paginators/ListTablesPublisher.html). `ListTablesPublisher` implements two interfaces, which provides many options to process responses. We'll look at methods of each interface.
#### Use a `Subscriber`
The following code example demonstrates how to process paginated results by using the `org.reactivestreams.Publisher` interface implemented by `ListTablesPublisher`. To learn more about the reactive streams model, see the [Reactive Streams GitHub repo](https://github.com/reactive-streams/reactive-streams-jvm/blob/v1.0.2/README.md).
The following imports apply to all examples in this asynchronous pagination section.
##### Imports
```
import io.reactivex.rxjava3.core.Flowable;
import org.reactivestreams.Subscriber;
import org.reactivestreams.Subscription;
import reactor.core.publisher.Flux;
import software.amazon.awssdk.core.async.SdkPublisher;
import software.amazon.awssdk.services.dynamodb.DynamoDbAsyncClient;
import software.amazon.awssdk.services.dynamodb.model.ListTablesRequest;
import software.amazon.awssdk.services.dynamodb.model.ListTablesResponse;
import software.amazon.awssdk.services.dynamodb.paginators.ListTablesPublisher;
import java.util.List;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ExecutionException;
```
The following code acquires a `ListTablesPublisher` instance.
```
// Creates a default client with credentials and region loaded from the
// environment.
final DynamoDbAsyncClient asyncClient = DynamoDbAsyncClient.create();
ListTablesRequest listTablesRequest = ListTablesRequest.builder().limit(3).build();
ListTablesPublisher publisher = asyncClient.listTablesPaginator(listTablesRequest);
```
The following code uses an anonymous implementation of `org.reactivestreams.Subscriber` to process the results for each page.
The `onSubscribe` method calls the `Subscription.request` method to initiate requests for data from the publisher. This method must be called to start getting data from the publisher.
The subscriber's `onNext` method processes a response page by accessing all the table names and printing out each one. After the page is processed, another page is requested from the publisher. This method that is called repeatedly until all pages are retrieved.
The `onError` method is triggered if an error occurs while retrieving data. Finally, the `onComplete` method is called when all pages have been requested.
```
// A Subscription represents a one-to-one life-cycle of a Subscriber subscribing
// to a Publisher.
publisher.subscribe(new Subscriber() {
// Maintain a reference to the subscription object, which is required to request
// data from the publisher.
private Subscription subscription;
@Override
public void onSubscribe(Subscription s) {
subscription = s;
// Request method should be called to demand data. Here we request a single
// page.
subscription.request(1);
}
@Override
public void onNext(ListTablesResponse response) {
response.tableNames().forEach(System.out::println);
// After you process the current page, call the request method to signal that
// you are ready for next page.
subscription.request(1);
}
@Override
public void onError(Throwable t) {
// Called when an error has occurred while processing the requests.
}
@Override
public void onComplete() {
// This indicates all the results are delivered and there are no more pages
// left.
}
});
```
See the [complete example](https://github.com/awsdocs/aws-doc-sdk-examples/blob/f807d60010caf3d14fe4cd0801b842fb8e9511ca/javav2/example_code/dynamodbasync/src/main/java/com/example/dynamodbasync/AsyncPagination.java#L83) on GitHub.
#### Use a `Consumer`
The `SdkPublisher` interface that `ListTablesPublisher` implements has a `subscribe` method that takes a `Consumer` and returns a `CompletableFuture`.
The `subscribe` method from this interface can be used for simple use cases when an `org.reactivestreams.Subscriber` might be too much overhead. As the code below consumes each page, it calls the `tableNames` method on each. The `tableNames` method returns a `java.util.List` of DynamoDB table names that are processed with the `forEach` method.
```
// Use a Consumer for simple use cases.
CompletableFuture future = publisher.subscribe(
response -> response.tableNames()
.forEach(System.out::println));
```
See the [complete example](https://github.com/awsdocs/aws-doc-sdk-examples/blob/f807d60010caf3d14fe4cd0801b842fb8e9511ca/javav2/example_code/dynamodbasync/src/main/java/com/example/dynamodbasync/AsyncPagination.java#L96) on GitHub.
### Iterate over table names
The following examples show ways to iterate over the objects returned in the response instead of the pages of the response. Similar to the synchronous Amazon S3 example previously shown with its `contents` method, the DynamoDB asynchronous result class, `ListTablesPublisher` has the `tableNames` convenience method to interact with the underlying item collection. The return type of the `tableNames` method is an [https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/core/async/SdkPublisher.html](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/core/async/SdkPublisher.html) that can be used to request items across all pages.
#### Use a `Subscriber`
The following code acquires an `SdkPublisher` of the underlying collection of table names.
```
// Create a default client with credentials and region loaded from the
// environment.
final DynamoDbAsyncClient asyncClient = DynamoDbAsyncClient.create();
ListTablesRequest listTablesRequest = ListTablesRequest.builder().limit(3).build();
ListTablesPublisher listTablesPublisher = asyncClient.listTablesPaginator(listTablesRequest);
SdkPublisher publisher = listTablesPublisher.tableNames();
```
The following code uses an anonymous implementation of `org.reactivestreams.Subscriber` to process the results for each page.
The subscriber's `onNext` method processes an individual element of the collection. In this case, it's a table name. After the table name is processed, another table name is requested from the publisher. This method that is called repeatedly until all table names are retrieved.
```
// Use a Subscriber.
publisher.subscribe(new Subscriber() {
private Subscription subscription;
@Override
public void onSubscribe(Subscription s) {
subscription = s;
subscription.request(1);
}
@Override
public void onNext(String tableName) {
System.out.println(tableName);
subscription.request(1);
}
@Override
public void onError(Throwable t) {
}
@Override
public void onComplete() {
}
});
```
See the [complete example](https://github.com/awsdocs/aws-doc-sdk-examples/blob/f807d60010caf3d14fe4cd0801b842fb8e9511ca/javav2/example_code/dynamodbasync/src/main/java/com/example/dynamodbasync/AsyncPagination.java#L147) on GitHub.
#### Use a `Consumer`
The following example uses the `subscribe` method of `SdkPublisher` that takes a `Consumer` to process each item.
```
// Use a Consumer.
CompletableFuture future = publisher.subscribe(System.out::println);
future.get();
```
See the [complete example](https://github.com/awsdocs/aws-doc-sdk-examples/blob/f807d60010caf3d14fe4cd0801b842fb8e9511ca/javav2/example_code/dynamodbasync/src/main/java/com/example/dynamodbasync/AsyncPagination.java#L161) on GitHub.
### Use third-party library
You can use other third party libraries instead of implementing a custom subscriber. This example demonstrates the use of RxJava, but any library that implements the reactive stream interfaces can be used. See the [RxJava wiki page on GitHub](https://github.com/ReactiveX/RxJava/wiki) for more information on that library.
To use the library, add it as a dependency. If using Maven, the example shows the POM snippet to use.
**POM Entry**
```
io.reactivex.rxjava3
rxjava
3.1.6
```
**Code**
```
DynamoDbAsyncClient asyncClient = DynamoDbAsyncClient.create();
ListTablesPublisher publisher = asyncClient.listTablesPaginator(ListTablesRequest.builder()
.build());
// The Flowable class has many helper methods that work with
// an implementation of an org.reactivestreams.Publisher.
List tables = Flowable.fromPublisher(publisher)
.flatMapIterable(ListTablesResponse::tableNames)
.toList()
.blockingGet();
System.out.println(tables);
```
See the [complete example](https://github.com/awsdocs/aws-doc-sdk-examples/blob/f807d60010caf3d14fe4cd0801b842fb8e9511ca/javav2/example_code/dynamodbasync/src/main/java/com/example/dynamodbasync/AsyncPagination.java#L198) on GitHub.
# Using waiters in the AWS SDK for Java 2.x
The waiters utility of the AWS SDK for Java 2.x enables you to validate that AWS resources are in a specified state before performing operations on those resources.
A *waiter* is an abstraction used to poll AWS resources, such as DynamoDB tables or Amazon S3 buckets, until a desired state is reached (or until a determination is made that the resource won’t ever reach the desired state). Instead of writing logic to continuously poll your AWS resources, which can be cumbersome and error-prone, you can use waiters to poll a resource and have your code continue to run after the resource is ready.
## Prerequisites
Before you can use waiters in a project with the AWS SDK for Java, you must complete the steps in [Setting up the AWS SDK for Java 2.x](setup.md).
You must also configure your project dependencies (for example, in your `pom.xml` or `build.gradle` file) to use version `2.15.0` or later of the AWS SDK for Java.
For example:
```
software.amazon.awssdk
bom
2.27.21
pom
import
```
## Using waiters
To instantiate a waiters object, first create a service client. Set the service client’s `waiter()` method as the value of the waiter object. Once the waiter instance exists, set its response options to execute the appropriate code.
### Synchronous programming
The following code snippet shows how to wait for a DynamoDB table to exist and be in an **ACTIVE** state.
```
DynamoDbClient dynamo = DynamoDbClient.create();
DynamoDbWaiter waiter = dynamo.waiter();
WaiterResponse waiterResponse =
waiter.waitUntilTableExists(r -> r.tableName("myTable"));
// print out the matched response with a tableStatus of ACTIVE
waiterResponse.matched().response().ifPresent(System.out::println);
```
### Asynchronous programming
The following code snippet shows how to wait for a DynamoDB table to no longer exist.
```
DynamoDbAsyncClient asyncDynamo = DynamoDbAsyncClient.create();
DynamoDbAsyncWaiter asyncWaiter = asyncDynamo.waiter();
CompletableFuture> waiterResponse =
asyncWaiter.waitUntilTableNotExists(r -> r.tableName("myTable"));
waiterResponse.whenComplete((r, t) -> {
if (t == null) {
// print out the matched ResourceNotFoundException
r.matched().exception().ifPresent(System.out::println);
}
}).join();
```
## Configure waiters
You can customize the configuration for a waiter by using the `overrideConfiguration()` on its builder. For some operations, you can apply a custom configuration when you make the request.
### Configure a waiter
The following code snippet shows how to override the configuration on a waiter.
```
// sync
DynamoDbWaiter waiter =
DynamoDbWaiter.builder()
.overrideConfiguration(b -> b.maxAttempts(10))
.client(dynamoDbClient)
.build();
// async
DynamoDbAsyncWaiter asyncWaiter =
DynamoDbAsyncWaiter.builder()
.client(dynamoDbAsyncClient)
.overrideConfiguration(o -> o.backoffStrategy(
FixedDelayBackoffStrategy.create(Duration.ofSeconds(2))))
.scheduledExecutorService(Executors.newScheduledThreadPool(3))
.build();
```
### Override configuration for a specific request
The following code snippet shows how to override the configuration for a waiter on a per-request basis. Note that only some operations have customizable configurations.
```
waiter.waitUntilTableNotExists(b -> b.tableName("myTable"),
o -> o.maxAttempts(10));
asyncWaiter.waitUntilTableExists(b -> b.tableName("myTable"),
o -> o.waitTimeout(Duration.ofMinutes(1)));
```
## Code examples
For a complete example using waiters with DynamoDB, see [CreateTable.java](https://github.com/awsdocs/aws-doc-sdk-examples/blob/869b7ddbc7c8f66c7c45acd5b813429aff37003e/javav2/example_code/dynamodb/src/main/java/com/example/dynamodb/CreateTable.java) in the AWS Code Examples Repository.
For a complete example using waiters with Amazon S3, see [S3BucketOps.java](https://github.com/awsdocs/aws-doc-sdk-examples/blob/869b7ddbc7c8f66c7c45acd5b813429aff37003e/javav2/example_code/s3/src/main/java/com/example/s3/S3BucketOps.java) in the AWS Code Examples Repository.
# Troubleshooting FAQs
As you use the AWS SDK for Java 2.x in your applications, you might encounter the runtime errors listed in this topic. Use the suggestions here to help you uncover the root cause and resolve the error.
## How do I fix "`java.net.SocketException`: Connection reset" or "server failed to complete the response" error?
A connection reset error indicates that your host, the AWS service, or any intermediary party (for example, a NAT gateway, a proxy, a load balancer) closed the connection before the request was complete. Because there are many causes, finding a solution requires that you know why the connection is being closed. The following items commonly cause a connection to be closed.
+ **The connection is inactive. **This is common for streaming operations, where data is not being written to or from the wire for a period of time, so an intermediary party detects the connection as dead and closes it. To prevent this, be sure your application actively downloads or uploads data.
+ **You've closed the HTTP or SDK client.** Be sure not to close resources while they are in use.
+ **A misconfigured proxy.** Try to bypass any proxies that you've configured to see if it fixes the problem. If this fixes the issue, the proxy is closing your connection for some reason. Research your specific proxy to determine why it's closing the connection.
If you cannot identify the problem, try running a TCP dump for an affected connection at the client edge of your network (for example, after any proxies that you control).
If you see that the AWS endpoint is sending a `TCP RST` (reset), [contact the affected service](https://aws.amazon.com/contact-us/) to see if they can determine why the reset is occurring. Be prepared to provide request IDs and timestamps of when the issue occurred. The AWS support team might also benefit from [wire logs](logging-slf4j.md#sdk-java-logging-verbose) that show exactly what bytes your application is sending and receiving and when.
## How do I fix "connection timeout"?
A connection timeout error indicates that your host, the AWS service, or any intermediary party (for example, a NAT gateway, a proxy, a load balancer) failed to establish a new connection with the server within the configured connection timeout. The following items describe common causes of this issue.
+ **The configured connection timeout is too low.** By default, the connection timeout is 2 seconds in the AWS SDK for Java 2.x. If you set the connection timeout too low, you may get this error. The recommended connection timeout is 1 second if you make only in-region calls and 3 seconds if you make cross-region requests.
+ **A misconfigured proxy.** Try to bypass any proxies that you configured to see if it fixes the problem. If this fixes the issue, the proxy is the reason for the connection timeout. Research your specific proxy to determine why that is happening
If you cannot identify the problem, try running a TCP dump for an affected connection at the client edge of your network (for example, after any proxies that you control) to investigate any network issue.
## How do I fix "`java.net.SocketTimeoutException`: Read timed out"?
A read timed out error indicates that the JVM attempted to read data from the underlying operating system, but data was not returned within the time configured via the SDK. This error can occur if the operating system, the AWS service, or any intermediary party (for example, a NAT gateway, a proxy, a load balancers) fails to send data within the time expected by the JVM. Because there are many causes, finding a solution requires that you know why the data is not being returned.
Try running a TCP dump for an affected connection at the client edge of your network (for example, after any proxies that you control).
If you see that the AWS endpoint is sending a `TCP RST` (reset), [contact the affected service](https://aws.amazon.com/contact-us/). Be prepared to provide request IDs and timestamps of when the issue occurred. The AWS support team might also benefit from [wire logs](logging-slf4j.md#sdk-java-logging-verbose) that show exactly what bytes your application is sending and receiving and when.
## How do I fix "Unable to execute HTTP request: Timeout waiting for connection from pool" error?
This error indicates that a request cannot get a connection from the pool within the specified maximum time. To troubleshoot the issue, we recommend that you [enable SDK client-side metrics](metrics.md) to publish metrics to Amazon CloudWatch. The HTTP metrics can help narrow down the root cause. The following items describe common causes of this error.
+ **Connection leak.** You can investigate this by checking `LeasedConcurrency` , `AvailableConcurrency`, and `MaxConcurrency` metrics. If `LeasedConcurrency` increases until it reaches `MaxConcurrency` but never decreases, there may be a connection leak. A common cause of a leak is because a streaming operation—such as a S3 `getObject` method—is not closed. We recommend that your application read all data from the input stream as soon as possible and [close the input stream afterwards](best-practices.md#bestpractice2). The following chart shows what SDK metrics might look like for connection leak.
![\[A screenshot of CloudWatch metrics that show a likely connection leak.\]](http://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/images/JavaDevGuide-connection-leak-metrics-chart.png)
+ **Connection pool starvation. **This can happen if your request rate is too high and the connection pool size that has been configured cannot meet the request demand. The default connection pool size is 50, and when the connections in the pool reach the maximum, the HTTP client queues incoming requests until connections become available. The following chart shows what SDK metrics might look like for connection pool starvation.
![\[A screenshot of CloudWatch metrics that shows how connection pool starvation might look like.\]](http://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/images/JavaDevGuide-connection-pool-starvation-chart.png)
To mitigate this issue, consider taking any of the following actions.
+ Increase the connection pool size,
+ Increase acquire timeout.
+ Slow the request rate.
By increasing the maximum number of connections, client throughput can increase (unless the network interface is already fully utilized). However, you can eventually hit operation system limitations on the number of file descriptors used by the process. If you already fully use your network interface or cannot further increase your connection count, try increasing the acquire timeout. With the increase, you gain extra time for requests to acquire a connection before timing out. If the connections don't free up, the subsequent requests will still timeout.
If you are unable to fix the issue by using the first two mechanisms, slow the request rate by trying the following options.
+ Smooth out your requests so that large traffic bursts don't overload the client.
+ Be more efficient with calls to AWS services.
+ Increase the number of hosts sending requests.
+ **I/O Threads are too busy.** This only applies if you are using an asynchronous SDK client with [https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/http/nio/netty/NettyNioAsyncHttpClient.html](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/http/nio/netty/NettyNioAsyncHttpClient.html). If the `AvailableConcurrency` metric is not low—indicating that connections are available in the pool—but `ConcurrencyAcquireDuration` is high, it might be because I/O threads are not able to handle the requests. Be sure you are not passing `Runnable:run` as a [future completion executor](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/core/client/config/SdkAdvancedAsyncClientOption.html#FUTURE_COMPLETION_EXECUTOR) and performing time-consuming task in the response future completion chain since this can block an I/O thread. If that is not the case, consider increasing the number of I/O threads by using the `[eventLoopGroupBuilder](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/http/nio/netty/NettyNioAsyncHttpClient.Builder.html#eventLoopGroupBuilder(software.amazon.awssdk.http.nio.netty.SdkEventLoopGroup.Builder))` method. For reference, the default number of I/O threads for a `NettyNioAsyncHttpClient` instance is twice the number of CPU cores of the host.
+ **High TLS handshake latency.** If your `AvailableConcurrency` metric is near 0 and `LeasedConcurrency` is lower than `MaxConcurrency`, it might be because the TLS handshake latency is high. The following chart shows what SDK metrics might look like for high TLS handshake latency.
![\[A screenshot of CloudWatch metrics that might indicate high TLS handshake latency.\]](http://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/images/JavaDevGuide-high-tls-latency-chart.png)
For HTTP clients offered by the Java SDK that are not based on CRT, try enabling [TLS logs](security-java-tls.md) to troubleshoot TLS issues. For the AWS CRT-based HTTP client, try enabling [AWS CRT logs](logging-slf4j.md#sdk-java-logging-verbose). If you see that the AWS endpoint seems to take a long time to perform a TLS handshake, you should [contact the affected service](https://aws.amazon.com/contact-us/).
## How do I fix a `NoClassDefFoundError`, `NoSuchMethodError` or `NoSuchFieldError`?
A `NoClassDefFoundError` indicates that a class could not be loaded at runtime. The two most common causes for this error are:
+ the class does not exist in the classpath because the JAR is missing or the wrong version of the JAR is on the classpath.
+ the class failed to load because its static initializer threw an exception.
Similarly, `NoSuchMethodError`s and `NoSuchFieldError`s typically result from a mismatched JAR version. We recommend that you perform the following steps.
1. **Check your dependencies** to make sure that you're using the *same version of all SDK jars*. The most common reason that a class, method, or field cannot be found is when you upgrade to a new client version but you continue to use an old 'shared' SDK dependency version. The new client version might attempt to use classes that exist only in newer 'shared' SDK dependencies. Try running `mvn dependency:tree` or `gradle dependencies` (for Gradle) to verify that the SDK library versions all match. To avoid this issue completely in the future, we recommend using [BOM (Bill of Materials)](setup-project-maven.md#sdk-as-dependency) to manage SDK module versions.
The following example shows you an example of mixed SDK versions.
```
[INFO] +- software.amazon.awssdk:dynamodb:jar:2.20.00:compile
[INFO] | +- software.amazon.awssdk:aws-core:jar:2.13.19:compile
[INFO] +- software.amazon.awssdk:netty-nio-client:jar:2.20.00:compile
```
The version of `dynamodb` is 2.20.00 and the version of `aws-core` is 2.13.19. The `aws-core` artifact version should also be 2.20.00.
1. **Check statements early in your logs** to see if a class is failing to load because of a static initialization failure. The first time the class fails to load, it may throw a different, more useful exception that specifies *why* the class cannot be loaded. This potentially useful exception occurs only once, so later log statements will only report that the class is not found.
1. **Check your deployment process** to make sure that it actually deploys required JAR files along with your application. It's possible that you're building with the correct version, but the process that creates the classpath for your application is excluding a required dependency.
## How do I fix a "`SignatureDoesNotMatch`" error or "The request signature we calculated does not match the signature you provided" error?
A `SignatureDoesNotMatch` error indicates that the signature generated by the AWS SDK for Java and the signature generated by the AWS service do not match. The following items describe potential causes.
+ A proxy or intermediary party modifies the request. For example, a proxy or load balancer might modify a header, path or query string that was signed by the SDK.
+ The service and SDK differ in the way they encode the request when each generates the string to sign.
To debug this issue, we recommend that you [enable debug logging](logging-slf4j.md#sdk-debug-level-logging) for the SDK. Try to reproduce the error and find the canonical request that the SDK generated. In the log, the canonical request is labeled with `AWS4 Canonical Request: ...` and the string to sign is labeled `AWS4 String to sign: ...` .
If you cannot enable debugging—for example, because it's only reproducible in production—add logic to your application that logs information about the request when the error occurs. You can then use that information to try to replicate the error outside of production in an integration test with debug logging enabled.
After you have collected the canonical request and string to sign, compare them against the [AWS Signature Version 4 specification](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_aws-signing.html) to determine if there are any issues in the way the SDK generated the string to sign. If something seems wrong, you can create a [GitHub bug report](https://github.com/aws/aws-sdk-java-v2/issues/new/choose) to the AWS SDK for Java.
If nothing appears wrong, you can compare the SDK's string to sign with the string to sign that some AWS services return as part of the failure response (Amazon S3, for example) . If this isn't available, you should [contact the affected service](https://aws.amazon.com/contact-us/) to see what canonical request and string to sign they generated for comparison. These comparisons can help to identify intermediary parties that might have modified the request or encoding differences between the service and client.
For more background information about signing requests, see [Signing AWS API requests](https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_aws-signing.html) in the AWS Identity and Access Management User Guide.
**Example of a canonical request**
```
PUT
/Example-Bucket/Example-Object
partNumber=19&uploadId=string
amz-sdk-invocation-id:f8c2799d-367c-f024-e8fa-6ad6d0a1afb9
amz-sdk-request:attempt=1; max=4
content-encoding:aws-chunked
content-length:51
content-type:application/octet-stream
host:xxxxx
x-amz-content-sha256:STREAMING-UNSIGNED-PAYLOAD-TRAILER
x-amz-date:20240308T034733Z
x-amz-decoded-content-length:10
x-amz-sdk-checksum-algorithm:CRC32
x-amz-trailer:x-amz-checksum-crc32
```
**Example of a string to sign**
```
AWS4-HMAC-SHA256
20240308T034435Z
20240308/us-east-1/s3/aws4_request
5f20a7604b1ef65dd89c333fd66736fdef9578d11a4f5d22d289597c387dc713
```
## How do I fix "`java.lang.IllegalStateException`: Connection pool shut down" error?
This error indicates the underlying Apache HTTP connection pool was closed. The following items describe potential causes.
+ **The SDK client was closed prematurely. **The SDK only closes the connection pool when the associated client is closed. Be sure not to close resources while they are in use.
+ **A `java.lang.Error` was thrown.** Errors such as `OutOfMemoryError` cause an Apache HTTP connection pool to [shut down](https://github.com/apache/httpcomponents-client/blob/6a741b4f8f23e6c5c7cc42c36c2acabfac19c3d6/httpclient/src/main/java/org/apache/http/impl/execchain/MainClientExec.java#L368). Examine your logs for error stack traces. Also review your code for places where it catches `Throwable`s or `Error`s but swallows the output that prevents the error from surfacing. If your code does not report errors, rewrite the code so information is logged. The logged information helps determine the root cause of the error.
+ **You attempted to use the credentials provider returned from `DefaultCredentialsProvider#create()` after it was closed**. [https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/auth/credentials/DefaultCredentialsProvider.html#create()](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/auth/credentials/DefaultCredentialsProvider.html#create()) returns a singleton instance, so if it's closed and your code calls the `resolveCredentials` method, the exception is thrown after cached credentials (or token) expire.
Check your code for places where the `DefaultCredentialsProvider` is closed, as shown in the following examples.
+ The singleton instance is closed by calling `DefaultCredentialsProvider#close().`
```
DefaultCredentialsProvider defaultCredentialsProvider = DefaultCredentialsProvider.create(); // Singleton instance returned.
AwsCredentials credentials = defaultCredentialsProvider.resolveCredentials();
// Make calls to AWS services.
defaultCredentialsProvider.close(); // Explicit close.
// Make calls to AWS services.
// After the credentials expire, either of the following calls eventually results in a "Connection pool shut down" exception.
credentials = defaultCredentialsProvider.resolveCredentials();
// Or
credentials = DefaultCredentialsProvider.create().resolveCredentials();
```
+ Invoke `DefaultCredentialsProvider#create()` in a try-with-resources block.
```
try (DefaultCredentialsProvider defaultCredentialsProvider = DefaultCredentialsProvider.create()) {
AwsCredentials credentials = defaultCredentialsProvider.resolveCredentials();
// Make calls to AWS services.
} // After the try-with-resources block exits, the singleton DefaultCredentialsProvider is closed.
// Make calls to AWS services.
DefaultCredentialsProvider defaultCredentialsProvider = DefaultCredentialsProvider.create(); // The closed singleton instance is returned.
// If the credentials (or token) has expired, the following call results in the error.
AwsCredentials credentials = defaultCredentialsProvider.resolveCredentials();
```
Create a new, non-singleton instance by calling `DefaultCredentialsProvider.builder().build()` if your code has closed the singleton instance and you need to resolve credentials by using a `DefaultCredentialsProvider`.
## How do I fix "Unable to load credentials from any of the providers in the chain AwsCredentialsProviderChain"?
This error indicates that the AWS SDK for Java 2.x could not find valid AWS credentials through any of the credential providers in the default credential provider chain. The SDK automatically searches for credentials in a specific order, and this error occurs when all providers in the chain fail to provide valid credentials.
The full error message typically looks like this (line endings and indents added to improve readability):
```
Unable to load credentials from any of the providers in the chain AwsCredentialsProviderChain(
credentialsProviders=[
SystemPropertyCredentialsProvider(),
EnvironmentVariableCredentialsProvider(),
WebIdentityTokenCredentialsProvider(),
ProfileCredentialsProvider(profileName=default, profileFile=ProfileFile(sections=[])),
ContainerCredentialsProvider(),
InstanceProfileCredentialsProvider()
]) : [
SystemPropertyCredentialsProvider(): Unable to load credentials from system settings.
Access key must be specified either via environment variable (AWS_ACCESS_KEY_ID)
or system property (aws.accessKeyId).,
EnvironmentVariableCredentialsProvider(): Unable to load credentials from system settings.
Access key must be specified either via environment variable (AWS_ACCESS_KEY_ID)
or system property (aws.accessKeyId).,
WebIdentityTokenCredentialsProvider(): To use web identity tokens, the 'sts' service module
must be on the class path.,
ProfileCredentialsProvider(profileName=default, profileFile=ProfileFile(sections=[])):
Profile file contained no credentials for profile 'default': ProfileFile(sections=[]),
ContainerCredentialsProvider(): Cannot fetch credentials from container - neither
AWS_CONTAINER_CREDENTIALS_FULL_URI or AWS_CONTAINER_CREDENTIALS_RELATIVE_URI environment
variables are set.,
InstanceProfileCredentialsProvider(): Failed to load credentials from IMDS.]
```
### Common causes and solutions
#### Review your credential configuration
When you use the default credentials provider (by calling `ServiceClient.create()` without explicitly configuring credentials), the SDK searches for credentials in a specific order. Review [how the default credential provider chain works](credentials-chain.md) to understand which credential sources the SDK checks and in what order.
Ensure that the credential configuration method you intend to use is properly set up in your environment:
##### For Amazon EC2 instances
+ **Check IAM role:** Verify that an IAM role is attached to your instance.
+ **Intermittent IMDS failures:** If you're experiencing intermittent failures (typically lasting a few hundred milliseconds), this usually indicates transient network issues reaching the Instance Metadata Service (IMDS).
Solutions:
+ Enable [debug logging](logging-slf4j.md#sdk-debug-level-logging) to analyze the timing and frequency of failures
+ Consider implementing retry logic in your application for credential-related failures
+ Check for network connectivity issues between your instance and the IMDS endpoint
##### For container environments
Confirm that task roles (Amazon ECS) or service accounts (Amazon EKS) are configured and that required environment variables are set.
##### For local development
Check that environment variables, credentials files, or IAM Identity Center configuration is in place.
##### For web identity federation
+ **Verify configuration: **Verify that the web identity token file exists and that the required environment variables are configured.
+ **Missing STS module dependency:** If you see the error `To use web identity tokens, the 'sts' service module must be on the class path`, you need to add the STS module as a dependency. This is common when using Amazon EKS Pod Identity or other web identity token authentication.
Solution: Add the STS module to your project dependencies:
+
```
software.amazon.awssdk
sts
```
For some services, you might also need the `aws-query-protocol` dependency:
```
software.amazon.awssdk
aws-query-protocol
```
#### Network or proxy connectivity issues
If you see `Connection refused` errors in the credential provider chain, this typically indicates network connectivity problems when the SDK tries to reach AWS endpoints.
**Solutions:**
+ Verify proxy configuration if you're using a proxy server
+ Check that your network allows outbound HTTPS connections to AWS endpoints
+ Enable [debug logging](logging-slf4j.md#sdk-debug-level-logging) to see detailed connection attempts
+ Test connectivity using tools like `curl` to verify network access to AWS endpoints
# Reduce SDK startup time for AWS Lambda
One of the goals of the AWS SDK for Java 2.x is to reduce the startup latency for AWS Lambda functions. The SDK contains changes that reduce startup time, which are discussed at the end of this topic.
First, this topic focuses on changes that you can make to reduce cold start times. These include making changes in your code structure and in the configuration of service clients.
## Use an AWS CRT-based HTTP client
For working with AWS Lambda, we recommend the [https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/http/crt/AwsCrtHttpClient.html](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/http/crt/AwsCrtHttpClient.html) for synchronous scenarios and the [https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/http/crt/AwsCrtAsyncHttpClient.html](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/http/crt/AwsCrtAsyncHttpClient.html) for asynchronous scenarios.
The [Configure AWS CRT-based HTTP clients](http-configuration-crt.md) topic in this guide describes the benefits of using the HTTP clients, how to add the dependency, and how configure their use by service clients.
## Remove unused HTTP client dependencies
Along with the explicit use of an AWS CRT-based client, you can remove other HTTP clients that the SDK brings in by default. Lambda startup time is reduced when fewer libraries need to be loaded, so you should remove any unused artifacts that the JVM needs to load.
The following snippet of a Maven `pom.xml` file shows the exclusion of the Apache-based HTTP client and the Netty-based HTTP client. (These clients aren't needed when you use an AWS CRT-based client.) This example excludes the HTTP client artifacts from the S3 client dependency and adds the `aws-crt-client` artifact to allow access to the AWS CRT-based HTTP clients.
```
2.27.21
software.amazon.awssdk
bom
${aws.java.sdk.version}
pom
import
software.amazon.awssdk
aws-crt-client
software.amazon.awssdk
s3
software.amazon.awssdk
netty-nio-client
software.amazon.awssdk
apache-client
```
**Note**
Add the `` element to all service client dependencies in your `pom.xml` file.
## Configure service clients to shortcut lookups
**Specify a region**
When you create a service client, call the `region` method on the service client builder. This shortcuts the SDK's default [Region lookup process](region-selection.md#default-region-provider-chain) that checks several places for the AWS Region information.
To keep the Lambda code independent of the region, use the following code inside the `region` method. This code accesses the `AWS_REGION` environment variable set by the Lambda container.
```
Region.of(System.getenv(SdkSystemSetting.AWS_REGION.environmentVariable()))
```
**Use the `EnvironmentVariableCredentialProvider`**
Much like the default lookup behavior for the Region information, the SDK looks in several places for credentials. By specifying the [https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/auth/credentials/EnvironmentVariableCredentialsProvider.html](https://sdk.amazonaws.com/java/api/latest/software/amazon/awssdk/auth/credentials/EnvironmentVariableCredentialsProvider.html) when you build a service client, you save time in the SDK's lookup process for credentials.
Using this credentials provider enables the code to be used in Lambda functions, but might not work on Amazon EC2 or other systems.
If you intend to use [Lambda SnapStart for Java](#lambda-quick-snapstart) at some point, you should rely on the default credentials provider chain to lookup credentials. If you specify the `EnvironmentVariableCredentialsProvider`, the initial credentials lookup works, but when SnapStart is activated, [the Java runtime sets container credentials environment variables](https://docs.aws.amazon.com/lambda/latest/dg/snapstart-activate.html#snapstart-credentials). On activation, the environment variables used by the `EnvironmentVariableCredentialsProvider`—access key environment variables—are not available to the Java SDK.
The following code snippet shows an S3 service client appropriately configured for use in a Lambda environment.
```
S3Client s3Client = S3Client.builder()
.region(Region.of(System.getenv(SdkSystemSetting.AWS_REGION.environmentVariable())))
.credentialsProvider(EnvironmentVariableCredentialsProvider.create())
.httpClient(AwsCrtHttpClient.builder().build())
.build();
```
## Initialize the SDK client outside of the Lambda function handler
We recommend initializing an SDK client outside of the Lambda handler method. This way, if the execution context is reused, the initialization of the service client can be skipped. By reusing the client instance and its connections, subsequent invocations of the handler method occur more quickly.
In the following example, the `S3Client` instance is initialized in the constructor using a static factory method. If the container that is managed by the Lambda environment is reused, the initialized `S3Client` instance is reused.
```
public class App implements RequestHandler {
private final S3Client s3Client;
public App() {
s3Client = DependencyFactory.s3Client();
}
@Override
public Object handle Request(final Object input, final Context context) {
ListBucketResponse response = s3Client.listBuckets();
// Process the response.
}
}
```
## Minimize dependency injection
Dependency injection (DI) frameworks might take additional time to complete the setup process. They might also require additional dependencies, which take time to load.
If a DI framework is needed, we recommend using lightweight DI frameworks such as [Dagger](https://dagger.dev/dev-guide/).
## Use a Maven Archetype targeting AWS Lambda
The AWS Java SDK team has developed a [Maven Archetype](https://github.com/aws/aws-sdk-java-v2/tree/master/archetypes/archetype-lambda) template to bootstrap a Lambda project with minimal startup time. You can build out a Maven project from the archetype and know that the dependencies are configured suitably for the Lambda environment.
To learn more about the archetype and work through an example deployment, see this [blog post](https://aws.amazon.com/blogs/developer/bootstrapping-a-java-lambda-application-with-minimal-aws-java-sdk-startup-time-using-maven/).
## Consider Lambda SnapStart for Java
If your runtime requirements are compatible, AWS offers [Lambda SnapStart for Java](https://docs.aws.amazon.com/lambda/latest/dg/snapstart.html). Lambda SnapStart is an infrastructure-based solution that improves startup performance for Java functions. When you publish a new version of a function, Lambda SnapStart initializes it and takes an immutable, encrypted snapshot of the memory and disk state. SnapStart then caches the snapshot for reuse.
## Version 2.x changes that affect startup time
In addition to changes that you make to your code, version 2.x of the SDK for Java includes three primary changes that reduce startup time:
+ Use of [jackson-jr](https://github.com/FasterXML/jackson-jr), which is a serialization library that improves initialization time
+ Use of the [java.time](https://docs.oracle.com/javase/8/docs/api/index.html?java/time.html) libraries for date and time objects, which is part of the JDK
+ Use of [Slf4j](https://www.slf4j.org/) for a logging facade
## Additional resources
The AWS Lambda Developer Guide contains a [section on best practices](https://docs.aws.amazon.com/lambda/latest/dg/best-practices.html) for developing Lambda functions that is not Java specific.
For an example of building a cloud-native application in Java that uses AWS Lambda, see this [workshop content](https://github.com/aws-samples/aws-lambda-java-workshop). The workshop discussion performance optimization and other best practices.
You can consider using static images that are compiled ahead of time to reduce startup latency. For example, you can use the SDK for Java 2.x and Maven to [build a GraalVM native image](setup-project-graalvm.md).
# Implement `ContentStreamProvider` in the AWS SDK for Java 2.x
`ContentStreamProvider` is an abstraction used in the AWS SDK for Java 2.x to allow multiple reads of input data. This topic explains how to implement a `ContentStreamProvider` correctly for your applications.
The SDK for Java 2.x uses the `ContentStreamProvider#newStream()` method each time it needs to read an entire stream. For this to work for the entire stream, the returned stream must always be at the start of the content and it must contain the same data.
In the following sections, we provide three approaches for how to implement this behavior correctly.
## Use `mark()` and `reset()`
In the example below, we use `mark(int)` in the constructor before reading begins to ensure that we can reset the stream back to the beginning. For each invocation of `newStream()` we reset the stream:
```
public class MyContentStreamProvider implements ContentStreamProvider {
private InputStream contentStream;
public MyContentStreamProvider(InputStream contentStream) {
this.contentStream = contentStream;
this.contentStream.mark(MAX_LEN);
}
@Override
public InputStream newStream() {
contentStream.reset();
return contentStream;
}
}
```
## Use buffering if `mark()` and `reset()` are not available
If your stream doesn't support `mark()` and `reset()` directly, you can still use the solution shown previously by first wrapping the stream in a `BufferedInputStream`:
```
public class MyContentStreamProvider implements ContentStreamProvider {
private BufferedReader contentStream;
public MyContentStreamProvider(InputStream contentStream) {
this.contentStream = new BufferedInputStream(contentStream);
this.contentStream.mark(MAX_LEN);
}
@Override
public InputStream newStream() {
contentStream.reset();
return contentStream;
}
}
```
## Create new streams
A simpler approach is to simply obtain a new stream to your data on each invocation and close the previous one:
```
public class MyContentStreamProvider implements ContentStreamProvider {
private InputStream contentStream;
@Override
public InputStream newStream() {
if (contentStream != null) {
contentStream.close();
}
contentStream = openStream();
return contentStream;
}
}
```
# Set the JVM TTL for DNS name lookups
The Java virtual machine (JVM) caches DNS name lookups. When the JVM resolves a hostname to an IP address, it caches the IP address for a specified period of time, known as the *time-to-live* (TTL).
Because AWS resources use DNS name entries that occasionally change, we recommend that you configure your JVM with a TTL value of 5 seconds. This ensures that when a resource’s IP address changes, your application will be able to receive and use the resource’s new IP address by requerying the DNS.
On some Java configurations, the JVM default TTL is set so that it will *never* refresh DNS entries until the JVM is restarted. Thus, if the IP address for an AWS resource changes while your application is still running, it won’t be able to use that resource until you *manually restart* the JVM and the cached IP information is refreshed. In this case, it’s crucial to set the JVM’s TTL so that it will periodically refresh its cached IP information.
## How to set the JVM TTL
To modify the JVM’s TTL, set the [networkaddress.cache.ttl](https://docs.oracle.com/en/java/javase/17/core/java-networking.html#GUID-A680DADB-C4C1-40F1-B568-D9A97C917F5D) security property value. Note that `networkaddress.cache.ttl` is a *security property*, not a system property, i.e., it cannot be set with the `-D` command-line flag.
### Option 1: Set it programmatically in your application
Call [https://docs.oracle.com/en/java/javase/21/docs/api/java.base/java/security/Security.html](https://docs.oracle.com/en/java/javase/21/docs/api/java.base/java/security/Security.html) early in your application startup, before any AWS SDK clients are created and before any network requests are made:
```
import java.security.Security;
public class MyApplication {
public static void main(String[] args) {
Security.setProperty("networkaddress.cache.ttl", "5");
// ... create SDK clients and run application
}
}
```
### Option 2: Set it in the java.security file
Set the `networkaddress.cache.ttl` property in the `$JAVA_HOME/jre/lib/security/java.security` file for Java 8 or `$JAVA_HOME/conf/security/java.security` file for Java 11 or higher.
The following is a snippet from a `java.security` file that shows the TTL cache set to 5 seconds.
```
#
# The Java-level namelookup cache policy for successful lookups:
#
# any negative value: caching forever
# any positive value: the number of seconds to cache an address for
# zero: do not cache
#
...
networkaddress.cache.ttl=5
...
```
All applications that run on the JVM represented by the `$JAVA_HOME` environment variable use this setting.
### Option 3: Use the JDK system properties fallback (command-line)
If you cannot modify the security configuration or code, you can use JDK system properties. These act as fallbacks if no security property is defined.
+ `sun.net.inetaddr.ttl` – Controls successful lookups (positive TTL)
+ `sun.net.inetaddr.negative.ttl` – Controls failed lookups (negative TTL)
```
java -Dsun.net.inetaddr.ttl=5 -Dsun.net.inetaddr.negative.ttl=1 -jar myapp.jar
```
**Note**
These are JDK-internal properties documented in the [Oracle Java 8 Networking Properties](https://docs.oracle.com/javase/8/docs/technotes/guides/net/properties.html) reference as private properties that "may not be supported in future releases". Use Options 1-2 when possible.
# Work with HTTP/2 in the AWS SDK for Java
HTTP/2 is a major revision of the HTTP protocol. This new version has several enhancements to improve performance:
+ Binary data encoding provides more efficient data transfer.
+ Header compression reduces the overhead bytes downloaded by the client, helping get the content to the client sooner. This is especially useful for mobile clients that are already constrained on bandwidth.
+ Bidirectional asynchronous communication (multiplexing) allows multiple requests and response messages between the client and AWS to be in flight at the same time over a single connection, instead of over multiple connections, which improves performance.
Developers upgrading to the latest SDKs will automatically use HTTP/2 when it’s supported by the service they’re working with. New programming interfaces seamlessly take advantage of HTTP/2 features and provide new ways to build applications.
The AWS SDK for Java 2.x features new APIs for event streaming that implement the HTTP/2 protocol. For examples of how to use these new APIs, see [Working with Kinesis](examples-kinesis.md).