Configure AWS IoT SiteWise Buffered using Amazon S3 - AWS IoT SiteWise

Configure AWS IoT SiteWise Buffered using Amazon S3

Configure the AWS CLI to use AWS IoT SiteWise Buffered using Amazon S3 when you need to store data locally before uploading it to AWS IoT SiteWise. This buffered mode is particularly useful in scenarios with intermittent or unreliable network connectivity. By using Amazon S3 as a buffer, you can ensure data continuity even when the connection to AWS IoT SiteWise is unavailable.

To set up AWS IoT SiteWise Buffered using Amazon S3:

  1. Create an Amazon S3 bucket to serve as your data buffer.

  2. Configure your AWS IoT SiteWise data source to use the StreamManager destination type.

  3. Set up the AWS IoT SiteWise publisher component to map the stream to your Amazon S3 bucket.

  4. Ensure that your IAM roles have the necessary permissions to interact with both AWS IoT SiteWise and Amazon S3.

When configuring your data source and publisher, it's important to use consistent stream names. The stream name specified in your OPC UA capability configuration must match the one in your publisher capability configuration.

This buffered mode offers several advantages:

  • Data resilience – Your data is stored in Amazon S3 when a connection to AWS IoT SiteWise is unavailable.

  • Bandwidth optimization – You can batch upload data, reducing the number of network requests.

  • Cost efficiency – Amazon S3 storage can be more cost-effective for large volumes of data.

In the following sections, we'll walk through the detailed configuration steps for both the OPC UA data source and the publisher component.

See Configure an OPC UA source (CLI) for the data source configuration model. In the data source configuration, the destination configuration determines where the incoming data values are transmitted to.

A pre-determined stream name is not required for this. However, the name of the stream must be identical in the configurations defined in Configure an OPC UA source (CLI) and Configure the AWS IoT SiteWise publisher component.

Example in the OPC UA capability configuration
...
  "destination": {
        "type": "StreamManager", 
        "streamName": "My_Unique_Stream_Name", 
        "streamBufferSize": integer 
    }, 
...
Example in the Publisher capability configuration
...
  "SiteWiseS3PublisherConfiguration": {
        ...
        "streamToS3ConfigMapping": [{
            "streamName": "My_Unique_Stream_Name",
            ...
        }] 
    }
...