# Building solutions with the AWS IoT Device SDKs
The tutorials in this section help walk you through the steps to develop an IoT solution that can be deployed to a production environment using AWS IoT.
These tutorials can take more time to complete than those in the section on [Building demos with the AWS IoT Device Client](iot-tutorials-dc-intro.md) because they use the AWS IoT Device SDKs and explain the concepts being applied in more detail to help you create secure and reliable solutions.
## Start building solutions with the AWS IoT Device SDKs
These tutorials walk you through different AWS IoT scenarios. Where appropriate, the tutorials use the AWS IoT Device SDKs.
**Topics**
+ [Start building solutions with the AWS IoT Device SDKs](#iot-sdk-tutorial-overview)
+ [Tutorial: Connecting a device to AWS IoT Core by using the AWS IoT Device SDK](sdk-tutorials.md)
+ [Creating AWS IoT rules to route device data to other services](iot-rules-tutorial.md)
+ [Retaining device state while the device is offline with Device Shadows](iot-shadows-tutorial.md)
+ [Tutorial: Creating a custom authorizer for AWS IoT Core](custom-auth-tutorial.md)
+ [Tutorial: Monitoring soil moisture with AWS IoT and Raspberry Pi](iot-moisture-tutorial.md)
# Tutorial: Connecting a device to AWS IoT Core by using the AWS IoT Device SDK
This tutorial demonstrates how to connect a device to AWS IoT Core so that it can send and receive data to and from AWS IoT. After you complete this tutorial, your device will be configured to connect to AWS IoT Core and you'll understand how devices communicate with AWS IoT.
**Topics**
+ [Pre-requisites](#sdk-tutorials-prereq)
+ [Prepare your device for AWS IoT](#sdk-tutorials-prepare)
+ [Review the MQTT protocol](#sdk-tutorials-mqtt-review)
+ [Review the pubsub.py Device SDK sample app](#sdk-tutorials-explore-sample)
+ [Connect your device and communicate with AWS IoT Core](#sdk-tutorials-experiment)
+ [Review the results](#sdk-tutorials-conclusion)
+ [Tutorial: Using the AWS IoT Device SDK for Embedded C](iot-embedded-c-sdk.md)
## Pre-requisites
Before you start this tutorial, make sure that you have:
+
**Completed [Getting started with AWS IoT Core tutorials](iot-gs.md)**
In the section of that tutorial where you must [Configure your device](configure-device.md), select the [Connect a Raspberry Pi or other device](connecting-to-existing-device.md) option for your device and use the Python language options to configure your device.
**Note**
Keep open the terminal window you use in that tutorial because you'll also use it in this tutorial.
+
**A device that can run the AWS IoT Device SDK v2 for Python.**
This tutorial shows how to connect a device to AWS IoT Core by using Python code examples, which require a relatively powerful device. If you are working with resource-constrained devices, these code examples might not work on them. In that case, you might have more success with the [Tutorial: Using the AWS IoT Device SDK for Embedded C](iot-embedded-c-sdk.md) tutorial.
+
**Obtained the required information to connect to the device**
To connect your device to AWS IoT, you must have information about the thing name, the host name, and the port number.
**Note**
You can also use custom authentication to connect devices to AWS IoT Core. The connection data you pass to your authorizer Lambda function depends on the protocol you use.
+ **Thing name**: The name of the AWS IoT thing that you want to connect to. You must have registered as your device as an AWS IoT thing. For more information, see [Managing devices with AWS IoT](iot-thing-management.md).
+ **Host name**: The host name for the account-specific IoT endpoint.
+ **Port number**: The port number to connect to.
You can use the `configureEndpoint` method in the AWS IoT Python SDK to configure the host name and port number.
```
myAWSIoTMQTTClient.configureEndpoint("random.iot.region.amazonaws.com", 8883)
```
## Prepare your device for AWS IoT
In [Getting started with AWS IoT Core tutorials](iot-gs.md), you prepared your device and AWS account so they could communicate. This section reviews the aspects of that preparation that apply to any device connection with AWS IoT Core.
For a device to connect to AWS IoT Core:
1. You must have an **AWS account**.
The procedure in [Set up AWS account](setting-up.md) describes how to create an AWS account if you don’t already have one.
1. In that account, you must have the following **AWS IoT resources** defined for the device in your AWS account and Region.
The procedure in [Create AWS IoT resources](create-iot-resources.md) describes how to create these resources for the device in your AWS account and Region.
+ A **device certificate** registered with AWS IoT and activated to authenticate the device.
The certificate is often created with, and attached to, an **AWS IoT thing object**. While a thing object is not required for a device to connect to AWS IoT, it makes additional AWS IoT features available to the device.
+ A **policy** attached to the device certificate that authorizes it to connect to AWS IoT Core and perform all the actions that you want it to.
1. An **internet connection** that can access your AWS account’s device endpoints.
The device endpoints are described in [AWS IoT device data and service endpoints](iot-connect-devices.md#iot-connect-device-endpoints) and can be seen in the [settings page of the AWS IoT console](https://console.aws.amazon.com/iot/home#/settings).
1. **Communication software** such as the AWS IoT Device SDKs provide. This tutorial uses the [AWS IoT Device SDK v2 for Python](https://github.com/aws/aws-iot-device-sdk-python-v2#aws-iot-device-sdk-v2-for-python).
## Review the MQTT protocol
Before we talk about the sample app, it helps to understand the MQTT protocol. The MQTT protocol offers some advantages over other network communication protocols, such as HTTP, which makes it a popular choice for IoT devices. This section reviews the key aspects of MQTT that apply to this tutorial. For information about how MQTT compares to HTTP, see [Choosing an application protocol for your device communication](protocols.md#protocol-selection).
**MQTT uses a publish/subscribe communication model**
The MQTT protocol uses a publish/subscribe communication model with its host. This model differs from the request/response model that HTTP uses. With MQTT, devices establish a session with the host that is identified by a unique client ID. To send data, devices publish messages identified by topics to a message broker in the host. To receive messages from the message broker, devices subscribe to topics by sending topic filters in subscription requests to the message broker.
**MQTT supports persistent sessions**
The message broker receives messages from devices and publishes messages to devices that have subscribed to them. With [persistent sessions](mqtt.md#mqtt-persistent-sessions) —sessions that remain active even when the initiating device is disconnected—devices can retrieve messages that were published while they were disconnected. On the device side, MQTT supports Quality of Service levels ([QoS](mqtt.md#mqtt-qos)) that ensure the host receives messages sent by the device.
## Review the pubsub.py Device SDK sample app
This section reviews the `pubsub.py` sample app from the **AWS IoT Device SDK v2 for Python** used in this tutorial. Here, we'll review how it connects to AWS IoT Core to publish and subscribe to MQTT messages. The next section presents some exercises to help you explore how a device connects and communicates with AWS IoT Core.
**The `pubsub.py` sample app demonstrates these aspects of an MQTT connection with AWS IoT Core:**
+ [Communication protocols](#sdk-tutorials-explore-protocols)
+ [Persistent sessions](#sdk-tutorials-explore-persistent)
+ [Quality of Service](#sdk-tutorials-explore-qos)
+ [Message publish](#sdk-tutorials-explore-publish)
+ [Message subscription](#sdk-tutorials-explore-subscribe)
+ [Device disconnection and reconnection](#sdk-tutorials-explore-connect)
### Communication protocols
The `pubsub.py` sample demonstrates an MQTT connection using the MQTT and MQTT over WSS protocols. The [AWS common runtime (AWS CRT)](https://github.com/awslabs/aws-crt-python#aws-crt-python) library provides the low-level communication protocol support and is included with the AWS IoT Device SDK v2 for Python.
#### MQTT
The `pubsub.py` sample calls `mtls_from_path` (shown here) in the [https://github.com/awslabs/aws-crt-python/blob/89207bcf1387177034e02fe29e8e469ca45e39b7/awscrt/awsiot_mqtt_connection_builder.py](https://github.com/awslabs/aws-crt-python/blob/89207bcf1387177034e02fe29e8e469ca45e39b7/awscrt/awsiot_mqtt_connection_builder.py) to establish a connection with AWS IoT Core by using the MQTT protocol. `mtls_from_path` uses X.509 certificates and TLS v1.2 to authenticate the device. The AWS CRT library handles the lower-level details of that connection.
```
mqtt_connection = mqtt_connection_builder.mtls_from_path(
endpoint=args.endpoint,
cert_filepath=args.cert,
pri_key_filepath=args.key,
ca_filepath=args.ca_file,
client_bootstrap=client_bootstrap,
on_connection_interrupted=on_connection_interrupted,
on_connection_resumed=on_connection_resumed,
client_id=args.client_id,
clean_session=False,
keep_alive_secs=6
)
```
`endpoint`
Your AWS account’s IoT device endpoint
In the sample app, this value is passed in from the command line.
`cert_filepath`
The path to the device’s certificate file
In the sample app, this value is passed in from the command line.
`pri_key_filepath`
The path to the device’s private key file that was created with its certificate file
In the sample app, this value is passed in from the command line.
`ca_filepath`
The path to the Root CA file. Required only if the MQTT server uses a certificate that's not already in your trust store.
In the sample app, this value is passed in from the command line.
`client_bootstrap`
The common runtime object that handles socket communication activities
In the sample app, this object is instantiated before the call to `mqtt_connection_builder.mtls_from_path`.
`on_connection_interrupted``on_connection_resumed`
The callback functions to call when the device’s connection is interrupted and resumed
`client_id`
The ID that uniquely identifies this device in the AWS Region
In the sample app, this value is passed in from the command line.
`clean_session`
Whether to start a new persistent session, or, if one is present, reconnect to an existing one
`keep_alive_secs`
The keep alive value, in seconds, to send in the `CONNECT` request. A ping will automatically be sent at this interval. If the server doesn't receive a ping after 1.5 times this value, it assumes that the connection is lost.
#### MQTT over WSS
The `pubsub.py` sample calls `websockets_with_default_aws_signing` (shown here) in the [https://github.com/awslabs/aws-crt-python/blob/89207bcf1387177034e02fe29e8e469ca45e39b7/awscrt/awsiot_mqtt_connection_builder.py](https://github.com/awslabs/aws-crt-python/blob/89207bcf1387177034e02fe29e8e469ca45e39b7/awscrt/awsiot_mqtt_connection_builder.py) to establish a connection with AWS IoT Core using the MQTT protocol over WSS. `websockets_with_default_aws_signing` creates an MQTT connection over WSS using [Signature V4](https://docs.aws.amazon.com/general/latest/gr/signature-version-4.html) to authenticate the device.
```
mqtt_connection = mqtt_connection_builder.websockets_with_default_aws_signing(
endpoint=args.endpoint,
client_bootstrap=client_bootstrap,
region=args.signing_region,
credentials_provider=credentials_provider,
websocket_proxy_options=proxy_options,
ca_filepath=args.ca_file,
on_connection_interrupted=on_connection_interrupted,
on_connection_resumed=on_connection_resumed,
client_id=args.client_id,
clean_session=False,
keep_alive_secs=6
)
```
`endpoint`
Your AWS account’s IoT device endpoint
In the sample app, this value is passed in from the command line.
`client_bootstrap`
The common runtime object that handles socket communication activities
In the sample app, this object is instantiated before the call to `mqtt_connection_builder.websockets_with_default_aws_signing`.
`region`
The AWS signing Region used by Signature V4 authentication. In `pubsub.py`, it passes the parameter entered in the command line.
In the sample app, this value is passed in from the command line.
`credentials_provider`
The AWS credentials provided to use for authentication
In the sample app, this object is instantiated before the call to `mqtt_connection_builder.websockets_with_default_aws_signing`.
`websocket_proxy_options`
HTTP proxy options, if using a proxy host
In the sample app, this value is initialized before the call to `mqtt_connection_builder.websockets_with_default_aws_signing`.
`ca_filepath`
The path to the Root CA file. Required only if the MQTT server uses a certificate that's not already in your trust store.
In the sample app, this value is passed in from the command line.
`on_connection_interrupted``on_connection_resumed`
The callback functions to call when the device’s connection is interrupted and resumed
`client_id`
The ID that uniquely identifies this device in the AWS Region.
In the sample app, this value is passed in from the command line.
`clean_session`
Whether to start a new persistent session, or, if one is present, reconnect to an existing one
`keep_alive_secs`
The keep alive value, in seconds, to send in the `CONNECT` request. A ping will automatically be sent at this interval. If the server doesn't receive a ping after 1.5 times this value, it assumes the connection is lost.
#### HTTPS
What about HTTPS? AWS IoT Core supports devices that publish HTTPS requests. From a programming perspective, devices send HTTPS requests to AWS IoT Core as would any other application. For an example of a Python program that sends an HTTP message from a device, see the [HTTPS code example](http.md#codeexample) using Python’s `requests` library. This example sends a message to AWS IoT Core using HTTPS such that AWS IoT Core interprets it as an MQTT message.
While AWS IoT Core supports HTTPS requests from devices, be sure to review the information about [Choosing an application protocol for your device communication](protocols.md#protocol-selection) so that you can make an informed decision on which protocol to use for your device communications.
### Persistent sessions
In the sample app, setting the `clean_session` parameter to `False` indicates that the connection should be persistent. In practice, this means that the connection opened by this call reconnects to an existing persistent session, if one exists. Otherwise, it creates and connects to a new persistent session.
With a persistent session, messages that are sent to the device are stored by the message broker while the device is not connected. When a device reconnects to a persistent session, the message broker sends to the device any stored messages to which it has subscribed.
Without a persistent session, the device will not receive messages that are sent while the device isn't connected. Which option to use depends on your application and whether messages that occur while a device is not connected must be communicated. For more information, see [MQTT persistent sessions](mqtt.md#mqtt-persistent-sessions).
### Quality of Service
When the device publishes and subscribes to messages, the preferred Quality of Service (QoS) can be set. AWS IoT supports QoS levels 0 and 1 for publish and subscribe operations. For more information about QoS levels in AWS IoT, see [MQTT Quality of Service (QoS) options](mqtt.md#mqtt-qos).
The AWS CRT runtime for Python defines these constants for the QoS levels that it supports:
**Python Quality of Service levels**
| MQTT QoS level | Python symbolic value used by SDK | Description |
| --- | --- | --- |
| QoS level 0 | mqtt.QoS.AT\$1MOST\$1ONCE | Only one attempt to send the message will be made, whether it is received or not. The message might not be sent at all, for example, if the device is not connected or there's a network error. |
| QoS level 1 | mqtt.QoS.AT\$1LEAST\$1ONCE | The message is sent repeatedly until a PUBACK acknowledgement is received. |
In the sample app, the publish and subscribe requests are made with a QoS level of 1 (`mqtt.QoS.AT_LEAST_ONCE`).
+
**QoS on publish**
When a device publishes a message with QoS level 1, it sends the message repeatedly until it receives a `PUBACK` response from the message broker. If the device isn't connected, the message is queued to be sent after it reconnects.
+
**QoS on subscribe**
When a device subscribes to a message with QoS level 1, the message broker saves the messages to which the device is subscribed until they can be sent to the device. The message broker resends the messages until it receives a `PUBACK` response from the device.
### Message publish
After successfully establishing a connection to AWS IoT Core, devices can publish messages. The `pubsub.py` sample does this by calling the `publish` operation of the `mqtt_connection` object.
```
mqtt_connection.publish(
topic=args.topic,
payload=message,
qos=mqtt.QoS.AT_LEAST_ONCE
)
```
`topic`
The message's topic name that identifies the message
In the sample app, this is passed in from the command line.
`payload`
The message payload formatted as a string (for example, a JSON document)
In the sample app, this is passed in from the command line.
A JSON document is a common payload format, and one that is recognized by other AWS IoT services; however, the data format of the message payload can be anything that the publishers and subscribers agree upon. Other AWS IoT services, however, only recognize JSON, and CBOR, in some cases, for most operations.
`qos`
The QoS level for this message
### Message subscription
To receive messages from AWS IoT and other services and devices, devices subscribe to those messages by their topic name. Devices can subscribe to individual messages by specifying a [topic name](topics.md#topicnames), and to a group of messages by specifying a [topic filter](topics.md#topicfilters), which can include wild card characters. The `pubsub.py` sample uses the code shown here to subscribe to messages and register the callback functions to process the message after it’s received.
```
subscribe_future, packet_id = mqtt_connection.subscribe(
topic=args.topic,
qos=mqtt.QoS.AT_LEAST_ONCE,
callback=on_message_received
)
subscribe_result = subscribe_future.result()
```
`topic`
The topic to subscribe to. This can be a topic name or a topic filter.
In the sample app, this is passed in from the command line.
`qos`
Whether the message broker should store these messages while the device is disconnected.
A value of `mqtt.QoS.AT_LEAST_ONCE` (QoS level 1), requires a persistent session to be specified (`clean_session=False`) when the connection is created.
`callback`
The function to call to process the subscribed message.
The `mqtt_connection.subscribe` function returns a future and a packet ID. If the subscription request was initiated successfully, the packet ID returned is greater than 0. To make sure that the subscription was received and registered by the message broker, you must wait for the result of the asynchronous operation to return, as shown in the code example.
**The callback function**
The callback in the `pubsub.py` sample processes the subscribed messages as the device receives them.
```
def on_message_received(topic, payload, **kwargs):
print("Received message from topic '{}': {}".format(topic, payload))
global received_count
received_count += 1
if received_count == args.count:
received_all_event.set()
```
`topic`
The message’s topic
This is the specific topic name of the message received, even if you subscribed to a topic filter.
`payload`
The message payload
The format for this is application specific.
`kwargs`
Possible additional arguments as described in [https://awslabs.github.io/aws-crt-python/api/mqtt.html#awscrt.mqtt.Connection.subscribe](https://awslabs.github.io/aws-crt-python/api/mqtt.html#awscrt.mqtt.Connection.subscribe).
In the `pubsub.py` sample, `on_message_received` only displays the topic and its payload. It also counts the messages received to end the program after the limit is reached.
Your app would evaluate the topic and the payload to determine what actions to perform.
### Device disconnection and reconnection
The `pubsub.py` sample includes callback functions that are called when the device is disconnected and when the connection is re-established. What actions your device takes on these events is application specific.
When a device connects for the first time, it must subscribe to topics to receive. If a device's session is present when it reconnects, its subscriptions are restored, and any stored messages from those subscriptions are sent to the device after it reconnects.
If a device's session no longer exists when it reconnects, it must resubscribe to its subscriptions. Persistent sessions have a limited lifetime and can expire when the device is disconnected for too long.
## Connect your device and communicate with AWS IoT Core
This section presents some exercises to help you explore different aspects of connecting your device to AWS IoT Core. For these exercises, you’ll use the [MQTT test client](https://console.aws.amazon.com/iot/home#/test) in the AWS IoT console to see what your device publishes and to publish messages to your device. These exercises use the [https://github.com/aws/aws-iot-device-sdk-python-v2/blob/master/samples/pubsub.py](https://github.com/aws/aws-iot-device-sdk-python-v2/blob/master/samples/pubsub.py) sample from the [AWS IoT Device SDK v2 for Python](https://github.com/aws/aws-iot-device-sdk-python-v2/tree/master/samples#sample-apps-for-the-aws-iot-device-sdk-v2-for-python) and build on your experience with [Getting started with AWS IoT Core tutorials](iot-gs.md) tutorials.
**Topics**
+ [Subscribe to wild card topic filters](#sdk-tutorials-experiment-wild)
+ [Process topic filter subscriptions](#sdk-tutorials-experiment-process)
+ [Publish messages from your device](#sdk-tutorials-experiment-publish)
For these exercises, you'll start from the `pubsub.py` sample program.
**Note**
These exercises assume that you completed the [Getting started with AWS IoT Core tutorials](iot-gs.md) tutorials and use the terminal window for your device from that tutorial.
### Subscribe to wild card topic filters
In this exercise, you’ll modify the command line used to call `pubsub.py` to subscribe to a wild card topic filter and process the messages received based on the message’s topic.
#### Exercise procedure
For this exercise, imagine that your device contains a temperature control and a light control. It uses these topic names to identify the messages about them.
1. Before starting the exercise, try running this command from the [Getting started with AWS IoT Core tutorials](iot-gs.md) tutorials on your device to make sure that everything is ready for the exercise.
```
cd ~/aws-iot-device-sdk-python-v2/samples
python3 pubsub.py --topic topic_1 --ca_file ~/certs/Amazon-root-CA-1.pem --cert ~/certs/device.pem.crt --key ~/certs/private.pem.key --endpoint your-iot-endpoint
```
You should see the same output as you saw in the [Getting started tutorial](connecting-to-existing-device.md#gs-device-node-app-run).
1. For this exercise, change these command line parameters.
[\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/iot/latest/developerguide/sdk-tutorials.html)
Making these changes to the initial command line results in this command line. Enter this command in the terminal window for your device.
```
python3 pubsub.py --message "" --count 2 --topic device/+/details --ca_file ~/certs/Amazon-root-CA-1.pem --cert ~/certs/device.pem.crt --key ~/certs/private.pem.key --endpoint your-iot-endpoint
```
The program should display something like this:
```
Connecting to a3qexamplesffp-ats.iot.us-west-2.amazonaws.com with client ID 'test-24d7cdcc-cc01-458c-8488-2d05849691e1'...
Connected!
Subscribing to topic 'device/+/details'...
Subscribed with QoS.AT_LEAST_ONCE
Waiting for all messages to be received...
```
If you see something like this on your terminal, your device is ready and listening for messages where the topic names start with `device` and end with `/detail`. So, let's test that.
1. Here are a couple of messages that your device might receive.
[\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/iot/latest/developerguide/sdk-tutorials.html)
1. Using the MQTT test client in the AWS IoT console, send the messages described in the previous step to your device.
1. Open the [MQTT test client](https://console.aws.amazon.com/iot/home#/test) in the AWS IoT console.
1. In **Subscribe to a topic**, in the **Subscription topic field**, enter the topic filter: **device/\$1/details**, and then choose **Subscribe to topic**.
1. In the **Subscriptions** column of the MQTT test client, choose **device/\$1/details**.
1. For each of the topics in the preceding table, do the following in the MQTT test client:
1. In **Publish**, enter the value from the **Topic name** column in the table.
1. In the message payload field below the topic name, enter the value from the **Message payload** column in the table.
1. Watch the terminal window where `pubsub.py` is running and, in the MQTT test client, choose **Publish to topic**.
You should see that the message was received by `pubsub.py` in the terminal window.
#### Exercise result
With this, `pubsub.py`, subscribed to the messages using a wild card topic filter, received them, and displayed them in the terminal window. Notice how you subscribed to a single topic filter, and the callback function was called to process messages having two distinct topics.
### Process topic filter subscriptions
Building on the previous exercise, modify the `pubsub.py` sample app to evaluate the message topics and process the subscribed messages based on the topic.
#### Exercise procedure
**To evaluate the message topic**
1. Copy `pubsub.py` to `pubsub2.py`.
1. Open `pubsub2.py` in your favorite text editor or IDE.
1. In `pubsub2.py`, find the `on_message_received` function.
1. In `on_message_received`, insert the following code after the line that starts with `print("Received message` and before the line that starts with `global received_count`.
```
topic_parsed = False
if "/" in topic:
parsed_topic = topic.split("/")
if len(parsed_topic) == 3:
# this topic has the correct format
if (parsed_topic[0] == 'device') and (parsed_topic[2] == 'details'):
# this is a topic we care about, so check the 2nd element
if (parsed_topic[1] == 'temp'):
print("Received temperature request: {}".format(payload))
topic_parsed = True
if (parsed_topic[1] == 'light'):
print("Received light request: {}".format(payload))
topic_parsed = True
if not topic_parsed:
print("Unrecognized message topic.")
```
1. Save your changes and run the modified program by using this command line.
```
python3 pubsub2.py --message "" --count 2 --topic device/+/details --ca_file ~/certs/Amazon-root-CA-1.pem --cert ~/certs/device.pem.crt --key ~/certs/private.pem.key --endpoint your-iot-endpoint
```
1. In the AWS IoT console, open the [MQTT test client](https://console.aws.amazon.com/iot/home#/test).
1. In **Subscribe to a topic**, in the **Subscription topic field**, enter the topic filter: **device/\$1/details**, and then choose **Subscribe to topic**.
1. In the **Subscriptions** column of the MQTT test client, choose **device/\$1/details**.
1. For each of the topics in this table, do the following in the MQTT test client:
[\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/iot/latest/developerguide/sdk-tutorials.html)
1. In **Publish**, enter the value from the **Topic name** column in the table.
1. In the message payload field below the topic name, enter the value from the **Message payload** column in the table.
1. Watch the terminal window where `pubsub.py` is running and, in the MQTT test client, choose **Publish to topic**.
You should see that the message was received by `pubsub.py` in the terminal window.
You should see something similar to this in your terminal window.
```
Connecting to a3qexamplesffp-ats.iot.us-west-2.amazonaws.com with client ID 'test-af794be0-7542-45a0-b0af-0b0ea7474517'...
Connected!
Subscribing to topic 'device/+/details'...
Subscribed with QoS.AT_LEAST_ONCE
Waiting for all messages to be received...
Received message from topic 'device/light/details': b'{ "desiredLight": 100, "currentLight": 50 }'
Received light request: b'{ "desiredLight": 100, "currentLight": 50 }'
Received message from topic 'device/temp/details': b'{ "desiredTemp": 20, "currentTemp": 15 }'
Received temperature request: b'{ "desiredTemp": 20, "currentTemp": 15 }'
2 message(s) received.
Disconnecting...
Disconnected!
```
#### Exercise result
In this exercise, you added code so the sample app would recognize and process multiple messages in the callback function. With this, your device could receive messages and act on them.
Another way for your device to receive and process multiple messages is to subscribe to different messages separately and assign each subscription to its own callback function.
### Publish messages from your device
You can use the pubsub.py sample app to publish messages from your device. While it will publish messages as it is, the messages can't be read as JSON documents. This exercise modifies the sample app to be able to publish JSON documents in the message payload that can be read by AWS IoT Core.
#### Exercise procedure
In this exercise, the following message will be sent with the `device/data` topic.
```
{
"timestamp": 1601048303,
"sensorId": 28,
"sensorData": [
{
"sensorName": "Wind speed",
"sensorValue": 34.2211224
}
]
}
```
**To prepare your MQTT test client to monitor the messages from this exercise**
1. In **Subscribe to a topic**, in the **Subscription topic field**, enter the topic filter: **device/data**, and then choose **Subscribe to topic**.
1. In the **Subscriptions** column of the MQTT test client, choose **device/data**.
1. Keep the MQTT test client window open to wait for messages from your device.
**To send JSON documents with the pubsub.py sample app**
1. On your device, copy `pubsub.py` to `pubsub3.py`.
1. Edit `pubsub3.py` to change how it formats the messages it publishes.
1. Open `pubsub3.py` in a text editor.
1. Locate this line of code:
`message = "{} [{}]".format(message_string, publish_count)`
1. Change it to:
`message = "{}".format(message_string)`
1. Locate this line of code:
`message_json = json.dumps(message)`
1. Change it to:
`message = "{}".json.dumps(json.loads(message))`
1. Save your changes.
1. On your device, run this command to send the message two times.
```
python3 pubsub3.py --ca_file ~/certs/Amazon-root-CA-1.pem --cert ~/certs/device.pem.crt --key ~/certs/private.pem.key --topic device/data --count 2 --message '{"timestamp":1601048303,"sensorId":28,"sensorData":[{"sensorName":"Wind speed","sensorValue":34.2211224}]}' --endpoint your-iot-endpoint
```
1. In the MQTT test client, check to see that it has interpreted and formatted the JSON document in the message payload, such as this:
![\[Image showing how a JSON message payload is displayed in the MQTT client of the AWS IoT console.\]](http://docs.aws.amazon.com/iot/latest/developerguide/images/mqtt-test-client-output.png)
By default, `pubsub3.py` also subscribes to the messages it sends. You should see that it received the messages in the app’s output. The terminal window should look something like this.
```
Connecting to a3qEXAMPLEsffp-ats.iot.us-west-2.amazonaws.com with client ID 'test-5cff18ae-1e92-4c38-a9d4-7b9771afc52f'...
Connected!
Subscribing to topic 'device/data'...
Subscribed with QoS.AT_LEAST_ONCE
Sending 2 message(s)
Publishing message to topic 'device/data': {"timestamp":1601048303,"sensorId":28,"sensorData":[{"sensorName":"Wind speed","sensorValue":34.2211224}]}
Received message from topic 'device/data': b'{"timestamp":1601048303,"sensorId":28,"sensorData":[{"sensorName":"Wind speed","sensorValue":34.2211224}]}'
Publishing message to topic 'device/data': {"timestamp":1601048303,"sensorId":28,"sensorData":[{"sensorName":"Wind speed","sensorValue":34.2211224}]}
Received message from topic 'device/data': b'{"timestamp":1601048303,"sensorId":28,"sensorData":[{"sensorName":"Wind speed","sensorValue":34.2211224}]}'
2 message(s) received.
Disconnecting...
Disconnected!
```
#### Exercise result
With this, your device can generate messages to send to AWS IoT Core to test basic connectivity and provide device messages for AWS IoT Core to process. For example, you could use this app to send test data from your device to test AWS IoT rule actions.
## Review the results
The examples in this tutorial gave you hands-on experience with the basics of how devices can communicate with AWS IoT Core—a fundamental part of your AWS IoT solution. When your devices are able to communicate with AWS IoT Core, they can pass messages to AWS services and other devices on which they can act. Likewise, AWS services and other devices can process information that results in messages sent back to your devices.
When you are ready to explore AWS IoT Core further, try these tutorials:
+ [Tutorial: Sending an Amazon SNS notification](iot-sns-rule.md)
+ [Tutorial: Storing device data in a DynamoDB table](iot-ddb-rule.md)
+ [Tutorial: Formatting a notification by using an AWS Lambda function](iot-lambda-rule.md)
# Tutorial: Using the AWS IoT Device SDK for Embedded C
This section describes how to run the AWS IoT Device SDK for Embedded C.
**Topics**
+ [Step1: Install the AWS IoT Device SDK for Embedded C](#install-embedded-c-sdk)
+ [Step 2: Configure the sample app](#iot-c-sdk-app-config)
+ [Step 3: Build and run the sample application](#iot-c-sdk-app-run)
## Step1: Install the AWS IoT Device SDK for Embedded C
The AWS IoT Device SDK for Embedded C is generally targeted at resource constrained devices that require an optimized C language runtime. You can use the SDK on any operating system and host it on any processor type (for example, MCUs and MPUs). If you have more memory and processing resources available, we recommend that you use one of the higher order AWS IoT Device and Mobile SDKs (for example, C\$1\$1, Java, JavaScript, and Python).
In general, the AWS IoT Device SDK for Embedded C is intended for systems that use MCUs or low-end MPUs that run embedded operating systems. For the programming example in this section, we assume your device uses Linux.
**Example**
1. Download the AWS IoT Device SDK for Embedded C to your device from [GitHub](https://github.com/aws/aws-iot-device-sdk-embedded-C).
```
git clone https://github.com/aws/aws-iot-device-sdk-embedded-c.git --recurse-submodules
```
This creates a directory named `aws-iot-device-sdk-embedded-c` in the current directory.
1. Navigate to that directory and checkout the latest release. Please see [ github.com/aws/aws-iot-device-sdk-embedded-C/tags](https://github.com/aws/aws-iot-device-sdk-embedded-C/tags) for the latest release tag.
```
cd aws-iot-device-sdk-embedded-c
git checkout latest-release-tag
```
1. Install OpenSSL version 1.1.0 or later. The OpenSSL development libraries are usually called "libssl-dev" or "openssl-devel" when installed through a package manager.
```
sudo apt-get install libssl-dev
```
## Step 2: Configure the sample app
The AWS IoT Device SDK for Embedded C includes sample applications for you to try. For simplicity, this tutorial uses the `mqtt_demo_mutual_auth` application, that illustrates how to connect to the AWS IoT Core message broker and subscribe and publish to MQTT topics.
1. Copy the certificate and private key you created in [Getting started with AWS IoT Core tutorials](iot-gs.md) into the `build/bin/certificates` directory.
**Note**
Device and root CA certificates are subject to expiration or revocation. If these certificates expire or are revoked, you must copy a new CA certificate or private key and device certificate onto your device.
1. You must configure the sample with your personal AWS IoT Core endpoint, private key, certificate, and root CA certificate. Navigate to the `aws-iot-device-sdk-embedded-c/demos/mqtt/mqtt_demo_mutual_auth` directory.
If you have the AWS CLI installed, you can use this command to find your account's endpoint URL.
```
aws iot describe-endpoint --endpoint-type iot:Data-ATS
```
If you don't have the AWS CLI installed, open your [AWS IoT console](https://console.aws.amazon.com/iot/home). From the navigation pane, choose **Manage**, and then choose **Things**. Choose the IoT thing for your device, and then choose **Interact**. Your endpoint is displayed in the ** HTTPS** section of the thing details page.
1. Open the `demo_config.h` file and update the values for the following:
AWS\$1IOT\$1ENDPOINT
Your personal endpoint.
CLIENT\$1CERT\$1PATH
Your certificate file path, for example `certificates/device.pem.crt"`.
CLIENT\$1PRIVATE\$1KEY\$1PATH
Your private key file name, for example `certificates/private.pem.key`.
For example:
```
// Get from demo_config.h
// =================================================
#define AWS_IOT_ENDPOINT "my-endpoint-ats.iot.us-east-1.amazonaws.com"
#define AWS_MQTT_PORT 8883
#define CLIENT_IDENTIFIER "testclient"
#define ROOT_CA_CERT_PATH "certificates/AmazonRootCA1.crt"
#define CLIENT_CERT_PATH "certificates/my-device-cert.pem.crt"
#define CLIENT_PRIVATE_KEY_PATH "certificates/my-device-private-key.pem.key"
// =================================================
```
1. Check to see if you have CMake installed on your device by using this command.
```
cmake --version
```
If you see the version information for the compiler, you can continue to the next section.
If you get an error or don't see any information, then you'll need to install the cmake package using this command.
```
sudo apt-get install cmake
```
Run the **cmake --version** command again and confirm that CMake has been installed and that you are ready to continue.
1. Check to see if you have the development tools installed on your device by using this command.
```
gcc --version
```
If you see the version information for the compiler, you can continue to the next section.
If you get an error or don't see any compiler information, you'll need to install the `build-essential` package using this command.
```
sudo apt-get install build-essential
```
Run the **gcc --version** command again and confirm that the build tools have been installed and that you are ready to continue.
## Step 3: Build and run the sample application
This procedure exaplains how to generate the `mqtt_demo_mutual_auth` application on your device and connect it to the [AWS IoT console](https://console.aws.amazon.com/iot/home) using the AWS IoT Device SDK for Embedded C.
**To run the AWS IoT Device SDK for Embedded C sample applications**
1. Navigate to `aws-iot-device-sdk-embedded-c` and create a build directory.
```
mkdir build && cd build
```
1. Enter the following CMake command to generate the Makefiles needed to build.
```
cmake ..
```
1. Enter the following command to build the executable app file.
```
make
```
1. Run the `mqtt_demo_mutual_auth` app with this command.
```
cd bin
./mqtt_demo_mutual_auth
```
You should see output similar to the following:
![\[Command line output for running the AWS IoT Device SDK for Embedded C sample application.\]](http://docs.aws.amazon.com/iot/latest/developerguide/images/successful-run2.png)
Your device is now connected to AWS IoT using the AWS IoT Device SDK for Embedded C.
You can also use the AWS IoT console to view the MQTT messages that the sample app is publishing. For information about how to use the MQTT client in the [AWS IoT console](https://console.aws.amazon.com/iot/home), see [View MQTT messages with the AWS IoT MQTT client](view-mqtt-messages.md) .
# Creating AWS IoT rules to route device data to other services
These tutorials show you how to create and test AWS IoT rules using some of the more common rule actions.
AWS IoT rules send data from your devices to other AWS services. They listen for specific MQTT messages, format the data in the message payloads, and send the result to other AWS services.
We recommend that you try these in the order they are shown here, even if your goal is to create a rule that uses a Lambda function or something more complex. The tutorials are presented in order from basic to complex. They present new concepts incrementally to help you learn the concepts you can use to create the rule actions that don't have a specific tutorial.
**Note**
AWS IoT rules help you send the data from your IoT devices to other AWS services. To do that successfully, however, you need a working knowledge of the other services where you want to send data. While these tutorials provide the necessary information to complete the tasks, you might find it helpful to learn more about the services you want to send data to before you use them in your solution. A detailed explanation of the other AWS services is outside of the scope of these tutorials.
**Tutorial scenario overview**
The scenario for these tutorials is that of a weather sensor device that periodically publishes its data. There are many such sensor devices in this imaginary system. The tutorials in this section, however, focus on a single device while showing how you might accommodate multiple sensors.
The tutorials in this section show you how to use AWS IoT rules to do the following tasks with this imaginary system of weather sensor devices.
+
**[Tutorial: Republishing an MQTT message](iot-repub-rule.md)**
This tutorial shows how to republish an MQTT message received from the weather sensors as a message that contains only the sensor ID and the temperature value. It uses only AWS IoT Core services and demonstrates a simple SQL query and how to use the MQTT client to test your rule.
+
**[Tutorial: Sending an Amazon SNS notification](iot-sns-rule.md)**
This tutorial shows how to send an SNS message when a value from a weather sensor device exceeds a specific value. It builds on the concepts presented in the previous tutorial and adds how to work with another AWS service, the [Amazon Simple Notification Service](https://docs.aws.amazon.com//sns/latest/dg/welcome.html) (Amazon SNS).
If you're new to Amazon SNS, review its [Getting started](https://docs.aws.amazon.com//sns/latest/dg/sns-getting-started.html) exercises before you start this tutorial.
+
**[Tutorial: Storing device data in a DynamoDB table](iot-ddb-rule.md)**
This tutorial shows how to store the data from the weather sensor devices in a database table. It uses the rule query statement and substitution templates to format the message data for the destination service, [Amazon DynamoDB](https://docs.aws.amazon.com//amazondynamodb/latest/developerguide/Introduction.html).
If you're new to DynamoDB, review its [Getting started](https://docs.aws.amazon.com//amazondynamodb/latest/developerguide/GettingStartedDynamoDB.html) exercises before you start this tutorial.
+
**[Tutorial: Formatting a notification by using an AWS Lambda function](iot-lambda-rule.md)**
This tutorial shows how to call a Lambda function to reformat the device data and then send it as a text message. It adds a Python script and AWS SDK functions in an [AWS Lambda](https://docs.aws.amazon.com//lambda/latest/dg/welcome.html) function to format with the message payload data from the weather sensor devices and send a text message.
If you're new to Lambda, review its [Getting started](https://docs.aws.amazon.com//lambda/latest/dg/getting-started.html) exercises before you start this tutorial.
**AWS IoT rule overview**
All of these tutorials create AWS IoT rules.
For an AWS IoT rule to send the data from a device to another AWS service, it uses:
+ A rule query statement that consists of:
+ A SQL SELECT clause that selects and formats the data from the message payload
+ A topic filter (the FROM object in the rule query statement) that identifies the messages to use
+ An optional conditional statement (a SQL WHERE clause) that specifies specific conditions on which to act
+ At least one rule action
Devices publish messages to MQTT topics. The topic filter in the SQL SELECT statement identifies the MQTT topics to apply the rule to. The fields specified in the SQL SELECT statement format the data from the incoming MQTT message payload for use by the rule's actions. For a complete list of rule actions, see [AWS IoT Rule Actions](iot-rule-actions.md).
**Topics**
+ [Tutorial: Republishing an MQTT message](iot-repub-rule.md)
+ [Tutorial: Sending an Amazon SNS notification](iot-sns-rule.md)
+ [Tutorial: Storing device data in a DynamoDB table](iot-ddb-rule.md)
+ [Tutorial: Formatting a notification by using an AWS Lambda function](iot-lambda-rule.md)
# Tutorial: Republishing an MQTT message
This tutorial demonstrates how to create an AWS IoT rule that publishes an MQTT message when a specified MQTT message is received. The incoming message payload can be modified by the rule before it's published. This makes it possible to create messages that are tailored to specific applications without the need to alter your device or its firmware. You can also use the filtering aspect of a rule to publish messages only when a specific condition is met.
The messages republished by a rule act like messages sent by any other AWS IoT device or client. Devices can subscribe to the republished messages the same way they can subscribe to any other MQTT message topic.
**What you'll learn in this tutorial:**
+ How to use simple SQL queries and functions in a rule query statement
+ How to use the MQTT client to test an AWS IoT rule
This tutorial takes about 30 minutes to complete.
**Topics**
+ [Review MQTT topics and AWS IoT rules](#iot-repub-rule-mqtt)
+ [Step 1: Create an AWS IoT rule to republish an MQTT message](#iot-repub-rule-define)
+ [Step 2: Test your new rule](#iot-repub-rule-test)
+ [Step 3: Review the results and next steps](#iot-repub-rule-review)
**Before you start this tutorial, make sure that you have:**
+
**[Set up AWS account](setting-up.md)**
You'll need your AWS account and AWS IoT console to complete this tutorial.
+
**Reviewed [View MQTT messages with the AWS IoT MQTT client](view-mqtt-messages.md)**
Be sure you can use the MQTT client to subscribe and publish to a topic. You'll use the MQTT client to test your new rule in this procedure.
## Review MQTT topics and AWS IoT rules
Before talking about AWS IoT rules, it helps to understand the MQTT protocol. In IoT solutions, the MQTT protocol offers some advantages over other network communication protocols, such as HTTP, which makes it a popular choice for use by IoT devices. This section reviews the key aspects of MQTT as they apply to this tutorial. For information about how MQTT compares to HTTP, see [Choosing an application protocol for your device communication](protocols.md#protocol-selection).
**MQTT protocol**
The MQTT protocol uses a publish/subscribe communication model with its host. To send data, devices publish messages that are identified by topics to the AWS IoT message broker. To receive messages from the message broker, devices subscribe to the topics they will receive by sending topic filters in subscription requests to the message broker. The AWS IoT rules engine receives MQTT messages from the message broker.
**AWS IoT rules**
AWS IoT rules consist of a rule query statement and one or more rule actions. When the AWS IoT rules engine receives an MQTT message, these elements act on the message as follows.
+
**Rule query statement**
The rule's query statement describes the MQTT topics to use, interprets the data from the message payload, and formats the data as described by a SQL statement that is similar to statements used by popular SQL databases. The result of the query statement is the data that is sent to the rule's actions.
+
**Rule action**
Each rule action in a rule acts on the data that results from the rule's query statement. AWS IoT supports [many rule actions](iot-rule-actions.md). In this tutorial, however, you'll concentrate on the [Republish](republish-rule-action.md) rule action, which publishes the result of the query statement as an MQTT message with a specific topic.
## Step 1: Create an AWS IoT rule to republish an MQTT message
The AWS IoT rule that you'll create in this tutorial subscribes to the `device/device_id/data` MQTT topics where *device\$1id* is the ID of the device that sent the message. These topics are described by a [topic filter](topics.md#topicfilters) as `device/+/data`, where the `+` is a wildcard character that matches any string between the two forward slash characters.
When the rule receives a message from a matching topic, it republishes the `device_id` and `temperature` values as a new MQTT message with the `device/data/temp` topic.
For example, the payload of an MQTT message with the `device/22/data` topic looks like this:
```
{
"temperature": 28,
"humidity": 80,
"barometer": 1013,
"wind": {
"velocity": 22,
"bearing": 255
}
}
```
The rule takes the `temperature` value from the message payload, and the `device_id` from the topic, and republishes them as an MQTT message with the `device/data/temp` topic and a message payload that looks like this:
```
{
"device_id": "22",
"temperature": 28
}
```
With this rule, devices that only need the device's ID and the temperature data subscribe to the `device/data/temp` topic to receive only that information.
**To create a rule that republishes an MQTT message**
1. Open [the **Rules** hub of the AWS IoT console](https://console.aws.amazon.com//iot/home#/rulehub).
1. In **Rules**, choose **Create** and start creating your new rule.
1. In the top part of **Create a rule**:
1. In **Name**, enter the rule's name. For this tutorial, name it **republish\$1temp**.
Remember that a rule name must be unique within your Account and Region, and it can't have any spaces. We've used an underscore character in this name to separate the two words in the rule's name.
1. In **Description**, describe the rule.
A meaningful description helps you remember what this rule does and why you created it. The description can be as long as needed, so be as detailed as possible.
1. In **Rule query statement** of **Create a rule**:
1. In **Using SQL version**, select **2016-03-23**.
1. In the **Rule query statement** edit box, enter the statement:
```
SELECT topic(2) as device_id, temperature FROM 'device/+/data'
```
This statement:
+ Listens for MQTT messages with a topic that matches the `device/+/data` topic filter.
+ Selects the second element from the topic string and assigns it to the `device_id` field.
+ Selects the value `temperature` field from the message payload and assigns it to the `temperature` field.
1. In **Set one or more actions**:
1. To open up the list of rule actions for this rule, choose **Add action**.
1. In **Select an action**, choose **Republish a message to an AWS IoT topic**.
1. At the bottom of the action list, choose **Configure action** to open the selected action's configuration page.
1. In **Configure action**:
1. In **Topic**, enter **device/data/temp**. This is the MQTT topic of the message that this rule will publish.
1. In **Quality of Service**, choose **0 - The message is delivered zero or more times**.
1. In **Choose or create a role to grant AWS IoT access to perform this action**:
1. Choose **Create Role**. The **Create a new role** dialog box opens.
1. Enter a name that describes the new role. In this tutorial, use **republish\$1role**.
When you create a new role, the correct policies to perform the rule action are created and attached to the new role. If you change the topic of this rule action or use this role in another rule action, you must update the policy for that role to authorize the new topic or action. To update an existing role, choose **Update role** in this section.
1. Choose **Create Role** to create the role and close the dialog box.
1. Choose **Add action** to add the action to the rule and return to the **Create a rule** page.
1. The **Republish a message to an AWS IoT topic** action is now listed in **Set one or more actions**.
In the new action's tile, below **Republish a message to an AWS IoT topic**, you can see the topic to which your republish action will publish.
This is the only rule action you'll add to this rule.
1. In **Create a rule**, scroll down to the bottom and choose **Create rule** to create the rule and complete this step.
## Step 2: Test your new rule
To test your new rule, you'll use the MQTT client to publish and subscribe to the MQTT messages used by this rule.
Open the [MQTT client in the AWS IoT console](https://console.aws.amazon.com//iot/home#/test) in a new window. This will let you edit the rule without losing the configuration of your MQTT client. The MQTT client does not retain any subscriptions or message logs if you leave it to go to another page in the console.
**To use the MQTT client to test your rule**
1. In the [MQTT client in the AWS IoT console](https://console.aws.amazon.com//iot/home#/test), subscribe to the input topics, in this case, `device/+/data`.
1. In the MQTT client, under **Subscriptions**, choose **Subscribe to a topic**.
1. In **Subscription topic**, enter the topic of the input topic filter, **device/\$1/data**.
1. Keep the rest of the fields at their default settings.
1. Choose **Subscribe to topic**.
In the **Subscriptions** column, under **Publish to a topic**, **device/\$1/data** appears.
1. Subscribe to the topic that your rule will publish: `device/data/temp`.
1. Under **Subscriptions**, choose **Subscribe to a topic** again, and in **Subscription topic**, enter the topic of the republished message, **device/data/temp**.
1. Keep the rest of the fields at their default settings.
1. Choose **Subscribe to topic**.
In the **Subscriptions** column, under **device/\$1/data**, **device/data/temp** appears.
1. Publish a message to the input topic with a specific device ID, **device/22/data**. You can't publish to MQTT topics that contain wildcard characters.
1. In the MQTT client, under **Subscriptions**, choose **Publish to topic**.
1. In the **Publish** field, enter the input topic name, **device/22/data**.
1. Copy the sample data shown here and, in the edit box below the topic name, paste the sample data.
```
{
"temperature": 28,
"humidity": 80,
"barometer": 1013,
"wind": {
"velocity": 22,
"bearing": 255
}
}
```
1. To send your MQTT message, choose **Publish to topic**.
1. Review the messages that were sent.
1. In the MQTT client, under **Subscriptions**, there is a green dot next to the two topics to which you subscribed earlier.
The green dots indicate that one or more new messages have been received since the last time you looked at them.
1. Under **Subscriptions**, choose **device/\$1/data** to check that the message payload matches what you just published and looks like this:
```
{
"temperature": 28,
"humidity": 80,
"barometer": 1013,
"wind": {
"velocity": 22,
"bearing": 255
}
}
```
1. Under **Subscriptions**, choose **device/data/temp** to check that your republished message payload looks like this:
```
{
"device_id": "22",
"temperature": 28
}
```
Notice that the `device_id` value is a quoted string and the `temperature` value is numeric. This is because the [https://docs.aws.amazon.com//iot/latest/developerguide/iot-sql-functions.html#iot-function-topic](https://docs.aws.amazon.com//iot/latest/developerguide/iot-sql-functions.html#iot-function-topic) function extracted the string from the input message's topic name while the `temperature` value uses the numeric value from the input message's payload.
If you want to make the `device_id` value a numeric value, replace `topic(2)` in the rule query statement with:
```
cast(topic(2) AS DECIMAL)
```
Note that casting the `topic(2)` value to a numeric value will only work if that part of the topic contains only numeric characters.
1. If you see that the correct message was published to the **device/data/temp** topic, then your rule worked. See what more you can learn about the Republish rule action in the next section.
If you don't see that the correct message was published to either the **device/\$1/data** or **device/data/temp** topics, check the troubleshooting tips.
### Troubleshooting your Republish message rule
Here are some things to check in case you're not seeing the results you expect.
+
**You got an error banner**
If an error appeared when you published the input message, correct that error first. The following steps might help you correct that error.
+
**You don't see the input message in the MQTT client**
Every time you publish your input message to the `device/22/data` topic, that message should appear in the MQTT client if you subscribed to the `device/+/data` topic filter as described in the procedure.
**Things to check**
+
**Check the topic filter you subscribed to**
If you subscribed to the input message topic as described in the procedure, you should see a copy of the input message every time you publish it.
If you don't see the message, check the topic name you subscribed to and compare it to the topic to which you published. Topic names are case sensitive and the topic to which you subscribed must be identical to the topic to which you published the message payload.
+
**Check the message publish function**
In the MQTT client, under **Subscriptions**, choose **device/\$1/data**, check the topic of the publish message, and then choose **Publish to topic**. You should see the message payload from the edit box below the topic appear in the message list.
+
**You don't see your republished message in the MQTT client**
For your rule to work, it must have the correct policy that authorizes it to receive and republish a message and it must receive the message.
**Things to check**
+
**Check the AWS Region of your MQTT client and the rule that you created**
The console in which you're running the MQTT client must be in the same AWS Region as the rule you created.
+
**Check the input message topic in the rule query statement**
For the rule to work, it must receive a message with the topic name that matches the topic filter in the FROM clause of the rule query statement.
Check the spelling of the topic filter in the rule query statement with that of the topic in the MQTT client. Topic names are case sensitive and the message's topic must match the topic filter in the rule query statement.
+
**Check the contents of the input message payload**
For the rule to work, it must find the data field in the message payload that is declared in the SELECT statement.
Check the spelling of the `temperature` field in the rule query statement with that of the message payload in the MQTT client. Field names are case sensitive and the `temperature` field in the rule query statement must be identical to the `temperature` field in the message payload.
Make sure that the JSON document in the message payload is correctly formatted. If the JSON has any errors, such as a missing comma, the rule will not be able to read it.
+
**Check the republished message topic in the rule action**
The topic to which the Republish rule action publishes the new message must match the topic to which you subscribed in the MQTT client.
Open the rule you created in the console and check the topic to which the rule action will republish the message.
+
**Check the role being used by the rule**
The rule action must have permission to receive the original topic and publish the new topic.
The policies that authorize the rule to receive message data and republish it are specific to the topics used. If you change the topic used to republish the message data, you must update the rule action's role to update its policy to match the current topic.
If you suspect this is the problem, edit the Republish rule action and create a new role. New roles created by the rule action receive the authorizations necessary to perform these actions.
## Step 3: Review the results and next steps
**In this tutorial**
+ You used a simple SQL query and a couple of functions in a rule query statement to produce a new MQTT message.
+ You created a rule that republished that new message.
+ You used the MQTT client to test your AWS IoT rule.
**Next steps**
After you republish a few messages with this rule, try experimenting with it to see how changing some aspects of the tutorial affect the republished message. Here are some ideas to get you started.
+ Change the *device\$1id* in the input message's topic and observe the effect in the republished message payload.
+ Change the fields selected in the rule query statement and observe the effect in the republished message payload.
+ Try the next tutorial in this series and learn how to [Tutorial: Sending an Amazon SNS notification](iot-sns-rule.md).
The Republish rule action used in this tutorial can also help you debug rule query statements. For example, you can add this action to a rule to see how its rule query statement is formatting the data used by its rule actions.
# Tutorial: Sending an Amazon SNS notification
This tutorial demonstrates how to create an AWS IoT rule that sends MQTT message data to an Amazon SNS topic so that it can be sent as an SMS text message.
In this tutorial, you create a rule that sends message data from a weather sensor to all subscribers of an Amazon SNS topic, whenever the temperature exceeds the value set in the rule. The rule detects when the reported temperature exceeds the value set by the rule, and then creates a new message payload that includes only the device ID, the reported temperature, and the temperature limit that was exceeded. The rule sends the new message payload as a JSON document to an SNS topic, which notifies all subscribers to the SNS topic.
**What you'll learn in this tutorial:**
+ How to create and test an Amazon SNS notification
+ How to call an Amazon SNS notification from an AWS IoT rule
+ How to use simple SQL queries and functions in a rule query statement
+ How to use the MQTT client to test an AWS IoT rule
This tutorial takes about 30 minutes to complete.
**Topics**
+ [Step 1: Create an Amazon SNS topic that sends a SMS text message](#iot-sns-rule-create-sns-topic)
+ [Step 2: Create an AWS IoT rule to send the text message](#iot-sns-rule-create-rule)
+ [Step 3: Test the AWS IoT rule and Amazon SNS notification](#iot-sns-rule-test-rule)
+ [Step 4: Review the results and next steps](#iot-sns-rule-review-results)
**Before you start this tutorial, make sure that you have:**
+
**[Set up AWS account](setting-up.md)**
You'll need your AWS account and AWS IoT console to complete this tutorial.
+
**Reviewed [View MQTT messages with the AWS IoT MQTT client](view-mqtt-messages.md)**
Be sure you can use the MQTT client to subscribe and publish to a topic. You'll use the MQTT client to test your new rule in this procedure.
+
**Reviewed the [Amazon Simple Notification Service](https://docs.aws.amazon.com//sns/latest/dg/welcome.html)**
If you haven't used Amazon SNS before, review [Setting up access for Amazon SNS](https://docs.aws.amazon.com//sns/latest/dg/sns-setting-up.html). If you've already completed other AWS IoT tutorials, your AWS account should already be configured correctly.
## Step 1: Create an Amazon SNS topic that sends a SMS text message
This procedure exaplains how to create the Amazon SNS topic your weather sensor can send message data to. The Amazon SNS topic will then notify all of its subscribers via a SMS text message of the temperature limit that was exceeded.
**To create an Amazon SNS topic that sends an SMS text message**
1. **Create an Amazon SNS topic.**
1. Sign in to the [Amazon SNS console](https://console.aws.amazon.com//sns/home).
1. In the left navigation pane, choose **Topics**.
1. On the **Topics** page, choose **Create topic**.
1. In **Details**, choose the **Standard** type. By default, the console creates a FIFO topic.
1. In **Name**, enter the SNS topic name. For this tutorial, enter **high\$1temp\$1notice**.
1. Scroll to the end of the page and choose **Create topic**.
The console opens the new topic's **Details** page.
1. **Create an Amazon SNS subscription.**
**Note**
The phone number that you use in this subscription might incur text messaging charges from the messages you will send in this tutorial.
1. In the **high\$1temp\$1notice** topic's details page, choose **Create subscription**.
1. In **Create subscription**, in the **Details** section, in the **Protocol** list, choose **SMS**.
1. In **Endpoint**, enter the number of a phone that can receive text messages. Be sure to enter it such that it starts with a `+`, includes the country and area code, and doesn't include any other punctuation characters.
1. Choose **Create subscription**.
1. **Test the Amazon SNS notification.**
1. In the [Amazon SNS console](https://console.aws.amazon.com//sns/home), in the left navigation pane, choose **Topics**.
1. To open the topic's details page, in **Topics**, in the list of topics, choose **high\$1temp\$1notice**.
1. To open the **Publish message to topic** page, in the **high\$1temp\$1notice** details page, choose **Publish message**.
1. In **Publish message to topic**, in the** Message body** section, in **Message body to send to the endpoint**, enter a short message.
1. Scroll down to the bottom of the page and choose **Publish message**.
1. On the phone with the number you used earlier when creating the subscription, confirm that the message was received.
If you did not receive the test message, double check the phone number and your phone's settings.
Make sure you can publish test messages from the [Amazon SNS console](https://console.aws.amazon.com//sns/home) before you continue the tutorial.
## Step 2: Create an AWS IoT rule to send the text message
The AWS IoT rule that you'll create in this tutorial subscribes to the `device/device_id/data` MQTT topics where `device_id` is the ID of the device that sent the message. These topics are described in a topic filter as `device/+/data`, where the `+` is a wildcard character that matches any string between the two forward slash characters. This rule also tests the value of the `temperature` field in the message payload.
When the rule receives a message from a matching topic, it takes the `device_id` from the topic name, the `temperature` value from the message payload, and adds a constant value for the limit it's testing, and sends these values as a JSON document to an Amazon SNS notification topic.
For example, an MQTT message from weather sensor device number 32 uses the `device/32/data` topic and has a message payload that looks like this:
```
{
"temperature": 38,
"humidity": 80,
"barometer": 1013,
"wind": {
"velocity": 22,
"bearing": 255
}
}
```
The rule's rule query statement takes the `temperature` value from the message payload, the `device_id` from the topic name, and adds the constant `max_temperature` value to send a message payload that looks like this to the Amazon SNS topic:
```
{
"device_id": "32",
"reported_temperature": 38,
"max_temperature": 30
}
```
**To create an AWS IoT rule to detect an over-limit temperature value and create the data to send to the Amazon SNS topic**
1. Open [the **Rules** hub of the AWS IoT console](https://console.aws.amazon.com//iot/home#/rulehub).
1. If this is your first rule, choose **Create**, or **Create a rule**.
1. In **Create a rule**:
1. In **Name**, enter **temp\$1limit\$1notify**.
Remember that a rule name must be unique within your AWS account and Region, and it can't have any spaces. We've used an underscore character in this name to separate the words in the rule's name.
1. In **Description**, describe the rule.
A meaningful description makes it easier to remember what this rule does and why you created it. The description can be as long as needed, so be as detailed as possible.
1. In **Rule query statement** of **Create a rule**:
1. In **Using SQL version**, select **2016-03-23**.
1. In the **Rule query statement** edit box, enter the statement:
```
SELECT topic(2) as device_id,
temperature as reported_temperature,
30 as max_temperature
FROM 'device/+/data'
WHERE temperature > 30
```
This statement:
+ Listens for MQTT messages with a topic that matches the `device/+/data` topic filter and that have a `temperature` value greater than 30.
+ Selects the second element from the topic string and assigns it to the `device_id` field.
+ Selects the value `temperature` field from the message payload and assigns it to the `reported_temperature` field.
+ Creates a constant value `30` to represent the limit value and assigns it to the `max_temperature` field.
1. To open up the list of rule actions for this rule, in **Set one or more actions**, choose **Add action**.
1. In **Select an action**, choose **Send a message as an SNS push notification**.
1. To open the selected action's configuration page, at the bottom of the action list, choose **Configure action**.
1. In **Configure action**:
1. In **SNS target**, choose **Select**, find your SNS topic named **high\$1temp\$1notice**, and choose **Select**.
1. In **Message format**, choose **RAW**.
1. In **Choose or create a role to grant AWS IoT access to perform this action**, choose **Create Role**.
1. In **Create a new role**, in **Name**, enter a unique name for the new role. For this tutorial, use **sns\$1rule\$1role**.
1. Choose **Create role**.
If you're repeating this tutorial or reusing an existing role, choose **Update role** before continuing. This updates the role's policy document to work with the SNS target.
1. Choose **Add action** and return to the **Create a rule** page.
In the new action's tile, below **Send a message as an SNS push notification**, you can see the SNS topic that your rule will call.
This is the only rule action you'll add to this rule.
1. To create the rule and complete this step, in **Create a rule**, scroll down to the bottom and choose **Create rule**.
## Step 3: Test the AWS IoT rule and Amazon SNS notification
To test your new rule, you'll use the MQTT client to publish and subscribe to the MQTT messages used by this rule.
Open the [MQTT client in the AWS IoT console](https://console.aws.amazon.com//iot/home#/test) in a new window. This will let you edit the rule without losing the configuration of your MQTT client. If you leave the MQTT client to go to another page in the console, it won't retain any subscriptions or message logs.
**To use the MQTT client to test your rule**
1. In the [MQTT client in the AWS IoT console](https://console.aws.amazon.com//iot/home#/test), subscribe to the input topics, in this case, `device/+/data`.
1. In the MQTT client, under **Subscriptions**, choose **Subscribe to a topic**.
1. In **Subscription topic**, enter the topic of the input topic filter, **device/\$1/data**.
1. Keep the rest of the fields at their default settings.
1. Choose **Subscribe to topic**.
In the **Subscriptions** column, under **Publish to a topic**, **device/\$1/data** appears.
1. Publish a message to the input topic with a specific device ID, **device/32/data**. You can't publish to MQTT topics that contain wildcard characters.
1. In the MQTT client, under **Subscriptions**, choose **Publish to topic**.
1. In the **Publish** field, enter the input topic name, **device/32/data**.
1. Copy the sample data shown here and, in the edit box below the topic name, paste the sample data.
```
{
"temperature": 38,
"humidity": 80,
"barometer": 1013,
"wind": {
"velocity": 22,
"bearing": 255
}
}
```
1. Choose **Publish to topic** to publish your MQTT message.
1. Confirm that the text message was sent.
1. In the MQTT client, under **Subscriptions**, there is a green dot next to the topic to which you subscribed earlier.
The green dot indicates that one or more new messages have been received since the last time you looked at them.
1. Under **Subscriptions**, choose **device/\$1/data** to check that the message payload matches what you just published and looks like this:
```
{
"temperature": 38,
"humidity": 80,
"barometer": 1013,
"wind": {
"velocity": 22,
"bearing": 255
}
}
```
1. Check the phone that you used to subscribe to the SNS topic and confirm the contents of the message payload look like this:
```
{"device_id":"32","reported_temperature":38,"max_temperature":30}
```
Notice that the `device_id` value is a quoted string and the `temperature` value is numeric. This is because the [https://docs.aws.amazon.com//iot/latest/developerguide/iot-sql-functions.html#iot-function-topic](https://docs.aws.amazon.com//iot/latest/developerguide/iot-sql-functions.html#iot-function-topic) function extracted the string from the input message's topic name while the `temperature` value uses the numeric value from the input message's payload.
If you want to make the `device_id` value a numeric value, replace `topic(2)` in the rule query statement with:
```
cast(topic(2) AS DECIMAL)
```
Note that casting the `topic(2)` value to a numeric, `DECIMAL` value will only work if that part of the topic contains only numeric characters.
1. Try sending an MQTT message in which the temperature does not exceed the limit.
1. In the MQTT client, under **Subscriptions**, choose **Publish to topic**.
1. In the **Publish** field, enter the input topic name, **device/33/data**.
1. Copy the sample data shown here and, in the edit box below the topic name, paste the sample data.
```
{
"temperature": 28,
"humidity": 80,
"barometer": 1013,
"wind": {
"velocity": 22,
"bearing": 255
}
}
```
1. To send your MQTT message, choose **Publish to topic**.
You should see the message that you sent in the **device/\$1/data** subscription. However, because the temperature value is below the max temperature in the rule query statement, you shouldn't receive a text message.
If you don't see the correct behavior, check the troubleshooting tips.
### Troubleshooting your SNS message rule
Here are some things to check, in case you're not seeing the results you expect.
+
**You got an error banner**
If an error appeared when you published the input message, correct that error first. The following steps might help you correct that error.
+
**You don't see the input message in the MQTT client**
Every time you publish your input message to the `device/22/data` topic, that message should appear in the MQTT client, if you subscribed to the `device/+/data` topic filter as described in the procedure.
**Things to check**
+
**Check the topic filter you subscribed to**
If you subscribed to the input message topic as described in the procedure, you should see a copy of the input message every time you publish it.
If you don't see the message, check the topic name you subscribed to and compare it to the topic to which you published. Topic names are case sensitive and the topic to which you subscribed must be identical to the topic to which you published the message payload.
+
**Check the message publish function**
In the MQTT client, under **Subscriptions**, choose **device/\$1/data**, check the topic of the publish message, and then choose **Publish to topic**. You should see the message payload from the edit box below the topic appear in the message list.
+
**You don't receive an SMS message**
For your rule to work, it must have the correct policy that authorizes it to receive a message and send an SNS notification, and it must receive the message.
**Things to check**
+
**Check the AWS Region of your MQTT client and the rule that you created**
The console in which you're running the MQTT client must be in the same AWS Region as the rule you created.
+
**Check that the temperature value in the message payload exceeds the test threshold**
If the temperature value is less than or equal to 30, as defined in the rule query statement, the rule will not perform any of its actions.
+
**Check the input message topic in the rule query statement**
For the rule to work, it must receive a message with the topic name that matches the topic filter in the FROM clause of the rule query statement.
Check the spelling of the topic filter in the rule query statement with that of the topic in the MQTT client. Topic names are case sensitive and the message's topic must match the topic filter in the rule query statement.
+
**Check the contents of the input message payload**
For the rule to work, it must find the data field in the message payload that is declared in the SELECT statement.
Check the spelling of the `temperature` field in the rule query statement with that of the message payload in the MQTT client. Field names are case sensitive and the `temperature` field in the rule query statement must be identical to the `temperature` field in the message payload.
Make sure that the JSON document in the message payload is correctly formatted. If the JSON has any errors, such as a missing comma, the rule will not be able to read it.
+
**Check the republished message topic in the rule action**
The topic to which the Republish rule action publishes the new message must match the topic to which you subscribed in the MQTT client.
Open the rule you created in the console and check the topic to which the rule action will republish the message.
+
**Check the role being used by the rule**
The rule action must have permission to receive the original topic and publish the new topic.
The policies that authorize the rule to receive message data and republish it are specific to the topics used. If you change the topic used to republish the message data, you must update the rule action's role to update its policy to match the current topic.
If you suspect this is the problem, edit the Republish rule action and create a new role. New roles created by the rule action receive the authorizations necessary to perform these actions.
## Step 4: Review the results and next steps
**In this tutorial:**
+ You created and tested an Amazon SNS notification topic and subscription.
+ You used a simple SQL query and functions in a rule query statement to create a new message for your notification.
+ You created an AWS IoT rule to send an Amazon SNS notification that used your customized message payload.
+ You used the MQTT client to test your AWS IoT rule.
**Next steps**
After you send a few text messages with this rule, try experimenting with it to see how changing some aspects of the tutorial affect the message and when it's sent. Here are some ideas to get you started.
+ Change the *device\$1id* in the input message's topic and observe the effect in the text message contents.
+ Change the fields selected in the rule query statement and observe the effect in the text message contents.
+ Change the test in the rule query statement to test for a minimum temperature instead of a maximum temperature. Remember to change the name of `max_temperature`\$1
+ Add a republish rule action to send an MQTT message when an SNS notification is sent.
+ Try the next tutorial in this series and learn how to [Tutorial: Storing device data in a DynamoDB table](iot-ddb-rule.md).
# Tutorial: Storing device data in a DynamoDB table
This tutorial demonstrates how to create an AWS IoT rule that sends message data to a DynamoDB table.
In this tutorial, you create a rule that sends message data from an imaginary weather sensor device to a DynamoDB table. The rule formats the data from many weather sensors such that they can be added to a single database table.
**What you'll learn in this tutorial**
+ How to create a DynamoDB table
+ How to send message data to a DynamoDB table from an AWS IoT rule
+ How to use substitution templates in an AWS IoT rule
+ How to use simple SQL queries and functions in a rule query statement
+ How to use the MQTT client to test an AWS IoT rule
This tutorial takes about 30 minutes to complete.
**Topics**
+ [Step 1: Create the DynamoDB table for this tutorial](#iot-ddb-rule-ddb-table)
+ [Step 2: Create an AWS IoT rule to send data to the DynamoDB table](#iot-ddb-rule-topic-rule)
+ [Step 3: Test the AWS IoT rule and DynamoDB table](#iot-ddb-rule-test)
+ [Step 4: Review the results and next steps](#iot-ddb-rule-review)
**Before you start this tutorial, make sure that you have:**
+
**[Set up AWS account](setting-up.md)**
You'll need your AWS account and AWS IoT console to complete this tutorial.
+
**Reviewed [View MQTT messages with the AWS IoT MQTT client](view-mqtt-messages.md)**
Be sure you can use the MQTT client to subscribe and publish to a topic. You'll use the MQTT client to test your new rule in this procedure.
+
**Reviewed the [Amazon DynamoDB](https://docs.aws.amazon.com//amazondynamodb/latest/developerguide/Introduction.html) overview**
If you've not used DynamoDB before, review [Getting Started with DynamoDB](https://docs.aws.amazon.com//amazondynamodb/latest/developerguide/GettingStartedDynamoDB.html) to become familiar with the basic concepts and operations of DynamoDB.
## Step 1: Create the DynamoDB table for this tutorial
In this tutorial, you'll create a DynamoDB table with these attributes to record the data from the imaginary weather sensor devices:
+ `sample_time` is a primary key and describes the time the sample was recorded.
+ `device_id` is a sort key and describes the device that provided the sample
+ `device_data` is the data received from the device and formatted by the rule query statement
**To create the DynamoDB table for this tutorial**
1. Open the [DynamoDB console](https://console.aws.amazon.com//dynamodb/home), and then choose **Create table**.
1. In **Create table**:
1. In **Table name**, enter the table name: **wx\$1data**.
1. In **Partition key**, enter **sample\$1time**, and in the option list next to the field, choose **Number**.
1. In **Sort key**, enter **device\$1id**, and in the option list next to the field, choose **Number**.
1. At the bottom of the page, choose **Create**.
You'll define `device_data` later, when you configure the DynamoDB rule action.
## Step 2: Create an AWS IoT rule to send data to the DynamoDB table
In this step, you'll use the rule query statement to format the data from the imaginary weather sensor devices to write to the database table.
A sample message payload received from a weather sensor device looks like this:
```
{
"temperature": 28,
"humidity": 80,
"barometer": 1013,
"wind": {
"velocity": 22,
"bearing": 255
}
}
```
For the database entry, you'll use the rule query statement to flatten the structure of the message payload to look like this:
```
{
"temperature": 28,
"humidity": 80,
"barometer": 1013,
"wind_velocity": 22,
"wind_bearing": 255
}
```
In this rule, you'll also use a couple of [Substitution templates](iot-substitution-templates.md). Substitution templates are expressions that let you insert dynamic values from functions and message data.
**To create the AWS IoT rule to send data to the DynamoDB table**
1. Open [the Rules hub of the AWS IoT console](https://console.aws.amazon.com//iot/home#/rulehub). Or, you can open the AWS IoT homepage within the AWS Management Console and navigate to **Message routing>Rules**.
1. To start creating your new rule in **Rules**, choose **Create rule**.
1. In **Rule properties**:
1. In **Rule name**, enter **wx\$1data\$1ddb**.
Remember that a rule name must be unique within your AWS account and Region, and it can't have any spaces. We've used an underscore character in this name to separate the two words in the rule's name.
1. In **Rule description**, describe the rule.
A meaningful description makes it easier to remember what this rule does and why you created it. The description can be as long as needed, so be as detailed as possible.
1. Choose **Next** to continue.
1. In **SQL statement**:
1. In **SQL version**, select **2016-03-23**.
1. In the **SQL statement** edit box, enter the statement:
```
SELECT temperature, humidity, barometer,
wind.velocity as wind_velocity,
wind.bearing as wind_bearing,
FROM 'device/+/data'
```
This statement:
+ Listens for MQTT messages with a topic that matches the `device/+/data` topic filter.
+ Formats the elements of the `wind` attribute as individual attributes.
+ Passes the `temperature`, `humidity`, and `barometer` attributes unchanged.
1. Choose **Next** to continue.
1. In **Rule actions**:
1. To open the list of rule actions for this rule, in **Action 1**, choose **DynamoDB**.
**Note**
Make sure that you choose DynamoDB and not DynamoDBv2 as the rule action.
1. In **Table name**, choose the name of the DynamoDB table you created in a previous step: **wx\$1data**.
The **Partition key type** and **Sort key type** fields are filled with the values from your DynamoDB table.
1. In **Partition key**, enter **sample\$1time**.
1. In **Partition key value**, enter **\$1\$1timestamp()\$1**.
This is the first of the [Substitution templates](iot-substitution-templates.md) you'll use in this rule. Instead of using a value from the message payload, it will use the value returned from the timestamp function. To learn more, see [timestamp](iot-sql-functions.md#iot-function-timestamp) in the *AWS IoT Core Developer Guide*.
1. In **Sort key**, enter **device\$1id**.
1. In **Sort key value**, enter **\$1\$1cast(topic(2) AS DECIMAL)\$1**.
This is the second one of the [Substitution templates](iot-substitution-templates.md) you'll use in this rule. It inserts the value of the second element in topic name, which is the device's ID, after it casts it to a DECIMAL value to match the numeric format of the key. To learn more about topics, see [topic](iot-sql-functions.md#iot-function-topic) in the *AWS IoT Core Developer Guide*. Or to learn more about casting, see [cast](iot-sql-functions.md#iot-sql-function-cast) in the *AWS IoT Core Developer Guide*.
1. In **Write message data to this column**, enter **device\$1data**.
This will create the `device_data` column in the DynamoDB table.
1. Leave **Operation** blank.
1. In **IAM role**, choose **Create new role**.
1. In the **Create role** dialog box, for **Role name**, enter **wx\$1ddb\$1role**. This new role will automatically contain a policy with a prefix of "aws-iot-rule" that will allow the **wx\$1data\$1ddb** rule to send data to the **wx\$1data** DynamoDB table you created.
1. In **IAM role**, choose **wx\$1ddb\$1role**.
1. At the bottom of the page, choose **Next**.
1. At the bottom of the **Review and create** page, choose **Create** to create the rule.
## Step 3: Test the AWS IoT rule and DynamoDB table
To test the new rule, you'll use the MQTT client to publish and subscribe to the MQTT messages used in this test.
Open the [MQTT client in the AWS IoT console](https://console.aws.amazon.com//iot/home#/test) in a new window. This will let you edit the rule without losing the configuration of your MQTT client. The MQTT client does not retain any subscriptions or message logs if you leave it to go to another page in the console. You'll also want a separate console window open to the [DynamoDB Tables hub in the AWS IoT console](https://console.aws.amazon.com//dynamodb/home#tables:) to view the new entries that your rule sends.
**To use the MQTT client to test your rule**
1. In the [MQTT client in the AWS IoT console](https://console.aws.amazon.com//iot/home#/test), subscribe to the input topic, `device/+/data`.
1. In the MQTT client, choose **Subscribe to a topic**.
1. For **Topic filter**, enter the topic of the input topic filter, **device/\$1/data**.
1. Choose **Subscribe**.
1. Now, publish a message to the input topic with a specific device ID, **device/22/data**. You can't publish to MQTT topics that contain wildcard characters.
1. In the MQTT client, choose **Publish to a topic**.
1. For **Topic name**, enter the input topic name, **device/22/data**.
1. For **Message payload**, enter the following sample data.
```
{
"temperature": 28,
"humidity": 80,
"barometer": 1013,
"wind": {
"velocity": 22,
"bearing": 255
}
}
```
1. To publish the MQTT message, choose **Publish**.
1. Now, in the MQTT client, choose **Subscribe to a topic**. In the **Subscribe** column, choose the **device/\$1/data** subscription. Confirm that the sample data from the previous step appears there.
1. Check to see the row in the DynamoDB table that your rule created.
1. In the [DynamoDB Tables hub in the AWS IoT console](https://console.aws.amazon.com//dynamodb/home#tables:), choose **wx\$1data**, and then choose the **Items** tab.
If you're already on the **Items** tab, you might need to refresh the display by choosing the refresh icon in the upper-right corner of the table's header.
1. Notice that the **sample\$1time** values in the table are links and open one. If you just sent your first message, it will be the only one in the list.
This link displays all the data in that row of the table.
1. Expand the **device\$1data** entry to see the data that resulted from the rule query statement.
1. Explore the different representations of the data that are available in this display. You can also edit the data in this display.
1. After you have finished reviewing this row of data, to save any changes you made, choose **Save**, or to exit without saving any changes, choose **Cancel**.
If you don't see the correct behavior, check the troubleshooting tips.
### Troubleshooting your DynamoDB rule
Here are some things to check in case you're not seeing the results you expect.
+
**You got an error banner**
If an error appeared when you published the input message, correct that error first. The following steps might help you correct that error.
+
**You don't see the input message in the MQTT client**
Every time you publish your input message to the `device/22/data` topic, that message should appear in the MQTT client if you subscribed to the `device/+/data` topic filter as described in the procedure.
**Things to check**
+
**Check the topic filter you subscribed to**
If you subscribed to the input message topic as described in the procedure, you should see a copy of the input message every time you publish it.
If you don't see the message, check the topic name you subscribed to and compare it to the topic to which you published. Topic names are case sensitive and the topic to which you subscribed must be identical to the topic to which you published the message payload.
+
**Check the message publish function**
In the MQTT client, under **Subscriptions**, choose **device/\$1/data**, check the topic of the publish message, and then choose **Publish to topic**. You should see the message payload from the edit box below the topic appear in the message list.
+
**You don't see your data in the DynamoDB table**
The first thing to do is to refresh the display by choosing the refresh icon in the upper-right corner of the table's header. If that doesn't display the data you're looking for, check the following.
**Things to check**
+
**Check the AWS Region of your MQTT client and the rule that you created**
The console in which you're running the MQTT client must be in the same AWS Region as the rule you created.
+
**Check the input message topic in the rule query statement**
For the rule to work, it must receive a message with the topic name that matches the topic filter in the FROM clause of the rule query statement.
Check the spelling of the topic filter in the rule query statement with that of the topic in the MQTT client. Topic names are case sensitive and the message's topic must match the topic filter in the rule query statement.
+
**Check the contents of the input message payload**
For the rule to work, it must find the data field in the message payload that is declared in the SELECT statement.
Check the spelling of the `temperature` field in the rule query statement with that of the message payload in the MQTT client. Field names are case sensitive and the `temperature` field in the rule query statement must be identical to the `temperature` field in the message payload.
Make sure that the JSON document in the message payload is correctly formatted. If the JSON has any errors, such as a missing comma, the rule will not be able to read it.
+
**Check the key and field names used in the rule action**
The field names used in the topic rule must match those found in the JSON message payload of the published message.
Open the rule you created in the console and check the field names in the rule action configuration with those used in the MQTT client.
+
**Check the role being used by the rule**
The rule action must have permission to receive the original topic and publish the new topic.
The policies that authorize the rule to receive message data and update the DynamoDB table are specific to the topics used. If you change the topic or DynamoDB table name used by the rule, you must update the rule action's role to update its policy to match.
If you suspect this is the problem, edit the rule action and create a new role. New roles created by the rule action receive the authorizations necessary to perform these actions.
## Step 4: Review the results and next steps
After you send a few messages to the DynamoDB table with this rule, try experimenting with it to see how changing some aspects from the tutorial affect the data written to the table. Here are some ideas to get you started.
+ Change the *device\$1id* in the input message's topic and observe the effect on the data. You could use this to simulate receiving data from multiple weather sensors.
+ Change the fields selected in the rule query statement and observe the effect on the data. You could use this to filter the data stored in the table.
+ Add a republish rule action to send an MQTT message for each row added to the table. You could use this for debugging.
After you have completed this tutorial, check out [Tutorial: Formatting a notification by using an AWS Lambda function](iot-lambda-rule.md).
# Tutorial: Formatting a notification by using an AWS Lambda function
This tutorial demonstrates how to send MQTT message data to an AWS Lambda action for formatting and sending to another AWS service. In this tutorial, the AWS Lambda action uses the AWS SDK to send the formatted message to the Amazon SNS topic you created in the tutorial about how to [Tutorial: Sending an Amazon SNS notification](iot-sns-rule.md).
In the tutorial about how to [Tutorial: Sending an Amazon SNS notification](iot-sns-rule.md), the JSON document that resulted from the rule's query statement was sent as the body of the text message. The result was a text message that looked something like this example:
```
{"device_id":"32","reported_temperature":38,"max_temperature":30}
```
In this tutorial, you'll use an AWS Lambda rule action to call an AWS Lambda function that formats the data from the rule query statement into a friendlier format, such as this example:
```
Device 32 reports a temperature of 38, which exceeds the limit of 30.
```
The AWS Lambda function you'll create in this tutorial formats the message string by using the data from the rule query statement and calls the [SNS publish](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/sns.html#SNS.Client.publish) function of the AWS SDK to create the notification.
**What you'll learn in this tutorial**
+ How to create and test an AWS Lambda function
+ How to use the AWS SDK in an AWS Lambda function to publish an Amazon SNS notification
+ How to use simple SQL queries and functions in a rule query statement
+ How to use the MQTT client to test an AWS IoT rule
This tutorial takes about 45 minutes to complete.
**Topics**
+ [Step 1: Create an AWS Lambda function that sends a text message](#iot-lambda-rule-create-lambda)
+ [Step 2: Create an AWS IoT rule with an AWS Lambda rule action](#iot-lambda-rule-create-rule)
+ [Step 3: Test the AWS IoT rule and AWS Lambda rule action](#iot-lambda-rule-test-rule)
+ [Step 4: Review the results and next steps](#iot-lambda-rule-next-steps)
**Before you start this tutorial, make sure that you have:**
+
**[Set up AWS account](setting-up.md)**
You'll need your AWS account and AWS IoT console to complete this tutorial.
+
**Reviewed [View MQTT messages with the AWS IoT MQTT client](view-mqtt-messages.md)**
Be sure you can use the MQTT client to subscribe and publish to a topic. You'll use the MQTT client to test your new rule in this procedure.
+
**Completed the other rules tutorials in this section**
This tutorial requires the SNS notification topic you created in the tutorial about how to [Tutorial: Sending an Amazon SNS notification](iot-sns-rule.md). It also assumes that you've completed the other rules-related tutorials in this section.
+
**Reviewed the [AWS Lambda](https://docs.aws.amazon.com//lambda/latest/dg/welcome.html) overview**
If you haven't used AWS Lambda before, review [AWS Lambda](https://docs.aws.amazon.com//lambda/latest/dg/welcome.html) and [Getting started with Lambda](https://docs.aws.amazon.com//lambda/latest/dg/getting-started.html) to learn its terms and concepts.
## Step 1: Create an AWS Lambda function that sends a text message
The AWS Lambda function in this tutorial receives the result of the rule query statement, inserts the elements into a text string, and sends the resulting string to Amazon SNS as the message in a notification.
Unlike the tutorial about how to [Tutorial: Sending an Amazon SNS notification](iot-sns-rule.md), which used an AWS IoT rule action to send the notification, this tutorial sends the notification from the Lambda function by using a function of the AWS SDK. The actual Amazon SNS notification topic used in this tutorial, however, is the same one that you used in the tutorial about how to [Tutorial: Sending an Amazon SNS notification](iot-sns-rule.md).
**To create an AWS Lambda function that sends a text message**
1. Create a new AWS Lambda function.
1. In the [AWS Lambda console](https://console.aws.amazon.com//lambda/home), choose **Create function**.
1. In **Create function**, select **Use a blueprint**.
Search for and select the **hello-world-python** blueprint, and then choose **Configure**.
1. In **Basic information**:
1. In **Function name**, enter the name of this function, **format-high-temp-notification**.
1. In **Execution role**, choose **Create a new role from AWS policy templates**.
1. In Role name, enter the name of the new role, **format-high-temp-notification-role**.
1. In **Policy templates - *optional***, search for and select **Amazon SNS publish policy**.
1. Choose **Create function**.
1. Modify the blueprint code to format and send an Amazon SNS notification.
1. After you created your function, you should see the **format-high-temp-notification** details page. If you don't, open it from the [Lambda **Functions**](https://console.aws.amazon.com//lambda/home#/functions) page.
1. In the **format-high-temp-notification** details page, choose the **Configuration** tab and scroll to the **Function code** panel.
1. In the **Function code** window, in the **Environment** pane, choose the Python file, `lambda_function.py`.
1. In the **Function code** window, delete all of the original program code from the blueprint and replace it with this code.
```
import boto3
#
# expects event parameter to contain:
# {
# "device_id": "32",
# "reported_temperature": 38,
# "max_temperature": 30,
# "notify_topic_arn": "arn:aws:sns:us-east-1:57EXAMPLE833:high_temp_notice"
# }
#
# sends a plain text string to be used in a text message
#
# "Device {0} reports a temperature of {1}, which exceeds the limit of {2}."
#
# where:
# {0} is the device_id value
# {1} is the reported_temperature value
# {2} is the max_temperature value
#
def lambda_handler(event, context):
# Create an SNS client to send notification
sns = boto3.client('sns')
# Format text message from data
message_text = "Device {0} reports a temperature of {1}, which exceeds the limit of {2}.".format(
str(event['device_id']),
str(event['reported_temperature']),
str(event['max_temperature'])
)
# Publish the formatted message
response = sns.publish(
TopicArn = event['notify_topic_arn'],
Message = message_text
)
return response
```
1. Choose **Deploy**.
1. In a new window, look up the Amazon Resource Name (ARN) of your Amazon SNS topic from the tutorial about how to [Tutorial: Sending an Amazon SNS notification](iot-sns-rule.md).
1. In a new window, open the [Topics page of the Amazon SNS console](https://console.aws.amazon.com//sns/v3/home#/topics).
1. In the **Topics** page, find the **high\$1temp\$1notice** notification topic in the list of Amazon SNS topics.
1. Find the **ARN** of the **high\$1temp\$1notice** notification topic to use in the next step.
1. Create a test case for your Lambda function.
1. In the [Lambda **Functions**](https://console.aws.amazon.com//lambda/home#/functions) page of the console, on the **format-high-temp-notification** details page, choose **Select a test event** in the upper right corner of the page (even though it looks disabled), and then choose **Configure test events**.
1. In **Configure test event**, choose **Create new test event**.
1. In **Event name**, enter **SampleRuleOutput**.
1. In the JSON editor below **Event name**, paste this sample JSON document. This is an example of what your AWS IoT rule will send to the Lambda function.
```
{
"device_id": "32",
"reported_temperature": 38,
"max_temperature": 30,
"notify_topic_arn": "arn:aws:sns:us-east-1:57EXAMPLE833:high_temp_notice"
}
```
1. Refer to the window that has the **ARN** of the **high\$1temp\$1notice** notification topic and copy the ARN value.
1. Replace the `notify_topic_arn` value in the JSON editor with the ARN from your notification topic.
Keep this window open so you can use this ARN value again when you create the AWS IoT rule.
1. Choose **Create**.
1. Test the function with sample data.
1. In the **format-high-temp-notification** details page, in the upper-right corner of the page, confirm that **SampleRuleOutput** appears next to the **Test** button. If it doesn't, choose it from the list of available test events.
1. To send the sample rule output message to your function, choose **Test**.
If the function and the notification both worked, you will get a text message on the phone that subscribed to the notification.
If you didn't get a text message on the phone, check the result of the operation. In the **Function code** panel, in the **Execution result** tab, review the response to find any errors that occurred. Don't continue to the next step until your function can send the notification to your phone.
## Step 2: Create an AWS IoT rule with an AWS Lambda rule action
In this step, you'll use the rule query statement to format the data from the imaginary weather sensor device to send to a Lambda function, which will format and send a text message.
A sample message payload received from the weather devices looks like this:
```
{
"temperature": 28,
"humidity": 80,
"barometer": 1013,
"wind": {
"velocity": 22,
"bearing": 255
}
}
```
In this rule, you'll use the rule query statement to create a message payload for the Lambda function that looks like this:
```
{
"device_id": "32",
"reported_temperature": 38,
"max_temperature": 30,
"notify_topic_arn": "arn:aws:sns:us-east-1:57EXAMPLE833:high_temp_notice"
}
```
This contains all the information the Lambda function needs to format and send the correct text message.
**To create the AWS IoT rule to call a Lambda function**
1. Open the [**Rules** hub of the AWS IoT console](https://console.aws.amazon.com//iot/home#/rulehub).
1. To start creating your new rule in **Rules**, choose **Create**.
1. In the top part of **Create a rule**:
1. In **Name**, enter the rule's name, **wx\$1friendly\$1text**.
Remember that a rule name must be unique within your AWS account and Region, and it can't have any spaces. We've used an underscore character in this name to separate the two words in the rule's name.
1. In **Description**, describe the rule.
A meaningful description makes it easier to remember what this rule does and why you created it. The description can be as long as needed, so be as detailed as possible.
1. In **Rule query statement** of **Create a rule**:
1. In **Using SQL version**, select **2016-03-23**.
1. In the **Rule query statement** edit box, enter the statement:
```
SELECT
cast(topic(2) AS DECIMAL) as device_id,
temperature as reported_temperature,
30 as max_temperature,
'arn:aws:sns:us-east-1:57EXAMPLE833:high_temp_notice' as notify_topic_arn
FROM 'device/+/data' WHERE temperature > 30
```
This statement:
+ Listens for MQTT messages with a topic that matches the `device/+/data` topic filter and that have a `temperature` value greater than 30.
+ Selects the second element from the topic string, converts it to a decimal number, and then assigns it to the `device_id` field.
+ Selects the value of the `temperature` field from the message payload and assigns it to the `reported_temperature` field.
+ Creates a constant value, `30`, to represent the limit value and assigns it to the `max_temperature` field.
+ Creates a constant value for the `notify_topic_arn` field.
1. Refer to the window that has the **ARN** of the **high\$1temp\$1notice** notification topic and copy the ARN value.
1. Replace the ARN value (*arn:aws:sns:us-east-1:57EXAMPLE833:high\$1temp\$1notice*) in the rule query statement editor with the ARN of your notification topic.
1. In **Set one or more actions**:
1. To open up the list of rule actions for this rule, choose **Add action**.
1. In **Select an action**, choose **Send a message to a Lambda function**.
1. To open the selected action's configuration page, at the bottom of the action list, choose **Configure action**.
1. In **Configure action**:
1. In **Function name**, choose **Select**.
1. Choose **format-high-temp-notification**.
1. At the bottom of **Configure action**, choose **Add action**.
1. To create the rule, at the bottom of **Create a rule**, choose **Create rule**.
## Step 3: Test the AWS IoT rule and AWS Lambda rule action
To test your new rule, you'll use the MQTT client to publish and subscribe to the MQTT messages used by this rule.
Open the [MQTT client in the AWS IoT console](https://console.aws.amazon.com//iot/home#/test) in a new window. Now you can edit the rule without losing the configuration of your MQTT client. If you leave the MQTT client to go to another page in the console, you'll lose your subscriptions or message logs.
**To use the MQTT client to test your rule**
1. In the [MQTT client in the AWS IoT console](https://console.aws.amazon.com//iot/home#/test), subscribe to the input topics, in this case, `device/+/data`.
1. In the MQTT client, under **Subscriptions**, choose **Subscribe to a topic**.
1. In **Subscription topic**, enter the topic of the input topic filter, **device/\$1/data**.
1. Keep the rest of the fields at their default settings.
1. Choose **Subscribe to topic**.
In the **Subscriptions** column, under **Publish to a topic**, **device/\$1/data** appears.
1. Publish a message to the input topic with a specific device ID, **device/32/data**. You can't publish to MQTT topics that contain wildcard characters.
1. In the MQTT client, under **Subscriptions**, choose **Publish to topic**.
1. In the **Publish** field, enter the input topic name, **device/32/data**.
1. Copy the sample data shown here and, in the edit box below the topic name, paste the sample data.
```
{
"temperature": 38,
"humidity": 80,
"barometer": 1013,
"wind": {
"velocity": 22,
"bearing": 255
}
}
```
1. To publish your MQTT message, choose **Publish to topic**.
1. Confirm that the text message was sent.
1. In the MQTT client, under **Subscriptions**, there is a green dot next to the topic to which you subscribed earlier.
The green dot indicates that one or more new messages have been received since the last time you looked at them.
1. Under **Subscriptions**, choose **device/\$1/data** to check that the message payload matches what you just published and looks like this:
```
{
"temperature": 38,
"humidity": 80,
"barometer": 1013,
"wind": {
"velocity": 22,
"bearing": 255
}
}
```
1. Check the phone that you used to subscribe to the SNS topic and confirm the contents of the message payload look like this:
```
Device 32 reports a temperature of 38, which exceeds the limit of 30.
```
If you change the topic ID element in the message topic, remember that casting the `topic(2)` value to a numeric value will only work if that element in the message topic contains only numeric characters.
1. Try sending an MQTT message in which the temperature does not exceed the limit.
1. In the MQTT client, under **Subscriptions**, choose **Publish to topic**.
1. In the **Publish** field, enter the input topic name, **device/33/data**.
1. Copy the sample data shown here and, in the edit box below the topic name, paste the sample data.
```
{
"temperature": 28,
"humidity": 80,
"barometer": 1013,
"wind": {
"velocity": 22,
"bearing": 255
}
}
```
1. To send your MQTT message, choose **Publish to topic**.
You should see the message that you sent in the **device/\$1/data** subscription; however, because the temperature value is below the max temperature in the rule query statement, you shouldn't receive a text message.
If you don't see the correct behavior, check the troubleshooting tips.
### Troubleshooting your AWS Lambda rule and notification
Here are some things to check, in case you're not seeing the results you expect.
+
**You got an error banner**
If an error appeared when you published the input message, correct that error first. The following steps might help you correct that error.
+
**You don't see the input message in the MQTT client**
Every time you publish your input message to the `device/32/data` topic, that message should appear in the MQTT client, if you subscribed to the `device/+/data` topic filter as described in the procedure.
**Things to check**
+
**Check the topic filter you subscribed to**
If you subscribed to the input message topic as described in the procedure, you should see a copy of the input message every time you publish it.
If you don't see the message, check the topic name you subscribed to and compare it to the topic to which you published. Topic names are case sensitive and the topic to which you subscribed must be identical to the topic to which you published the message payload.
+
**Check the message publish function**
In the MQTT client, under **Subscriptions**, choose **device/\$1/data**, check the topic of the publish message, and then choose **Publish to topic**. You should see the message payload from the edit box below the topic appear in the message list.
+
**You don't receive an SMS message**
For your rule to work, it must have the correct policy that authorizes it to receive a message and send an SNS notification, and it must receive the message.
**Things to check**
+
**Check the AWS Region of your MQTT client and the rule that you created**
The console in which you're running the MQTT client must be in the same AWS Region as the rule you created.
+
**Check that the temperature value in the message payload exceeds the test threshold**
If the temperature value is less than or equal to 30, as defined in the rule query statement, the rule will not perform any of its actions.
+
**Check the input message topic in the rule query statement**
For the rule to work, it must receive a message with the topic name that matches the topic filter in the FROM clause of the rule query statement.
Check the spelling of the topic filter in the rule query statement with that of the topic in the MQTT client. Topic names are case sensitive and the message's topic must match the topic filter in the rule query statement.
+
**Check the contents of the input message payload**
For the rule to work, it must find the data field in the message payload that is declared in the SELECT statement.
Check the spelling of the `temperature` field in the rule query statement with that of the message payload in the MQTT client. Field names are case sensitive and the `temperature` field in the rule query statement must be identical to the `temperature` field in the message payload.
Make sure that the JSON document in the message payload is correctly formatted. If the JSON has any errors, such as a missing comma, the rule will not be able to read it.
+
**Check the Amazon SNS notification**
In [Step 1: Create an Amazon SNS topic that sends a SMS text message](iot-sns-rule.md#iot-sns-rule-create-sns-topic), refer to step 3 that describes how to test the Amazon SNS notification and test the notification to make sure the notification works.
+
**Check the Lambda function**
In [Step 1: Create an AWS Lambda function that sends a text message](#iot-lambda-rule-create-lambda), refer to step 5 that describes how to test the Lambda function using test data and test the Lambda function.
+
**Check the role being used by the rule**
The rule action must have permission to receive the original topic and publish the new topic.
The policies that authorize the rule to receive message data and republish it are specific to the topics used. If you change the topic used to republish the message data, you must update the rule action's role to update its policy to match the current topic.
If you suspect this is the problem, edit the Republish rule action and create a new role. New roles created by the rule action receive the authorizations necessary to perform these actions.
## Step 4: Review the results and next steps
**In this tutorial:**
+ You created an AWS IoT rule to call a Lambda function that sent an Amazon SNS notification that used your customized message payload.
+ You used a simple SQL query and functions in a rule query statement to create a new message payload for your Lambda function.
+ You used the MQTT client to test your AWS IoT rule.
**Next steps**
After you send a few text messages with this rule, try experimenting with it to see how changing some aspects of the tutorial affect the message and when it's sent. Here are some ideas to get you started.
+ Change the *device\$1id* in the input message's topic and observe the effect in the text message contents.
+ Change the fields selected in the rule query statement, update the Lambda function to use them in a new message, and observe the effect in the text message contents.
+ Change the test in the rule query statement to test for a minimum temperature instead of a maximum temperature. Update the Lambda function to format a new message and remember to change the name of `max_temperature`.
+ To learn more about how to find errors that might occur while you're developing and using AWS IoT rules, see [Monitoring AWS IoT](monitoring_overview.md).
# Retaining device state while the device is offline with Device Shadows
These tutorials show you how to use the AWS IoT Device Shadow service to store and update the state information of a device. The Shadow document, which is a JSON document, shows the change in the device's state based on the messages published by a device, local app, or service. In this tutorial, the Shadow document shows the change in the color of a light bulb. These tutorials also show how the shadow stores this information even when the device is disconnected from the internet, and passes the latest state information back to the device when it comes back online and requests this information.
We recommend that you try these tutorials in the order they're shown here, starting with the AWS IoT resources you need to create and the necessary hardware setup, which also helps you learn the concepts incrementally. These tutorials show how to configure and connect a Raspberry Pi device for use with AWS IoT. If you don't have the required hardware, you can follow these tutorials by adapting them to a device of your choice or by [creating a virtual device with Amazon EC2](creating-a-virtual-thing.md).
**Tutorial scenario overview**
The scenario for these tutorials is a local app or service that changes the color of a light bulb and that publishes its data to reserved shadow topics. These tutorials are similar to the Device Shadow functionality described in the [interactive getting started tutorial](interactive-demo.md) and are implemented on a Raspberry Pi device. The tutorials in this section focus on a single, classic shadow while showing how you might accommodate named shadows or multiple devices.
The following tutorials will help you learn how to use the AWS IoT Device Shadow service.
+
**[Tutorial: Preparing your Raspberry Pi to run the shadow application](create-resources-shadow.md)**
This tutorial shows how to set up a Raspberry Pi device for connecting with AWS IoT. You'll also create an AWS IoT policy document and a thing resource, download the certificates, and then attach the policy to that thing resource. This tutorial takes about 30 minutes to complete.
+
**[Tutorial: Installing the Device SDK and running the sample application for Device Shadows](lightbulb-shadow-application.md)**
This tutorial shows how to install the required tools, software, and the AWS IoT Device SDK for Python, and then run the sample shadow application. This tutorial builds on concepts presented in [Connect a Raspberry Pi or other device](connecting-to-existing-device.md) and takes 20 minutes to complete.
+
**[Tutorial: Interacting with Device Shadow using the sample app and the MQTT test client](interact-lights-device-shadows.md)**
This tutorial shows how you use the `shadow.py` sample app and **AWS IoT console** to observe the interaction between AWS IoT Device Shadows and the state changes of the light bulb. The tutorial also shows how to send MQTT messages to the Device Shadow's reserved topics. This tutorial can take 45 minutes to complete.
**AWS IoT Device Shadow overview**
A Device Shadow is a persistent, virtual representation of a device that is managed by a [thing resource](iot-thing-management.md) you create in the AWS IoT registry. The Shadow document is a JSON or a JavaScript notation doc that is used to store and retrieve the current state information for a device. You can use the shadow to get and set the state of a device over MQTT topics or HTTP REST APIs, regardless of whether the device is connected to the internet.
A Shadow document contains a `state` property that describes these aspects of the device's state.
+ `desired`: Apps specify the desired states of device properties by updating the `desired` object.
+ `reported`: Devices report their current state in the `reported` object.
+ `delta`: AWS IoT reports differences between the desired and the reported state in the `delta` object.
Here is an example of a Shadow state document.
```
{
"state": {
"desired": {
"color": "green"
},
"reported": {
"color": "blue"
},
"delta": {
"color": "green"
}
}
}
```
To update a device's Shadow document, you can use the [reserved MQTT topics](reserved-topics.md#reserved-topics-shadow), the [Device Shadow REST APIs](device-shadow-rest-api.md) that support the `GET`, `UPDATE`, and `DELETE` operations with HTTP, and the [AWS IoT CLI](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/iot-data/index.html).
In the previous example, say you want to change the `desired` color to `yellow`. To do this, send a request to the [UpdateThingShadow](device-shadow-rest-api.md#API_UpdateThingShadow) API or publish a message to the [Update](device-shadow-mqtt.md#update-pub-sub-topic) topic, `$aws/things/THING_NAME/shadow/update`.
```
{
"state": {
"desired": {
"color": yellow
}
}
}
```
Updates affect only the fields specified in the request. After successfully updating the Device Shadow, AWS IoT publishes the new `desired` state to the `delta` topic, `$aws/things/THING_NAME/shadow/delta`. The Shadow document in this case looks like this:
```
{
"state": {
"desired": {
"color": yellow
},
"reported": {
"color": green
},
"delta": {
"color": yellow
}
}
}
```
The new state is then reported to the AWS IoT Device Shadow using the `Update` topic `$aws/things/THING_NAME/shadow/update` with the following JSON message:
```
{
"state": {
"reported": {
"color": yellow
}
}
}
```
If you want to get the current state information, send a request to the [GetThingShadow](device-shadow-rest-api.md#API_GetThingShadow) API or publish an MQTT message to the [Get](device-shadow-mqtt.md#get-pub-sub-topic) topic, `$aws/things/THING_NAME/shadow/get`.
For more information about using the Device Shadow service, see [AWS IoT Device Shadow service](iot-device-shadows.md).
For more information about using Device Shadows in devices, apps, and services, see [Using shadows in devices](device-shadow-comms-device.md) and [Using shadows in apps and services](device-shadow-comms-app.md).
For information about interacting with AWS IoT shadows, see [Interacting with shadows](device-shadow-data-flow.md).
For information about the MQTT reserved topics and HTTP REST APIs, see [Device Shadow MQTT topics](device-shadow-mqtt.md) and [Device Shadow REST API](device-shadow-rest-api.md).
# Tutorial: Preparing your Raspberry Pi to run the shadow application
This tutorial demonstrates how to set up and configure a Raspberry Pi device and create the AWS IoT resources that a device requires to connect and exchange MQTT messages.
**Note**
If you're planning to [Create a virtual device with Amazon EC2](creating-a-virtual-thing.md), you can skip this page and continue to [Configure your device](configure-device.md). You'll create these resources when you create your virtual thing. If you would like to use a different device instead of the Raspberry Pi, you can try to follow these tutorials by adapting them to a device of your choice.
**In this tutorial, you'll learn how to:**
+ Set up a Raspberry Pi device and configure it for use with AWS IoT.
+ Create an AWS IoT policy document, which authorizes your device to interact with AWS IoT services.
+ Create a thing resource in AWS IoT the X.509 device certificates, and then attach the policy document.
The thing is the virtual representation of your device in the AWS IoT registry. The certificate authenticates your device to AWS IoT Core, and the policy document authorizes your device to interact with AWS IoT.
**How to run this tutorial**
To run the `shadow.py` sample application for Device Shadows, you'll need a Raspberry Pi device that connects to AWS IoT. We recommend that you follow this tutorial in the order it's presented here, starting with setting up the Raspberry Pi and it's accessories, and then creating a policy and attaching the policy to a thing resource that you create. You can then follow this tutorial by using the graphical user interface (GUI) supported by the Raspberry Pi to open the AWS IoT console on the device's web browser, which also makes it easier to download the certificates directly to your Raspberry Pi for connecting to AWS IoT.
**Before you start this tutorial, make sure that you have:**
+ An AWS account. If you don't have one, complete the steps described in [Set up AWS account](setting-up.md) before you continue. You'll need your AWS account and AWS IoT console to complete this tutorial.
+ The Raspberry Pi and its necessary accessories. You'll need:
+ A [Raspberry Pi 3 Model B](https://www.raspberrypi.com/products/) or more recent model. This tutorial might work on earlier versions of the Raspberry Pi, but we haven't tested it.
+ [Raspberry Pi OS (32-bit)](https://www.raspberrypi.com/software/operating-systems/) or later. We recommend using the latest version of the Raspberry Pi OS. Earlier versions of the OS might work, but we haven't tested it.
+ An Ethernet or Wi-Fi connection.
+ Keyboard, mouse, monitor, cables, and power supplies.
This tutorial takes about 30 minutes to complete.
## Step 1: Set up and configure Raspberry Pi device
In this section, we'll configure a Raspberry Pi device for use with AWS IoT.
**Important**
Adapting these instructions to other devices and operating systems can be challenging. You'll need to understand your device well enough to be able to interpret these instructions and apply them to your device. If you encounter difficulties, you might try one of the other device options as an alternative, such as [Create a virtual device with Amazon EC2](creating-a-virtual-thing.md) or [Use your Windows or Linux PC or Mac as an AWS IoT device](using-laptop-as-device.md).
You'll need to configure your Raspberry Pi such that it can start the operating system (OS), connect to the internet, and allow you to interact with it at a command line interface. You can also use the graphical user interface (GUI) supported with the Raspberry Pi to open the AWS IoT console and run the rest of this tutorial.
**To set up the Raspberry Pi**
1. Insert the SD card into the MicroSD card slot on the Raspberry Pi. Some SD cards come pre-loaded with an installation manager that prompts you with a menu to install the OS after booting up the board. You can also use the Raspberry Pi imager to install the OS on your card.
1. Connect an HDMI TV or monitor to the HDMI cable that connects to the HDMI port of the Raspberry Pi.
1. Connect the keyboard and mouse to the USB ports of the Raspberry Pi and then plug in the power adapter to boot up the board.
After the Raspberry Pi boots up, if the SD card came pre-loaded with the installation manager, a menu appears to install the operating system. If you have trouble installing the OS, you can try the following steps. For more information about setting up the Raspberry Pi, see [Setting up your Raspberry Pi](https://projects.raspberrypi.org/en/projects/raspberry-pi-setting-up/).
**If you're having trouble setting up the Raspberry Pi:**
+ Check whether you inserted the SD card before booting up the board. If you plug in the SD card after booting up the board, the installation menu might not appear.
+ Make sure that the TV or monitor is turned on and the correct input is selected.
+ Ensure that you are using Raspberry Pi compatible software.
After you have installed and configured the Raspberry Pi OS, open the Raspberry Pi's web browser and navigate to the AWS IoT Core console to continue the rest of the steps in this tutorial.
If you can open the AWS IoT Core console, you're Raspberry Pi is ready and you can continue to [Tutorial: Provisioning your device in AWS IoT](shadow-provision-cloud.md).
If you're having trouble or need additional help, see [Getting help for your Raspberry Pi](https://projects.raspberrypi.org/en/projects/raspberry-pi-setting-up/5).
# Tutorial: Provisioning your device in AWS IoT
This section creates the AWS IoT Core resources that your tutorial will use.
**Topics**
+ [Step 1: Create an AWS IoT policy for the Device Shadow](#create-policy-shadow)
+ [Step 2: Create a thing resource and attach the policy to the thing](#create-thing-shadow)
+ [Step 3: Review the results and next steps](#resources-shadow-review)
## Step 1: Create an AWS IoT policy for the Device Shadow
X.509 certificates authenticate your device with AWS IoT Core. AWS IoT policies are attached to the certificate that permits the device to perform AWS IoT operations, such as subscribing or publishing to MQTT reserved topics used by the Device Shadow service. Your device presents its certificate when it connects and sends messages to AWS IoT Core.
In this procedure, you'll create a policy that allows your device to perform the AWS IoT operations necessary to run the example program. We recommend that you create a policy that grants only the permissions required to perform the task. You create the AWS IoT policy first, and then attach it to the device certificate that you'll create later.
**To create an AWS IoT policy**
1. On the left menu, choose **Secure**, and then choose **Policies**. If your account has existing policies, choose **Create**, otherwise, on the **You don’t have a policy yet** page, choose **Create a policy**.
1. On the **Create a policy** page:
1. Enter a name for the policy in the **Name** field (for example, **My\$1Device\$1Shadow\$1policy**). Do not use personally identifiable information in your policy names.
1. In the policy document, you describe connect, subscribe, receive, and publish actions that give the device permission to publish and subscribe to the MQTT reserved topics.
Copy the following sample policy and paste it in your policy document. Replace `thingname` with the name of the thing that you'll create (for example, `My_light_bulb`), `region` with the AWS IoT Region where you're using the services, and `account` with your AWS account number. For more information about AWS IoT policies, see [AWS IoT Core policies](iot-policies.md).
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"iot:Publish"
],
"Resource": [
"arn:aws:iot:us-east-1:123456789012:topic/$aws/things/thingname/shadow/get",
"arn:aws:iot:us-east-1:123456789012:topic/$aws/things/thingname/shadow/update"
]
},
{
"Effect": "Allow",
"Action": [
"iot:Receive"
],
"Resource": [
"arn:aws:iot:us-east-1:123456789012:topic/$aws/things/thingname/shadow/get/accepted",
"arn:aws:iot:us-east-1:123456789012:topic/$aws/things/thingname/shadow/get/rejected",
"arn:aws:iot:us-east-1:123456789012:topic/$aws/things/thingname/shadow/update/accepted",
"arn:aws:iot:us-east-1:123456789012:topic/$aws/things/thingname/shadow/update/rejected",
"arn:aws:iot:us-east-1:123456789012:topic/$aws/things/thingname/shadow/update/delta"
]
},
{
"Effect": "Allow",
"Action": [
"iot:Subscribe"
],
"Resource": [
"arn:aws:iot:us-east-1:123456789012:topicfilter/$aws/things/thingname/shadow/get/accepted",
"arn:aws:iot:us-east-1:123456789012:topicfilter/$aws/things/thingname/shadow/get/rejected",
"arn:aws:iot:us-east-1:123456789012:topicfilter/$aws/things/thingname/shadow/update/accepted",
"arn:aws:iot:us-east-1:123456789012:topicfilter/$aws/things/thingname/shadow/update/rejected",
"arn:aws:iot:us-east-1:123456789012:topicfilter/$aws/things/thingname/shadow/update/delta"
]
},
{
"Effect": "Allow",
"Action": "iot:Connect",
"Resource": "arn:aws:iot:us-east-1:123456789012:client/test-*"
}
]
}
```
## Step 2: Create a thing resource and attach the policy to the thing
Devices connected to AWS IoT can be represented by *thing resources* in the AWS IoT registry. A *thing resource* represents a specific device or logical entity, such as the light bulb in this tutorial.
To learn how to create a thing in AWS IoT, follow the steps described in [Create a thing object](create-iot-resources.md#create-aws-thing). Here are some key things to note as you follow the steps in that tutorial:
1. Choose **Create a single thing**, and in the **Name** field, enter a name for the thing that is the same as the `thingname` (for example, `My_light_bulb`) you specified when you created the policy earlier.
You can't change a thing name after it has been created. If you gave it a different name other than `thingname`, create a new thing with name as `thingname` and delete the old thing.
**Note**
Do not use personally identifiable information in your thing name. The thing name can appear in unencrypted communications and reports.
1. We recommend that you download each of the certificate files on the **Certificate created\$1** page into a location where you can easily find them. You'll need to install these files for running the sample application.
We recommend that you download the files into a `certs` subdirectory in your `home` directory on the Raspberry Pi and name each of them with a simpler name as suggested in the following table.
**Certificate file names**
[\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/iot/latest/developerguide/shadow-provision-cloud.html)
1. After you activate the certificate to enable connections to AWS IoT, choose **Attach a policy** and make sure you attach the policy that you created earlier (for example, **My\$1Device\$1Shadow\$1policy**) to the thing.
After you've created a thing, you can see your thing resource displayed in the list of things in the AWS IoT console.
## Step 3: Review the results and next steps
**In this tutorial, you learned how to:**
+ Set up and configure the Raspberry Pi device.
+ Create an AWS IoT policy document that authorizes your device to interact with AWS IoT services.
+ Create a thing resource and associated X.509 device certificate, and attach the policy document to it.
**Next steps**
You can now install the AWS IoT device SDK for Python, run the `shadow.py` sample application, and use Device Shadows to control the state. For more information about how to run this tutorial, see [Tutorial: Installing the Device SDK and running the sample application for Device Shadows](lightbulb-shadow-application.md).
# Tutorial: Installing the Device SDK and running the sample application for Device Shadows
This section shows how you can install the required software and the AWS IoT Device SDK for Python and run the `shadow.py` sample application to edit the Shadow document and control the shadow's state.
**In this tutorial, you'll learn how to:**
+ Use the installed software and AWS IoT Device SDK for Python to run the sample app.
+ Learn how entering a value using the sample app publishes the desired value in the AWS IoT console.
+ Review the `shadow.py` sample app and how it uses the MQTT protocol to update the shadow's state.
**Before you run this tutorial:**
You must have set up your AWS account, configured your Raspberry Pi device, and created an AWS IoT thing and policy that gives the device permissions to publish and subscribe to the MQTT reserved topics of the Device Shadow service. For more information, see [Tutorial: Preparing your Raspberry Pi to run the shadow application](create-resources-shadow.md).
You must have also installed Git, Python, and the AWS IoT Device SDK for Python. This tutorial builds on the concepts presented in the tutorial [Connect a Raspberry Pi or other device](connecting-to-existing-device.md). If you haven't tried that tutorial, we recommend that you follow the steps described in that tutorial to install the certificate files and Device SDK and then come back to this tutorial to run the `shadow.py` sample app.
**Topics**
+ [Step 1: Run the shadow.py sample app](#run-sample-application-shadows)
+ [Step 2: Review the shadow.py Device SDK sample app](#review-shadow-sample-code)
+ [Step 3: Troubleshoot problems with the `shadow.py` sample app](#shadow-sample-app-troubleshoot)
+ [Step 4: Review the results and next steps](#sample-app-shadow-review)
This tutorial takes about 20 minutes to complete.
## Step 1: Run the shadow.py sample app
Before you run the `shadow.py` sample app, you'll need the following information in addition to the names and location of the certificate files that you installed.
**Application parameter values**
| Parameter | Where to find the value |
| --- | --- |
| your-iot-thing-name | Name of the AWS IoT thing that you created earlier in [Step 2: Create a thing resource and attach the policy to the thing](shadow-provision-cloud.md#create-thing-shadow). To find this value, in the [AWS IoT console](https://console.aws.amazon.com/iot/home), choose **Manage**, and then choose **Things**. |
| your-iot-endpoint | The *your-iot-endpoint* value has a format of: `endpoint_id-ats.iot.region.amazonaws.com`, for example, `a3qj468EXAMPLE-ats.iot.us-west-2.amazonaws.com`. To find this value: [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/iot/latest/developerguide/lightbulb-shadow-application.html) |
**Install and run the sample app**
1. Navigate to the sample app directory.
```
cd ~/aws-iot-device-sdk-python-v2/samples/service-clients
```
1. In the command line window, replace *your-iot-endpoint* and *your-iot-thing-name* as indicated and run this command.
```
python3 shadow.py --ca_file ~/certs/Amazon-root-CA-1.pem --cert ~/certs/device.pem.crt --key ~/certs/private.pem.key --endpoint your-iot-endpoint --thing_name your-iot-thing-name
```
1. Observe that the sample app:
1. Connects to the AWS IoT service for your account.
1. Subscribes to `Delta` events and `Update` and `Get` responses.
1. Prompts you to enter a desired value in the terminal.
1. Displays output similar to the following:
```
Connecting to a3qEXAMPLEffp-ats.iot.us-west-2.amazonaws.com with client ID 'test-0c8ae2ff-cc87-49d2-a82a-ae7ba1d0ca5a'...
Connected!
Subscribing to Delta events...
Subscribing to Update responses...
Subscribing to Get responses...
Requesting current shadow state...
Launching thread to read user input...
Finished getting initial shadow state.
Shadow contains reported value 'off'.
Enter desired value:
```
**Note**
If you're having trouble running the `shadow.py` sample app, review [Step 3: Troubleshoot problems with the `shadow.py` sample app](#shadow-sample-app-troubleshoot). To get additional information that might help you correct the problem, add the `--verbosity debug` parameter to the command line so the sample app displays detailed messages about what it’s doing.
**Enter values and observe the updates in Shadow document**
You can enter values in the terminal to specify the `desired` value, which also updates the `reported` value. Say you enter the color `yellow` in the terminal. The `reported` value is also updated to the color `yellow`. The following shows the messages displayed in the terminal:
```
Enter desired value:
yellow
Changed local shadow value to 'yellow'.
Updating reported shadow value to 'yellow'...
Update request published.
Finished updating reported shadow value to 'yellow'.
```
When you publish this update request, AWS IoT creates a default, classic shadow for the thing resource. You can observe the update request that you published to the `reported` and `desired` values in the AWS IoT console by looking at the Shadow document for the thing resource that you created (for example, `My_light_bulb`). To see the update in the Shadow document:
1. In the AWS IoT console, choose **Manage** and then choose **Things**.
1. In the list of things displayed, select the thing that you created, choose **Shadows**, and then choose **Classic Shadow**.
The Shadow document should look similar to the following, showing the `reported` and `desired` values set to the color `yellow`. You see these values in the **Shadow state** section of the document.
```
{
"desired": {
"welcome": "aws-iot",
"color": "yellow"
},
"reported": {
"welcome": "aws-iot",
"color": "yellow"
}
}
```
You also see a **Metadata** section that contains the timestamp information and version number of the request.
You can use the state document version to ensure you are updating the most recent version of a device's Shadow document. If you send another update request, the version number increments by 1. When you supply a version with an update request, the service rejects the request with an HTTP 409 conflict response code if the current version of the state document doesn't match the version supplied.
```
{
"metadata": {
"desired": {
"welcome": {
"timestamp": 1620156892
},
"color": {
"timestamp": 1620156893
}
},
"reported": {
"welcome": {
"timestamp": 1620156892
},
"color": {
"timestamp": 1620156893
}
}
},
"version": 10
}
```
To learn more about the Shadow document and observe changes to the state information, proceed to the next tutorial [Tutorial: Interacting with Device Shadow using the sample app and the MQTT test client](interact-lights-device-shadows.md) as described in the [Step 4: Review the results and next steps](#sample-app-shadow-review) section of this tutorial. Optionally, you can also learn about the `shadow.py` sample code and how it uses the MQTT protocol in the following section.
## Step 2: Review the shadow.py Device SDK sample app
This section reviews the `shadow.py` sample app from the **AWS IoT Device SDK v2 for Python** used in this tutorial. Here, we'll review how it connects to AWS IoT Core by using the MQTT and MQTT over WSS protocol. The [AWS common runtime (AWS-CRT)](https://github.com/awslabs/aws-crt-python#aws-crt-python) library provides the low-level communication protocol support and is included with the AWS IoT Device SDK v2 for Python.
While this tutorial uses MQTT and MQTT over WSS, AWS IoT supports devices that publish HTTPS requests. For an example of a Python program that sends an HTTP message from a device, see the [HTTPS code example](http.md#codeexample) using Python’s `requests` library.
For information about how you can make an informed decision about which protocol to use for your device communications, review the [Choosing an application protocol for your device communication](protocols.md#protocol-selection).
**MQTT**
The `shadow.py` sample calls `mtls_from_path` (shown here) in the [https://github.com/awslabs/aws-crt-python/blob/89207bcf1387177034e02fe29e8e469ca45e39b7/awscrt/awsiot_mqtt_connection_builder.py](https://github.com/awslabs/aws-crt-python/blob/89207bcf1387177034e02fe29e8e469ca45e39b7/awscrt/awsiot_mqtt_connection_builder.py) to establish a connection with AWS IoT Core by using the MQTT protocol. `mtls_from_path` uses X.509 certificates and TLS v1.2 to authenticate the device. The AWS-CRT library handles the lower-level details of that connection.
```
mqtt_connection = mqtt_connection_builder.mtls_from_path(
endpoint=args.endpoint,
cert_filepath=args.cert,
pri_key_filepath=args.key,
ca_filepath=args.ca_file,
client_bootstrap=client_bootstrap,
on_connection_interrupted=on_connection_interrupted,
on_connection_resumed=on_connection_resumed,
client_id=args.client_id,
clean_session=False,
keep_alive_secs=6
)
```
+ `endpoint` is your AWS IoT endpoint that you passed in from the command line and `client_id` is the ID that uniquely identifies this device in the AWS Region.
+ `cert_filepath`, `pri_key_filepath`, and `ca_filepath` are the paths to the device's certificate and private key files, and the root CA file.
+ `client_bootstrap` is the common runtime object that handles socket communication activities, and is instantiated prior to the call to `mqtt_connection_builder.mtls_from_path`.
+ `on_connection_interrupted` and `on_connection_resumed` are callback functions to call when the device’s connection is interrupted and resumed.
+ `clean_session` is whether to start a new, persistent session, or if one is present, reconnect to an existing one. `keep_alive_secs` is the keep alive value, in seconds, to send in the `CONNECT` request. A ping will automatically be sent at this interval. The server assumes that the connection is lost if it doesn't receive a ping after 1.5 times this value.
The `shadow.py` sample also calls `websockets_with_default_aws_signing` in the [https://github.com/awslabs/aws-crt-python/blob/89207bcf1387177034e02fe29e8e469ca45e39b7/awscrt/awsiot_mqtt_connection_builder.py](https://github.com/awslabs/aws-crt-python/blob/89207bcf1387177034e02fe29e8e469ca45e39b7/awscrt/awsiot_mqtt_connection_builder.py) to establish a connection with AWS IoT Core using MQTT protocol over WSS. MQTT over WSS also uses the same parameters as MQTT and takes these additional parameters:
+ `region` is the AWS signing Region used by Signature V4 authentication, and `credentials_provider` is the AWS credentials provided to use for authentication. The Region is passed from the command line, and the `credentials_provider` object is instantiated just prior to the call to `mqtt_connection_builder.websockets_with_default_aws_signing`.
+ `websocket_proxy_options` is the HTTP proxy options, if using a proxy host. In the `shadow.py` sample app, this value is instantiated just prior to the call to `mqtt_connection_builder.websockets_with_default_aws_signing`.
**Subscribe to Shadow topics and events**
The `shadow.py` sample attempts to establish a connection and waits to be fully connected. If it's not connected, commands are queued up. Once connected, the sample subscribes to delta events and update and get messages, and publishes messages with a Quality of Service (QoS) level of 1 (`mqtt.QoS.AT_LEAST_ONCE`).
When a device subscribes to a message with QoS level 1, the message broker saves the messages that the device is subscribed to until they can be sent to the device. The message broker resends the messages until it receives a `PUBACK` response from the device.
For more information about the MQTT protocol, see [Review the MQTT protocol](sdk-tutorials.md#sdk-tutorials-mqtt-review) and [MQTT](mqtt.md).
For more information about how MQTT, MQTT over WSS, persistent sessions, and QoS levels that are used in this tutorial, see [Review the pubsub.py Device SDK sample app](sdk-tutorials.md#sdk-tutorials-explore-sample).
## Step 3: Troubleshoot problems with the `shadow.py` sample app
When you run the `shadow.py` sample app, you should see some messages displayed in the terminal and a prompt to enter a `desired` value. If the program throws an error, then to debug the error, you can start by checking whether you ran the correct command for your system.
In some cases, the error message might indicate connection issues and look similar to: `Host name was invalid for dns resolution` or `Connection was closed unexpectedly`. In such cases, here are some things you can check:
+
**Check the endpoint address in the command**
Review the `endpoint` argument in the command you entered to run the sample app, (for example, `a3qEXAMPLEffp-ats.iot.us-west-2.amazonaws.com`) and check this value in the **AWS IoT console**.
To check whether you used the correct value:
1. In the **AWS IoT console**, choose **Manage** and then choose **Things**.
1. Choose the thing you created for your sample app (for example, **My\$1light\$1bulb**) and then choose **Interact**.
On the thing details page, your endpoint is displayed in the **HTTPS** section. You should also see a message that says: `This thing already appears to be connected.`
+
**Check certificate activation**
Certificates authenticate your device with AWS IoT Core.
To check whether your certificate is active:
1. In the **AWS IoT console**, choose **Manage** and then choose **Things**.
1. Choose the thing you created for your sample app (for example, **My\$1light\$1bulb**) and then choose **Security**.
1. Select the certificate and then, from the certificate's details page, choose Select the certificate and then, from the certificate's details page, choose **Actions**.
If in the dropdown list **Activate** isn't available and you can only choose **Deactivate**, your certificate is active. If not, choose **Activate** and rerun the sample program.
If the program still doesn't run, check the certificate file names in the `certs` folder.
+
**Check the policy attached to the thing resource**
While certificates authenticate your device, AWS IoT policies permit the device to perform AWS IoT operations, such as subscribing or publishing to MQTT reserved topics.
To check whether the correct policy is attached:
1. Find the certificate as described previously, and then choose **Policies**.
1. Choose the policy displayed and check whether it describes the `connect`, `subscribe`, `receive`, and `publish` actions that give the device permission to publish and subscribe to the MQTT reserved topics.
For a sample policy, see [Step 1: Create an AWS IoT policy for the Device Shadow](shadow-provision-cloud.md#create-policy-shadow).
If you see error messages that indicate trouble connecting to AWS IoT, it could be because of the permissions you're using for the policy. If that's the case, we recommend that you start with a policy that provides full access to AWS IoT resources and then rerun the sample program. You can either edit the current policy, or choose the current policy, choose **Detach**, and then create another policy that provides full access and attach it to your thing resource. You can later restrict the policy to only the actions and policies you need to run the program.
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"iot:*"
],
"Resource": "*"
}
]
}
```
+
**Check your Device SDK installation**
If the program still doesn't run, you can reinstall the Device SDK to make sure that your SDK installation is complete and correct.
## Step 4: Review the results and next steps
**In this tutorial, you learned how to:**
+ Install the required software, tools, and the AWS IoT Device SDK for Python.
+ Understand how the sample app, `shadow.py`, uses the MQTT protocol for retrieving and updating the shadow's current state.
+ Run the sample app for Device Shadows and observe the update to the Shadow document in the AWS IoT console. You also learned to troubleshoot any issues and fix errors when running the program.
**Next steps**
You can now run the `shadow.py` sample application and use Device Shadows to control the state. You can observe the updates to the Shadow document in the AWS IoT Console and observe delta events that the sample app responds to. Using the MQTT test client, you can subscribe to the reserved shadow topics and observe messages received by the topics when running the sample program. For more information about how to run this tutorial, see [Tutorial: Interacting with Device Shadow using the sample app and the MQTT test client](interact-lights-device-shadows.md).
# Tutorial: Interacting with Device Shadow using the sample app and the MQTT test client
To interact with the `shadow.py` sample app, enter a value in the terminal for the `desired` value. For example, you can specify colors that resemble the traffic lights and AWS IoT responds to the request and updates the reported values.
**In this tutorial, you'll learn how to:**
+ Use the `shadow.py` sample app to specify desired states and update the shadow's current state.
+ Edit the Shadow document to observe delta events and how the `shadow.py` sample app responds to it.
+ Use the MQTT test client to subscribe to shadow topics and observe updates when you run the sample program.
**Before you run this tutorial, you must have:**
Set up your AWS account, configured your Raspberry Pi device, and created an AWS IoT thing and policy. You must have also installed the required software, Device SDK, certificate files, and run the sample program in the terminal. For more information, see the previous tutorials [Tutorial: Preparing your Raspberry Pi to run the shadow application](create-resources-shadow.md) and [Step 1: Run the shadow.py sample app](lightbulb-shadow-application.md#run-sample-application-shadows). You must complete these tutorials if you haven't already.
**Topics**
+ [Step 1: Update desired and reported values using `shadow.py` sample app](#update-desired-shadow-sample)
+ [Step 2: View messages from the `shadow.py` sample app in the MQTT test client](#shadow-sample-view-msg)
+ [Step 3: Troubleshoot errors with Device Shadow interactions](#shadow-observe-messages-troubleshoot)
+ [Step 4: Review the results and next steps](#sample-shadow-review)
This tutorial takes about 45 minutes to complete.
## Step 1: Update desired and reported values using `shadow.py` sample app
In the previous tutorial [Step 1: Run the shadow.py sample app](lightbulb-shadow-application.md#run-sample-application-shadows), you learned how to observe a message published to the Shadow document in the AWS IoT console when you enter a desired value as described in the section [Tutorial: Installing the Device SDK and running the sample application for Device Shadows](lightbulb-shadow-application.md).
In the previous example, we set the desired color to `yellow`. After you enter each value, the terminal prompts you to enter another `desired` value. If you again enter the same value (`yellow`), the app recognizes this and prompts you to enter a new `desired` value.
```
Enter desired value:
yellow
Local value is already 'yellow'.
Enter desired value:
```
Now, say that you enter the color `green`. AWS IoT responds to the request and updates the `reported` value to `green`. This is how the update happens when the `desired` state is different from the `reported` state, causing a delta.
**How the `shadow.py` sample app simulates Device Shadow interactions:**
1. Enter a `desired` value (say `yellow`) in the terminal to publish the desired state.
1. As the `desired` state is different from the `reported` state (say the color `green`), a delta occurs, and the app that is subscribed to the delta receives this message.
1. The app responds to the message and updates its state to the `desired` value, `yellow`.
1. The app then publishes an update message with the new reported value of the device's state, `yellow`.
Following shows the messages displayed in the terminal that shows how the update request is published.
```
Enter desired value:
green
Changed local shadow value to 'green'.
Updating reported shadow value to 'green'...
Update request published.
Finished updating reported shadow value to 'green'.
```
In the AWS IoT console, the Shadow document reflects the updated value to `green` for both the `reported` and `desired` fields, and the version number is incremented by 1. For example, if the previous version number was displayed as 10, the current version number will display as 11.
**Note**
Deleting a shadow doesn't reset the version number to 0. You'll see that the shadow version is incremented by 1 when you publish an update request or create another shadow with the same name.
**Edit the Shadow document to observe delta events**
The `shadow.py` sample app is also subscribed to `delta` events, and responds when there is a change to the `desired` value. For example, you can change the `desired` value to the color `red`. To do this, in the AWS IoT console, edit the Shadow document by clicking **Edit** and then set the `desired` value to `red` in the JSON, while keeping the `reported` value to `green`. Before you save the changes, keep the terminal on the Raspberry Pi open as you'll see messages displayed in the terminal when the change occurs.
```
{
"desired": {
"welcome": "aws-iot",
"color": "red"
},
"reported": {
"welcome": "aws-iot",
"color": "green"
}
}
```
After you save the new value, the `shadow.py` sample app responds to this change and displays messages in the terminal indicating the delta. You should then see the following messages appear below the prompt for entering the `desired` value.
```
Enter desired value:
Received shadow delta event.
Delta reports that desired value is 'red'. Changing local value...
Changed local shadow value to 'red'.
Updating reported shadow value to 'red'...
Finished updating reported shadow value to 'red'.
Enter desired value:
Update request published.
Finished updating reported shadow value to 'red'.
```
## Step 2: View messages from the `shadow.py` sample app in the MQTT test client
You can use the **MQTT test client** in the **AWS IoT console** to monitor MQTT messages that are passed in your AWS account. By subscribing to reserved MQTT topics used by the Device Shadow service, you can observe the messages received by the topics when running the sample app.
If you haven't already used the MQTT test client, you can review [View MQTT messages with the AWS IoT MQTT client](view-mqtt-messages.md). This helps you learn how to use the **MQTT test client** in the **AWS IoT console** to view MQTT messages as they pass through the message broker.
1.
**Open the MQTT test client**
Open the [MQTT test client in the AWS IoT console](https://console.aws.amazon.com//iot/home#/test) in a new window so that you can observe the messages received by the MQTT topics without losing the configuration of your MQTT test client. The MQTT test client doesn't retain any subscriptions or message logs if you leave it to go to another page in the console. For this section of the tutorial, you can have the Shadow document of your AWS IoT thing and the MQTT test client open in separate windows to more easily observe the interaction with Device Shadows.
1.
**Subscribe to the MQTT reserved Shadow topics**
You can use the MQTT test client to enter the names of the Device Shadow's MQTT reserved topics and subscribe to them to receive updates when running the `shadow.py` sample app. To subscribe to the topics:
1. In the **MQTT test client** in the **AWS IoT console**, choose **Subscribe to a topic**.
1. In the **Topic filter** section, enter: **\$1aws/things/*thingname*/shadow/update/\$1**. Here, `thingname` is the name of the thing resource that you created earlier (for example, `My_light_bulb`).
1. Keep the default values for the additional configuration settings, and then choose **Subscribe**.
By using the **\$1** wildcard in the topic subscription, you can subscribe to multiple MQTT topics at the same time and observe all the messages that are exchanged between the device and its Shadow in a single window. For more information about the wildcard characters and their use, see [MQTT topics](topics.md).
1.
**Run `shadow.py` sample program and observe messages**
In your command line window of the Raspberry Pi, if you've disconnected the program, run the sample app again and watch the messages in the **MQTT test client** in the **AWS IoT console**.
1. Run the following command to restart the sample program. Replace *your-iot-thing-name* and *your-iot-endpoint* with the names of the AWS IoT thing that you created earlier (for example, `My_light_bulb`), and the endpoint to interact with the device.
```
cd ~/aws-iot-device-sdk-python-v2/samples/service-clients
python3 shadow.py --ca_file ~/certs/Amazon-root-CA-1.pem --cert ~/certs/device.pem.crt --key ~/certs/private.pem.key --endpoint your-iot-endpoint --thing_name your-iot-thing-name
```
The `shadow.py` sample app then runs and retrieves the current shadow state. If you've deleted the shadow or cleared the current states, the program sets the current value to `off` and then prompts you to enter a `desired` value.
```
Connecting to a3qEXAMPLEffp-ats.iot.us-west-2.amazonaws.com with client ID 'test-0c8ae2ff-cc87-49d2-a82a-ae7ba1d0ca5a'...
Connected!
Subscribing to Delta events...
Subscribing to Update responses...
Subscribing to Get responses...
Requesting current shadow state...
Launching thread to read user input...
Finished getting initial shadow state.
Shadow document lacks 'color' property. Setting defaults...
Changed local shadow value to 'off'.
Updating reported shadow value to 'off'...
Update request published.
Finished updating reported shadow value to 'off'...
Enter desired value:
```
On the other hand, if the program was running and you restarted it, you'll see the latest color value reported in the terminal. In the MQTT test client, you'll see an update to the topics **\$1aws/things/*thingname*/shadow/get** and **\$1aws/things/*thingname*/shadow/get/accepted**.
Suppose that the latest color reported was `green`. Following shows the contents of the **\$1aws/things/*thingname*/shadow/get/accepted** JSON file.
```
{
"state": {
"desired": {
"welcome": "aws-iot",
"color": "green"
},
"reported": {
"welcome": "aws-iot",
"color": "green"
}
},
"metadata": {
"desired": {
"welcome": {
"timestamp": 1620156892
},
"color": {
"timestamp": 1620161643
}
},
"reported": {
"welcome": {
"timestamp": 1620156892
},
"color": {
"timestamp": 1620161643
}
}
},
"version": 10,
"timestamp": 1620173908
}
```
1. Enter a `desired` value in the terminal, such as `yellow`. The `shadow.py` sample app responds and displays the following messages in the terminal that show the change in the `reported` value to `yellow`.
```
Enter desired value:
yellow
Changed local shadow value to 'yellow'.
Updating reported shadow value to 'yellow'...
Update request published.
Finished updating reported shadow value to 'yellow'.
```
In the **MQTT test client** in the **AWS IoT console**, under **Subscriptions**, you see that the following topics received a message:
+ **\$1aws/things/*thingname*/shadow/update**: shows that both `desired` and `updated` values change to the color `yellow`.
+ **\$1aws/things/*thingname*/shadow/update/accepted**: shows the current values of the `desired` and `reported` states and their metadata and version information.
+ **\$1aws/things/*thingname*/shadow/update/documents**: shows the previous and current values of the `desired` and `reported` states and their metadata and version information.
As the document **\$1aws/things/*thingname*/shadow/update/documents** also contains information that is contained in the other two topics, we can review it to see the state information. The previous state shows the reported value set to `green`, its metadata and version information, and the current state that shows the reported value updated to `yellow`.
```
{
"previous": {
"state": {
"desired": {
"welcome": "aws-iot",
"color": "green"
},
"reported": {
"welcome": "aws-iot",
"color": "green"
}
},
"metadata": {
"desired": {
"welcome": {
"timestamp": 1617297888
},
"color": {
"timestamp": 1617297898
}
},
"reported": {
"welcome": {
"timestamp": 1617297888
},
"color": {
"timestamp": 1617297898
}
}
},
"version": 10
},
"current": {
"state": {
"desired": {
"welcome": "aws-iot",
"color": "yellow"
},
"reported": {
"welcome": "aws-iot",
"color": "yellow"
}
},
"metadata": {
"desired": {
"welcome": {
"timestamp": 1617297888
},
"color": {
"timestamp": 1617297904
}
},
"reported": {
"welcome": {
"timestamp": 1617297888
},
"color": {
"timestamp": 1617297904
}
}
},
"version": 11
},
"timestamp": 1617297904
}
```
1. Now, if you enter another `desired` value, you see further changes to the `reported` values and message updates received by these topics. The version number also increments by 1. For example, if you enter the value `green`, the previous state reports the value `yellow` and the current state reports the value `green`.
1.
**Edit Shadow document to observe delta events**
To observe changes to the delta topic, edit the Shadow document in the AWS IoT console. For example, you can change the `desired` value to the color `red`. To do this, in the AWS IoT console, choose **Edit** and then set the `desired` value to red in the JSON, while keeping the `reported` value set to `green`. Before you save the change, keep the terminal open as you'll see the delta message reported in the terminal.
```
{
"desired": {
"welcome": "aws-iot",
"color": "red"
},
"reported": {
"welcome": "aws-iot",
"color": "green"
}
}
```
The `shadow.py` sample app responds to this change and displays messages in the terminal indicating the delta. In the MQTT test client, the `update` topics will have received a message showing changes to the `desired` and `reported` values.
You also see that the topic **\$1aws/things/*thingname*/shadow/update/delta** received a message. To see the message, choose this topic, which is listed under **Subscriptions**.
```
{
"version": 13,
"timestamp": 1617318480,
"state": {
"color": "red"
},
"metadata": {
"color": {
"timestamp": 1617318480
}
}
}
```
## Step 3: Troubleshoot errors with Device Shadow interactions
When you run the Shadow sample app, you might encounter issues with observing interactions with the Device Shadow service.
If the program runs successfully and prompts you to enter a `desired` value, you should be able to observe the Device Shadow interactions by using the Shadow document and the MQTT test client as described previously. However, if you're unable to see the interactions, here are some things you can check:
+
**Check the thing name and its shadow in the AWS IoT console**
If you don't see the messages in the Shadow document, review the command and make sure it matches the thing name in the **AWS IoT console**. You can also check whether you have a classic shadow by choosing your thing resource and then choosing **Shadows**. This tutorial focuses primarily on interactions with the classic shadow.
You can also confirm that the device you used is connected to the internet. In the **AWS IoT console**, choose the thing you created earlier, and then choose **Interact**. On the thing details page, you should see a message here that says: `This thing already appears to be connected.`
+
**Check the MQTT reserved topics you subscribed to**
If you don't see the messages appear in the MQTT test client, check whether the topics you subscribed to are formatted correctly. MQTT Device Shadow topics have a format **\$1aws/things/*thingname*/shadow/** and might have `update`, `get`, or `delete` following it depending on actions you want to perform on the shadow. This tutorial uses the topic **\$1aws/things/*thingname*/shadow/\$1** so make sure you entered it correctly when subscribing to the topic in the **Topic filter** section of the test client.
As you enter the topic name, make sure that the *thingname* is the same as the name of the AWS IoT thing that you created earlier. You can also subscribe to additional MQTT topics to see if an update has been successfully performed. For example, you can subscribe to the topic **\$1aws/things/*thingname*/shadow/update/rejected** to receive a message whenever an update request failed so that you can debug connection issues. For more information about the reserved topics, see [Shadow topics](reserved-topics.md#reserved-topics-shadow) and [Device Shadow MQTT topics](device-shadow-mqtt.md).
## Step 4: Review the results and next steps
**In this tutorial, you learned how to:**
+ Use the `shadow.py` sample app to specify desired states and update the shadow's current state.
+ Edit the Shadow document to observe delta events and how the `shadow.py` sample app responds to it.
+ Use the MQTT test client to subscribe to shadow topics and observe updates when you run the sample program.
**Next steps**
You can subscribe to additional MQTT reserved topics to observe updates to the shadow application. For example, if you only subscribe to the topic **\$1aws/things/*thingname*/shadow/update/accepted**, you'll see only the current state information when an update is successfully performed.
You can also subscribe to additional shadow topics to debug issues or learn more about the Device Shadow interactions and also debug any issues with the Device Shadow interactions. For more information, see [Shadow topics](reserved-topics.md#reserved-topics-shadow) and [Device Shadow MQTT topics](device-shadow-mqtt.md).
You can also choose to extend your application by using named shadows or by using additional hardware connected with the Raspberry Pi for the LEDs and observe changes to their state using messages sent from the terminal.
For more information about the Device Shadow service and using the service in devices, apps, and services, see [AWS IoT Device Shadow service](iot-device-shadows.md), [Using shadows in devices](device-shadow-comms-device.md), and [Using shadows in apps and services](device-shadow-comms-app.md).
# Tutorial: Creating a custom authorizer for AWS IoT Core
This tutorial demonstrates the steps to create, validate, and use Custom Authentication by using the AWS CLI. Optionally, using this tutorial, you can use Postman to send data to AWS IoT Core by using the HTTP Publish API.
This tutorial show you how to create a sample Lambda function that implements the authorization and authentication logic and a custom authorizer using the **create-authorizer** call with token signing enabled. The authorizer is then validated using the **test-invoke-authorizer**, and finally you can send data to AWS IoT Core by using the HTTP Publish API to a test MQTT topic. Sample request will specify the authorizer to invoke by using the `x-amz-customauthorizer-name` header and pass the token-key-name and `x-amz-customauthorizer-signature` in request headers.
**What you'll learn in this tutorial:**
+ How to create a Lambda function to be a custom authorizer handler
+ How to create a custom authorizer using the AWS CLI with token signing enabled
+ How to test your custom authorizer using the **test-invoke-authorizer** command
+ How to publish an MQTT topic by using [Postman](https://www.postman.com/) and validate the request with your custom authorizer
This tutorial takes about 60 minutes to complete.
**Topics**
+ [Step 1: Create a Lambda function for your custom authorizer](#custom-auth-tutorial-define)
+ [Step 2: Create a public and private key pair for your custom authorizer](#custom-auth-tutorial-keys)
+ [Step 3: Create a custom authorizer resource and its authorization](#custom-auth-tutorial-authorizer)
+ [Step 4: Test the authorizer by calling test-invoke-authorizer](#custom-auth-tutorial-test)
+ [Step 5: Test publishing MQTT message using Postman](#custom-auth-tutorial-postman)
+ [Step 6: View messages in MQTT test client](#custom-auth-tutorial-testclient)
+ [Step 7: Review the results and next steps](#custom-auth-tutorial-review)
+ [Step 8: Clean up](#custom-auth-tutorial-cleanup)
**Before you start this tutorial, make sure that you have:**
+
**[Set up AWS account](setting-up.md)**
You'll need your AWS account and AWS IoT console to complete this tutorial.
The account you use for this tutorial works best when it includes at least these AWS managed policies:
+ [https://console.aws.amazon.com//iam/home#/policies/arn:aws:iam::aws:policy/IAMFullAccess$jsonEditor](https://console.aws.amazon.com//iam/home#/policies/arn:aws:iam::aws:policy/IAMFullAccess$jsonEditor)
+ [https://console.aws.amazon.com//iam/home#/policies/arn:aws:iam::aws:policy/AWSIoTFullAccess$jsonEditor](https://console.aws.amazon.com//iam/home#/policies/arn:aws:iam::aws:policy/AWSIoTFullAccess$jsonEditor)
+ [https://console.aws.amazon.com//iam/home#/policies/arn:aws:iam::aws:policy/AWSLambda_FullAccess$jsonEditor](https://console.aws.amazon.com//iam/home#/policies/arn:aws:iam::aws:policy/AWSLambda_FullAccess$jsonEditor)
**Important**
The IAM policies used in this tutorial are more permissive than you should follow in a production implementation. In a production environment, make sure that your account and resource policies grant only the necessary permissions.
When you create IAM policies for production, determine what access users and roles need, and then design the policies that allow them to perform only those tasks.
For more information, see [Security best practices in IAM](https://docs.aws.amazon.com//IAM/latest/UserGuide/best-practices.html)
+
**Installed the AWS CLI**
For information about how to install the AWS CLI, see [Installing the AWS CLI](https://docs.aws.amazon.com//cli/latest/userguide/cli-chap-install.html). This tutorial requires AWS CLI version `aws-cli/2.1.3 Python/3.7.4 Darwin/18.7.0 exe/x86_64` or later.
+
**OpenSSL tools**
The examples in this tutorial use [LibreSSL 2.6.5](https://www.libressl.org/). You can also use [OpenSSL v1.1.1i](https://www.openssl.org/) tools for this tutorial.
+
**Reviewed the [AWS Lambda](https://docs.aws.amazon.com//lambda/latest/dg/welcome.html) overview**
If you haven't used AWS Lambda before, review [AWS Lambda](https://docs.aws.amazon.com//lambda/latest/dg/welcome.html) and [Getting started with Lambda](https://docs.aws.amazon.com//lambda/latest/dg/getting-started.html) to learn its terms and concepts.
+
**Reviewed how to build requests in Postman**
For more information, see [Building requests](https://learning.postman.com/docs/sending-requests/requests/).
+
**Removed custom authorizers from previous tutorial**
Your AWS account can have only a limited number of custom authorizers configured at one time. For information about how to remove a custom authorizer, see [Step 8: Clean up](#custom-auth-tutorial-cleanup).
## Step 1: Create a Lambda function for your custom authorizer
Custom authentication in AWS IoT Core uses [authorizer resources](https://docs.aws.amazon.com//iot/latest/apireference/API_AuthorizerDescription.html) that you create to authenticate and authorize clients. The function you'll create in this section authenticates and authorizes clients as they connect to AWS IoT Core and access AWS IoT resources.
The Lambda function does the following:
+ If a request comes from **test-invoke-authorizer**, it returns an IAM policy with a `Deny` action.
+ If a request comes from Postman using HTTP and the `actionToken` parameter has a value of `allow`, it returns an IAM policy with an `Allow` action. Otherwise, it returns an IAM policy with a `Deny` action.
**To create the Lambda function for your custom authorizer**
1. In the [Lambda](https://console.aws.amazon.com//lambda/home#) console, open [Functions](https://console.aws.amazon.com//lambda/home#/functions).
1. Choose **Create function**.
1. Confirm **Author from scratch** is selected.
1. Under **Basic information**:
1. In **Function name**, enter **custom-auth-function**.
1. In **Runtime**, confirm **Node.js 18.x**
1. Choose **Create function**.
Lambda creates a Node.js function and an [execution role](https://docs.aws.amazon.com//lambda/latest/dg/lambda-intro-execution-role.html) that grants the function permission to upload logs. The Lambda function assumes the execution role when you invoke your function and uses the execution role to create credentials for the AWS SDK and to read data from event sources.
1. To see the function's code and configuration in the [AWS Cloud9](https://docs.aws.amazon.com/cloud9/latest/user-guide/welcome.html) editor, choose **custom-auth-function** in the designer window, and then choose **index.js** in the navigation pane of the editor.
For scripting languages such as Node.js, Lambda includes a basic function that returns a success response. You can use the [AWS Cloud9](https://docs.aws.amazon.com/cloud9/latest/user-guide/welcome.html) editor to edit your function as long as your source code doesn't exceed 3 MB.
1. Replace the **index.js** code in the editor with the following code:
```
// A simple Lambda function for an authorizer. It demonstrates
// How to parse a CLI and Http password to generate a response.
export const handler = async (event, context, callback) => {
//Http parameter to initiate allow/deny request
const HTTP_PARAM_NAME='actionToken';
const ALLOW_ACTION = 'Allow';
const DENY_ACTION = 'Deny';
//Event data passed to Lambda function
var event_str = JSON.stringify(event);
console.log('Complete event :'+ event_str);
//Read protocolData from the event json passed to Lambda function
var protocolData = event.protocolData;
console.log('protocolData value---> ' + protocolData);
//Get the dynamic account ID from function's ARN to be used
// as full resource for IAM policy
var ACCOUNT_ID = context.invokedFunctionArn.split(":")[4];
console.log("ACCOUNT_ID---"+ACCOUNT_ID);
//Get the dynamic region from function's ARN to be used
// as full resource for IAM policy
var REGION = context.invokedFunctionArn.split(":")[3];
console.log("REGION---"+REGION);
//protocolData data will be undefined if testing is done via CLI.
// This will help to test the set up.
if (protocolData === undefined) {
//If CLI testing, pass deny action as this is for testing purpose only.
console.log('Using the test-invoke-authorizer cli for testing only');
callback(null, generateAuthResponse(DENY_ACTION,ACCOUNT_ID,REGION));
} else{
//Http Testing from Postman
//Get the query string from the request
var queryString = event.protocolData.http.queryString;
console.log('queryString values -- ' + queryString);
/* global URLSearchParams */
const params = new URLSearchParams(queryString);
var action = params.get(HTTP_PARAM_NAME);
if(action!=null && action.toLowerCase() === 'allow'){
callback(null, generateAuthResponse(ALLOW_ACTION,ACCOUNT_ID,REGION));
}else{
callback(null, generateAuthResponse(DENY_ACTION,ACCOUNT_ID,REGION));
}
}
};
// Helper function to generate the authorization IAM response.
var generateAuthResponse = function(effect,ACCOUNT_ID,REGION) {
var full_resource = "arn:aws:iot:"+ REGION + ":" + ACCOUNT_ID + ":*";
console.log("full_resource---"+full_resource);
var authResponse = {};
authResponse.isAuthenticated = true;
authResponse.principalId = 'principalId';
var policyDocument = {};
policyDocument.Version = '2012-10-17';
policyDocument.Statement = [];
var statement = {};
statement.Action = 'iot:*';
statement.Effect = effect;
statement.Resource = full_resource;
policyDocument.Statement[0] = statement;
authResponse.policyDocuments = [policyDocument];
authResponse.disconnectAfterInSeconds = 3600;
authResponse.refreshAfterInSeconds = 600;
console.log('custom auth policy function called from http');
console.log('authResponse --> ' + JSON.stringify(authResponse));
console.log(authResponse.policyDocuments[0]);
return authResponse;
}
```
1. Choose **Deploy**.
1. After **Changes deployed** appears above the editor:
1. Scroll to the **Function overview** section above the editor.
1. Copy the **Function ARN** and save it to use later in this tutorial.
1. Test your function.
1. Choose the **Test** tab.
1. Using the default test settings, choose **Invoke**.
1. If the test succeeded, in the **Execution results**, open the **Details** view. You should see the policy document that the function returned.
If the test failed or you don't see a policy document, review the code to find and correct the errors.
## Step 2: Create a public and private key pair for your custom authorizer
Your custom authorizer requires a public and private key to authenticate it. The commands in this section use OpenSSL tools to create this key pair.
**To create the public and private key pair for your custom authorizer**
1. Create the private key file.
```
openssl genrsa -out private-key.pem 4096
```
1. Verify the private key file you just created.
```
openssl rsa -check -in private-key.pem -noout
```
If the command doesn't display any errors, the private key file is valid.
1. Create the public key file.
```
openssl rsa -in private-key.pem -pubout -out public-key.pem
```
1. Verify the public key file.
```
openssl pkey -inform PEM -pubin -in public-key.pem -noout
```
If the command doesn't display any errors, the public key file is valid.
## Step 3: Create a custom authorizer resource and its authorization
The AWS IoT custom authorizer is the resource that ties together all the elements created in the previous steps. In this section, you'll create a custom authorizer resource and give it permission to run the Lambda function you created earlier. You can create a custom authorizer resource by using the AWS IoT console, the AWS CLI, or the AWS API.
For this tutorial, you only need to create one custom authorizer. This section describes how to create by using the AWS IoT console and the AWS CLI, so you can use the method that is most convenient for you. There's no difference between the custom authorizer resources created by either method.
### Create a custom authorizer resource
**Choose one of these options to create your custom authorizer resource**
+ [Create a custom authorizer by using the AWS IoT console](#create-custom-auth-in-console)
+ [Create a custom authorizer using the AWS CLI](#create-custom-auth-in-cli)
**To create a custom authorizer (console)**
1. Open the [Custom authorizer page of the AWS IoT console](https://console.aws.amazon.com//iot/home#/authorizerhub), and choose **Create Authorizer**.
1. In **Create Authorizer**:
1. In **Authorizer name**, enter **my-new-authorizer**.
1. In **Authorizer status**, check **Active**.
1. In **Authorizer function**, choose the Lambda function you created earlier.
1. In **Token validation - optional**:
1. Toggle on **Token validation**.
1. In **Token key name**, enter **tokenKeyName**.
1. Choose **Add key**.
1. In **Key name**, enter **FirstKey**.
1. In **Public key**, enter the contents of the `public-key.pem` file. Be sure to include the lines from the file with `-----BEGIN PUBLIC KEY-----` and `-----END PUBLIC KEY-----` and don't add or remove any line feeds, carriage returns, or other characters from the file contents. The string that you enter should look something like this example.
```
-----BEGIN PUBLIC KEY-----
MIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEAvEBzOk4vhN+3LgslvEWt
sLCqNmt5Damas3bmiTRvq2gjRJ6KXGTGQChqArAJwL1a9dkS9+maaXC3vc6xzx9z
QPu/vQOe5tyzz1MsKdmtFGxMqQ3qjEXAMPLEOmqyUKPP5mff58k6ePSfXAnzBH0q
lg2HioefrpU5OSAnpuRAjYKofKjbc2Vrn6N2G7hV+IfTBvCElf0csalS/Rk4phD5
oa4Y0GHISRnevypg5C8n9Rrz91PWGqP6M/q5DNJJXjMyleG92hQgu1N696bn5Dw8
FhedszFa6b2x6xrItZFzewNQkPMLMFhNrQIIyvshtT/F1LVCS5+v8AQ8UGGDfZmv
QeqAMAF7WgagDMXcfgKSVU8yid2sIm56qsCLMvD2Sq8Lgzpey9N5ON1o1Cvldwvc
KrJJtgwW6hVqRGuShnownLpgG86M6neZ5sRMbVNZO8OzcobLngJ0Ibw9KkcUdklW
gvZ6HEJqBY2XE70iEXAMPLETPHzhqvK6Ei1HGxpHsXx6BNft582J1VpgYjXha8oa
/NN7l7Zbj/euAb41IVtmX8JrD9z613d1iM5L8HluJlUzn62Q+VeNV2tdA7MfPfMC
8btGYladFAnitThaz6+F0VSBJPu7pZQoLnqyEp5zLMtF+kFl2yOBmGAP0RBivRd9
JWBUCG0bqcLQPeQyjbXSOfUCAwEAAQ==
-----END PUBLIC KEY-----
```
1. Choose **Create authorizer**.
1. If the custom authorizer resource was created, you'll see the list of custom authorizers and your new custom authorizer should appear in the list and you can continue to the next section to test it.
If you see an error, review the error and try to create your custom authorizer again and double-check the entries. Note that each custom authorizer resource must have a unique name.
**To create a custom authorizer (AWS CLI)**
1. Substitute your values for `authorizer-function-arn` and `token-signing-public-keys`, and then run the following command:
```
aws iot create-authorizer \
--authorizer-name "my-new-authorizer" \
--token-key-name "tokenKeyName" \
--status ACTIVE \
--no-signing-disabled \
--authorizer-function-arn "arn:aws:lambda:Region:57EXAMPLE833:function:custom-auth-function" \
--token-signing-public-keys FirstKey="-----BEGIN PUBLIC KEY-----
MIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEAvEBzOk4vhN+3LgslvEWt
sLCqNmt5Damas3bmiTRvq2gjRJ6KXGTGQChqArAJwL1a9dkS9+maaXC3vc6xzx9z
QPu/vQOe5tyzz1MsKdmtFGxMqQ3qjEXAMPLEOmqyUKPP5mff58k6ePSfXAnzBH0q
lg2HioefrpU5OSAnpuRAjYKofKjbc2Vrn6N2G7hV+IfTBvCElf0csalS/Rk4phD5
oa4Y0GHISRnevypg5C8n9Rrz91PWGqP6M/q5DNJJXjMyleG92hQgu1N696bn5Dw8
FhedszFa6b2x6xrItZFzewNQkPMLMFhNrQIIyvshtT/F1LVCS5+v8AQ8UGGDfZmv
QeqAMAF7WgagDMXcfgKSVU8yid2sIm56qsCLMvD2Sq8Lgzpey9N5ON1o1Cvldwvc
KrJJtgwW6hVqRGuShnownLpgG86M6neZ5sRMbVNZO8OzcobLngJ0Ibw9KkcUdklW
gvZ6HEJqBY2XE70iEXAMPLETPHzhqvK6Ei1HGxpHsXx6BNft582J1VpgYjXha8oa
/NN7l7Zbj/euAb41IVtmX8JrD9z613d1iM5L8HluJlUzn62Q+VeNV2tdA7MfPfMC
8btGYladFAnitThaz6+F0VSBJPu7pZQoLnqyEp5zLMtF+kFl2yOBmGAP0RBivRd9
JWBUCG0bqcLQPeQyjbXSOfUCAwEAAQ==
-----END PUBLIC KEY-----"
```
**Where:**
+ The `authorizer-function-arn` value is the Amazon Resource Name (ARN) of the Lambda function you created for your custom authorizer.
+ The `token-signing-public-keys` value includes the name of the key, **FirstKey**, and the contents of the `public-key.pem` file. Be sure to include the lines from the file with `-----BEGIN PUBLIC KEY-----` and `-----END PUBLIC KEY-----` and don't add or remove any line feeds, carriage returns, or other characters from the file contents.
Note: be careful entering the public key as any alteration to the public key value makes it unusable.
1. If the custom authorizer is created, the command returns the name and ARN of the new resource, such as the following.
```
{
"authorizerName": "my-new-authorizer",
"authorizerArn": "arn:aws:iot:Region:57EXAMPLE833:authorizer/my-new-authorizer"
}
```
Save the `authorizerArn` value for use in the next step.
Remember that each custom authorizer resource must have a unique name.
### Authorize the custom authorizer resource
In this section, you'll grant permission the custom authorizer resource that you just created permission to run the Lambda function. To grant the permission, you can use the [add-permission](https://docs.aws.amazon.com//cli/latest/reference/lambda/add-permission.html) CLI command.
**Grant permission to your Lambda function using the AWS CLI**
1. After inserting your values, enter the following command. Note that the `statement-id` value must be unique. Replace `Id-1234` with another value if you have run this tutorial before or if you get a `ResourceConflictException` error.
```
aws lambda add-permission \
--function-name "custom-auth-function" \
--principal "iot.amazonaws.com" \
--action "lambda:InvokeFunction" \
--statement-id "Id-1234" \
--source-arn authorizerArn
```
1. If the command succeeds, it returns a permission statement, such as this example. You can continue to the next section to test the custom authorizer.
```
{
"Statement": "{\"Sid\":\"Id-1234\",\"Effect\":\"Allow\",\"Principal\":{\"Service\":\"iot.amazonaws.com\"},\"Action\":\"lambda:InvokeFunction\",\"Resource\":\"arn:aws:lambda:Region:57EXAMPLE833:function:custom-auth-function\",\"Condition\":{\"ArnLike\":{\"AWS:SourceArn\":\"arn:aws:lambda:Region:57EXAMPLE833:function:custom-auth-function\"}}}"
}
```
If the command doesn't succeed, it returns an error, such as this example. You'll need to review and correct the error before you continue.
```
An error occurred (AccessDeniedException) when calling the AddPermission operation: User: arn:aws:iam::57EXAMPLE833:user/EXAMPLE-1 is not authorized to perform: lambda:AddPer
mission on resource: arn:aws:lambda:Region:57EXAMPLE833:function:custom-auth-function
```
## Step 4: Test the authorizer by calling test-invoke-authorizer
With all the resources defined, in this section, you'll call test-invoke-authorizer from the command line to test the authorization pass.
Note that when invoking the authorizer from the command line, `protocolData` is not defined, so the authorizer will always return a DENY document. This test does, however, confirm that your custom authorizer and Lambda function are configured correctly--even if it doesn't fully test the Lambda function.
**To test your custom authorizer and its Lambda function by using the AWS CLI**
1. In the directory that has the `private-key.pem` file you created in a previous step, run the following command.
```
echo -n "tokenKeyValue" | openssl dgst -sha256 -sign private-key.pem | openssl base64 -A
```
This command creates a signature string to use in the next step. The signature string looks something like this:
```
dBwykzlb+fo+JmSGdwoGr8dyC2qB/IyLefJJr+rbCvmu9Jl4KHAA9DG+V+MMWu09YSA86+64Y3Gt4tOykpZqn9mn
VB1wyxp+0bDZh8hmqUAUH3fwi3fPjBvCa4cwNuLQNqBZzbCvsluv7i2IMjEg+CPY0zrWt1jr9BikgGPDxWkjaeeh
bQHHTo357TegKs9pP30Uf4TrxypNmFswA5k7QIc01n4bIyRTm90OyZ94R4bdJsHNig1JePgnuOBvMGCEFE09jGjj
szEHfgAUAQIWXiVGQj16BU1xKpTGSiTAwheLKUjITOEXAMPLECK3aHKYKY+d1vTvdthKtYHBq8MjhzJ0kggbt29V
QJCb8RilN/P5+vcVniSXWPplyB5jkYs9UvG08REoy64AtizfUhvSul/r/F3VV8ITtQp3aXiUtcspACi6ca+tsDuX
f3LzCwQQF/YSUy02u5XkWn+sto6KCkpNlkD0wU8gl3+kOzxrthnQ8gEajd5Iylx230iqcXo3osjPha7JDyWM5o+K
EWckTe91I1mokDr5sJ4JXixvnJTVSx1li49IalW4en1DAkc1a0s2U2UNm236EXAMPLELotyh7h+flFeloZlAWQFH
xRlXsPqiVKS1ZIUClaZWprh/orDJplpiWfBgBIOgokJIDGP9gwhXIIk7zWrGmWpMK9o=
```
Copy this signature string to use in the next step. Be careful not to include any extra characters or leave any out.
1. In this command, replace the `token-signature` value with the signature string from the previous step and run this command to test your authorizer.
```
aws iot test-invoke-authorizer \
--authorizer-name my-new-authorizer \
--token tokenKeyValue \
--token-signature dBwykzlb+fo+JmSGdwoGr8dyC2qB/IyLefJJr+rbCvmu9Jl4KHAA9DG+V+MMWu09YSA86+64Y3Gt4tOykpZqn9mnVB1wyxp+0bDZh8hmqUAUH3fwi3fPjBvCa4cwNuLQNqBZzbCvsluv7i2IMjEg+CPY0zrWt1jr9BikgGPDxWkjaeehbQHHTo357TegKs9pP30Uf4TrxypNmFswA5k7QIc01n4bIyRTm90OyZ94R4bdJsHNig1JePgnuOBvMGCEFE09jGjjszEHfgAUAQIWXiVGQj16BU1xKpTGSiTAwheLKUjITOEXAMPLECK3aHKYKY+d1vTvdthKtYHBq8MjhzJ0kggbt29VQJCb8RilN/P5+vcVniSXWPplyB5jkYs9UvG08REoy64AtizfUhvSul/r/F3VV8ITtQp3aXiUtcspACi6ca+tsDuXf3LzCwQQF/YSUy02u5XkWn+sto6KCkpNlkD0wU8gl3+kOzxrthnQ8gEajd5Iylx230iqcXo3osjPha7JDyWM5o+KEWckTe91I1mokDr5sJ4JXixvnJTVSx1li49IalW4en1DAkc1a0s2U2UNm236EXAMPLELotyh7h+flFeloZlAWQFHxRlXsPqiVKS1ZIUClaZWprh/orDJplpiWfBgBIOgokJIDGP9gwhXIIk7zWrGmWpMK9o=
```
If the command is successful, it returns the information generated by your custom authorizer function, such as this example.
```
{
"isAuthenticated": true,
"principalId": "principalId",
"policyDocuments": [
"{\"Version\":\"2012-10-17\", \"Statement\":[{\"Action\":\"iot:*\",\"Effect\":\"Deny\",\"Resource\":\"arn:aws:iot:Region:57EXAMPLE833:*\"}]}"
],
"refreshAfterInSeconds": 600,
"disconnectAfterInSeconds": 3600
}
```
If the command returns an error, review the error and double-check the commands you used in this section.
## Step 5: Test publishing MQTT message using Postman
1. To get your device data endpoint from the command line, call [describe-endpoint](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/iot/describe-endpoint.html) as shown here
```
aws iot describe-endpoint --output text --endpoint-type iot:Data-ATS
```
Save this address for use as the *device\$1data\$1endpoint\$1address* in a later step.
1. Open a new Postman window and create a new HTTP POST request.
1. From your computer, open the Postman app.
1. In Postman, in the **File** menu, choose **New...**.
1. In the **New** dialog box, choose **Request**.
1. In Save request,
1. In **Request name** enter **Custom authorizer test request**.
1. In **Select a collection or folder to save to:** choose or create a collection into which to save this request.
1. Choose **Save to *collection\$1name***.
1. Create the POST request to test your custom authorizer.
1. In the request method selector next to the URL field, choose **POST**.
1. In the URL field, create the URL for your request by using the following URL with the *device\$1data\$1endpoint\$1address* from the [describe-endpoint](https://awscli.amazonaws.com/v2/documentation/api/latest/reference/iot/describe-endpoint.html) command in a previous step.
```
https://device_data_endpoint_address:443/topics/test/cust-auth/topic?qos=0&actionToken=allow
```
Note that this URL includes the `actionToken=allow` query parameter that will tell your Lambda function to return a policy document that allows access to AWS IoT. After you enter the URL, the query parameters also appear in the **Params** tab of Postman.
1. In the **Auth** tab, in the **Type** field, choose **No Auth**.
1. In the Headers tab:
1. If there's a **Host** key that's checked, uncheck this one.
1. At the bottom of the list of headers add these new headers and confirm they are checked. Replace the **Host** value with your *device\$1data\$1endpoint\$1address* and the **x-amz-customauthorizer-signature** value with the signature string that you used with the **test-invoke-authorize** command in the previous section.
[\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/iot/latest/developerguide/custom-auth-tutorial.html)
1. In the Body tab:
1. In the data format option box, choose **Raw**.
1. In the data type list, choose **JavaScript**.
1. In the text field, enter this JSON message payload for your test message:
```
{
"data_mode": "test",
"vibration": 200,
"temperature": 40
}
```
1. Choose **Send** to send the request.
If the request was successful, it returns:
```
{
"message": "OK",
"traceId": "ff35c33f-409a-ea90-b06f-fbEXAMPLE25c"
}
```
The successful response indicates that your custom authorizer allowed the connection to AWS IoT and that the test message was delivered to broker in AWS IoT Core.
If it returns an error, review error message, the *device\$1data\$1endpoint\$1address*, the signature string, and the other header values.
Keep this request in Postman for use in the next section.
## Step 6: View messages in MQTT test client
In the previous step, you sent simulated device messages to AWS IoT by using Postman. The successful response indicated that your custom authorizer allowed the connection to AWS IoT and that the test message was delivered to broker in AWS IoT Core. In this section, you'll use the MQTT test client in the AWS IoT console to see the message contents from that message as other devices and services might.
**To see the test messages authorized by your custom authorizer**
1. In the AWS IoT console, open the [MQTT test client](https://console.aws.amazon.com//iot/home#/test).
1. In the **Subscribe to topic** tab, in **Topic filter**, enter **test/cust-auth/topic**, which is the message topic used in the Postman example from the previous section.
1. Choose **Subscribe**.
Keep this window visible for the next step.
1. In Postman, in the request you created for the previous section, choose **Send**.
Review the response to make sure it was successful. If not, troubleshoot the error as the previous section describes.
1. In the **MQTT test client**, you should see a new entry that shows the message topic and, if expanded, the message payload from the request you sent from Postman.
If you don't see your messages in the **MQTT test client**, here are some things to check:
+ Make sure your Postman request returned successfully. If AWS IoT rejects the connection and returns an error, the message in the request doesn't get passed to the message broker.
+ Make sure the AWS account and AWS Region used to open the AWS IoT console are the same as you're using in the Postman URL.
+ Make sure that you're using the appropriate endpoint for the custom authorizer. The default IoT endpoint might not support using custom authorizers with Lambda functions. Instead, you can use domain configurations to define a new endpoint and then specify that endpoint for the custom authorizer.
+ Make sure you've entered the topic correctly in the **MQTT test client**. The topic filter is case-sensitive. If in doubt, you can also subscribe to the **\$1** topic, which subscribes to all MQTT messages that pass through the message broker the AWS account and AWS Region used to open the AWS IoT console.
## Step 7: Review the results and next steps
**In this tutorial:**
+ You created a Lambda function to be a custom authorizer handler
+ You created a custom authorizer with token signing enabled
+ You tested your custom authorizer using the **test-invoke-authorizer** command
+ You published an MQTT topic by using [Postman](https://www.postman.com/) and validate the request with your custom authorizer
+ You used the **MQTT test client** to view the messages sent from your Postman test
**Next steps**
After you send some messages from Postman to verify that the custom authorizer is working, try experimenting to see how changing different aspects of this tutorial affect the results. Here are some examples to get you started.
+ Change the signature string so that it's no longer valid to see how unauthorized connection attempts are handled. You should get an error response, such as this one, and the message should not appear in the **MQTT test client**.
```
{
"message": "Forbidden",
"traceId": "15969756-a4a4-917c-b47a-5433e25b1356"
}
```
+ To learn more about how to find errors that might occur while you're developing and using AWS IoT rules, see [Monitoring AWS IoT](monitoring_overview.md).
## Step 8: Clean up
If you'd like repeat this tutorial, you might need to remove some of your custom authorizers. Your AWS account can have only a limited number of custom authorizers configured at one time and you can get a `LimitExceededException` when you try to add a new one without removing an existing custom authorizer.
**To remove a custom authorizer (console)**
1. Open the [Custom authorizer page of the AWS IoT console](https://console.aws.amazon.com//iot/home#/authorizerhub), and in the list of custom authorizers, find the custom authorizer to remove.
1. Open the Custom authorizer details page and, from the **Actions** menu, choose **Edit**.
1. Uncheck the **Activate authorizer**, and then choose **Update**.
You can't delete a custom authorizer while it's active.
1. From the Custom authorizer details page, open the **Actions** menu, and choose **Delete**.
**To remove a custom authorizer (AWS CLI)**
1. List the custom authorizers that you have installed and find the name of the custom authorizer you want to delete.
```
aws iot list-authorizers
```
1. Set the custom authorizer to `inactive` by running this command after replacing `Custom_Auth_Name` with the `authorizerName` of the custom authorizer to delete.
```
aws iot update-authorizer --status INACTIVE --authorizer-name Custom_Auth_Name
```
1. Delete the custom authorizer by running this command after replacing `Custom_Auth_Name` with the `authorizerName` of the custom authorizer to delete.
```
aws iot delete-authorizer --authorizer-name Custom_Auth_Name
```
# Tutorial: Monitoring soil moisture with AWS IoT and Raspberry Pi
This tutorial shows you how to use a [Raspberry Pi](https://www.raspberrypi.org/), a moisture sensor, and AWS IoT to monitor the soil moisture level for a house plant or garden. The Raspberry Pi runs code that reads the moisture level and temperature from the sensor and then sends the data to AWS IoT. You create a rule in AWS IoT that sends an email to an address subscribed to an Amazon SNS topic when the moisture level falls below a threshold.
**Note**
This tutorial might not be up to date. Some references might have been superseded since this topic was originally published.
**Contents**
+ [Prerequisites](#iot-moisture-prereqs)
+ [Setting up AWS IoT](iot-moisture-setup.md)
+ [Step 1: Create the AWS IoT policy](iot-moisture-policy.md)
+ [Step 2: Create the AWS IoT thing, certificate, and private key](iot-moisture-create-thing.md)
+ [Step 3: Create an Amazon SNS topic and subscription](iot-moisture-create-sns-topic.md)
+ [Step 4: Create an AWS IoT rule to send an email](iot-moisture-create-rule.md)
+ [Setting up your Raspberry Pi and moisture sensor](iot-moisture-raspi-setup.md)
## Prerequisites
To complete this tutorial, you need:
+ An AWS account.
+ An IAM user with administrator permissions.
+ A development computer running Windows, macOS, Linux, or Unix to access the [AWS IoT console](https://console.aws.amazon.com/iot/home).
+ A [Raspberry Pi 3B or 4B](https://www.raspberrypi.com/products/) running the latest [Raspberry Pi OS](https://www.raspberrypi.com/software/operating-systems/). For installation instructions, see [Install an operating system](https://www.raspberrypi.com/documentation/computers/getting-started.html#installing-the-operating-system) on the Raspberry Pi website.
+ A monitor, keyboard, mouse, and Wi-Fi network or Ethernet connection for your Raspberry Pi.
+ A Raspberry Pi-compatible moisture sensor. The sensor used in this tutorial is an [Adafruit STEMMA I2C Capacitive Moisture Sensor](https://www.adafruit.com/product/4026) with a [JST 4-pin to female socket cable header](https://www.adafruit.com/product/3950).
# Setting up AWS IoT
To complete this tutorial, you need to create the following resources. To connect a device to AWS IoT, you create an IoT thing, a device certificate, and an AWS IoT policy.
+ An AWS IoT thing.
A thing represents a physical device (in this case, your Rasberry Pi) and contains static metadata about the device.
+ A device certificate.
All devices must have a device certificate to connect to and authenticate with AWS IoT.
+ An AWS IoT policy.
Each device certificate has one or more AWS IoT policies associated with it. These policies determine which AWS IoT resources the device can access.
+ An AWS IoT root CA certificate.
Devices and other clients use an AWS IoT root CA certificate to authenticate the AWS IoT server with which they are communicating. For more information, see [Server authentication](server-authentication.md).
+ An AWS IoT rule.
A rule contains a query and one or more rule actions. The query extracts data from device messages to determine if the message data should be processed. The rule action specifies what to do if the data matches the query.
+ An Amazon SNS topic and topic subscription.
The rule listens for moisture data from your Raspberry Pi. If the value is below a threshold, it sends a message to the Amazon SNS topic. Amazon SNS sends that message to all email addresses subscribed to the topic.
# Step 1: Create the AWS IoT policy
Create an AWS IoT policy that allows your Raspberry Pi to connect and send messages to AWS IoT.
1. In the [AWS IoT console](https://console.aws.amazon.com/iot), if a **Get started** button appears, choose it. Otherwise, in the navigation pane, expand ** Security**, and then choose **Policies**.
1. If a **You don't have any policies yet** dialog box appears, choose **Create a policy**. Otherwise, choose **Create**.
1. Enter a name for the AWS IoT policy (for example, **MoistureSensorPolicy**).
1. In the **Add statements** section, replace the existing policy with the following JSON. Replace *region* and *account* with your AWS Region and AWS account number.
****
```
{
"Version":"2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": "iot:Connect",
"Resource": "arn:aws:iot:us-east-1:123456789012:client/RaspberryPi"
},
{
"Effect": "Allow",
"Action": "iot:Publish",
"Resource": [
"arn:aws:iot:us-east-1:123456789012:topic/$aws/things/RaspberryPi/shadow/update",
"arn:aws:iot:us-east-1:123456789012:topic/$aws/things/RaspberryPi/shadow/delete",
"arn:aws:iot:us-east-1:123456789012:topic/$aws/things/RaspberryPi/shadow/get"
]
},
{
"Effect": "Allow",
"Action": "iot:Receive",
"Resource": [
"arn:aws:iot:us-east-1:123456789012:topic/$aws/things/RaspberryPi/shadow/update/accepted",
"arn:aws:iot:us-east-1:123456789012:topic/$aws/things/RaspberryPi/shadow/delete/accepted",
"arn:aws:iot:us-east-1:123456789012:topic/$aws/things/RaspberryPi/shadow/get/accepted",
"arn:aws:iot:us-east-1:123456789012:topic/$aws/things/RaspberryPi/shadow/update/rejected",
"arn:aws:iot:us-east-1:123456789012:topic/$aws/things/RaspberryPi/shadow/delete/rejected"
]
},
{
"Effect": "Allow",
"Action": "iot:Subscribe",
"Resource": [
"arn:aws:iot:us-east-1:123456789012:topicfilter/$aws/things/RaspberryPi/shadow/update/accepted",
"arn:aws:iot:us-east-1:123456789012:topicfilter/$aws/things/RaspberryPi/shadow/delete/accepted",
"arn:aws:iot:us-east-1:123456789012:topicfilter/$aws/things/RaspberryPi/shadow/get/accepted",
"arn:aws:iot:us-east-1:123456789012:topicfilter/$aws/things/RaspberryPi/shadow/update/rejected",
"arn:aws:iot:us-east-1:123456789012:topicfilter/$aws/things/RaspberryPi/shadow/delete/rejected"
]
},
{
"Effect": "Allow",
"Action": [
"iot:GetThingShadow",
"iot:UpdateThingShadow",
"iot:DeleteThingShadow"
],
"Resource": "arn:aws:iot:us-east-1:123456789012:thing/RaspberryPi"
}
]
}
```
1. Choose **Create**.
# Step 2: Create the AWS IoT thing, certificate, and private key
Create a thing in the AWS IoT registry to represent your Raspberry Pi.
1. In the [AWS IoT console](https://console.aws.amazon.com/iot/home), in the navigation pane, choose **Manage**, and then choose **Things**.
1. If a **You don't have any things yet** dialog box is displayed, choose **Register a thing**. Otherwise, choose **Create**.
1. On the **Creating AWS IoT things** page, choose **Create a single thing**.
1. On the **Add your device to the device registry** page, enter a name for your IoT thing (for example, **RaspberryPi**), and then choose **Next**. You can't change the name of a thing after you create it. To change a thing's name, you must create a new thing, give it the new name, and then delete the old thing.
1. On the **Add a certificate for your thing** page, choose **Create certificate**.
1. Choose the **Download** links to download the certificate, private key, and root CA certificate.
**Important**
This is the only time you can download your certificate and private key.
1. To activate the certificate, choose **Activate**. The certificate must be active for a device to connect to AWS IoT.
1. Choose **Attach a policy**.
1. For **Add a policy for your thing**, choose **MoistureSensorPolicy**, and then choose **Register Thing**.
# Step 3: Create an Amazon SNS topic and subscription
Create an Amazon SNS topic and subscription.
1. From the [AWS SNS console](https://console.aws.amazon.com/sns/home), in the navigation pane, choose **Topics**, and then choose **Create topic**.
1. Choose type as **Standard** and enter a name for the topic (for example, **MoistureSensorTopic**).
1. Enter a display name for the topic (for example, **Moisture Sensor Topic**). This is the name displayed for your topic in the Amazon SNS console.
1. Choose **Create topic**.
1. In the Amazon SNS topic detail page, choose **Create subscription**.
1. For **Protocol**, choose **Email**.
1. For **Endpoint**, enter your email address.
1. Choose **Create subscription**.
1. Open your email client and look for a message with the subject **MoistureSensorTopic**. Open the email and click the **Confirm subscription** link.
**Important**
You won't receive any email alerts from this Amazon SNS topic until you confirm the subscription.
You should receive an email message with the text you typed.
# Step 4: Create an AWS IoT rule to send an email
An AWS IoT rule defines a query and one or more actions to take when a message is received from a device. The AWS IoT rules engine listens for messages sent by devices and uses the data in the messages to determine if some action should be taken. For more information, see [Rules for AWS IoT](iot-rules.md).
In this tutorial, your Raspberry Pi publishes messages on `aws/things/RaspberryPi/shadow/update`. This is an internal MQTT topic used by devices and the Thing Shadow service. The Raspberry Pi publishes messages that have the following form:
```
{
"reported": {
"moisture" : moisture-reading,
"temp" : temperature-reading
}
}
```
You create a query that extracts the moisture and temperature data from the incoming message. You also create an Amazon SNS action that takes the data and sends it to Amazon SNS topic subscribers if the moisture reading is below a threshold value.
**Create an Amazon SNS rule**
1. In the [AWS IoT console](https://console.aws.amazon.com/iot/home), choose **Message routing** and then choose **Rules**. If a **You don't have any rules yet** dialog box appears, choose **Create a rule**. Otherwise, choose **Create rule**.
1. In the **Rule properties** page, enter a **Rule name** such as **MoistureSensorRule**, and provide a short **Rule description** such as **Sends an alert when soil moisture level readings are too low**.
1. Choose **Next** and configure your SQL statement. Choose **SQL version** as **2016-03-23**, and enter the following AWS IoT SQL query statement:
```
SELECT * FROM '$aws/things/RaspberryPi/shadow/update/accepted' WHERE state.reported.moisture < 400
```
This statement triggers the rule action when the `moisture` reading is less than `400`.
**Note**
You might have to use a different value. After you have the code running on your Raspberry Pi, you can see the values that you get from your sensor by touching the sensor, placing it in water, or placing it in a planter.
1. Choose **Next** and attach rule actions. For **Action 1**, choose **Simple Notification Service**. The description for this rule action is **Send a message as an SNS push notification**.
1. For **SNS topic**, choose the topic that you created in [Step 3: Create an Amazon SNS topic and subscription](iot-moisture-create-sns-topic.md), **MoistureSensorTopic**, and leave the **Message format** as **RAW**. For **IAM role**, choose **Create a new role**. Enter a name for the role, for example, **LowMoistureTopicRole**, and then choose **Create role**.
1. Choose **Next** to review and then choose **Create** to create the rule.
# Setting up your Raspberry Pi and moisture sensor
Insert your microSD card into the Raspberry Pi, connect your monitor, keyboard, mouse, and, if you're not using Wi-Fi, Ethernet cable. Do not connect the power cable yet.
Connect the JST jumper cable to the moisture sensor. The other side of the jumper has four wires:
+ Green: I2C SCL
+ White: I2C SDA
+ Red: power (3.5 V)
+ Black: ground
Hold the Raspberry Pi with the Ethernet jack on the right. In this orientation, there are two rows of GPIO pins at the top. Connect the wires from the moisture sensor to the bottom row of pins in the following order. Starting at the left-most pin, connect red (power), white (SDA), and green (SCL). Skip one pin, and then connect the black (ground) wire. For more information, see [Python Computer Wiring](https://learn.adafruit.com/adafruit-stemma-soil-sensor-i2c-capacitive-moisture-sensor/python-circuitpython-test).
Attach the power cable to the Raspberry Pi and plug the other end into a wall socket to turn it on.
**Configure your Raspberry Pi**
1. On **Welcome to Raspberry Pi**, choose **Next**.
1. Choose your country, language, timezone, and keyboard layout. Choose **Next**.
1. Enter a password for your Raspberry Pi, and then choose **Next**.
1. Choose your Wi-Fi network, and then choose **Next**. If you aren't using a Wi-Fi network, choose **Skip**.
1. Choose **Next** to check for software updates. When the updates are complete, choose **Restart** to restart your Raspberry Pi.
After your Raspberry Pi starts up, enable the I2C interface.
1. In the upper left corner of the Raspbian desktop, click the Raspberry icon, choose **Preferences**, and then choose **Raspberry Pi Configuration**.
1. On the **Interfaces** tab, for **I2C**, choose **Enable**.
1. Choose **OK**.
The libraries for the Adafruit STEMMA moisture sensor are written for CircuitPython. To run them on a Raspberry Pi, you need to install the latest version of Python 3.
1. Run the following commands from a command prompt to update your Raspberry Pi software:
`sudo apt-get update`
`sudo apt-get upgrade`
1. Run the following command to update your Python 3 installation:
`sudo pip3 install --upgrade setuptools`
1. Run the following command to install the Raspberry Pi GPIO libraries:
`pip3 install RPI.GPIO`
1. Run the following command to install the Adafruit Blinka libraries:
`pip3 install adafruit-blinka`
For more information, see [Installing CircuitPython Libraries on Raspberry Pi](https://learn.adafruit.com/circuitpython-on-raspberrypi-linux/installing-circuitpython-on-raspberry-pi).
1. Run the following command to install the Adafruit Seesaw libraries:
`sudo pip3 install adafruit-circuitpython-seesaw`
1. Run the following command to install the AWS IoT Device SDK for Python:
`pip3 install AWSIoTPythonSDK`
Your Raspberry Pi now has all of the required libraries. Create a file called **moistureSensor.py** and copy the following Python code into the file:
```
from adafruit_seesaw.seesaw import Seesaw
from AWSIoTPythonSDK.MQTTLib import AWSIoTMQTTShadowClient
from board import SCL, SDA
import logging
import time
import json
import argparse
import busio
# Shadow JSON schema:
#
# {
# "state": {
# "desired":{
# "moisture":,
# "temp":
# }
# }
# }
# Function called when a shadow is updated
def customShadowCallback_Update(payload, responseStatus, token):
# Display status and data from update request
if responseStatus == "timeout":
print("Update request " + token + " time out!")
if responseStatus == "accepted":
payloadDict = json.loads(payload)
print("~~~~~~~~~~~~~~~~~~~~~~~")
print("Update request with token: " + token + " accepted!")
print("moisture: " + str(payloadDict["state"]["reported"]["moisture"]))
print("temperature: " + str(payloadDict["state"]["reported"]["temp"]))
print("~~~~~~~~~~~~~~~~~~~~~~~\n\n")
if responseStatus == "rejected":
print("Update request " + token + " rejected!")
# Function called when a shadow is deleted
def customShadowCallback_Delete(payload, responseStatus, token):
# Display status and data from delete request
if responseStatus == "timeout":
print("Delete request " + token + " time out!")
if responseStatus == "accepted":
print("~~~~~~~~~~~~~~~~~~~~~~~")
print("Delete request with token: " + token + " accepted!")
print("~~~~~~~~~~~~~~~~~~~~~~~\n\n")
if responseStatus == "rejected":
print("Delete request " + token + " rejected!")
# Read in command-line parameters
def parseArgs():
parser = argparse.ArgumentParser()
parser.add_argument("-e", "--endpoint", action="store", required=True, dest="host", help="Your device data endpoint")
parser.add_argument("-r", "--rootCA", action="store", required=True, dest="rootCAPath", help="Root CA file path")
parser.add_argument("-c", "--cert", action="store", dest="certificatePath", help="Certificate file path")
parser.add_argument("-k", "--key", action="store", dest="privateKeyPath", help="Private key file path")
parser.add_argument("-p", "--port", action="store", dest="port", type=int, help="Port number override")
parser.add_argument("-n", "--thingName", action="store", dest="thingName", default="Bot", help="Targeted thing name")
parser.add_argument("-id", "--clientId", action="store", dest="clientId", default="basicShadowUpdater", help="Targeted client id")
args = parser.parse_args()
return args
# Configure logging
# AWSIoTMQTTShadowClient writes data to the log
def configureLogging():
logger = logging.getLogger("AWSIoTPythonSDK.core")
logger.setLevel(logging.DEBUG)
streamHandler = logging.StreamHandler()
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
streamHandler.setFormatter(formatter)
logger.addHandler(streamHandler)
# Parse command line arguments
args = parseArgs()
if not args.certificatePath or not args.privateKeyPath:
parser.error("Missing credentials for authentication.")
exit(2)
# If no --port argument is passed, default to 8883
if not args.port:
args.port = 8883
# Init AWSIoTMQTTShadowClient
myAWSIoTMQTTShadowClient = None
myAWSIoTMQTTShadowClient = AWSIoTMQTTShadowClient(args.clientId)
myAWSIoTMQTTShadowClient.configureEndpoint(args.host, args.port)
myAWSIoTMQTTShadowClient.configureCredentials(args.rootCAPath, args.privateKeyPath, args.certificatePath)
# AWSIoTMQTTShadowClient connection configuration
myAWSIoTMQTTShadowClient.configureAutoReconnectBackoffTime(1, 32, 20)
myAWSIoTMQTTShadowClient.configureConnectDisconnectTimeout(10) # 10 sec
myAWSIoTMQTTShadowClient.configureMQTTOperationTimeout(5) # 5 sec
# Initialize Raspberry Pi's I2C interface
i2c_bus = busio.I2C(SCL, SDA)
# Intialize SeeSaw, Adafruit's Circuit Python library
ss = Seesaw(i2c_bus, addr=0x36)
# Connect to AWS IoT
myAWSIoTMQTTShadowClient.connect()
# Create a device shadow handler, use this to update and delete shadow document
deviceShadowHandler = myAWSIoTMQTTShadowClient.createShadowHandlerWithName(args.thingName, True)
# Delete current shadow JSON doc
deviceShadowHandler.shadowDelete(customShadowCallback_Delete, 5)
# Read data from moisture sensor and update shadow
while True:
# read moisture level through capacitive touch pad
moistureLevel = ss.moisture_read()
# read temperature from the temperature sensor
temp = ss.get_temp()
# Display moisture and temp readings
print("Moisture Level: {}".format(moistureLevel))
print("Temperature: {}".format(temp))
# Create message payload
payload = {"state":{"reported":{"moisture":str(moistureLevel),"temp":str(temp)}}}
# Update shadow
deviceShadowHandler.shadowUpdate(json.dumps(payload), customShadowCallback_Update, 5)
time.sleep(1)
```
Save the file to a place you can find it. Run `moistureSensor.py` from the command line with the following parameters:
endpoint
Your custom AWS IoT endpoint. For more information, see [Device Shadow REST API](device-shadow-rest-api.md).
rootCA
The full path to your AWS IoT root CA certificate.
cert
The full path to your AWS IoT device certificate.
key
The full path to your AWS IoT device certificate private key.
thingName
Your thing name (in this case, `RaspberryPi`).
clientId
The MQTT client ID. Use `RaspberryPi`.
The command line should look like this:
`python3 moistureSensor.py --endpoint your-endpoint --rootCA ~/certs/AmazonRootCA1.pem --cert ~/certs/raspberrypi-certificate.pem.crt --key ~/certs/raspberrypi-private.pem.key --thingName RaspberryPi --clientId RaspberryPi`
Try touching the sensor, putting it in a planter, or putting it in a glass of water to see how the sensor responds to various levels of moisture. If needed, you can change the threshold value in the `MoistureSensorRule`. When the moisture sensor reading goes below the value specified in your rule's SQL query statement, AWS IoT publishes a message to the Amazon SNS topic. You should receive an email message that contains the moisture and temperature data.
After you have verified receipt of email messages from Amazon SNS, press **CTRL\$1C** to stop the Python program. It is unlikely that the Python program will send enough messages to incur charges, but it is a best practice to stop the program when you are done.