# Deploy containerized applications on AWS IoT Greengrass V2 running as a Docker container
*Salih Bakir, Giuseppe Di Bella, and Gustav Svalander, Amazon Web Services*
## Summary
AWS IoT Greengrass Version 2, when deployed as a Docker container, doesn't natively support running Docker application containers. This pattern shows you how to create a custom container image based on the latest version of AWS IoT Greengrass V2 that enables Docker-in-Docker (DinD) functionality. With DinD, you can run containerized applications within the AWS IoT Greengrass V2 environment.
You can deploy this pattern as a stand-alone solution or integrate it with container orchestration platforms like Amazon ECS Anywhere. In either deployment model, you maintain full AWS IoT Greengrass V2 functionality including AWS IoT SiteWise Edge processing capabilities, while enabling scalable container-based deployments.
## Prerequisites and limitations
**Prerequisites **
+ An active AWS account.
+ For general AWS IoT Greengrass Version 2 prerequisites, see [Prerequisites](https://docs.aws.amazon.com/greengrass/v2/developerguide/getting-started-prerequisites.html) in the AWS IoT Greengrass Version 2 documentation.
+ Docker Engine, installed and configured on Linux, macOS, or Windows.
+ Docker Compose (if you use the Docker Compose command line interface (CLI) to run Docker images).
+ A Linux operating system.
+ A hypervisor with a host server that supports virtualization.
+ System requirements:
+ 2 GB of RAM (minimum)
+ 5 GB of available disk space (minimum)
+ For AWS IoT SiteWise Edge, an x86\_64 quad-core CPU with 16 GB of RAM and 50 GB of available disk space. For more information about AWS IoT SiteWise data processing, see [Data processing pack requirements](https://docs.aws.amazon.com/iot-sitewise/latest/userguide/configure-gateway-ggv2.html#w2aac17c19c13b7) in the AWS IoT SiteWise documentation.
**Product versions**
+ AWS IoT Greengrass Version 2 version 2.5.3 or later
+ Docker-in-Docker version 1.0.0 or later
+ Docker Compose version 1.22 or later
+ Docker Engine version 20.10.12 or later
**Limitations**
+ Some AWS services aren’t available in all AWS Regions. For Region availability, see [AWS Services by Region](https://aws.amazon.com/about-aws/global-infrastructure/regional-product-services/). For specific endpoints, see [Service endpoints and quotas](https://docs.aws.amazon.com/general/latest/gr/aws-service-information.html), and choose the link for the service.
## Architecture
**Target technology stack**
+ **Data sources** – IoT devices, sensors, or industrial equipment that generates data for processing
+ **AWS IoT Greengrass V2** – Running as a Docker container with D-in-D capabilities, deployed on edge infrastructures
+ **Containerized applications** – Custom applications running within the AWS IoT Greengrass V2 environment as nested Docker containers
+ **(Optional) Amazon ECS Anywhere** – Container orchestration that manages the AWS IoT Greengrass V2 container deployment
+ **Other AWS services** – AWS IoT Core, AWS IoT SiteWise, and other AWS services for data processing and management
**Target architecture **
The following diagram shows an example target deployment architecture that uses Amazon ECS Anywhere, which is a container management tool.
The diagram shows the following workflow:
**1: Container image storage** – Amazon ECR stores the AWS IoT Greengrass container images and any custom application containers needed for edge processing.
**2 **and** 3: Container deployment** – Amazon ECS Anywhere deploys the AWS IoT Greengrass container image from Amazon ECR to the edge location, managing the container lifecycle and deployment process.
**4: Component deployment** – The deployed AWS IoT Greengrass core automatically deploys its relevant components based on its configuration. Components include AWS IoT SiteWise Edge and other necessary edge processing components within the containerized environment.
**5: Data ingestion** – After it’s fully configured, AWS IoT Greengrass begins ingesting telemetry and sensor data from various IoT data sources at the edge location.
**6: Data processing and cloud integration** – The containerized AWS IoT Greengrass core processes data locally using its deployed components (including AWS IoT SiteWise Edge for industrial data). Then, it sends processed data to AWS Cloud services for further analysis and storage.
## Tools
**AWS services**
+ [Amazon ECS Anywhere](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/launch-type-external.html) helps you deploy, use, and manage Amazon ECS tasks and services on your own infrastructure.
+ [Amazon Elastic Compute Cloud (Amazon EC2)](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/concepts.html) provides scalable computing capacity in the AWS Cloud. You can launch as many virtual servers as you need and quickly scale them up or down.
+ [Amazon Elastic Container Registry (Amazon ECR)](https://docs.aws.amazon.com/AmazonECR/latest/userguide/what-is-ecr.html) is a managed container image registry service that’s secure, scalable, and reliable.
+ [AWS IoT Greengrass](https://docs.aws.amazon.com/greengrass/v2/developerguide/what-is-iot-greengrass.html) is an open source Internet of Things (IoT) edge runtime and cloud service that helps you build, deploy, and manage IoT applications on your devices.
+ [AWS IoT SiteWise](https://docs.aws.amazon.com/iot-sitewise/latest/userguide/what-is-sitewise.html) helps you collect, model, analyze, and visualize data from industrial equipment at scale.
**Other tools**
+ [Docker](https://www.docker.com/) is a set of platform as a service (PaaS) products that use virtualization at the operating-system level to deliver software in containers.
+ [Docker Compose](https://docs.docker.com/compose/) is a tool for defining and running multi-container applications.
+ [Docker Engine](https://docs.docker.com/engine/) is an open source containerization technology for building and containerizing applications.
**Code repository**
The code for this pattern is available in the GitHub [AWS IoT Greengrass v2 Docker-in-Docker](https://github.com/aws-samples/aws-iot-greengrass-docker-in-docker) repository.
## Epics
### Build the AWS IoT Greengrass V2 Docker-in-Docker image
| Task | Description | Skills required |
| --- | --- | --- |
| Clone and navigate to the repository. | To clone the repository, use the following command:
`git clone https://github.com/aws-samples/aws-iot-greengrass-docker-in-docker.git`
To navigate to the `docker` directory, use the following command:
`cd aws-iot-greengrass-docker-in-docker/docker` | DevOps engineer, AWS DevOps |
| Build the Docker image. | To build the Docker image with the default (latest) version, run the following command:
`docker build -t x86_64/aws-iot-greengrass:latest .`
Or, to build the Docker image with a specific version, run the following command:
`docker build --build-arg GREENGRASS_RELEASE_VERSION=2.12.0 -t x86_64/aws-iot-greengrass:2.12.0 .`
To verify the build, run the following command:
`docker images \| grep aws-iot-greengrass`
| AWS DevOps, DevOps engineer, App developer |
| (Optional) Push to Amazon ECR. | 1. To create the repository, run the following command:
`aws ecr create-repository --repository-name aws-iot-greengrass-dind --region us-east-1`
2. To authenticate, run the following command:
`aws ecr get-login-password --region us-east-1 \| docker login --username AWS --password-stdin .dkr.ecr.us-east-1.amazonaws.com`
3. To tag and push, run the following commands:
`docker tag x86_64/aws-iot-greengrass:latest .dkr.ecr.us-east-1.amazonaws.com/aws-iot-greengrass-dind:latest`
`docker push .dkr.ecr.us-east-1.amazonaws.com/aws-iot-greengrass-dind:latest` | App developer, AWS DevOps, DevOps engineer |
### Configure AWS credentials
| Task | Description | Skills required |
| --- | --- | --- |
| Select authentication method. | Choose one of the following options:+ **Option 1 (Recommended)**: IAM roles – Use if running on Amazon EC2, Amazon ECS, or Amazon EKS with proper IAM roles. No additional configuration needed.
+ **Option 2**: Environment variables – For testing and development.
+ **Option 3**: Credentials file – Not recommended for production.
+ **Option 4**: Legacy `env.cfg` file. | AWS administrator |
| Configure authentication method. | For the authentication method you selected, use the following configuration guidance:+ **Option 2 (Environment variables)** – Prepare `AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, and optionally `AWS_SESSION_TOKEN`.
+ **Option 3 (Credentials file)** – Create credentials directory and file:(a) Run the following command:
`mkdir -p ~/greengrass-v2-credentials`(b) Create the following file at `~/greengrass-v2-credentials/credentials`:
`[default]`
`aws_access_key_id=YourAccessKey`
`aws_secret_access_key=YourSecretKey`(c) Secure the file as follows:
`chmod 600 ~/greengrass-v2-credentials/credentials`
+ **Option 4 (**`env.cfg`**)** – Create the `env.cfg` file in the `docker` directory as follows:
`GGC_ROOT_PATH=/greengrass/v2`
`AWS_REGION=us-east-1`
`PROVISION=true`
`THING_NAME=MyGreengrassCore`
`THING_GROUP_NAME=MyGreengrassCoreGroup`
`TES_ROLE_NAME=GreengrassV2TokenExchangeRole`
`TES_ROLE_ALIAS_NAME=GreengrassCoreTokenExchangeRoleAlias`
`COMPONENT_DEFAULT_USER=ggc_user:ggc_group` | AWS administrator |
### Run with Docker Compose
| Task | Description | Skills required |
| --- | --- | --- |
| Configure `docker-compose.yml`. | Update the `docker-compose.yml` file with environment variables as follows:+ Key variables: `PROVISION,` `AWS_REGION`, `THING_NAME`, `THING_GROUP_NAME`, `TES_ROLE_NAME`, `TES_ROLE_ALIAS_NAME`
+ For credentials through environment variables, uncomment the following:
`environment:`
` - AWS_ACCESS_KEY_ID=YourKey`
` - AWS_SECRET_ACCESS_KEY=YourSecret`+ For credentials file, uncomment the following:
`volumes:`
` - ~/greengrass-v2-credentials:/root/.aws/:ro`+ For log persistence, add the following:
`volumes:`
` - ./logs:/greengrass/v2/logs` | DevOps engineer |
| Start and verify container. | To start in the foreground, run the following command:
`docker-compose up --build`
Or, to start in the background, run the following command:
`docker-compose up --build -d`
To verify status, run the following command:
`docker-compose ps`
To monitor logs, run the following command:
`docker-compose logs -f` | DevOps engineer |
### Run with Docker CLI
| Task | Description | Skills required |
| --- | --- | --- |
| Run container with Docker CLI. | + To initiate a basic run without provisioning, run the following command:
`docker run --init --privileged -it --name aws-iot-greengrass x86_64/aws-iot-greengrass:latest`+ To initiate a run with environment variables, run the following command:
`docker run --init --privileged -it --name aws-iot-greengrass -e PROVISION=true -e AWS_REGION=us-east-1 -e AWS_ACCESS_KEY_ID=YourKey -e AWS_SECRET_ACCESS_KEY=YourSecret -e THING_NAME=MyGreengrassCore -e THING_GROUP_NAME=MyGreengrassCoreGroup -p 8883:8883 x86_64/aws-iot-greengrass:latest`+ To initiate a run with a credentials file, run the following command:
`docker run --init --privileged -it --name aws-iot-greengrass -v ~/greengrass-v2-credentials:/root/.aws:ro -e PROVISION=true -e AWS_REGION=us-east-1 -e THING_NAME=MyGreengrassCore -p 8883:8883 x86_64/aws-iot-greengrass:latest`+ To initiate a run with an `env.cfg` file, run the following command:
`docker run --init --privileged -it --name aws-iot-greengrass -v ~/greengrass-v2-credentials:/root/.aws:ro --env-file env.cfg -p 8883:8883 x86_64/aws-iot-greengrass:latest`+ To initiate a run with persistent logs, run the following command:
`docker run --init --privileged -it --name aws-iot-greengrass -v \$(pwd)/logs:/greengrass/v2/logs x86_64/aws-iot-greengrass:latest` | DevOps engineer |
| Verify container. | + To check the container’s status, run the following command:
`docker ps \| grep aws-iot-greengrass`+ To monitor logs, run the following command:
`docker logs -f aws-iot-greengrass` | DevOps engineer |
### Manage containerized applications
| Task | Description | Skills required |
| --- | --- | --- |
| Deploy applications. | + To deploy the application container, run the following command:
`docker run -d --name my-app nginx:alpine`+ To verify deployment, run the following command:
`docker ps`+ To access logs, run the following command:
`cd /greengrass/v2/logs && cat greengrass.log` | App developer |
| Access and test Docker-in-Docker. | 1. To open a shell in the container, run the following command:
`docker exec -it aws-iot-greengrass sh`
2. To verify Docker, run the following commands:
`docker --version`
`docker ps`
3. To test Docker-in-Docker, run the following command:
`docker run --rm hello-world` | DevOps engineer |
### (Optional) Integrate with Amazon ECS Anywhere
| Task | Description | Skills required |
| --- | --- | --- |
| Set up Amazon ECS cluster. | + To create an Amazon ECS cluster, run the following command:
`aws ecs create-cluster --cluster-name greengrass-cluster --region us-east-1`+ To create SSM activation, run the following command:
`aws ssm create-activation --default-instance-name greengrass-edge --iam-role GreengrassECSAnywhereRole --registration-limit 10 --region us-east-1`+ To install SSM Agent on an edge device, run the following command:
`curl -o install.sh https://s3.amazonaws.com/aws-ssm-downloads/latest/linux_amd64/install.sh`
`chmod +x install.sh`
`./install.sh activation-code activation-id us-east-1` | AWS administrator |
| Deploy Amazon ECS task. | + Create `greengrass-task-definition.json` with task configuration including `containerDefinitions` with `privileged:true`, environment variables, and volume mounts for `/var/lib/docker` and `/greengrass/v2.Register` task. Run the following command:
`aws ecs register-task-definition --cli-input-json file://greengrass-task-definition.json`+ To create the service, run the following command:
`aws ecs create-service --cluster greengrass-cluster --service-name greengrass-service --task-definition greengrass-dind --desired-count 1 --launch-type EXTERNAL --region us-east-1`+ To verify deployment, run the following command:
`aws ecs describe-services --cluster greengrass-cluster --services greengrass-service --region us-east-1` | AWS administrator |
### Stop and cleanup
| Task | Description | Skills required |
| --- | --- | --- |
| Stop container. | + To stop the container by using Docker Compose, run the following commands:
`docker-compose stop`
`docker-compose down`
`docker-compose down -v`+ To stop the container (removes volumes) by using Docker CLI, run the following commands:
`docker stop aws-iot-greengrass`
`docker rm aws-iot-greengrass`+ To stop the container by using force remove, run the following command:
`docker rm -f aws-iot-greengrass` | DevOps engineer |
## Troubleshooting
| Issue | Solution |
| --- | --- |
| Container fails to start with permission errors. | 1. To make sure that the container runs with the `--privileged` flag required for Docker-in-Docker, run the following command:
`docker run --init --privileged -it --name aws-iot-greengrass x86_64/aws-iot-greengrass:latest`
2. In the `docker-compose.yml` file, set the following:
`privileged: true``--privileged` grants extended privileges to the container. |
| Provisioning fails with credential errors. | To verify credentials are configured correctly, use the following steps:1. Check IAM role:
`aws sts get-caller-identity`
2. Verify environment variables:
`docker exec -it aws-iot-greengrass sh -c 'echo \$AWS_ACCESS_KEY_ID'`
3. Check credentials file:
`docker exec -it aws-iot-greengrass sh -c 'cat /root/.aws/credentials'`
Make sure that IAM permissions include `iot:CreateThing`, `iot:CreatePolicy`, `iot:AttachPolicy`, `iam:CreateRole`, and `iam:AttachRolePolicy`. |
| Cannot connect to Docker daemon inside container. | 1. Check the Docker daemon status:
`docker exec -it aws-iot-greengrass sh -c 'ps aux \| grep dockerd'`
2. Verify that the Docker socket exists:
`docker exec -it aws-iot-greengrass sh -c 'ls -la /var/run/docker.sock'`
3. Manually start if needed:
`docker exec -it aws-iot-greengrass sh -c 'dockerd > /var/log/docker.log 2>&1 &'` |
| Container runs out of disk space. | 1. Check disk space:
`df -h`
2. Clean Docker resources:
`docker system prune -a --volumes`
3. Remove old Greengrass artifacts:
`docker exec -it aws-iot-greengrass sh -c 'rm -rf /greengrass/v2/work/*`'Ensure minimum disk space: 5 GB for basic operations and 50 GB for AWS IoT SiteWise Edge |
| Build issues. | 1. Make sure that you’re in the `docker` directory:
`cd docker && pwd`
2. Verify that `Dockerfile` exists:
`ls -la Dockerfile`
3. Check that Docker is installed and running:
`docker --version && docker ps` |
| Network connectivity issues. | 1. Test DNS resolution:
`docker exec -it aws-iot-greengrass sh -c 'nslookup google.com'`
2. Test AWS connectivity:
`docker exec -it aws-iot-greengrass sh -c 'curl https://iot.us-east-1.amazonaws.com'`Verify that the firewall allows outbound HTTPS (443) and MQTT (8883) traffic. |
| Greengrass components fail to deploy. | 1. Check Greengrass logs:
`docker exec -it aws-iot-greengrass sh -c 'cat /greengrass/v2/logs/greengrass.log'`
2. Verify TES role configuration:
`aws iot describe-role-alias --role-alias TES_ROLE_ALIAS_NAME`Check component-specific logs in the `/greengrass/v2/logs/` directory. |
| Container exits immediately after starting. | 1. Check container logs for errors:
`docker logs aws-iot-greengrass`
2. Run with interactive shell for debugging:
`docker run --init --privileged -it --entrypoint sh x86_64/aws-iot-greengrass:latest`Verify all required environment variables are set correctly if `PROVISION=true`. Make sure that the `--init` flag is used when starting the container. |
## Related resources
**AWS resources**
+ [Amazon Elastic Container Service](https://aws.amazon.com/ecs/)
+ [Configure edge data processing for AWS IoT SiteWise models and assets](https://docs.aws.amazon.com/iot-sitewise/latest/userguide/edge-processing.html)
+ [What is AWS IoT Greengrass](https://docs.aws.amazon.com/greengrass/v2/developerguide/what-is-iot-greengrass.html)
**Other resources**
+ [Docker documentation](https://docs.docker.com/)
## Additional information
+ For AWS IoT SiteWise Edge data processing, Docker must be available within the AWS IoT Greengrass environment.
+ To run a nested container, you must run the AWS IoT Greengrass container with administrator-level credentials.