Concatenating multiline or stack-trace Amazon ECS log messages

To use the JSON policy editor to create a policy

1. Sign in to the AWS Management Console and open the IAM console at https://console.aws.amazon.com/iam/.

2. In the navigation pane on the left, choose Policies.

   If this is your first time choosing Policies, the Welcome to Managed Policies page appears. Choose Get Started.

3. At the top of the page, choose Create policy.

4. In the Policy editor section, choose the JSON option.

5. Enter the following JSON policy document:

   {
   	"Version": "2012-10-17",
   	"Statement": [{
   		"Effect": "Allow",
   		"Action": [
   			"logs:CreateLogStream",
   			"logs:CreateLogGroup",
   			"logs:PutLogEvents"
   		],
   		"Resource": "*"
   	}]
   }

6. Choose Next.

   Note

   You can switch between the Visual and JSON editor options anytime. However, if you make changes or choose Next in the Visual editor, IAM might restructure your policy to optimize it for the visual editor. For more information, see Policy restructuring in the IAM User Guide.

7. On the Review and create page, enter a Policy name and a Description (optional) for the policy that you are creating. Review Permissions defined in this policy to see the permissions that are granted by your policy.

8. Choose Create policy to save your new policy.

Build and upload the image for a Fluent Bit container

This image will include the parser file where you specify the regular expression and a configuration file that references the parser file.

1. Create a folder with the name FluentBitDockerImage.

2. Within the folder, create a parser file that contains the rules to parse the log and concatenate lines that belong in the same message.

   1. Paste the following contents in the parser file:

      [MULTILINE_PARSER]
          name          multiline-regex-test
          type          regex
          flush_timeout 1000
          #
          # Regex rules for multiline parsing
          # ---------------------------------
          #
          # configuration hints:
          #
          #  - first state always has the name: start_state
          #  - every field in the rule must be inside double quotes
          #
          # rules |   state name  | regex pattern                  | next state
          # ------|---------------|--------------------------------------------
          rule      "start_state"   "/(Dec \d+ \d+\:\d+\:\d+)(.*)/"  "cont"
          rule      "cont"          "/^\s+at.*/"                     "cont"

      As you customize your regex pattern, we recommend you use a regular expression editor to test the expression.

   2. Save the file as parsers_multiline.conf.

3. Within the FluentBitDockerImage folder, create a custom configuration file that references the parser file that you created in the previous step.

   For more information about the custom configuration file, see Specifying a custom configuration file in the Amazon Elastic Container Service Developer Guide

   1. Paste the following contents in the file:

      [SERVICE]
          flush                 1
          log_level             info
          parsers_file          /parsers_multiline.conf
          
      [FILTER]
          name                  multiline
          match                 *
          multiline.key_content log
          multiline.parser      multiline-regex-test

      Note

      You must use the absolute path of the parser.

   2. Save the file as extra.conf.

4. Within the FluentBitDockerImage folder, create the Dockerfile with the Fluent Bit image and the parser and configuration files that you created.

   1. Paste the following contents in the file:

      FROM public.ecr.aws/aws-observability/aws-for-fluent-bit:latest
      
      ADD parsers_multiline.conf /parsers_multiline.conf
      ADD extra.conf /extra.conf

   2. Save the file as Dockerfile.

5. Using the Dockerfile, build a custom Fluent Bit image with the parser and custom configuration files included.

   Note

   You can place the parser file and configuration file anywhere in the Docker image except /fluent-bit/etc/fluent-bit.conf as this file path is used by FireLens.

   1. Build the image: docker build -t fluent-bit-multiline-image .

      Where: fluent-bit-multiline-image is the name for the image in this example.

   2. Verify that the image was created correctly: docker images —filter reference=fluent-bit-multiline-image

      If successful, the output shows the image and the latest tag.

6. Upload the custom Fluent Bit image to Amazon Elastic Container Registry.

   1. Create an Amazon ECR repository to store the image: aws ecr create-repository --repository-name fluent-bit-multiline-repo --region us-east-1

      Where: fluent-bit-multiline-repo is the name for the repository and us-east-1 is the region in this example.

      The output gives you the details of the new repository.

   2. Tag your image with the repositoryUri value from the previous output: docker tag fluent-bit-multiline-image repositoryUri

      Example: docker tag fluent-bit-multiline-image xxxxxxxxxxxx.dkr.ecr.us-east-1.amazonaws.com/fluent-bit-multiline-repo

   3. Run the docker image to verify it ran correctly: docker images —filter reference=repositoryUri

      In the output, the repository name changes from fluent-bit-multiline-repo to the repositoryUri.

   4. Authenticate to Amazon ECR by running the aws ecr get-login-password command and specifying the registry ID you want to authenticate to: aws ecr get-login-password | docker login --username AWS --password-stdin registry ID.dkr.ecr.region.amazonaws.com

      Example: ecr get-login-password | docker login --username AWS --password-stdin xxxxxxxxxxxx.dkr.ecr.us-east-1.amazonaws.com

      A successful login message appears.

   5. Push the image to Amazon ECR: docker push registry ID.dkr.ecr.region.amazonaws.com/repository name

      Example: docker push xxxxxxxxxxxx.dkr.ecr.us-east-1.amazonaws.com/fluent-bit-multiline-repo

Build and upload the image for a demo multiline application

This image will include a Python script file that runs the application and a sample log file.

When you run the task, the application simulates runs, then fails and creates a stack trace.

1. Create a folder named multiline-app: mkdir multiline-app

2. Create a Python script file.

   1. Within the multiline-app folder, create a file and name it main.py.

   2. Paste the following contents in the file:

      import os
      import time
      file1 = open('/test.log', 'r')
      Lines = file1.readlines()
       
      count = 0
      
      for i in range(10):
          print("app running normally...")
          time.sleep(1)
      
      # Strips the newline character
      for line in Lines:
          count += 1
          print(line.rstrip())
      print(count)
      print("app terminated.")
                                      

   3. Save the main.py file.

3. Create a sample log file.

   1. Within the multiline-app folder, create a file and name it test.log.

   2. Paste the following contents in the file:

      single line...
      Dec 14 06:41:08 Exception in thread "main" java.lang.RuntimeException: Something has gone wrong, aborting!
          at com.myproject.module.MyProject.badMethod(MyProject.java:22)
          at com.myproject.module.MyProject.oneMoreMethod(MyProject.java:18)
          at com.myproject.module.MyProject.anotherMethod(MyProject.java:14)
          at com.myproject.module.MyProject.someMethod(MyProject.java:10)
          at com.myproject.module.MyProject.main(MyProject.java:6)
      another line...
      

   3. Save the test.log file.

4. Within the multiline-app folder, create the Dockerfile.

   1. Paste the following contents in the file:

      FROM public.ecr.aws/amazonlinux/amazonlinux:latest
      ADD test.log /test.log
      
      RUN yum upgrade -y && yum install -y python3
      
      WORKDIR /usr/local/bin
      
      COPY main.py .
      
      CMD ["python3", "main.py"]

   2. Save the Dockerfile file.

5. Using the Dockerfile, build an image.

   1. Build the image: docker build -t multiline-app-image .

      Where: multiline-app-image is the name for the image in this example.

   2. Verify that the image was created correctly: docker images —filter reference=multiline-app-image

      If successful, the output shows the image and the latest tag.

6. Upload the image to Amazon Elastic Container Registry.

   1. Create an Amazon ECR repository to store the image: aws ecr create-repository --repository-name multiline-app-repo --region us-east-1

      Where: multiline-app-repo is the name for the repository and us-east-1 is the region in this example.

      The output gives you the details of the new repository. Note the repositoryUri value as you will need it in the next steps.

   2. Tag your image with the repositoryUri value from the previous output: docker tag multiline-app-image repositoryUri

      Example: docker tag multiline-app-image xxxxxxxxxxxx.dkr.ecr.us-east-1.amazonaws.com/multiline-app-repo

   3. Run the docker image to verify it ran correctly: docker images —filter reference=repositoryUri

      In the output, the repository name changes from multiline-app-repo to the repositoryUri value.

   4. Push the image to Amazon ECR: docker push aws_account_id.dkr.ecr.region.amazonaws.com/repository name

      Example: docker push xxxxxxxxxxxx.dkr.ecr.us-east-1.amazonaws.com/multiline-app-repo

Create the task definition and run the task

1. Create a task definition file with the file name multiline-task-definition.json.

2. Paste the following contents in the multiline-task-definition.json file:

   {
       "family": "firelens-example-multiline",
       "taskRoleArn": "task role ARN,
       "executionRoleArn": "execution role ARN",
       "containerDefinitions": [
           {
               "essential": true,
               "image": "aws_account_id.dkr.ecr.us-east-1.amazonaws.com/fluent-bit-multiline-image:latest",
               "name": "log_router",
               "firelensConfiguration": {
                   "type": "fluentbit",
                   "options": {
                       "config-file-type": "file",
                       "config-file-value": "/extra.conf"
                   }
               },
               "memoryReservation": 50
           },
           {
               "essential": true,
               "image": "aws_account_id.dkr.ecr.us-east-1.amazonaws.com/multiline-app-image:latest",
               "name": "app",
               "logConfiguration": {
                   "logDriver": "awsfirelens",
                   "options": {
                       "Name": "cloudwatch_logs",
                       "region": "us-east-1",
                       "log_group_name": "multiline-test/application",
                       "auto_create_group": "true",
                       "log_stream_prefix": "multiline-"
                   }
               },
               "memoryReservation": 100
           }
       ],
       "requiresCompatibilities": ["FARGATE"],
       "networkMode": "awsvpc",
       "cpu": "256",
       "memory": "512"
   }

   Replace the following in the multiline-task-definition.json task definition:

   1. task role ARN

      To find the task role ARN, go to the IAM console. Choose Roles and find the ecs-task-role-for-firelens task role that you created. Choose the role and copy the ARN that appears in the Summary section.

   2. execution role ARN

      To find the execution role ARN, go to the IAM console. Choose Roles and find the ecsTaskExecutionRole role. Choose the role and copy the ARN that appears in the Summary section.

   3. aws_account_id

      To find your aws_account_id, log into the AWS Management Console. Choose your user name on the top right and copy your Account ID.

   4. us-east-1

      Replace the region if necessary.

3. Register the task definition file: aws ecs register-task-definition --cli-input-json file://multiline-task-definition.json --region region

4. Open the console at https://console.aws.amazon.com/ecs/v2.

5. In the navigation pane, choose Task Definitions and then choose the firelens-example-multiline family because we registered the task definition to this family in the first line of the task definition above.

6. Choose the latest version.

7. Choose the Deploy, Run task.

8. On the Run Task page, For Cluster, choose the cluster, and then under Networking, for Subnets, choose the available subnets for your task.

9. Choose Create.

Verify that multiline log messages in Amazon CloudWatch appear concatenated

1. Open the CloudWatch console at https://console.aws.amazon.com/cloudwatch/.

2. From the navigation pane, expand Logs and choose Log groups.

3. Choose the multiline-test/applicatio log group.

4. Choose the log. View messages. Lines that matched the rules in the parser file are concatenated and appear as a single message.

   The following log snippet shows lines concatenated in a single Java stack trace event:

   {
       "container_id": "xxxxxx",
       "container_name": "app",
       "source": "stdout",
       "log": "Dec 14 06:41:08 Exception in thread \"main\" java.lang.RuntimeException: Something has gone wrong, aborting!\n    at com.myproject.module.MyProject.badMethod(MyProject.java:22)\n    at com.myproject.module.MyProject.oneMoreMethod(MyProject.java:18)\n    at com.myproject.module.MyProject.anotherMethod(MyProject.java:14)\n    at com.myproject.module.MyProject.someMethod(MyProject.java:10)\n    at com.myproject.module.MyProject.main(MyProject.java:6)",
       "ecs_cluster": "default",
       "ecs_task_arn": "arn:aws:ecs:us-east-1:xxxxxxxxxxxx:task/default/xxxxxx",
       "ecs_task_definition": "firelens-example-multiline:2"
   }

   The following log snippet shows how the same message appears with just a single line if you run an Amazon ECS container that is not configured to concatenate multiline log messages.

   {
       "log": "Dec 14 06:41:08 Exception in thread \"main\" java.lang.RuntimeException: Something has gone wrong, aborting!",
       "container_id": "xxxxxx-xxxxxx",
       "container_name": "app",
       "source": "stdout",
       "ecs_cluster": "default",
       "ecs_task_arn": "arn:aws:ecs:us-east-1:xxxxxxxxxxxx:task/default/xxxxxx",
       "ecs_task_definition": "firelens-example-multiline:3"
   }

Build and upload the image for a Fluent Bit container

This image will include a configuration file that references the Fluent Bit parser.

1. Create a folder with the name FluentBitDockerImage.

2. Within the FluentBitDockerImage folder, create a custom configuration file that references the Fluent Bit built-in parser file.

   For more information about the custom configuration file, see Specifying a custom configuration file in the Amazon Elastic Container Service Developer Guide

   1. Paste the following contents in the file:

      [FILTER]
          name                  multiline
          match                 *
          multiline.key_content log
          multiline.parser      go

   2. Save the file as extra.conf.

3. Within the FluentBitDockerImage folder, create the Dockerfile with the Fluent Bit image and the parser and configuration files that you created.

   1. Paste the following contents in the file:

      FROM public.ecr.aws/aws-observability/aws-for-fluent-bit:latest
      ADD extra.conf /extra.conf

   2. Save the file as Dockerfile.

4. Using the Dockerfile, build a custom Fluent Bit image with the custom configuration file included.

   Note

   You can place the configuration file anywhere in the Docker image except /fluent-bit/etc/fluent-bit.conf as this file path is used by FireLens.

   1. Build the image: docker build -t fluent-bit-multiline-image .

      Where: fluent-bit-multiline-image is the name for the image in this example.

   2. Verify that the image was created correctly: docker images —filter reference=fluent-bit-multiline-image

      If successful, the output shows the image and the latest tag.

5. Upload the custom Fluent Bit image to Amazon Elastic Container Registry.

   1. Create an Amazon ECR repository to store the image: aws ecr create-repository --repository-name fluent-bit-multiline-repo --region us-east-1

      Where: fluent-bit-multiline-repo is the name for the repository and us-east-1 is the region in this example.

      The output gives you the details of the new repository.

   2. Tag your image with the repositoryUri value from the previous output: docker tag fluent-bit-multiline-image repositoryUri

      Example: docker tag fluent-bit-multiline-image xxxxxxxxxxxx.dkr.ecr.us-east-1.amazonaws.com/fluent-bit-multiline-repo

   3. Run the docker image to verify it ran correctly: docker images —filter reference=repositoryUri

      In the output, the repository name changes from fluent-bit-multiline-repo to the repositoryUri.

   4. Authenticate to Amazon ECR by running the aws ecr get-login-password command and specifying the registry ID you want to authenticate to: aws ecr get-login-password | docker login --username AWS --password-stdin registry ID.dkr.ecr.region.amazonaws.com

      Example: ecr get-login-password | docker login --username AWS --password-stdin xxxxxxxxxxxx.dkr.ecr.us-east-1.amazonaws.com

      A successful login message appears.

   5. Push the image to Amazon ECR: docker push registry ID.dkr.ecr.region.amazonaws.com/repository name

      Example: docker push xxxxxxxxxxxx.dkr.ecr.us-east-1.amazonaws.com/fluent-bit-multiline-repo

Build and upload the image for a demo multiline application

This image will include a Python script file that runs the application and a sample log file.

1. Create a folder named multiline-app: mkdir multiline-app

2. Create a Python script file.

   1. Within the multiline-app folder, create a file and name it main.py.

   2. Paste the following contents in the file:

      import os
      import time
      file1 = open('/test.log', 'r')
      Lines = file1.readlines()
       
      count = 0
      
      for i in range(10):
          print("app running normally...")
          time.sleep(1)
      
      # Strips the newline character
      for line in Lines:
          count += 1
          print(line.rstrip())
      print(count)
      print("app terminated.")
                                      

   3. Save the main.py file.

3. Create a sample log file.

   1. Within the multiline-app folder, create a file and name it test.log.

   2. Paste the following contents in the file:

      panic: my panic
      
      goroutine 4 [running]:
      panic(0x45cb40, 0x47ad70)
        /usr/local/go/src/runtime/panic.go:542 +0x46c fp=0xc42003f7b8 sp=0xc42003f710 pc=0x422f7c
      main.main.func1(0xc420024120)
        foo.go:6 +0x39 fp=0xc42003f7d8 sp=0xc42003f7b8 pc=0x451339
      runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:2337 +0x1 fp=0xc42003f7e0 sp=0xc42003f7d8 pc=0x44b4d1
      created by main.main
        foo.go:5 +0x58
      
      goroutine 1 [chan receive]:
      runtime.gopark(0x4739b8, 0xc420024178, 0x46fcd7, 0xc, 0xc420028e17, 0x3)
        /usr/local/go/src/runtime/proc.go:280 +0x12c fp=0xc420053e30 sp=0xc420053e00 pc=0x42503c
      runtime.goparkunlock(0xc420024178, 0x46fcd7, 0xc, 0x1000f010040c217, 0x3)
        /usr/local/go/src/runtime/proc.go:286 +0x5e fp=0xc420053e70 sp=0xc420053e30 pc=0x42512e
      runtime.chanrecv(0xc420024120, 0x0, 0xc420053f01, 0x4512d8)
        /usr/local/go/src/runtime/chan.go:506 +0x304 fp=0xc420053f20 sp=0xc420053e70 pc=0x4046b4
      runtime.chanrecv1(0xc420024120, 0x0)
        /usr/local/go/src/runtime/chan.go:388 +0x2b fp=0xc420053f50 sp=0xc420053f20 pc=0x40439b
      main.main()
        foo.go:9 +0x6f fp=0xc420053f80 sp=0xc420053f50 pc=0x4512ef
      runtime.main()
        /usr/local/go/src/runtime/proc.go:185 +0x20d fp=0xc420053fe0 sp=0xc420053f80 pc=0x424bad
      runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:2337 +0x1 fp=0xc420053fe8 sp=0xc420053fe0 pc=0x44b4d1
      
      goroutine 2 [force gc (idle)]:
      runtime.gopark(0x4739b8, 0x4ad720, 0x47001e, 0xf, 0x14, 0x1)
        /usr/local/go/src/runtime/proc.go:280 +0x12c fp=0xc42003e768 sp=0xc42003e738 pc=0x42503c
      runtime.goparkunlock(0x4ad720, 0x47001e, 0xf, 0xc420000114, 0x1)
        /usr/local/go/src/runtime/proc.go:286 +0x5e fp=0xc42003e7a8 sp=0xc42003e768 pc=0x42512e
      runtime.forcegchelper()
        /usr/local/go/src/runtime/proc.go:238 +0xcc fp=0xc42003e7e0 sp=0xc42003e7a8 pc=0x424e5c
      runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:2337 +0x1 fp=0xc42003e7e8 sp=0xc42003e7e0 pc=0x44b4d1
      created by runtime.init.4
        /usr/local/go/src/runtime/proc.go:227 +0x35
      
      goroutine 3 [GC sweep wait]:
      runtime.gopark(0x4739b8, 0x4ad7e0, 0x46fdd2, 0xd, 0x419914, 0x1)
        /usr/local/go/src/runtime/proc.go:280 +0x12c fp=0xc42003ef60 sp=0xc42003ef30 pc=0x42503c
      runtime.goparkunlock(0x4ad7e0, 0x46fdd2, 0xd, 0x14, 0x1)
        /usr/local/go/src/runtime/proc.go:286 +0x5e fp=0xc42003efa0 sp=0xc42003ef60 pc=0x42512e
      runtime.bgsweep(0xc42001e150)
        /usr/local/go/src/runtime/mgcsweep.go:52 +0xa3 fp=0xc42003efd8 sp=0xc42003efa0 pc=0x419973
      runtime.goexit()
        /usr/local/go/src/runtime/asm_amd64.s:2337 +0x1 fp=0xc42003efe0 sp=0xc42003efd8 pc=0x44b4d1
      created by runtime.gcenable
        /usr/local/go/src/runtime/mgc.go:216 +0x58
      one more line, no multiline

   3. Save the test.log file.

4. Within the multiline-app folder, create the Dockerfile.

   1. Paste the following contents in the file:

      FROM public.ecr.aws/amazonlinux/amazonlinux:latest
      ADD test.log /test.log
      
      RUN yum upgrade -y && yum install -y python3
      
      WORKDIR /usr/local/bin
      
      COPY main.py .
      
      CMD ["python3", "main.py"]

   2. Save the Dockerfile file.

5. Using the Dockerfile, build an image.

   1. Build the image: docker build -t multiline-app-image .

      Where: multiline-app-image is the name for the image in this example.

   2. Verify that the image was created correctly: docker images —filter reference=multiline-app-image

      If successful, the output shows the image and the latest tag.

6. Upload the image to Amazon Elastic Container Registry.

   1. Create an Amazon ECR repository to store the image: aws ecr create-repository --repository-name multiline-app-repo --region us-east-1

      Where: multiline-app-repo is the name for the repository and us-east-1 is the region in this example.

      The output gives you the details of the new repository. Note the repositoryUri value as you will need it in the next steps.

   2. Tag your image with the repositoryUri value from the previous output: docker tag multiline-app-image repositoryUri

      Example: docker tag multiline-app-image xxxxxxxxxxxx.dkr.ecr.us-east-1.amazonaws.com/multiline-app-repo

   3. Run the docker image to verify it ran correctly: docker images —filter reference=repositoryUri

      In the output, the repository name changes from multiline-app-repo to the repositoryUri value.

   4. Push the image to Amazon ECR: docker push aws_account_id.dkr.ecr.region.amazonaws.com/repository name

      Example: docker push xxxxxxxxxxxx.dkr.ecr.us-east-1.amazonaws.com/multiline-app-repo

Create the task definition and run the task

1. Create a task definition file with the file name multiline-task-definition.json.

2. Paste the following contents in the multiline-task-definition.json file:

   {
       "family": "firelens-example-multiline",
       "taskRoleArn": "task role ARN,
       "executionRoleArn": "execution role ARN",
       "containerDefinitions": [
           {
               "essential": true,
               "image": "aws_account_id.dkr.ecr.us-east-1.amazonaws.com/fluent-bit-multiline-image:latest",
               "name": "log_router",
               "firelensConfiguration": {
                   "type": "fluentbit",
                   "options": {
                       "config-file-type": "file",
                       "config-file-value": "/extra.conf"
                   }
               },
               "memoryReservation": 50
           },
           {
               "essential": true,
               "image": "aws_account_id.dkr.ecr.us-east-1.amazonaws.com/multiline-app-image:latest",
               "name": "app",
               "logConfiguration": {
                   "logDriver": "awsfirelens",
                   "options": {
                       "Name": "cloudwatch_logs",
                       "region": "us-east-1",
                       "log_group_name": "multiline-test/application",
                       "auto_create_group": "true",
                       "log_stream_prefix": "multiline-"
                   }
               },
               "memoryReservation": 100
           }
       ],
       "requiresCompatibilities": ["FARGATE"],
       "networkMode": "awsvpc",
       "cpu": "256",
       "memory": "512"
   }

   Replace the following in the multiline-task-definition.json task definition:

   1. task role ARN

      To find the task role ARN, go to the IAM console. Choose Roles and find the ecs-task-role-for-firelens task role that you created. Choose the role and copy the ARN that appears in the Summary section.

   2. execution role ARN

      To find the execution role ARN, go to the IAM console. Choose Roles and find the ecsTaskExecutionRole role. Choose the role and copy the ARN that appears in the Summary section.

   3. aws_account_id

      To find your aws_account_id, log into the AWS Management Console. Choose your user name on the top right and copy your Account ID.

   4. us-east-1

      Replace the region if necessary.

3. Register the task definition file: aws ecs register-task-definition --cli-input-json file://multiline-task-definition.json --region us-east-1

4. Open the console at https://console.aws.amazon.com/ecs/v2.

5. In the navigation pane, choose Task Definitions and then choose the firelens-example-multiline family because we registered the task definition to this family in the first line of the task definition above.

6. Choose the latest version.

7. Choose the Deploy, Run task.

8. On the Run Task page, For Cluster, choose the cluster, and then under Networking, for Subnets, choose the available subnets for your task.

9. Choose Create.

Verify that multiline log messages in Amazon CloudWatch appear concatenated

1. Open the CloudWatch console at https://console.aws.amazon.com/cloudwatch/.

2. From the navigation pane, expand Logs and choose Log groups.

3. Choose the multiline-test/applicatio log group.

4. Choose the log and view the messages. Lines that matched the rules in the parser file are concatenated and appear as a single message.

   The following log snippet shows a Go stack trace that is concatenated into a single event:

   {
       "log": "panic: my panic\n\ngoroutine 4 [running]:\npanic(0x45cb40, 0x47ad70)\n  /usr/local/go/src/runtime/panic.go:542 +0x46c fp=0xc42003f7b8 sp=0xc42003f710 pc=0x422f7c\nmain.main.func1(0xc420024120)\n  foo.go:6 +0x39 fp=0xc42003f7d8 sp=0xc42003f7b8 pc=0x451339\nruntime.goexit()\n  /usr/local/go/src/runtime/asm_amd64.s:2337 +0x1 fp=0xc42003f7e0 sp=0xc42003f7d8 pc=0x44b4d1\ncreated by main.main\n  foo.go:5 +0x58\n\ngoroutine 1 [chan receive]:\nruntime.gopark(0x4739b8, 0xc420024178, 0x46fcd7, 0xc, 0xc420028e17, 0x3)\n  /usr/local/go/src/runtime/proc.go:280 +0x12c fp=0xc420053e30 sp=0xc420053e00 pc=0x42503c\nruntime.goparkunlock(0xc420024178, 0x46fcd7, 0xc, 0x1000f010040c217, 0x3)\n  /usr/local/go/src/runtime/proc.go:286 +0x5e fp=0xc420053e70 sp=0xc420053e30 pc=0x42512e\nruntime.chanrecv(0xc420024120, 0x0, 0xc420053f01, 0x4512d8)\n  /usr/local/go/src/runtime/chan.go:506 +0x304 fp=0xc420053f20 sp=0xc420053e70 pc=0x4046b4\nruntime.chanrecv1(0xc420024120, 0x0)\n  /usr/local/go/src/runtime/chan.go:388 +0x2b fp=0xc420053f50 sp=0xc420053f20 pc=0x40439b\nmain.main()\n  foo.go:9 +0x6f fp=0xc420053f80 sp=0xc420053f50 pc=0x4512ef\nruntime.main()\n  /usr/local/go/src/runtime/proc.go:185 +0x20d fp=0xc420053fe0 sp=0xc420053f80 pc=0x424bad\nruntime.goexit()\n  /usr/local/go/src/runtime/asm_amd64.s:2337 +0x1 fp=0xc420053fe8 sp=0xc420053fe0 pc=0x44b4d1\n\ngoroutine 2 [force gc (idle)]:\nruntime.gopark(0x4739b8, 0x4ad720, 0x47001e, 0xf, 0x14, 0x1)\n  /usr/local/go/src/runtime/proc.go:280 +0x12c fp=0xc42003e768 sp=0xc42003e738 pc=0x42503c\nruntime.goparkunlock(0x4ad720, 0x47001e, 0xf, 0xc420000114, 0x1)\n  /usr/local/go/src/runtime/proc.go:286 +0x5e fp=0xc42003e7a8 sp=0xc42003e768 pc=0x42512e\nruntime.forcegchelper()\n  /usr/local/go/src/runtime/proc.go:238 +0xcc fp=0xc42003e7e0 sp=0xc42003e7a8 pc=0x424e5c\nruntime.goexit()\n  /usr/local/go/src/runtime/asm_amd64.s:2337 +0x1 fp=0xc42003e7e8 sp=0xc42003e7e0 pc=0x44b4d1\ncreated by runtime.init.4\n  /usr/local/go/src/runtime/proc.go:227 +0x35\n\ngoroutine 3 [GC sweep wait]:\nruntime.gopark(0x4739b8, 0x4ad7e0, 0x46fdd2, 0xd, 0x419914, 0x1)\n  /usr/local/go/src/runtime/proc.go:280 +0x12c fp=0xc42003ef60 sp=0xc42003ef30 pc=0x42503c\nruntime.goparkunlock(0x4ad7e0, 0x46fdd2, 0xd, 0x14, 0x1)\n  /usr/local/go/src/runtime/proc.go:286 +0x5e fp=0xc42003efa0 sp=0xc42003ef60 pc=0x42512e\nruntime.bgsweep(0xc42001e150)\n  /usr/local/go/src/runtime/mgcsweep.go:52 +0xa3 fp=0xc42003efd8 sp=0xc42003efa0 pc=0x419973\nruntime.goexit()\n  /usr/local/go/src/runtime/asm_amd64.s:2337 +0x1 fp=0xc42003efe0 sp=0xc42003efd8 pc=0x44b4d1\ncreated by runtime.gcenable\n  /usr/local/go/src/runtime/mgc.go:216 +0x58",
       "container_id": "xxxxxx-xxxxxx",
       "container_name": "app",
       "source": "stdout",
       "ecs_cluster": "default",
       "ecs_task_arn": "arn:aws:ecs:us-east-1:xxxxxxxxxxxx:task/default/xxxxxx",
       "ecs_task_definition": "firelens-example-multiline:2"
   }

   The following log snippet shows how the same event appears if you run an ECS container that is not configured to concatenate multiline log messages. The log field contains a single line.

   {
       "log": "panic: my panic",
       "container_id": "xxxxxx-xxxxxx",
       "container_name": "app",
       "source": "stdout",
       "ecs_cluster": "default",
       "ecs_task_arn": "arn:aws:ecs:us-east-1:xxxxxxxxxxxx:task/default/xxxxxx",
       "ecs_task_definition": "firelens-example-multiline:3"