Log manager - AWS IoT Greengrass
VersionsTypeOperating systemRequirementsDependenciesConfigurationUsageLocal log fileChangelog

Log manager

Warning

We recommend upgrading to Log Manager v2.3.5 or later. Version 2.3.5 optimizes Log Manager configuration writes, reducing IO operations and improving log upload speed, overall device performance and possibly extending device life.

The log manager component (aws.greengrass.LogManager) uploads logs from AWS IoT Greengrass core devices to Amazon CloudWatch Logs. You can upload logs from the Greengrass nucleus, other Greengrass components, and other applications and services that aren't Greengrass components. For more information about how to monitor logs in CloudWatch Logs and on the local file system, see Monitor AWS IoT Greengrass logs.

The following considerations apply when you use the log manager component to write to CloudWatch Logs:

  • Log delays

    The log manager component version 2.2.8 (and earlier) processes and uploads logs from only rotated log files. By default, the AWS IoT Greengrass Core software rotates log files every hour or after they are 1,024 KB. As a result, the log manager component uploads logs only after the AWS IoT Greengrass Core software or a Greengrass component writes over 1,024 KB worth of logs. You can configure a lower log file size limit to cause log files to rotate more often. This causes the log manager component to upload logs to CloudWatch Logs more frequently.

    The log manager component version 2.3.0 (and later) processes and uploads all logs. When you write a new log, log manager version 2.3.0 (and later) processes and directly uploads that active log file instead of waiting for it to be rotated. This means that you can view the new log in 5 minutes or less.

    The log manager component uploads new logs periodically. By default, the log manager component uploads new logs every 5 minutes. You can configure a lower upload interval, so the log manager component uploads logs to CloudWatch Logs more frequently by configuring the periodicUploadIntervalSec. For more information about how to configure this periodic interval, see Configuration.

    Logs can be uploaded in near real-time from the same Greengrass file system. If you need to observe logs in real time, consider using file system logs.

    Note

    If you're using different file systems to write logs to, log manager reverts back to the behavior in log manager component versions 2.2.8 and earlier. For information about accessing file system logs, see Access file system logs.

  • Clock skew

    The log manager component uses the standard Signature Version 4 signing process to create API requests to CloudWatch Logs. If the system time on a core device is out of sync by more than 15 minutes, then CloudWatch Logs rejects the requests. For more information, see Signature Version 4 signing process in the AWS General Reference.

For information about the log groups and log streams to which this component uploads logs, see Usage.

Versions

This component has the following versions:

  • 2.3.x

  • 2.2.x

  • 2.1.x

  • 2.0.x

Type

This component is a plugin component (aws.greengrass.plugin). The Greengrass nucleus runs this component in the same Java Virtual Machine (JVM) as the nucleus. The nucleus restarts when you change this component's version on the core device.

This component uses the same log file as the Greengrass nucleus. For more information, see Monitor AWS IoT Greengrass logs.

For more information, see Component types.

Operating system

This component can be installed on core devices that run the following operating systems:

  • Linux

  • Windows

Requirements

This component has the following requirements:

  • The Greengrass device role must allow the logs:CreateLogGroup, logs:CreateLogStream, logs:PutLogEvents, and logs:DescribeLogStreams actions, as shown in the following example IAM policy.

    {
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": [
        "logs:CreateLogGroup",
        "logs:CreateLogStream",
        "logs:PutLogEvents",
        "logs:DescribeLogStreams"
      ],
      "Effect": "Allow",
      "Resource": "arn:aws:logs:*:*:*"
    }
  ]
}
    Note

    The Greengrass device role that you create when you install the AWS IoT Greengrass Core software includes the permissions in this example policy by default.

    For more information, see Using identity-based policies (IAM policies) for CloudWatch Logs in the Amazon CloudWatch Logs User Guide.

  • The log manager component is supported to run in a VPC. To deploy this component in a VPC, the following is required.

    • The log manager component must have connectivity to logs.region.amazonaws.com which has the VPC endpoint of com.amazonaws.us-east-1.logs.

Endpoints and ports

This component must be able to perform outbound requests to the following endpoints and ports, in addition to endpoints and ports required for basic operation. For more information, see Allow device traffic through a proxy or firewall.

Endpoint Port Required Description

logs.region.amazonaws.com

 443 No

Required if you write logs to CloudWatch Logs.

Dependencies

When you deploy a component, AWS IoT Greengrass also deploys compatible versions of its dependencies. This means that you must meet the requirements for the component and all of its dependencies to successfully deploy the component. This section lists the dependencies for the released versions of this component and the semantic version constraints that define the component versions for each dependency. You can also view the dependencies for each version of the component in the AWS IoT Greengrass console. On the component details page, look for the Dependencies list.

2.3.8

The following table lists the dependencies for version 2.3.8 of this component.

Dependency Compatible versions Dependency type
Greengrass nucleus >=2.1.0 <2.14.0 Soft
2.3.7

The following table lists the dependencies for version 2.3.7 of this component.

Dependency Compatible versions Dependency type
Greengrass nucleus >=2.1.0 <2.13.0 Soft
2.3.5 and 2.3.6

The following table lists the dependencies for versions 2.3.5 and 2.3.6 of this component.

Dependency Compatible versions Dependency type
Greengrass nucleus >=2.1.0 <2.12.0 Soft
2.3.3 – 2.3.4

The following table lists the dependencies for versions 2.3.3 to 2.3.4 of this component.

Dependency Compatible versions Dependency type
Greengrass nucleus >=2.1.0 <2.11.0 Soft
2.2.8 – 2.3.2

The following table lists the dependencies for versions 2.2.8 to 2.3.2 of this component.

Dependency Compatible versions Dependency type
Greengrass nucleus >=2.1.0 <2.10.0 Soft
2.2.7

The following table lists the dependencies for version 2.2.7 of this component.

Dependency Compatible versions Dependency type
Greengrass nucleus >=2.1.0 <2.9.0 Soft
2.2.6

The following table lists the dependencies for version 2.2.6 of this component.

Dependency Compatible versions Dependency type
Greengrass nucleus >=2.1.0 <2.8.0 Soft
2.2.5

The following table lists the dependencies for version 2.2.5 of this component.

Dependency Compatible versions Dependency type
Greengrass nucleus >=2.1.0 <2.7.0 Soft
2.2.1 - 2.2.4

The following table lists the dependencies for versions 2.2.1 - 2.2.4 of this component.

Dependency Compatible versions Dependency type
Greengrass nucleus >=2.1.0 <2.6.0 Soft
2.1.3 and 2.2.0

The following table lists the dependencies for versions 2.1.3 and 2.2.0 of this component.

Dependency Compatible versions Dependency type
Greengrass nucleus >=2.1.0 <2.5.0 Soft
2.1.2

The following table lists the dependencies for version 2.1.2 of this component.

Dependency Compatible versions Dependency type
Greengrass nucleus >=2.1.0 <2.4.0 Soft
2.1.1

The following table lists the dependencies for version 2.1.1 of this component.

Dependency Compatible versions Dependency type
Greengrass nucleus >=2.1.0 <2.3.0 Soft
2.1.0

The following table lists the dependencies for version 2.1.0 of this component.

Dependency Compatible versions Dependency type
Greengrass nucleus >=2.1.0 <2.2.0 Soft
2.0.x

The following table lists the dependencies for version 2.0.x of this component.

Dependency Compatible versions Dependency type
Greengrass nucleus >=2.0.3 <2.1.0 Soft

For more information about component dependencies, see the component recipe reference.

Configuration

This component provides the following configuration parameters that you can customize when you deploy the component.

v2.3.6 – v2.3.7
logsUploaderConfiguration

(Optional) The configuration for logs that the log manager component uploads. This object contains the following information:

systemLogsConfiguration

(Optional) The configuration for AWS IoT Greengrass Core software system logs, which include logs from the Greengrass nucleus and plugin components. Specify this configuration to enable the log manager component to manage system logs. This object contains the following information:

uploadToCloudWatch

(Optional) You can upload system logs to CloudWatch Logs.

Default: false

minimumLogLevel

(Optional) The minimum level of log messages to upload. This minimum level applies only if you configure the Greengrass nucleus component to output JSON format logs. To enable JSON format logs, specify JSON for the logging format parameter (logging.format).

Choose from the following log levels, listed here in level order:

  • DEBUG

  • INFO

  • WARN

  • ERROR

Default: INFO

diskSpaceLimit

(Optional) The maximum total size of Greengrass system log files, in the unit you specify in diskSpaceLimitUnit. After the total size of Greengrass system log files exceeds this maximum total size, the AWS IoT Greengrass Core software deletes the oldest Greengrass system log files.

This parameter is equivalent to the log size limit parameter (totalLogsSizeKB) of the Greengrass nucleus component. The AWS IoT Greengrass Core software uses the minimum of the two values as the maximum total Greengrass system log size.

diskSpaceLimitUnit

(Optional) The unit for the diskSpaceLimit. Choose from the following options:

  • KB – kilobytes

  • MB – megabytes

  • GB – gigabytes

Default: KB

deleteLogFileAfterCloudUpload

(Optional) You can delete a log file after the log manager component uploads the logs to CloudWatch Logs.

Default: false

componentLogsConfigurationMap

(Optional) A map of log configurations for components on the core device. Each componentName object in this map defines the log configuration for the component or application. The log manager component uploads these component logs to CloudWatch Logs.

Important

We strongly recommend using a single configuration key per component. You should only target a group of files that have only one log file that's actively being written to when using the logFileRegex. Not following this recommendation may lead to duplicate logs getting uploaded to CloudWatch. If you are targeting multiple active log files with a single regex, we recommend you upgrade to log manager v2.3.1 or later and consider changing your configuration using the example configuration.

Note

If you're upgrading from a version of log manager earlier than v2.2.0, you can continue to use the componentLogsConfiguration list instead of componentLogsConfigurationMap. However, we strongly recommend that you use the map format so that you can use merge and reset updates to modify configurations for specific components. For information about the componentLogsConfiguration parameter, see the configuration parameters for v2.1.x of this component.

componentName

The log configuration for the componentName component or application for this log configuration. You can specify the name of a Greengrass component or another value to identify this log group.

Each object contains the following information:

minimumLogLevel

(Optional) The minimum level of log messages to upload. This minimum level applies only if this component's logs use a specific JSON format, which you can find in the AWS IoT Greengrass logging module repository on GitHub.

Choose from the following log levels, listed here in level order:

  • DEBUG

  • INFO

  • WARN

  • ERROR

Default: INFO

diskSpaceLimit

(Optional) The maximum total size of all log files for this component, in the unit you specify in diskSpaceLimitUnit. After the total size of this component's log files exceeds this maximum total size, the AWS IoT Greengrass Core software deletes this component's oldest log files.

This parameter is related to the log size limit parameter (totalLogsSizeKB) of the Greengrass nucleus component. The AWS IoT Greengrass Core software uses the minimum of the two values as the maximum total log size for this component.

diskSpaceLimitUnit

(Optional) The unit for the diskSpaceLimit. Choose from the following options:

  • KB – kilobytes

  • MB – megabytes

  • GB – gigabytes

Default: KB

logFileDirectoryPath

(Optional) The path to the folder that contains this component's log files.

You don't need to specify this parameter for Greengrass components that print to standard output (stdout) and standard error (stderr).

Default: /greengrass/v2/logs.

logFileRegex

(Optional) A regular expression that specifies the log file name format that the component or application uses. The log manager component uses this regular expression to identify log files in the folder at logFileDirectoryPath.

You don't need to specify this parameter for Greengrass components that print to standard output (stdout) and standard error (stderr).

If your component or application rotates log files, specify a regex that matches the rotated log file names. For example, you might specify hello_world\\\\w*.log to upload logs for a Hello World application. The \\\\w* pattern matches zero or more word characters, which includes alphanumeric characters and underscores. This regex matches log files with and without timestamps in their name. In this example, the log manager uploads the following log files:

  • hello_world.log – The most recent log file for the Hello World application.

  • hello_world_2020_12_15_17_0.log – An older log file for the Hello World application.

Default: componentName\\\\w*.log, where componentName is the name of the component for this log configuration.

deleteLogFileAfterCloudUpload

(Optional) You can delete a log file after the log manager component uploads the logs to CloudWatch Logs.

Default: false

multiLineStartPattern

(Optional) A regular expression that identifies when a log message on a new line is a new log message. If the regular expression doesn't match the new line, the log manager component appends the new line to the log message for the previous line.

By default, the log manager component checks if the line starts with a whitespace character, such as a tab or space. If it doesn't, the log manager handles that line as a new log message. Otherwise, it appends that line to the current log message. This behavior ensures that the log manager component doesn't split messages that span multiple lines, such as stack traces.

periodicUploadIntervalSec

(Optional) The period in seconds at which the log manager component checks for new log files to upload.

Default: 300 (5 minutes)

Minimum: 0.000001 (1 microsecond)

deprecatedVersionSupport

Indicates whether the log manager should use logging speed improvements introduced in log manager v2.3.5. Set the value to false to use the improvements.

If you set this value to false when you upgrade from log manager v2.3.1 or earlier duplicate log entries may be uploaded.

The default is true.

Example: Configuration merge update

The following example configuration specifies to upload system logs and com.example.HelloWorld component logs to CloudWatch Logs.

{
  "logsUploaderConfiguration": {
    "systemLogsConfiguration": {
      "uploadToCloudWatch": "true",
      "minimumLogLevel": "INFO",
      "diskSpaceLimit": "10",
      "diskSpaceLimitUnit": "MB",
      "deleteLogFileAfterCloudUpload": "false"
    },
    "componentLogsConfigurationMap": {
      "com.example.HelloWorld": {
        "minimumLogLevel": "INFO",
        "diskSpaceLimit": "20",
        "diskSpaceLimitUnit": "MB",
        "deleteLogFileAfterCloudUpload": "false"
      }
    }
  },
  "periodicUploadIntervalSec": "300",
  "deprecatedVersionSupport": "false"
}
Example: Configuration to upload multiple active log files using log manager v2.3.1

The following example configuration is the recommended example if you want to target multiple active log files. This example configuration specifies what active log files you want to upload to CloudWatch. Using this configuration example configuration will also upload any rotated files that match the logFileRegex. This example configuration is supported on log manager v2.3.1.

{
  "logsUploaderConfiguration": {
    "componentLogsConfigurationMap": {
      "com.example.A": {
        "logFileRegex": "com.example.A\\w*.log",
        "deleteLogFileAfterCloudUpload": "false"
      }
      "com.example.B": {
        "logFileRegex": "com.example.B\\w*.log",
        "deleteLogFileAfterCloudUpload": "false"
      }
    }
  },
  "periodicUploadIntervalSec": "10"
}
v2.3.x
logsUploaderConfiguration

(Optional) The configuration for logs that the log manager component uploads. This object contains the following information:

systemLogsConfiguration

(Optional) The configuration for AWS IoT Greengrass Core software system logs, which include logs from the Greengrass nucleus and plugin components. Specify this configuration to enable the log manager component to manage system logs. This object contains the following information:

uploadToCloudWatch

(Optional) You can upload system logs to CloudWatch Logs.

Default: false

minimumLogLevel

(Optional) The minimum level of log messages to upload. This minimum level applies only if you configure the Greengrass nucleus component to output JSON format logs. To enable JSON format logs, specify JSON for the logging format parameter (logging.format).

Choose from the following log levels, listed here in level order:

  • DEBUG

  • INFO

  • WARN

  • ERROR

Default: INFO

diskSpaceLimit

(Optional) The maximum total size of Greengrass system log files, in the unit you specify in diskSpaceLimitUnit. After the total size of Greengrass system log files exceeds this maximum total size, the AWS IoT Greengrass Core software deletes the oldest Greengrass system log files.

This parameter is equivalent to the log size limit parameter (totalLogsSizeKB) of the Greengrass nucleus component. The AWS IoT Greengrass Core software uses the minimum of the two values as the maximum total Greengrass system log size.

diskSpaceLimitUnit

(Optional) The unit for the diskSpaceLimit. Choose from the following options:

  • KB – kilobytes

  • MB – megabytes

  • GB – gigabytes

Default: KB

deleteLogFileAfterCloudUpload

(Optional) You can delete a log file after the log manager component uploads the logs to CloudWatch Logs.

Default: false

componentLogsConfigurationMap

(Optional) A map of log configurations for components on the core device. Each componentName object in this map defines the log configuration for the component or application. The log manager component uploads these component logs to CloudWatch Logs.

Important

We strongly recommend using a single configuration key per component. You should only target a group of files that have only one log file that's actively being written to when using the logFileRegex. Not following this recommendation may lead to duplicate logs getting uploaded to CloudWatch. If you are targeting multiple active log files with a single regex, we recommend you upgrade to log manager v2.3.1 and consider changing your configuration using the example configuration.

Note

If you're upgrading from a version of log manager earlier than v2.2.0, you can continue to use the componentLogsConfiguration list instead of componentLogsConfigurationMap. However, we strongly recommend that you use the map format so that you can use merge and reset updates to modify configurations for specific components. For information about the componentLogsConfiguration parameter, see the configuration parameters for v2.1.x of this component.

componentName

The log configuration for the componentName component or application for this log configuration. You can specify the name of a Greengrass component or another value to identify this log group.

Each object contains the following information:

minimumLogLevel

(Optional) The minimum level of log messages to upload. This minimum level applies only if this component's logs use a specific JSON format, which you can find in the AWS IoT Greengrass logging module repository on GitHub.

Choose from the following log levels, listed here in level order:

  • DEBUG

  • INFO

  • WARN

  • ERROR

Default: INFO

diskSpaceLimit

(Optional) The maximum total size of all log files for this component, in the unit you specify in diskSpaceLimitUnit. After the total size of this component's log files exceeds this maximum total size, the AWS IoT Greengrass Core software deletes this component's oldest log files.

This parameter is related to the log size limit parameter (totalLogsSizeKB) of the Greengrass nucleus component. The AWS IoT Greengrass Core software uses the minimum of the two values as the maximum total log size for this component.

diskSpaceLimitUnit

(Optional) The unit for the diskSpaceLimit. Choose from the following options:

  • KB – kilobytes

  • MB – megabytes

  • GB – gigabytes

Default: KB

logFileDirectoryPath

(Optional) The path to the folder that contains this component's log files.

You don't need to specify this parameter for Greengrass components that print to standard output (stdout) and standard error (stderr).

Default: /greengrass/v2/logs.

logFileRegex

(Optional) A regular expression that specifies the log file name format that the component or application uses. The log manager component uses this regular expression to identify log files in the folder at logFileDirectoryPath.

You don't need to specify this parameter for Greengrass components that print to standard output (stdout) and standard error (stderr).

If your component or application rotates log files, specify a regex that matches the rotated log file names. For example, you might specify hello_world\\\\w*.log to upload logs for a Hello World application. The \\\\w* pattern matches zero or more word characters, which includes alphanumeric characters and underscores. This regex matches log files with and without timestamps in their name. In this example, the log manager uploads the following log files:

  • hello_world.log – The most recent log file for the Hello World application.

  • hello_world_2020_12_15_17_0.log – An older log file for the Hello World application.

Default: componentName\\\\w*.log, where componentName is the name of the component for this log configuration.

deleteLogFileAfterCloudUpload

(Optional) You can delete a log file after the log manager component uploads the logs to CloudWatch Logs.

Default: false

multiLineStartPattern

(Optional) A regular expression that identifies when a log message on a new line is a new log message. If the regular expression doesn't match the new line, the log manager component appends the new line to the log message for the previous line.

By default, the log manager component checks if the line starts with a whitespace character, such as a tab or space. If it doesn't, the log manager handles that line as a new log message. Otherwise, it appends that line to the current log message. This behavior ensures that the log manager component doesn't split messages that span multiple lines, such as stack traces.

periodicUploadIntervalSec

(Optional) The period in seconds at which the log manager component checks for new log files to upload.

Default: 300 (5 minutes)

Minimum: 0.000001 (1 microsecond)

Example: Configuration merge update

The following example configuration specifies to upload system logs and com.example.HelloWorld component logs to CloudWatch Logs.

{
  "logsUploaderConfiguration": {
    "systemLogsConfiguration": {
      "uploadToCloudWatch": "true",
      "minimumLogLevel": "INFO",
      "diskSpaceLimit": "10",
      "diskSpaceLimitUnit": "MB",
      "deleteLogFileAfterCloudUpload": "false"
    },
    "componentLogsConfigurationMap": {
      "com.example.HelloWorld": {
        "minimumLogLevel": "INFO",
        "diskSpaceLimit": "20",
        "diskSpaceLimitUnit": "MB",
        "deleteLogFileAfterCloudUpload": "false"
      }
    }
  },
  "periodicUploadIntervalSec": "300"
}
Example: Configuration to upload multiple active log files using log manager v2.3.1

The following example configuration is the recommended example if you want to target multiple active log files. This example configuration specifies what active log files you want to upload to CloudWatch. Using this configuration example configuration will also upload any rotated files that match the logFileRegex. This example configuration is supported on log manager v2.3.1.

{
  "logsUploaderConfiguration": {
    "componentLogsConfigurationMap": {
      "com.example.A": {
        "logFileRegex": "com.example.A\\w*.log",
        "deleteLogFileAfterCloudUpload": "false"
      }
      "com.example.B": {
        "logFileRegex": "com.example.B\\w*.log",
        "deleteLogFileAfterCloudUpload": "false"
      }
    }
  },
  "periodicUploadIntervalSec": "10"
}
v2.2.x
logsUploaderConfiguration

(Optional) The configuration for logs that the log manager component uploads. This object contains the following information:

systemLogsConfiguration

(Optional) The configuration for AWS IoT Greengrass Core software system logs, which include logs from the Greengrass nucleus and plugin components. Specify this configuration to enable the log manager component to manage system logs. This object contains the following information:

uploadToCloudWatch

(Optional) You can upload system logs to CloudWatch Logs.

Default: false

minimumLogLevel

(Optional) The minimum level of log messages to upload. This minimum level applies only if you configure the Greengrass nucleus component to output JSON format logs. To enable JSON format logs, specify JSON for the logging format parameter (logging.format).

Choose from the following log levels, listed here in level order:

  • DEBUG

  • INFO

  • WARN

  • ERROR

Default: INFO

diskSpaceLimit

(Optional) The maximum total size of Greengrass system log files, in the unit you specify in diskSpaceLimitUnit. After the total size of Greengrass system log files exceeds this maximum total size, the AWS IoT Greengrass Core software deletes the oldest Greengrass system log files.

This parameter is equivalent to the log size limit parameter (totalLogsSizeKB) of the Greengrass nucleus component. The AWS IoT Greengrass Core software uses the minimum of the two values as the maximum total Greengrass system log size.

diskSpaceLimitUnit

(Optional) The unit for the diskSpaceLimit. Choose from the following options:

  • KB – kilobytes

  • MB – megabytes

  • GB – gigabytes

Default: KB

deleteLogFileAfterCloudUpload

(Optional) You can delete a log file after the log manager component uploads the logs to CloudWatch Logs.

Default: false

componentLogsConfigurationMap

(Optional) A map of log configurations for components on the core device. Each componentName object in this map defines the log configuration for the component or application. The log manager component uploads these component logs to CloudWatch Logs.

Note

If you're upgrading from a version of log manager earlier than v2.2.0, you can continue to use the componentLogsConfiguration list instead of componentLogsConfigurationMap. However, we strongly recommend that you use the map format so that you can use merge and reset updates to modify configurations for specific components. For information about the componentLogsConfiguration parameter, see the configuration parameters for v2.1.x of this component.

componentName

The log configuration for the componentName component or application for this log configuration. You can specify the name of a Greengrass component or another value to identify this log group.

Each object contains the following information:

minimumLogLevel

(Optional) The minimum level of log messages to upload. This minimum level applies only if this component's logs use a specific JSON format, which you can find in the AWS IoT Greengrass logging module repository on GitHub.

Choose from the following log levels, listed here in level order:

  • DEBUG

  • INFO

  • WARN

  • ERROR

Default: INFO

diskSpaceLimit

(Optional) The maximum total size of all log files for this component, in the unit you specify in diskSpaceLimitUnit. After the total size of this component's log files exceeds this maximum total size, the AWS IoT Greengrass Core software deletes this component's oldest log files.

This parameter is related to the log size limit parameter (totalLogsSizeKB) of the Greengrass nucleus component. The AWS IoT Greengrass Core software uses the minimum of the two values as the maximum total log size for this component.

diskSpaceLimitUnit

(Optional) The unit for the diskSpaceLimit. Choose from the following options:

  • KB – kilobytes

  • MB – megabytes

  • GB – gigabytes

Default: KB

logFileDirectoryPath

(Optional) The path to the folder that contains this component's log files.

You don't need to specify this parameter for Greengrass components that print to standard output (stdout) and standard error (stderr).

Default: /greengrass/v2/logs.

logFileRegex

(Optional) A regular expression that specifies the log file name format that the component or application uses. The log manager component uses this regular expression to identify log files in the folder at logFileDirectoryPath.

You don't need to specify this parameter for Greengrass components that print to standard output (stdout) and standard error (stderr).

If your component or application rotates log files, specify a regex that matches the rotated log file names. For example, you might specify hello_world\\\\w*.log to upload logs for a Hello World application. The \\\\w* pattern matches zero or more word characters, which includes alphanumeric characters and underscores. This regex matches log files with and without timestamps in their name. In this example, the log manager uploads the following log files:

  • hello_world.log – The most recent log file for the Hello World application.

  • hello_world_2020_12_15_17_0.log – An older log file for the Hello World application.

Default: componentName\\\\w*.log, where componentName is the name of the component for this log configuration.

deleteLogFileAfterCloudUpload

(Optional) You can delete a log file after the log manager component uploads the logs to CloudWatch Logs.

Default: false

multiLineStartPattern

(Optional) A regular expression that identifies when a log message on a new line is a new log message. If the regular expression doesn't match the new line, the log manager component appends the new line to the log message for the previous line.

By default, the log manager component checks if the line starts with a whitespace character, such as a tab or space. If it doesn't, the log manager handles that line as a new log message. Otherwise, it appends that line to the current log message. This behavior ensures that the log manager component doesn't split messages that span multiple lines, such as stack traces.

periodicUploadIntervalSec

(Optional) The period in seconds at which the log manager component checks for new log files to upload.

Default: 300 (5 minutes)

Minimum: 0.000001 (1 microsecond)

Example: Configuration merge update

The following example configuration specifies to upload system logs and com.example.HelloWorld component logs to CloudWatch Logs.

{
  "logsUploaderConfiguration": {
    "systemLogsConfiguration": {
      "uploadToCloudWatch": "true",
      "minimumLogLevel": "INFO",
      "diskSpaceLimit": "10",
      "diskSpaceLimitUnit": "MB",
      "deleteLogFileAfterCloudUpload": "false"
    },
    "componentLogsConfigurationMap": {
      "com.example.HelloWorld": {
        "minimumLogLevel": "INFO",
        "diskSpaceLimit": "20",
        "diskSpaceLimitUnit": "MB",
        "deleteLogFileAfterCloudUpload": "false"
      }
    }
  },
  "periodicUploadIntervalSec": "300"
}
v2.1.x
logsUploaderConfiguration

(Optional) The configuration for logs that the log manager component uploads. This object contains the following information:

systemLogsConfiguration

(Optional) The configuration for AWS IoT Greengrass Core software system logs, which include logs from the Greengrass nucleus and plugin components. Specify this configuration to enable the log manager component to manage system logs. This object contains the following information:

uploadToCloudWatch

(Optional) You can upload system logs to CloudWatch Logs.

Default: false

minimumLogLevel

(Optional) The minimum level of log messages to upload. This minimum level applies only if you configure the Greengrass nucleus component to output JSON format logs. To enable JSON format logs, specify JSON for the logging format parameter (logging.format).

Choose from the following log levels, listed here in level order:

  • DEBUG

  • INFO

  • WARN

  • ERROR

Default: INFO

diskSpaceLimit

(Optional) The maximum total size of Greengrass system log files, in the unit you specify in diskSpaceLimitUnit. After the total size of Greengrass system log files exceeds this maximum total size, the AWS IoT Greengrass Core software deletes the oldest Greengrass system log files.

This parameter is equivalent to the log size limit parameter (totalLogsSizeKB) of the Greengrass nucleus component. The AWS IoT Greengrass Core software uses the minimum of the two values as the maximum total Greengrass system log size.

diskSpaceLimitUnit

(Optional) The unit for the diskSpaceLimit. Choose from the following options:

  • KB – kilobytes

  • MB – megabytes

  • GB – gigabytes

Default: KB

deleteLogFileAfterCloudUpload

(Optional) You can delete a log file after the log manager component uploads the logs to CloudWatch Logs.

Default: false

componentLogsConfiguration

(Optional) A list of log configurations for components on the core device. Each configuration in this list defines the log configuration for a component or application. The log manager component uploads these component logs to CloudWatch Logs

Each object contains the following information:

componentName

The name of the component or application for this log configuration. You can specify the name of a Greengrass component or another value to identify this log group.

minimumLogLevel

(Optional) The minimum level of log messages to upload. This minimum level applies only if this component's logs use a specific JSON format, which you can find in the AWS IoT Greengrass logging module repository on GitHub.

Choose from the following log levels, listed here in level order:

  • DEBUG

  • INFO

  • WARN

  • ERROR

Default: INFO

diskSpaceLimit

(Optional) The maximum total size of all log files for this component, in the unit you specify in diskSpaceLimitUnit. After the total size of this component's log files exceeds this maximum total size, the AWS IoT Greengrass Core software deletes this component's oldest log files.

This parameter is related to the log size limit parameter (totalLogsSizeKB) of the Greengrass nucleus component. The AWS IoT Greengrass Core software uses the minimum of the two values as the maximum total log size for this component.

diskSpaceLimitUnit

(Optional) The unit for the diskSpaceLimit. Choose from the following options:

  • KB – kilobytes

  • MB – megabytes

  • GB – gigabytes

Default: KB

logFileDirectoryPath

(Optional) The path to the folder that contains this component's log files.

You don't need to specify this parameter for Greengrass components that print to standard output (stdout) and standard error (stderr).

Default: /greengrass/v2/logs.

logFileRegex

(Optional) A regular expression that specifies the log file name format that the component or application uses. The log manager component uses this regular expression to identify log files in the folder at logFileDirectoryPath.

You don't need to specify this parameter for Greengrass components that print to standard output (stdout) and standard error (stderr).

If your component or application rotates log files, specify a regex that matches the rotated log file names. For example, you might specify hello_world\\\\w*.log to upload logs for a Hello World application. The \\\\w* pattern matches zero or more word characters, which includes alphanumeric characters and underscores. This regex matches log files with and without timestamps in their name. In this example, the log manager uploads the following log files:

  • hello_world.log – The most recent log file for the Hello World application.

  • hello_world_2020_12_15_17_0.log – An older log file for the Hello World application.

Default: componentName\\\\w*.log, where componentName is the name of the component for this log configuration.

deleteLogFileAfterCloudUpload

(Optional) You can delete a log file after the log manager component uploads the logs to CloudWatch Logs.

Default: false

multiLineStartPattern

(Optional) A regular expression that identifies when a log message on a new line is a new log message. If the regular expression doesn't match the new line, the log manager component appends the new line to the log message for the previous line.

By default, the log manager component checks if the line starts with a whitespace character, such as a tab or space. If it doesn't, the log manager handles that line as a new log message. Otherwise, it appends that line to the current log message. This behavior ensures that the log manager component doesn't split messages that span multiple lines, such as stack traces.

periodicUploadIntervalSec

(Optional) The period in seconds at which the log manager component checks for new log files to upload.

Default: 300 (5 minutes)

Minimum: 0.000001 (1 microsecond)

Example: Configuration merge update

The following example configuration specifies to upload system logs and com.example.HelloWorld component logs to CloudWatch Logs.

{
  "logsUploaderConfiguration": {
    "systemLogsConfiguration": {
      "uploadToCloudWatch": "true",
      "minimumLogLevel": "INFO",
      "diskSpaceLimit": "10",
      "diskSpaceLimitUnit": "MB",
      "deleteLogFileAfterCloudUpload": "false"
    },
    "componentLogsConfiguration": [
      {
        "componentName": "com.example.HelloWorld",
        "minimumLogLevel": "INFO",
        "diskSpaceLimit": "20",
        "diskSpaceLimitUnit": "MB",
        "deleteLogFileAfterCloudUpload": "false"
      }
    ]
  },
  "periodicUploadIntervalSec": "300"
}
v2.0.x
logsUploaderConfiguration

(Optional) The configuration for logs that the log manager component uploads. This object contains the following information:

systemLogsConfiguration

(Optional) The configuration for AWS IoT Greengrass Core software system logs. Specify this configuration to enable the log manager component to manage system logs. This object contains the following information:

uploadToCloudWatch

(Optional) You can upload system logs to CloudWatch Logs.

Default: false

minimumLogLevel

(Optional) The minimum level of log messages to upload. This minimum level applies only if you configure the Greengrass nucleus component to output JSON format logs. To enable JSON format logs, specify JSON for the logging format parameter (logging.format).

Choose from the following log levels, listed here in level order:

  • DEBUG

  • INFO

  • WARN

  • ERROR

Default: INFO

diskSpaceLimit

(Optional) The maximum total size of Greengrass system log files, in the unit you specify in diskSpaceLimitUnit. After the total size of Greengrass system log files exceeds this maximum total size, the AWS IoT Greengrass Core software deletes the oldest Greengrass system log files.

This parameter is equivalent to the log size limit parameter (totalLogsSizeKB) of the Greengrass nucleus component. The AWS IoT Greengrass Core software uses the minimum of the two values as the maximum total Greengrass system log size.

diskSpaceLimitUnit

(Optional) The unit for the diskSpaceLimit. Choose from the following options:

  • KB – kilobytes

  • MB – megabytes

  • GB – gigabytes

Default: KB

deleteLogFileAfterCloudUpload

(Optional) You can delete a log file after the log manager component uploads the logs to CloudWatch Logs.

Default: false

componentLogsConfiguration

(Optional) A list of log configurations for components on the core device. Each configuration in this list defines the log configuration for a component or application. The log manager component uploads these component logs to CloudWatch Logs

Each object contains the following information:

componentName

The name of the component or application for this log configuration. You can specify the name of a Greengrass component or another value to identify this log group.

minimumLogLevel

(Optional) The minimum level of log messages to upload. This minimum level applies only if this component's logs use a specific JSON format, which you can find in the AWS IoT Greengrass logging module repository on GitHub.

Choose from the following log levels, listed here in level order:

  • DEBUG

  • INFO

  • WARN

  • ERROR

Default: INFO

diskSpaceLimit

(Optional) The maximum total size of all log files for this component, in the unit you specify in diskSpaceLimitUnit. After the total size of this component's log files exceeds this maximum total size, the AWS IoT Greengrass Core software deletes this component's oldest log files.

This parameter is related to the log size limit parameter (totalLogsSizeKB) of the Greengrass nucleus component. The AWS IoT Greengrass Core software uses the minimum of the two values as the maximum total log size for this component.

diskSpaceLimitUnit

(Optional) The unit for the diskSpaceLimit. Choose from the following options:

  • KB – kilobytes

  • MB – megabytes

  • GB – gigabytes

Default: KB

logFileDirectoryPath

The path to the folder that contains this component's log files.

To upload a Greengrass component's logs, specify /greengrass/v2/logs, and replace /greengrass/v2 with your Greengrass root folder.

logFileRegex

A regular expression that specifies the log file name format that the component or application uses. The log manager component uses this regular expression to identify log files in the folder at logFileDirectoryPath.

To upload a Greengrass component's logs, specify a regex that matches the rotated log file names. For example, you might specify com.example.HelloWorld\\w*.log to upload logs for a Hello World component. The \\w* pattern matches zero or more word characters, which includes alphanumeric characters and underscores. This regex matches log files with and without timestamps in their name. In this example, the log manager uploads the following log files:

  • com.example.HelloWorld.log – The most recent log file for the Hello World component.

  • com.example.HelloWorld_2020_12_15_17_0.log – An older log file for the Hello World component. The Greengrass nucleus adds a rotating timestamp to the log files.

deleteLogFileAfterCloudUpload

(Optional) You can delete a log file after the log manager component uploads the logs to CloudWatch Logs.

Default: false

multiLineStartPattern

(Optional) A regular expression that identifies when a log message on a new line is a new log message. If the regular expression doesn't match the new line, the log manager component appends the new line to the log message for the previous line.

By default, the log manager component checks if the line starts with a whitespace character, such as a tab or space. If it doesn't, the log manager handles that line as a new log message. Otherwise, it appends that line to the current log message. This behavior ensures that the log manager component doesn't split messages that span multiple lines, such as stack traces.

periodicUploadIntervalSec

(Optional) The period in seconds at which the log manager component checks for new log files to upload.

Default: 300 (5 minutes)

Minimum: 0.000001 (1 microsecond)

Example: Configuration merge update

The following example configuration specifies to upload system logs and com.example.HelloWorld component logs to CloudWatch Logs.

{
  "logsUploaderConfiguration": {
    "systemLogsConfiguration": {
      "uploadToCloudWatch": "true",
      "minimumLogLevel": "INFO",
      "diskSpaceLimit": "10",
      "diskSpaceLimitUnit": "MB",
      "deleteLogFileAfterCloudUpload": "false"
    },
    "componentLogsConfiguration": [
      {
        "componentName": "com.example.HelloWorld",
        "minimumLogLevel": "INFO",
        "logFileDirectoryPath": "/greengrass/v2/logs",
        "logFileRegex": "com.example.HelloWorld\\w*.log",
        "diskSpaceLimit": "20",
        "diskSpaceLimitUnit": "MB",
        "deleteLogFileAfterCloudUpload": "false"
      }
    ]
  },
  "periodicUploadIntervalSec": "300"
}

Usage

The log manager component uploads to the following log groups and log streams.

2.1.0 and later
Log group name
/aws/greengrass/componentType/region/componentName

The log group name uses the following variables:

  • componentType – The type of the component, which can be one of the following:

    • GreengrassSystemComponent – This log group includes logs for the nucleus and plugin components, which run in the same JVM as the Greengrass nucleus. The component is part of the Greengrass nucleus.

    • UserComponent – This log group includes logs for generic components, Lambda components, and other applications on the device. The component isn't part of the Greengrass nucleus.

    For more information, see Component types.

  • region – The AWS Region that the core device uses.

  • componentName – The name of the component. For system logs, this value is System.

Log stream name
/date/thing/thingName

The log stream name uses the following variables:

  • date – The date of the log, such as 2020/12/15. The log manager component uses the yyyy/MM/dd format.

  • thingName – The name of the core device.

Note

If a thing name contains a colon (:), the log manager replaces the colon with a plus (+).

2.0.x
Log group name
/aws/greengrass/componentType/region/componentName

The log group name uses the following variables:

  • componentType – The type of the component, which can be one of the following:

    • GreengrassSystemComponent – The component is part of the Greengrass nucleus.

    • UserComponent – The component isn't part of the Greengrass nucleus. The log manager uses this type for Greengrass components and other applications on the device.

  • region – The AWS Region that the core device uses.

  • componentName – The name of the component. For system logs, this value is System.

Log stream name
/date/deploymentTargets/thingName

The log stream name uses the following variables:

  • date – The date of the log, such as 2020/12/15. The log manager component uses the yyyy/MM/dd format.

  • deploymentTargets – The things whose deployments include the component. The log manager component separates each target by a slash. If the component runs on the core device as the result of a local deployment, this value is LOCAL_DEPLOYMENT.

    Consider an example where you have a core device named MyGreengrassCore, and the core device has two deployments:

    • A deployment that targets the core device, MyGreengrassCore.

    • A deployment that targets a thing group named MyGreengrassCoreGroup, which contains the core device.

    The deploymentTargets for this core device are thing/MyGreengrassCore/thinggroup/MyGreengrassCoreGroup.

  • thingName – The name of the core device.

Formats for log entries.

The Greengrass nucleus writes log files in either string or JSON format. For system logs, you control the format by setting the format field of the logging entry. You can find the logging entry in the Greengrass nucleus component's configuration file. For more information, see Greengrass nucleus configuration.

The text format is free-form and accepts any string. The following fleet status service message is an example of string formatted logging:

2023-03-26T18:18:27.271Z [INFO] (pool-1-thread-2)
com.aws.greengrass.status.FleetStatusService: fss-status-update-published. 
Status update published to FSS. {trigger=CADENCE, serviceName=FleetStatusService, 
currentState=RUNNING}

You should use the JSON format if you want to view logs with the Greengrass CLI logs command or interact with logs programmatically. The following example outlines the JSON shape:

{
   "loggerName": <string>,
   "level": <"DEBUG" | "INFO" | "ERROR" | "TRACE" | "WARN">,
   "eventType": <string, optional>,
   "cause": <string, optional>,
   "contexts": {},
   "thread": <string>,
   "message": <string>,
   "timestamp": <epoch time> # Needs to be epoch time
}

To control the output of your component's logs, you can use the minimumLogLevel configuration option. To use this option, your component must write its log entries in JSON format. You should use the same format as the system log file.

Local log file

This component uses the same log file as the Greengrass nucleus component.

Linux
/greengrass/v2/logs/greengrass.log
Windows
C:\greengrass\v2\logs\greengrass.log
To view this component's logs

  • Run the following command on the core device to view this component's log file in real time. Replace /greengrass/v2 or C:\greengrass\v2 with the path to the AWS IoT Greengrass root folder.

    Linux
    sudo tail -f /greengrass/v2/logs/greengrass.log
    Windows (PowerShell)
    Get-Content C:\greengrass\v2\logs\greengrass.log -Tail 10 -Wait

Changelog

The following table describes the changes in each version of the component.

Version

Changes

2.3.8

Version updated for Greengrass nucleus version 2.13.0 release.

2.3.7

Version updated for Greengrass nucleus version 2.12.0 release.

2.3.6
Bug fixes and improvements

  • Adjusts log levels for certain errors.

2.3.5
Improvements

Improves log upload speed.

Version updated for Greengrass nucleus version 2.11.0 release.

2.3.4
Bug fixes and improvements

  • Adds support for setting the periodicUploadIntervalSec parameter to fractional values. The minimum is 1 microsecond.

  • Fixes an issue where log manager doesn't respect the CloudWatch putLogEvents limits.

2.3.3

Version updated for Greengrass nucleus version 2.10.0 release.

2.3.2
Bug fixes and improvements

  • Improves space management so that log files are not deleted before they are uploaded.

  • Fixes issues with cache management.

  • Additional minor bug fixes and improvements.

2.3.1
Bug fixes and improvements

  • Fixes an issue where s that target file groups with multiples active log files upload duplicate entries to CloudWatch.

  • Additional minor bug fixes and improvements.

2.3.0
Note

We recommend that you upgrade to Greengrass nucleus 2.9.1 when you upgrade to log manager 2.3.0.

New features

Reduces log delays by processing and directly uploading active log files instead of waiting for new files to be rotated.

Bug fixes and improvements

  • Improves support of log rotation when rotating files with a unique name.

  • Additional minor bug fixes and improvements.

2.2.8

 Version updated for Greengrass nucleus version 2.9.0 release.

2.2.7

 Version updated for Greengrass nucleus version 2.8.0 release.

2.2.6

Version updated for Greengrass nucleus version 2.7.0 release.

2.2.5

Version updated for Greengrass nucleus version 2.6.0 release.

2.2.4
Bug fixes and improvements

  • Improves stability when handling invalid configurations.

  • Additional minor fixes and improvements.

2.2.3
Bug fixes and improvements

  • Improves stability in certain scenarios where the component restarts or encounters errors.

  • Fixes issues where large log messages and large log files fail to upload in certain scenarios.

  • Fixes issues with how this component handles configuration reset updates.

  • Fixes an issue where a null diskSpaceLimit configuration value prevented the component from deploying.

2.2.2
Bug fixes and improvements

  • Adds support for log messages that are larger than 256 kilobytes. The log manager component splits these large log messages into multiple messages with the same log event timestamp.

2.2.1

Version updated for Greengrass nucleus version 2.5.0 release.

2.2.0
New feature

  • Adds the componentLogsConfigurationMap configuration parameter to support a map format for component log configurations. Each componentName object in the map defines the log configuration for a component or application.

2.1.3

Version updated for Greengrass nucleus version 2.4.0 release.

2.1.2

Version updated for Greengrass nucleus version 2.3.0 release.

2.1.1
Bug fixes and improvements

  • Fixes an issue where the system log configuration wasn't updated in certain cases.

2.1.0
Bug fixes and improvements

  • Use defaults for logFileDirectoryPath and logFileRegex that work for Greengrass components that print to standard output (stdout) and standard error (stderr).

  • Correctly route traffic through a configured network proxy when uploading logs to CloudWatch Logs.

  • Correctly handle colon characters (:) in log stream names. CloudWatch Logs log stream names don't support colons.

  • Simplify log stream names by removing thing group names from the log stream.

  • Remove an error log message that prints during normal behavior.

2.0.x

Initial version.