Troubleshooting SSM Agent

If you experience problems running operations on your managed nodes, there might be a problem with AWS Systems Manager Agent (SSM Agent). Use the following information to help you view SSM Agent log files and troubleshoot the agent.

SSM Agent is out of date

An updated version of SSM Agent is released whenever new capabilities are added to Systems Manager or updates are made to existing capabilities. Failing to use the latest version of the agent can prevent your managed node from using various Systems Manager capabilities and features. For that reason, we recommend that you automate the process of keeping SSM Agent up to date on your machines. For information, see Automating updates to SSM Agent. Subscribe to the SSM Agent Release Notes page on GitHub to get notifications about SSM Agent updates.

Troubleshoot issues using SSM Agent log files

SSM Agent logs information in the following files. The information in these files can also help you troubleshoot problems. For more information about SSM Agent log files, including how to turn on debug logging, see Viewing SSM Agent logs.

For Linux managed nodes, you might find more information in the messages file written to the following directory: /var/log.

For additional information about troubleshooting using agent logs, see How do I use SSM Agent logs to troubleshoot issues with SSM Agent in my managed instance? in the AWS re:Post Knowledge Center.

Agent log files don't rotate (Windows)

If you specify date-based log file rotation in the seelog.xml file (on Windows Server managed nodes) and the logs don't rotate, specify the fullname=true parameter. Here is an example of a seelog.xml configuration file with the fullname=true parameter specified.

<seelog type="adaptive" mininterval="2000000" maxinterval="100000000" critmsgcount="500" minlevel="debug">
   <exceptions>
      <exception filepattern="test*" minlevel="error" />
   </exceptions>
   <outputs formatid="fmtinfo">
      <console formatid="fmtinfo" />
      <rollingfile type="date" datepattern="200601021504" maxrolls="4" filename="C:\ProgramData\Amazon\SSM\Logs\amazon-ssm-agent.log" fullname=true />
      <filter levels="error,critical" formatid="fmterror">
         <rollingfile type="date" datepattern="200601021504" maxrolls="4" filename="C:\ProgramData\Amazon\SSM\Logs\errors.log" fullname=true />
      </filter>
   </outputs>
   <formats>
      <format id="fmterror" format="%Date %Time %LEVEL [%FuncShort @ %File.%Line] %Msg%n" />
      <format id="fmtdebug" format="%Date %Time %LEVEL [%FuncShort @ %File.%Line] %Msg%n" />
      <format id="fmtinfo" format="%Date %Time %LEVEL %Msg%n" />
   </formats>
</seelog>

Unable to connect to SSM endpoints

SSM Agent must allow HTTPS (port 443) outbound traffic to the following endpoints:

region represents the identifier for an AWS Region supported by AWS Systems Manager, such as us-east-2 for the US East (Ohio) Region. For a list of supported region values, see the Region column in Systems Manager service endpoints in the Amazon Web Services General Reference.

SSM Agent won't work if it can't communicate with the preceding endpoints, as described, even if you use AWS provided Amazon Machine Images (AMIs) such as Amazon Linux 2 or Amazon Linux 2023. Your network configuration must have open internet access or you must have custom virtual private cloud (VPC) endpoints configured. If you don't plan on creating a custom VPC endpoint, check your internet gateways or NAT gateways. For more information about how to manage VPC endpoints, see Improve the security of EC2 instances by using VPC endpoints for Systems Manager.

Verify your VPC configuration

In order to manage EC2 instances with Systems Manager, your VPC endpoints must be configured properly for ssm.region.amazonaws.com, ssmmessages.region.amazonaws.com, and in some cases explained earlier in this topic in Unable to connect to SSM endpoints, ec2messages.region.amazonaws.com. Your network configuration must have open internet access or you must have these virtual private cloud (VPC) endpoints configured.

To troubleshoot issues with your VPC endpoints, do the following:

For more information, see the following topics:

Verify your VPC DNS-related attributes

As part of verifying your VPC configuration, ensure that the attributes enableDnsSupport and enableDnsHostnames are enabled.

You can enable these attributes using the Amazon EC2 ModifyVPCAttribute API action or the AWS CLI command modify-vpc-attribute.

For information about enabling these attributes in the Amazon VPC Console, see View and update DNS attributes for your VPC in the Amazon VPC User Guide.

Verify ingress rules on endpoint security groups

Ensure that any VPC endpoints you have configured (ssm, ssmmessages, and ec2messages) include an ingress rule on their security groups to allow traffic in on port 443. If necessary, you can create a new security group in the VPC with an ingress rule to allow traffic on port 443 for the Classless Inter-Domain Routing (CIDR) block for the VPC. After you create the security group, attach it to each VPC endpoint.

For more information, see the following topics:

Use ssm-cli to troubleshoot managed node availability

Starting with SSM Agent version 3.1.501.0, you can use ssm-cli to determine whether a managed node meets the primary requirements to be managed by Systems Manager, and to appear in lists of managed nodes in Fleet Manager. The ssm-cli is a standalone command line tool included in the SSM Agent installation. Preconfigured commands are included that gather the required information to help you diagnose why an Amazon EC2 instance or non-EC2 machine that you have confirmed is running isn't included in your lists of managed nodes in Systems Manager. These commands are run when you specify the get-diagnostics option.

For more information, see Troubleshooting managed node availability using ssm-cli.