Test GuardDuty findings in dedicated accounts - Amazon GuardDuty
ConsiderationsGuardDuty findings tester script can generateStep 1 - PrerequisitesStep 2 - Deploy AWS resourcesStep 3 - Run tester scriptsStep 4 - Clean up AWS test resourcesTroubleshooting common issues

Test GuardDuty findings in dedicated accounts

Use this document to run a tester script that generates GuardDuty findings against test resources that will be deployed in your AWS account. You can perform these steps when you want to understand and learn about certain GuardDuty finding types and how the finding details look for actual resources in your account. This experience is different from generating Sample findings. For more information about the experience of testing GuardDuty findings, see Considerations.

Considerations

Before you proceed, take the following considerations into account:

  • GuardDuty recommends deploying the tester in a dedicated non-production AWS account. This approach will ensure that you are able to properly identify GuardDuty findings generated by the tester. Additionally, the GuardDuty tester deploys a variety of resources which may require IAM permissions beyond what is allowed in other accounts. Using a dedicated account ensures that permissions can be properly scoped with a clear account boundary.

  • The tester script generates over 100 GuardDuty findings with different AWS resource combinations. Presently, this does't include all the GuardDuty finding types. For a list of finding types that you can generate with this tester script, see GuardDuty findings tester script can generate.

  • For the GuardDuty tester to work as expected, GuardDuty needs to be enabled in the account where the tester resources are deployed. Depending on the tests that will run, the tester evaluates whether or not the appropriate GuardDuty protection plans are enabled. For any protection plan that is not enabled, GuardDuty will request permission to enable the necessary protection plans long enough for GuardDuty to perform the tests that will generate findings. Later, GuardDuty will disable the protection plan once the testing is complete.

    Enabling GuardDuty for the first time

    When GuardDuty gets enabled in your dedicated account for the first time in a specific Region, your account will be automatically enrolled in a 30-day free trial.

    GuardDuty offers optional protection plans. At the time of enabling GuardDuty, certain protection plans also get enabled and are included in the GuardDuty 30-day free trial. For more information, see Using GuardDuty 30-day free trial.

    GuardDuty is already enabled in your account prior to running the tester script

    When GuardDuty is already enabled, then based on the parameters, the tester script will check the configuration status of certain protection plans and other account level settings that are required to generate the findings.

    By running this tester script, certain protection plans may get enabled for the first time in your dedicated account in a Region. This will start the 30-day free trial for that protection plan. For information about free trial associated with each protection plan, see Using GuardDuty 30-day free trial.

  • As long as the GuardDuty tester infrastructure is deployed, you may occasionally receive UnauthorizedAccess:EC2/TorClient findings from the PenTest instance.

GuardDuty findings tester script can generate

Presently, the tester script generates following finding types that are related to Amazon EC2, Amazon EKS, Amazon S3, IAM, and EKS audit logs:

Show moreShow less

Step 1 - Prerequisites

To prepare your test environment, you will need the following items:

  • Git – Install git command line tool based on the operating system that you use. This is required to clone the amazon-guardduty-tester repository.

  • AWS Command Line Interface – An open source tool that enables you to interact with AWS services by using commands in your command-line shell. For more information, see Get started with AWS CLI in the AWS Command Line Interface User Guide.

  • AWS Systems Manager – To initiate Session Manager sessions with your managed nodes by using AWS CLI you must install the Session Manager plugin on your local machine. For more information, see Install Session Manager plugin for AWS CLI in the AWS Systems Manager User Guide.

  • Node Package Manager (NPM) – Install NPM to install all the dependencies.

  • Docker – You must have Docker installed. For installation instructions, see the Docker website.

    To verify that Docker has been installed, run the following command and confirm there is an output similar to the following output:

    $ docker --version
Docker version 19.03.1

  • Subscribe to Kali Linux image in the AWS Marketplace.

Step 2 - Deploy AWS resources

This section provides a list of key concepts and the steps to deploy certain AWS resources in your dedicated account.

Concepts

The following list provides key concepts related to the commands that help you deploy the resources:

  • AWS Cloud Development Kit (AWS CDK) – CDK is an open-source software development framework for defining cloud infrastructure in code and provisioning it through AWS CloudFormation. CDK supports a couple of programming languages to define reusable cloud components known as constructs. You can compose these together into stacks and apps. Then, you can deploy your CDK applications to AWS CloudFormation to provision or update your resources. For more information, see What is the AWS CDK? in the AWS Cloud Development Kit (AWS CDK) Developer Guide.

  • Bootstrapping – It is the process of preparing your AWS environment for usage with AWS CDK. Before you deploy a CDK stack into an AWS environment, the environment must first be bootstrapped. This process of provisioning specific AWS resources in your environment that are used by AWS CDK is part of the steps that you will perform in the next section - Steps to deploy AWS resources.

    For more information about how bootstrapping works, see Bootstrapping in the AWS Cloud Development Kit (AWS CDK) Developer Guide.

Steps to deploy AWS resources

Perform the following steps to start deploying the resources:

  1. Set up your AWS CLI default account and Region unless the dedicated account Region variables are manually set in the bin/cdk-gd-tester.ts file. For more information, see Environments in the AWS Cloud Development Kit (AWS CDK) Developer Guide.

  2. Run the following commands to deploy the resources:

    git clone https://github.com/awslabs/amazon-guardduty-tester && cd amazon-guardduty-tester
npm install
cdk bootstrap
cdk deploy

    The last command (cdk deploy) creates a AWS CloudFormation stack on your behalf. The name of this stack is GuardDutyTesterStack.

    As a part of this script, GuardDuty creates new resources to generate GuardDuty findings in your account. It also adds the following tag key:value pair to the Amazon EC2 instances:

    CreatedBy:GuardDuty Test Script

    The Amazon EC2 instances also include the EC2 instances that host EKS nodes and ECS clusters.

    Instance types

    GuardDuty creates t3.micro for all resources with an exception to the Amazon EKS node group. Because EKS requires at least 2 cores, the EKS node has t3.medium instance type. For more information about instance types, see Available sizes in the Amazon EC2 Instances Types Guide.

Step 3 - Run tester scripts

This is a two-step process where you first need to start a session with test driver and then, run scripts to generate GuardDuty findings with specific resource combinations.

  1. After your resources are deployed, save the Region code to a variable in your current terminal session. Use the following command and replace us-east-1 with the Region code where you deployed the resources:

    $ REGION=us-east-1

  2. The tester script is available only through AWS Systems Manager (SSM). To start an interactive shell on the tester host instance, query the host InstanceId.

  3. Use the following command to begin your session for the tester script:

    aws ssm start-session 
  --region $REGION 
  --document-name AWS-StartInteractiveCommand 
  --parameters command="cd /home/ssm-user/py_tester && bash -l" 
  --target $(aws ec2 describe-instances 
    --region $REGION 
    --filters "Name=tag:Name,Values=Driver-GuardDutyTester" 
    --query "Reservations[].Instances[?State.Name=='running'].InstanceId" 
    --output text)

The tester script is a Python-based program that dynamically builds a bash script to generate findings based on your input. You have flexibility to generate findings based on one or more AWS resource types, GuardDuty protection plans, Threat Purposes (tactics), Foundational data sources, or GuardDuty findings tester script can generate.

Use the following command examples as reference, and run one or more commands to generate findings that you want to explore:

python3 guardduty_tester.py 
python3 guardduty_tester.py --all 
python3 guardduty_tester.py --s3 
python3 guardduty_tester.py --tactics discovery 
python3 guardduty_tester.py --ec2 --eks --tactics backdoor policy execution 
python3 guardduty_tester.py --eks --runtime only 
python3 guardduty_tester.py --ec2 --runtime only --tactics impact 
python3 guardduty_tester.py --log-source dns vpc-flowlogs 
python3 guardduty_tester.py --finding 'CryptoCurrency:EC2/BitcoinTool.B!DNS'

For more information about valid parameters, you can run the following help command:

python3 guardduty_tester.py --help

Choose a preferred method to view the generated findings in your account.

GuardDuty console

  1. Sign in to the AWS Management Console and open the GuardDuty console at https://console.aws.amazon.com/guardduty/.

  2. In the navigation pane, choose Findings.

  3. From the findings table, select a finding for which you want to view the details. This will open up the finding details panel. For information, see Understanding and generating Amazon GuardDuty findings.

  4. If you want to filter these findings, use the resource tag key and value. For example, to filter the findings generated for the Amazon EC2 instances, use CreatedBy:GuardDuty Test Script tag key:value pair for Instance tag key and Instance tag key.

API
AWS CLI

  • Run the following AWS CLI command to view the generated findings and replace us-east-1 and 12abc34d567e8fa901bc2d34EXAMPLE with suitable values:

    aws guardduty list-findings --region us-east-1 --detector-id 12abc34d567e8fa901bc2d34EXAMPLE

    To find the detectorId for your account and current Region, see the Settings page in the https://console.aws.amazon.com/guardduty/ console, or run the ListDetectors API.

    For more information about the parameters that you can use to filter findings, see list-findings in the AWS CLI Command Reference.

Step 4 - Clean up AWS test resources

The account-level settings and other configuration status updates made during Step 3 - Run tester scripts return to the original state when the tester script concludes.

After you have run the tester script, you can choose to clean up the AWS test resources. You can choose to do this by using one of the following methods:

Troubleshooting common issues

GuardDuty has identified common issues and recommends troubleshooting steps:

  • Cloud assembly schema version mismatch – Update AWS CDK CLI to a version compatible with the required cloud assembly version, or to the latest available version. For more information, see AWS CDK CLI compatibility.

  • Docker permission denied – Add the dedicated account user to the docker or docker-users so that the dedicated account can run the commands. For more information about steps, see Daemon socket option.

  • Your requested instance type is not supported in your requested Availability Zone – Some Availability zones don't support particular instance types. To identify which availability zones support your preferred instance type and reattempt to deploy AWS resources, perform the following steps:

    1. Choose a preferred method to determine which Availability zones support your instance type:

      Console
      To identify Availability zones that support preferred instance type

      1. Sign in to the AWS Management Console and open the Amazon EC2 console at https://console.aws.amazon.com/ec2/.

      2. By using the AWS Region selector in the upper-right corner of the page, choose the Region where you want to launch the instance.

      3. In the navigation pane, under Instances, choose Instance Types.

      4. From the Instance types table, choose a preferred instance type.

      5. Under Networking, view the Regions listed under Availability zones.

        Based on this information, you might need to choose a new Region where you can deploy the resources.

      AWS CLI

      Run the following command to view a list of Availability zones. Make sure to specify your preferred instance type and the Region (us-east-1).

      aws ec2 describe-instance-type-offerings --location-type availability-zone  --filters Name=instance-type,Values=Preferred instance type --region us-east-1 --output table

      For more information about this command, see describe-instance-type-offerings in the AWS CLI Command Reference.

      When running this command, if you receive an error, make sure you are using the latest version of AWS CLI. For more information, see Troubleshooting in the AWS Command Line Interface User Guide.

    2. Attempt deploying the AWS resources again and specify an Availability zone that supports your preferred instance type.

      To re-attempt deploying AWS resources

      1. Set up the default Region in the bin/cdk-gd-tester.ts file.

      2. To specify the Availability zone, open the amazon-guardduty-tester/lib/common/network/vpc.ts file.

      3. In this file, replace maxAzs: 2, with availabilityZones: ['us-east-1a', 'us-east-1c'], where you must specify the Availability zones for your instance type.

      4. Continue with the remaining steps under Steps to deploy AWS resources.