View a markdown version of this page

Create a penetration test - AWS Security Agent (now part of AWS Continuum)
PrerequisitesStart creating a penetration testName your penetration testConfigure penetration test scopeConfigure IAM RoleAutomatic code remediationConfigure VPC resources (optional)Configure authentication credentials (optional)Review network scopeAttach additional resources (optional)Set a maximum task-hours limitExclude risk types (optional)Create the penetration test

Create a penetration test

Set up automated penetration testing for your web applications by configuring test scope, target domains, and AWS resource access. Penetration tests help identify security vulnerabilities in running applications by simulating real-world attack scenarios against your verified domains.

AWS Security Agent performs comprehensive security testing against your web applications based on configured scope and permissions, providing detailed findings about exploitable vulnerabilities before attackers can discover them.

In this procedure, you’ll create a penetration test by configuring test details, defining the test scope, and setting up required permissions.

Prerequisites

Before you begin, ensure you have:

  • Access to the AWS Security Agent web application

  • At least one verified domain for testing

  • IAM role with appropriate permissions for AWS Security Agent

  • Understanding of your application’s architecture and critical paths

Start creating a penetration test

Navigate to the penetration test creation page in the web application.

  1. Log in to the AWS Security Agent web application.

  2. Navigate to the Penetration tests section.

  3. Choose Create a penetration test.

Tip

Only verified domains can be included in penetration tests. Ask your admin to verify the domain in AWS management console. See Enable an application domain for penetration testing.

When you complete the Credentials and Network configuration steps, AWS Security Agent tests more of your application and stays away from anything you do not want reached. The wizard has five steps:

  1. Penetration test details – Name the test, set the target URLs, and choose the service role and log group.

  2. (Optional) VPC Resources – Configure network access for a target that is not publicly reachable.

  3. (Optional) Credentials – Add the credentials the agent uses to sign in, and test that each one works.

  4. Network configuration – Review every domain the test may reach, including the domains discovered while your credentials were signed in.

  5. (Optional) Additional configuration – Add application context and set run options.

Choose Save & Next to move to the next step. Each Save & Next saves your progress, so you can leave the wizard and come back later.

A test you started but did not finish appears in the Penetration tests list with a Draft badge. Choose Continue setup on that test to return to the wizard. You cannot start a draft until you finish the wizard and create the test.

Name your penetration test

Provide a descriptive name that helps identify the purpose and scope of this penetration test.

  1. In the Penetration test name field, enter a descriptive name for your penetration test.

    Example

    The name should clearly identify the application, environment, or component being tested. Maximum 100 characters.

Configure penetration test scope

Define which domains the penetration test is allowed to attack.

The Penetration testing scope section holds the target URLs. Its Advanced network access subsection holds custom HTTP headers.

You classify the other domains the test may reach on the Network configuration step, after the agent has tested your credentials. For more information, see Review network scope.

Add target domains

Specify the verified domains that will be actively tested for security vulnerabilities.

  1. In the Penetration testing scope section, locate Target URLs.

  2. Expand the Verified domains section to view available domains.

  3. In the Target URL field, enter a target domain URL.

    Important

    Only verified domains can be tested. The URL must be under a domain you’ve previously verified in AWS Security Agent. Sub-domains of a verified domain do not require separate verification.

    Non-standard ports

    If your application serves traffic on a non-standard port, include the port in the URL. For example, https://example.com:8443. Domain verification applies to the host. You don’t need to verify the domain again to test a different port.

  4. To add multiple target domains:

    1. Choose Add domain.

    2. Enter each additional domain URL.

  5. To remove a target domain, choose Remove next to the domain URL.

Tip

For best results, include all domains that are part of your application’s user flow, including subdomains for APIs, authentication services, and content delivery. Sub-domains of a verified parent domain do not require separate verification.

Add custom HTTP headers (optional)

Specify custom HTTP headers that will be added to any requests made by AWS Security Agent during penetration testing.

  1. Expand Advanced network access, then locate the Custom HTTP headers section.

  2. In the Custom HTTP headers input field, enter a header name and value that will be associated with outbound requests.

    Note

    By default, AWS Security Agent adds a custom header for User-Agent set to securityagent unless a different custom User-Agent header value is specified.

  3. To add multiple custom headers:

    1. Choose Add header.

    2. Enter each additional custom header.

  4. To remove a custom header, choose Remove next to the header.

Configure IAM Role

Select the pre-configured service role for this penetration test. AWS Security Agent uses an Agent Space-based permission model where administrators configure IAM roles when setting up your Agent Space. You select from roles that are already configured and ready to use.

  1. In the Permissions section, locate the Service roles dropdown.

  2. Select the IAM role that grants AWS Security Agent access to required AWS resources.

    Important

    The selected IAM role must have permissions to access VPC resources, CloudWatch Logs, and any other AWS services needed for the penetration test. Verify that the role has the correct trust relationship with AWS Security Agent.

  3. Locate the CloudWatch log group dropdown.

  4. Select the log group where penetration test logs will be stored. (optional)

    Note

    The selected CloudWatch log group will store detailed logs of the penetration test execution, including requests made, responses received, and vulnerabilities discovered.

    If you don’t select a log group, a new CloudWatch log group will be automatically created with the /aws/securityagent prefix to store the penetration test logs.

Automatic code remediation

Select the Enable automatic remediation checkbox.

Important

To remediate security findings in your source code repositories, AWS Security Agent may submit pull requests to your repositories. The pull requests may be visible to all users who have read access to the repositories.

Configure VPC resources (optional)

If your target domains are private and hosted within a VPC, configure the VPC settings where AWS Security Agent should run penetration tests. This step is only necessary for applications that are not publicly accessible.

Note

Skip this step if your target domains are publicly accessible. VPC configuration is only required for testing private applications hosted within an Amazon Virtual Private Cloud.

Choose the VPC, subnets, and security groups for the penetration test environment.

  1. In the VPC section, locate the VPC ID dropdown.

  2. Select the VPC where your target domains are hosted.

    Important

    The selected VPC must contain the target domains you specified in Step 3. Ensure the VPC has appropriate routing and network configuration to allow AWS Security Agent to access your applications.

  3. Locate the Subnets dropdown.

  4. Select one or more subnets where the penetration test should run.

    Note

    Choose subnets that have network access to your target applications. The penetration test will execute from resources deployed in these subnets.

  5. Locate the Security group dropdown.

  6. Select the security group that controls network access for the penetration test.

    Important

    The selected security group must allow outbound traffic to your target domains and any accessible domains. Ensure the security group rules permit the necessary network access for comprehensive testing.

Configure authentication credentials (optional)

If your target domains require authentication, provide credentials to allow AWS Security Agent to access protected areas of your application during penetration testing. This step is only necessary for applications that require user authentication.

Note

Skip this step if your target domains do not require authentication or if all areas you want tested are publicly accessible. Configure credentials only when you need AWS Security Agent to test authenticated sections of your application.

Credentials appear in a table on the Credentials step. The table lists each credential’s name, test status, and authentication type.

Important

When AWS Security Agent signs in with your credentials, it tests everything your application exposes to that signed-in user. Authenticated paths can reach shared services and third-party domains that you are not authorized to test. Only provide credentials for applications you are authorized to test.

Test against a non-production environment, and do not provide credentials that have access to production systems. For more information, see Use non-production environments for penetration testing.

Add credentials

Provide authentication credentials that AWS Security Agent will use to access your application.

  1. On the Credentials step, choose Add credential.

  2. In the Credential name field, enter a name that is unique within this penetration test. AWS Security Agent uses this name to report the credential’s test results and findings.

  3. Select a credential input method:

    • Input credentials - Enter your credentials directly into AWS Security Agent.

    • Advanced setting - For sensitive credential information, use advanced options such as AWS Secrets Manager or AWS Lambda functions. See Provide authentication credentials for penetration testing for details.

      Tip

      For production environments or sensitive credentials, we recommend using the advanced setting option to securely reference credentials stored in AWS Secrets Manager or Systems Manager Parameter Store.

  4. For Input credentials, enter the Username and Password for the authenticated account. Use an account whose access matches a typical user rather than an administrator.

  5. In the Access URL dropdown, select the URL where these credentials will be used. You can select only a URL that is already in Target URLs. To use a different URL, add it as a target URL first.

  6. Under Agent Space login prompt, enter instructions that describe how to sign in with these credentials. A login prompt is required for each credential, because it tells AWS Security Agent how to reach your login, complete it, and confirm that it worked. To start from a worked example, choose one of the buttons below the field, such as Website login or API request, then edit the inserted text. Choosing an example replaces whatever is in the field.

  7. Choose Save, or choose Save and test to save the credential and immediately test it.

To change a credential later, choose the edit icon in its row. To delete one, choose the remove icon.

Test credentials

Test a credential to confirm that AWS Security Agent can sign in with it before the penetration test runs. Testing also discovers the domains your application reaches while signed in. AWS Security Agent then suggests those domains on the Network configuration step.

  1. In the credential’s row, choose Test. To test every credential at once, choose Test all credentials.

  2. Watch the Status column. A credential moves through Starting and In progress to one of these results:

    • Success – AWS Security Agent signed in with the credential.

    • Failed – Sign-in did not succeed. Choose the status to see the reason.

    • Timed out – AWS Security Agent could not complete the sign-in in time. A test that cannot sign in stops after 15 minutes.

    • Unverified – The credential has not been tested, or you changed it after its last test.

  3. To see what the agent did during a test, choose View logs in the credential’s row.

Most sign-in tests finish in 2 to 3 minutes. A test keeps running if you leave the Credentials step, so you can continue through the wizard while it finishes.

If a credential fails, check that the username and password are correct and that the login prompt describes the steps to reach and complete your login. Then edit the credential and test it again.

A timeout here does not stop you from starting the penetration test. During the penetration test, the agent gets an hour to sign in, four times what it gets for a credential test. A sign-in that was only slow often succeeds with that extra time.

When you leave the Credentials step with credentials that have not been tested, AWS Security Agent asks whether to test them. Choose Test credentials to test the remaining ones, or Skip testing to continue without testing. An untested credential discovers nothing, so any domain reachable only behind it is missing from the suggestions on the Network configuration step and you must add it yourself.

Add multiple credentials (optional)

If your application requires multiple sets of credentials or different domains need separate authentication, add additional credential sets.

  1. Choose Add credential again for each additional credential.

  2. Repeat the credential configuration steps for each one.

  3. To remove a credential, choose the remove icon in its row.

Tip

Configure multiple credentials when testing different user roles, accessing multiple authenticated domains, or verifying role-based access controls in your application.

Review network scope

On the Network configuration step, classify every domain the penetration test may reach. If you tested credentials on the previous step, the domains your application reached while signed in are already listed for you and marked Suggested. Review each one, then add anything the tests did not reach.

Every domain has one of three classifications:

  • Target – A URL the penetration test is allowed to attack. Target URLs come from the Penetration test details step and are marked with a Target badge. To change one, go back to that step.

  • Accessible – A URL the penetration test can access but must not attack. Use this for a hosted login page, a content delivery network (CDN), or another supporting service your application depends on.

  • Out of scope – A URL the penetration test must not access at all.

By default, only your target URLs are reachable. Add an accessible entry for anything else your application needs. An entry can be a whole domain, such as auth.example.com, or a specific path, such as https://example.com/callback. An out-of-scope entry always takes precedence over an accessible one.

Accessible domains do not require ownership verification, even if they belong to a different domain than your target. Only target domains require verified ownership. AWS Security Agent does not penetration test accessible domains; it uses them only for login and navigation.

Non-standard ports for accessible domains

To make a non-standard port accessible, enter a full URL that includes the scheme and the port. For example, https://auth.example.com:8443. If you enter a domain without a scheme, such as auth.example.com:8443, AWS Security Agent allows the domain only on standard ports.

Classify a domain

  1. On the Network configuration step, review the Network scope table.

  2. For each domain, select Accessible or Out of scope.

  3. To add a domain the tests did not discover, choose Add domain, then enter the URL and select its classification.

  4. To remove a domain you added, choose the remove icon in its row. You cannot remove a target URL here.

Warning

Mark as out of scope any path that should not be accessed during testing, such as a destructive operation or a sensitive administrative function. AWS Security Agent excludes the specified path and all paths nested beneath it. For example, if you mark https://example.com/admin as out of scope, https://example.com/admin/tools is also out of scope.

To review the same domains grouped by classification instead of in a table, choose Grouped in Domain view. In grouped view, choose Edit to make changes, then choose Save.

The Derived examples list at the bottom of the section shows endpoints that cannot receive traffic from the penetration test, based on how you classified each domain. Use it to confirm your classifications before you continue.

Attach additional resources (optional)

Provide supplementary resources to help AWS Security Agent conduct more thorough and accurate penetration testing. Additional resources can include architecture diagrams, API documentation, configuration files, GitHub repositories, or S3-hosted materials that give context about your application.

Note

Additional resources are optional but recommended. Providing comprehensive information about your application helps ensure thorough test coverage, reduces false positives, and delivers more actionable results.

Add resources to the penetration test

Select existing resources or upload new files that will help guide the penetration test.

  1. In the Connected resources section, you can:

    • Choose Select from available to choose from resources already connected to AWS Security Agent (such as GitHub repositories or S3 buckets).

    • Choose Upload to add new files directly from your local system.

Tip

Useful resources include API documentation, architecture diagrams, OpenAPI/Swagger specifications, configuration files, authentication flow diagrams, and any other materials that describe your application’s structure and behavior.

Select from available resources

Choose from resources that are already integrated with AWS Security Agent.

  1. Choose Select from existing resources.

  2. Browse the list of available resources from connected sources such as:

    • GitHub repositories, under the GitHub repositories tab

    • S3 buckets

    • Previously uploaded files

    • Documentation repositories

  3. Select the resources you want to include in the penetration test.

  4. Choose Add to penetration test to attach the selected resources.

Example

We recommend selecting and adding relevant GitHub repositories to your penetration test, so AWS Security Agent can develop an understanding of your application context, and generate ready-to-implement code fixes through pull requests (when enabled)

Tip

When you select a repository, you can specify a branch. By default, the service uses the primary branch. To use a different branch, enter the branch name in the field next to the repository.

Note

Resources selected from available sources remain synchronized with their original location. If you update a GitHub repository or S3 file, the penetration test will use the updated version.

Note

If you have a private VPC associated with your penetration test and a GitHub repository configured, ensure that GitHub is accessible through your private VPC for pulling GitHub resources. In most cases, you need to either ensure that outbound traffic through VPC NAT Gateway is allowed by default or configure specific rules to allow outbound traffic for GitHub IPs (see GitHub Meta API Endpoint)

Upload new resources

Upload files directly from your local system or provide plain text content to AWS Security Agent.

  1. Choose Upload.

  2. Choose one of the following input methods:

    • Upload local files - Select one or more files from your local system.

    • Paste plain text - Type or paste text content directly into the input field. Choose Upload.

  3. Then choose Add to complete uploading.

  4. The uploaded resources appear in the Connected resources table.

Tip

Use the plain text option when you want to quickly provide API endpoint lists, URL patterns, test instructions, or other text-based information without creating a separate file.

Important

Ensure uploaded files and pasted content do not contain sensitive information such as production credentials, private keys, or personally identifiable information (PII). Use sanitized versions of configuration files and documentation.

Connect existing resources

Existing resources can be from what you’ve previously uploaded to AWS Security Agent, from your S3 bucket, and your integrated GitHub repositories. Choose Select from existing resources to select them.

Manage connected resources

Review, organize, and remove resources attached to the penetration test.

The Connected resources table displays all resources included in the penetration test with the following information:

  • Name - The filename or resource identifier

  • Type - The resource category (Uploaded files, S3 resources, GitHub repositories, etc.)

To manage resources:

  1. Select one or more resources using the checkboxes.

  2. Choose Remove from penetration test to detach selected resources.

Note

You can sort the table by Name or Type by clicking the column headers. This helps organize resources when working with many files.

Set a maximum task-hours limit

Limit how much work AWS Security Agent performs for a penetration test by setting a maximum number of task hours. Task hours measure the cumulative time the agent spends actively working on the test, including time across parallel testing. Task hours are not the same as elapsed wall-clock time, because AWS Security Agent runs multiple tasks at once. Total task hours is also the unit that AWS Security Agent bills for a run, so setting a limit caps how much a test can cost. The smallest limit you can set is 20 hours.

  1. (Optional) In the Max task hours section, choose a limit:

    • Choose a preset value, such as 20 or 30 hours.

    • Choose No limit to run the test to completion without a task-hours limit.

    • Choose Custom to enter your own value. A custom limit must be at least 20 hours.

Tip

For larger applications, or to get the most complete results, set a higher limit such as 30 hours or more. AWS Security Agent bills only for the task hours a run actually uses, so a higher limit does not increase cost unless the test needs the extra time.

Note

When a test reaches its maximum task hours, AWS Security Agent stops working on the test and keeps the findings already discovered. The run finishes with a status of Completed. You can review these findings or create and run a new test.

Exclude risk types (optional)

Choose specific risk categories to exclude from testing if they’re not applicable to your application.

  1. On the Additional configuration step, locate the Exclude risk types field.

  2. Choose the dropdown to view available risk types.

  3. Select one or more risk types to exclude from the penetration test.

    Note

    Excluding risk types limits the scope of testing. Only exclude risk types that are not relevant to your application or that you want to test separately.

Create the penetration test

Finalize and launch your penetration test configuration.

After configuring all settings, you’re ready to create the penetration test.

  1. Review all configuration sections to ensure accuracy.

  2. Choose one of the following options:

    • Choose Create penetration to save the configuration without running it immediately.

    • Choose Create and execute to save the configuration and immediately start the penetration test.

    • Choose Cancel to discard the penetration test configuration.

Important

Before running a penetration test, verify that:

  • All target domains are correctly verified and accessible

  • IAM roles have appropriate permissions

  • Out-of-scope paths are properly configured to prevent testing destructive operations

  • You have authorization to perform security testing on all target domains

Note

After the penetration test starts, you can monitor its progress from the Penetration test runs section. The test can take several hours to complete, depending on the scope and complexity of your application. If you set a maximum task-hours limit, AWS Security Agent stops the test when it reaches that limit. AWS Security Agent preserves the findings discovered so far.