Tutorial: Gate an Azure DevOps deployment with a scoped penetration test

This tutorial walks through a complete, reproducible setup that runs a continuous, scoped AWS Security Agent penetration test as a post-deployment gate in Azure DevOps Pipelines. You deploy an intentionally vulnerable sample application to a staging environment, then let AWS Security Agent test the deployed change and block promotion to production when it finds a vulnerability.

For the concepts behind the integration — how the test is scoped to the deployed change, how OIDC authentication works, and how gating decisions are made — see Run penetration tests from your CI/CD pipeline. This tutorial applies those concepts to a concrete Azure DevOps project.

Prerequisites

Before you start, make sure you have the following:

Throughout this tutorial, replace the following placeholders with your own values:

Step 1: Create the vulnerable sample application

Create a small Flask application with one endpoint that builds a SQL query by string concatenation — a classic SQL injection. In your repository, add app.py:

import sqlite3
from flask import Flask, request

app = Flask(__name__)


@app.route("/products")
def products():
    # Intentionally vulnerable: user input is concatenated directly into SQL.
    category = request.args.get("category", "")
    conn = sqlite3.connect("products.db")
    cursor = conn.cursor()
    query = "SELECT name, price FROM products WHERE category = '" + category + "'"
    rows = cursor.execute(query).fetchall()
    conn.close()
    return {"products": rows}


if __name__ == "__main__":
    app.run(host="0.0.0.0", port=8080)

Add a deploy.sh script that deploys this application to your staging environment. The details depend on your infrastructure; the gate does not require any particular deployment mechanism, only that the application is running and reachable when the test runs.

Step 2: Set up the IAM role for Azure DevOps OIDC

Azure DevOps authenticates to AWS through an AWS service connection that uses workload identity federation — no long-lived access keys. Azure DevOps issues a signed OIDC token for the pipeline, and AWS STS exchanges it for temporary credentials through an IAM role you create.

Step 3: Create the AWS service connection

In Azure DevOps, go to Project settings, Service connections, and create a new AWS service connection provided by the AWS Toolkit. Choose workload identity federation (OIDC) rather than access keys, and set:

Name the connection (for example, aws-pentest-gate). Azure DevOps generates the issuer and subject values that you used in the trust policy in Step 2; if you created the role before the connection, update the trust policy’s subject to match the value the connection shows.

Step 4: Add the pipeline task

Edit your pipeline YAML. Run the deploy stage, then a gate stage that depends on it. In the gate stage, run the AWS Toolkit task for the integration, referencing your service connection.

trigger:
  branches:
    include: [main]

stages:
  - stage: DeployStaging
    jobs:
      - job: deploy
        pool:
          vmImage: ubuntu-latest
        steps:
          - checkout: self
            persistCredentials: false
          - script: ./deploy.sh staging
            displayName: Deploy to staging

  - stage: SecurityGate
    dependsOn: DeployStaging
    jobs:
      - job: pentest
        pool:
          vmImage: ubuntu-latest
        steps:
          - task: RunPentest@1
            inputs:
              awsCredentials: aws-pentest-gate
              regionName: us-west-2
              agentSpaceId: "as-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
              pentestId: "pt-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
              severityThreshold: HIGH
              onScopeConflict: block
              timeoutMinutes: "45"

  - stage: DeployProd
    dependsOn: SecurityGate
    jobs:
      - job: deploy_prod
        pool:
          vmImage: ubuntu-latest
        steps:
          - checkout: self
            persistCredentials: false
          - script: ./deploy.sh production
            displayName: Deploy to production

Because DeployProd declares dependsOn: SecurityGate and a failed task fails its stage, the production stage does not run when the gate fails. The awsCredentials input references the service connection you created in Step 3, which supplies temporary credentials through workload identity federation.

Step 5: Validate your setup with a dry run

Before you rely on the gate, run it once in dry-run mode. Add the dry-run input to the task and run the pipeline:

          - task: RunPentest@1
            inputs:
              awsCredentials: aws-pentest-gate
              regionName: us-west-2
              agentSpaceId: "as-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
              pentestId: "pt-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
              dryRun: "true"

The dry run checks AWS authentication, the Region, and that your repository resolves to its integration — without starting a billable penetration test job. It surfaces the common setup mistakes (a bad OIDC trust policy, a Region mismatch, or a repository that is not connected) in seconds. When it passes, remove the dry-run input and continue. For more information, see Validate your setup before you trust the gate.

Step 6: Trigger the gate and read the result

Push a commit to main that changes the vulnerable endpoint. This triggers the pipeline: the DeployStaging stage deploys the change, then the SecurityGate stage starts a penetration test scoped to that change.

Open the pipeline run and watch the gate task:

A scoped penetration test typically runs for tens of minutes, so the task runs for a while. That is expected, not a hang.

Because the deployed change introduced a SQL injection vulnerability, AWS Security Agent returns an in scope decision with a HIGH (or higher) finding. The gate task fails, the SecurityGate stage fails, and any later production stage that depends on it does not run.

The run’s Extensions tab records the verdict: the scope decision, the severity threshold the run was evaluated against, the finding counts, and whether the gate blocked.

For the full gating behavior — including how authentication failures always block and how you choose fail-closed or fail-open for infrastructure errors — see Choose how the pipeline responds.

Step 7: Fix the finding and confirm the gate passes

Fix the vulnerability by using a parameterized query instead of string concatenation. In app.py, replace the query construction:

    category = request.args.get("category", "")
    conn = sqlite3.connect("products.db")
    cursor = conn.cursor()
    query = "SELECT name, price FROM products WHERE category = ?"
    rows = cursor.execute(query, (category,)).fetchall()
    conn.close()
    return {"products": rows}

Push the fix to main. The pipeline redeploys the change and runs the gate again against the new commit. With the injection removed, AWS Security Agent finds no findings at or above HIGH, the gate task passes, and the production stage proceeds.

The Extensions tab shows the same checks for the new commit, with no findings at or above the threshold and a passing verdict.

Clean up

To avoid ongoing cost and to remove the intentionally vulnerable application:

Related topics