View AWS Network Firewall logs and metrics by using Splunk - AWS Prescriptive Guidance
SummaryPrerequisites and limitationsArchitectureToolsEpicsRelated resources

View AWS Network Firewall logs and metrics by using Splunk

Created by Ivo Pinto

Summary

Many organizations use Splunk Enterprise as a centralized aggregation and visualization tool for logs and metrics from different sources. This pattern helps you configure Splunk to fetch AWS Network Firewall logs and metrics from Amazon CloudWatch Logs by using the Splunk Add-On for AWS. 

To achieve this, you create a read-only AWS Identity and Access Management (IAM) role. Splunk Add-On for AWS uses this role to access CloudWatch. You configure the Splunk Add-On for AWS to fetch metrics and logs from CloudWatch. Finally, you create visualizations in Splunk from the retrieved log data and metrics.

Prerequisites and limitations

Prerequisites

  • A Splunk account

  • A Splunk Enterprise instance, version 8.2.2 or later 

  • An active AWS account

  • Network Firewall, set up and configured to send logs to CloudWatch Logs

Limitations

  • Splunk Enterprise must be deployed as a cluster of Amazon Elastic Compute Cloud (Amazon EC2) instances in the AWS Cloud.

  • Collecting data by using an automatically discovered IAM role for Amazon EC2 is not supported in the AWS China Regions.

Architecture

AWS Network Firewall and Splunk logging architecture

The diagram illustrates the following:

  1. Network Firewall publishes logs to CloudWatch Logs.

  2. Splunk Enterprise retrieves metrics and logs from CloudWatch.

To populate example metrics and logs in this architecture, a workload generates traffic that passes through the Network Firewall endpoint to go to the internet. This is achieved by the use of route tables. Although this pattern uses a single Amazon EC2 instance as the workload, this pattern can apply to any architecture as long as Network Firewall is configured to send logs to CloudWatch Logs.

This architecture also uses a Splunk Enterprise instance in another virtual private cloud (VPC). However, the Splunk instance can be in another location, such as in the same VPC as the workload, as long as it can reach the CloudWatch APIs.

Tools

AWS services

  • Amazon CloudWatch Logs helps you centralize the logs from all your systems, applications, and AWS services so you can monitor them and archive them securely.

  • Amazon Elastic Compute Cloud (Amazon EC2) provides scalable computing capacity in the AWS Cloud. You can launch as many virtual servers as you need and quickly scale them up or down.

  • AWS Network Firewall is a stateful, managed, network firewall and intrusion detection and prevention service for VPCs in the AWS Cloud.

Other tools

  • Splunk helps you monitor, visualize, and analyze log data.

Epics

TaskDescriptionSkills required

Create the IAM policy.

Follow the instructions in Creating policies using the JSON editor to create the IAM policy that grants read-only access to the CloudWatch Logs data and CloudWatch metrics. Paste the following policy into the JSON editor.

{
    "Statement": [
        {
            "Action": [
                "cloudwatch:List*",
                "cloudwatch:Get*",
                "network-firewall:List*",
                "logs:Describe*",
                "logs:Get*",
                "logs:List*",
                "logs:StartQuery",
                "logs:StopQuery",
                "logs:TestMetricFilter",
                "logs:FilterLogEvents",
                "network-firewall:Describe*"
            ],
            "Effect": "Allow",
            "Resource": "*"
        }
    ],
    "Version": "2012-10-17"
}
AWS administrator

Create a new IAM role.

Follow the instructions in Creating a role to delegate permissions to an AWS service to create the IAM role that the Splunk Add-On for AWS uses to access CloudWatch. For Permissions policies, choose the policy that you created previously.

AWS administrator

Assign the IAM role to the EC2 instances in the Splunk cluster.

  1. Open the Amazon EC2 console at https://console.aws.amazon.com/ec2/.

  2. In the navigation pane, choose Instances.

  3. Select the EC2 instances in the Splunk cluster.

  4. Choose Actions, Security, and then Modify IAM role.

  5. Select the IAM role that you created previously, and then choose Save.

AWS administrator
TaskDescriptionSkills required

Install the add-on.

  1. In the Splunk dashboard, navigate to Splunk Apps.

  2. Search for Splunk Add-on for Amazon Web Services.

  3. Choose Install.

  4. Provide your Splunk credentials.

Splunk administrator

Configure the AWS credentials.

  1. In the Splunk dashboard, navigate to Splunk Add-on for AWS.

  2. Choose Configuration.

  3. In the Autodiscovered IAM Role column, select the IAM role that you created previously.

For more information, see Find an IAM role within your Splunk platform instance in the Splunk documentation.

Splunk administrator
TaskDescriptionSkills required

Configure the retrieval of Network Firewall logs from CloudWatch Logs.

  1. In the Splunk dashboard, navigate to Splunk Add-on for AWS.

  2. Choose Input.

  3. Choose Create New Input.

  4. In the list, choose Custom Data Type, and then choose CloudWatch Logs.

  5. Provide the Name, AWS Account, AWS Region, and Log Group for your Network Firewall logs.

  6. Choose Save.

By default, Splunk fetches the log data every 10 minutes. This is a configurable parameter under Advanced Settings. For more information, see Configure a CloudWatch Logs input using Splunk Web in the Splunk documentation.

Splunk administrator

Configure the retrieval of Network Firewall metrics from CloudWatch.

  1. In the Splunk dashboard, navigate to Splunk Add-on for AWS.

  2. Choose Input.

  3. Choose Create New Input.

  4. In the list, choose CloudWatch.

  5. Provide the Name, AWS Account, and AWS Region for your Network Firewall metrics.

  6. Next to Metric Configuration, choose Edit in advanced mode.

  7. (Optional) Delete all preconfigured namespaces. 

  8. Choose Add Namespace, and then name it AWS/NetworkFirewall.

  9. In Dimension Value, add the following.

    [{"AvailabilityZone":[".*"],"Engine":[".*"],"FirewallName":[".*"]}]

  10. For Metrics, choose All.

  11. For Metric Statistics, choose Sum.

  12. Choose OK.

  13. Choose Save.

By default, Splunk fetches the metric data every 5 minutes. This is a configurable parameter under Advanced Settings. For more information, see Configure a CloudWatch input using Splunk Web in the Splunk documentation.

Splunk administrator
TaskDescriptionSkills required

View the top source IP addresses.

  1. In the Splunk dashboard, navigate to Search & Reporting.

  2. In the enter search here box, enter the following.

    sourcetype="aws:cloudwatchlogs" | top event.src_ip

    This query displays a table of the source IP addresses with the most traffic, in descending order.

  3. For a graphical representation, choose Visualization.

Splunk administrator

View packet statistics.

  1. In the Splunk dashboard, navigate to Search & Reporting.

  2. In the enter search here box, enter the following.

    sourcetype="aws:cloudwatch"| timechart sum(Sum) by metric_name

    This query displays a table of the metrics DroppedPackets, PassedPackets, and ReceivedPackets per minute.

  3. For a graphical representation, choose Visualization.

Splunk administrator

View the most-used source ports.

  1. In the Splunk dashboard, navigate to Search & Reporting.

  2. In the enter search here box, enter the following.

    sourcetype="aws:cloudwatchlogs" | top event.dest_port

    This query displays a table of the source ports with the most traffic, in descending order.

  3. For a graphical representation, choose Visualization.

Splunk administrator

Related resources

AWS documentation

AWS blog posts

AWS Marketplace