# Migrating from X-Ray instrumentation to OpenTelemetry instrumentation
**Note**
X-Ray SDK/Daemon Maintenance Notice – On February 25th, 2026, the AWS X-Ray SDKs/Daemon will enter maintenance mode, where AWS will limit X-Ray SDK and Daemon releases to address security issues only. For more information on the support timeline, see [X-Ray SDK and Daemon Support timeline](xray-sdk-daemon-timeline.md).
X-Ray is transitioning to OpenTelemetry (OTel) as its primary instrumentation standard for application tracing and observability. This strategic shift aligns AWS with industry best practices and offers customers a more comprehensive, flexible, and future-ready solution for their observability needs. OpenTelemetry's wide adoption in the industry enables tracing of requests across diverse systems, including those outside AWS that may not directly integrate with X-Ray.
This chapter provides recommendations for a smooth transition, and emphasizes the importance of migrating to OpenTelemetry-based solutions to ensure continued support and access to the latest features in application instrumentation and observability.
It is recommended to adopt OpenTelemetry as the observability solution for instrumenting your application.
**Topics**
+ [Understanding OpenTelemetry](#migration-to-opentelemetry)
+ [Understanding OpenTelemetry concepts for migration](#opentelemetry-concepts)
+ [Migration overview](#migration-overview)
+ [Migrating from X-Ray Daemon to AWS CloudWatch agent or OpenTelemetry collector](#xray-Daemon-migration)
+ [Migrating to OpenTelemetry Java](xray-migration-opentelemetry.md)
+ [Migrate to OpenTelemetry Go](manual-instrumentation-go.md)
+ [Migrate to OpenTelemetry Node.js](migrate-xray-to-opentelemetry-nodejs.md)
+ [Migrate to OpenTelemetry .NET](introduction-dotnet.md)
+ [Migrate to OpenTelemetry Python](migrate-xray-to-opentelemetry-python.md)
+ [Migrate to OpenTelemetry Ruby](migrate-xray-to-opentelemetry-ruby.md)
## Understanding OpenTelemetry
OpenTelemetry is an industry-standard observability framework that provides standardized protocols and tools for collecting telemetry data. It offers a unified approach to instrumenting, generating, collecting, and exporting telemetry data such as metrics, logs, and traces.
When you migrate from X-Ray SDKs to OpenTelemetry, you get the following benefits:
+ Enhanced framework and library instrumentation support
+ Support for additional programming languages
+ Automatic instrumentation capabilities
+ Flexible sampling configuration options
+ Unified collection of metrics, logs, and traces
The OpenTelemetry collector provides more options for data collection formats and export destinations than the X-Ray daemon.
### OpenTelemetry support in AWS
AWS provides multiple solutions for working with OpenTelemetry:
+ AWS Distro for OpenTelemetry
Export OpenTelemetry traces as segments to X-Ray.
For more information, see [AWS Distro for OpenTelemetry](https://aws-otel.github.io/).
+ CloudWatch Application Signals
Export customized OpenTelemetry traces and metrics to monitor application health.
For more information, see [Working with Application Signals](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-Application-Monitoring-Sections.html).
+ CloudWatch OTel Endpoint
Export OpenTelemetry traces to X-Ray using the HTTP OTel endpoint with native OpenTelemetry instrumentation.
For more information, see [Using OTel endpoints](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-OTLPEndpoint.html).
#### Using OpenTelemetry with AWS CloudWatch
AWS CloudWatch supports OpenTelemetry traces through client-side application instrumentation and native AWS CloudWatch services such as Application Signals, Trace, Map, Metrics, and Logs. For more information, see [OpenTelemetry](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-OpenTelemetry-Sections.html).
## Understanding OpenTelemetry concepts for migration
The following table maps X-Ray concepts to their OpenTelemetry equivalents. Understanding these mappings helps you translate your existing X-Ray instrumentation to OpenTelemetry:
| X-Ray concept | OpenTelemetry concept |
| --- | --- |
| X-Ray Recorder | Tracer Provider and Tracers |
| Service Plugins | Resource Detector |
| Segment | (Server) Span |
| Sub-segment | (non-Server) Span |
| X-Ray Sampling Rules | OpenTelemetry Sampling (Customizable) |
| X-Ray Emitter | Span Exporter (Customizable) |
| Annotations/Metadata | Attributes |
| Library Instrumentation | Library Instrumentation |
| X-Ray Trace Context | Span Context |
| X-Ray Trace Context Propagation | W3C Trace Context Propagation |
| X-Ray Trace Sampling | OpenTelemetry Trace Sampling |
| N/A | Span Processing |
| N/A | Baggage |
| X-Ray Daemon | OpenTelemetry Collector |
**Note**
For more information about OpenTelemetry concepts, see the [OpenTelemetry documentation](https://opentelemetry.io/docs).
### Comparing features
The following table shows which features are supported in both services. Use this information to identify any gaps you need to address during migration:
| Feature | X-Ray instrumentation | OpenTelemetry instrumentation |
| --- | --- | --- |
| Library instrumentation | Supported | Supported |
| X-Ray sampling | Supported | Supported in OTel Java/.NET/Go Supported in ADOT Java/.NET/Python/Node.js |
| X-Ray trace context propagation | Supported | Supported |
| Resource detection | Supported | Supported |
| Segment annotations | Supported | Supported |
| Segment metadata | Supported | Supported |
| Zero-code auto-instrumentation | Supported in Java | Supported in OTel Java/.NET/Python/Node.js Supported in ADOT Java/.NET/Python/Node.js |
| Manually trace creation | Supported | Supported |
### Setting up and configuring tracing
To create traces in OpenTelemetry, you need a tracer. You get a tracer by initializing a *Tracer Provider* in your application. This is similar to how you use the X-Ray Recorder to configure X-Ray and create segments and subsegments in an X-Ray trace.
**Note**
The OpenTelemetry *Tracer Provider* offers more configuration options than the X-Ray Recorder.
#### Understanding trace data structure
After understanding the basic concepts and feature mappings, you can learn about specific implementation details like trace data structure and sampling.
OpenTelemetry uses *spans* instead of segments and subsegments to structure trace data. Each span includes the following components:
+ Name
+ Unique ID
+ Start and end timestamps
+ Span kind
+ Span context
+ Attributes (key-value metadata)
+ Events (timestamped logs)
+ Links to other spans
+ Status information
+ Parent span references
When you migrate to OpenTelemetry, your spans are automatically converted to X-Ray segments or subsegments. This ensures your existing CloudWatch console experience remains unchanged.
##### Working with span attributes
The X-Ray SDK provides two ways to add data to segments and subsegments:
Annotations
Key-value pairs that are indexed for filtering and searching
Metadata
Key-value pairs containing complex data that isn't indexed for searching
By default, OpenTelemetry span attributes are converted to metadata in X-Ray raw data. To convert specific attributes to annotations instead, add their keys to the `aws.xray.annotations` attributes list.
+ For more information about OpenTelemetry concepts, see [OpenTelemetry Traces](https://opentelemetry.io/docs/concepts/signals/traces/)
+ For details about how OpenTelemetry data maps to X-Ray data, see [OpenTelemetry to X-Ray data model translation](https://aws-otel.github.io/docs/getting-started/X-Ray#otel-to-X-Ray-data-model-translation-behavior-of-aws-X-Ray-exporter)
### Detecting resources in your environment
OpenTelemetry uses *Resource Detectors* to collect metadata about the resources that generate telemetry data. This metadata is stored as *Resource Attributes*. For example, an entity producing telemetry could be an Amazon ECS cluster or an Amazon EC2 instance, and the Resource Attributes that can be recorded from these entities can include the Amazon ECS Cluster ARN or Amazon EC2 Instance ID.
+ For information about supported resource types, see [OpenTelemetry Resource Semantic Conventions](https://opentelemetry.io/docs/reference/specification/resource/semantic_conventions/)
+ For information about X-Ray service plugins, see [Configuring the X-Ray SDK](https://docs.aws.amazon.com/xray/latest/devguide/xray-sdk-python-configuration.html)
### Managing sampling strategies
Trace sampling helps you manage costs by collecting data from a representative subset of requests instead of all requests. Both OpenTelemetry and X-Ray support sampling, but implement it differently.
**Note**
Sampling fewer than 100% of traces reduces your observability costs while maintaining meaningful insights into your application's performance.
OpenTelemetry provides several built-in sampling strategies and lets you create custom ones. You can also configure an X-Ray *Remote Sampler* in some SDK languages to use X-Ray sampling rules with OpenTelemetry.
The additional sampling strategies from OpenTelemetry are:
+ Parent-based Sampling – Respects the parent span's sampling decision before applying additional sampling strategies
+ Trace ID Ratio Based Sampling – >Randomly samples a specified percentage of spans
+ Tail sampling – Applies sampling rules to complete traces in the OpenTelemetry Collector
+ Custom samplers – Implement your own sampling logic using the sampling interface
For information about X-Ray sampling rules, see [Sampling rules in the X-Ray console](https://docs.aws.amazon.com/xray/latest/devguide/xray-console-sampling.html)
For information about OpenTelemetry tail sampling, see [Tail sampling processor](https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/processor/tailsamplingprocessor)
### Managing trace context
X-Ray SDKs manage the Segment Context to correctly handle parent-child relationships between Segments and Subsegments in a trace. OpenTelemetry uses a similar mechanism to ensure that spans have the correct parent span. It stores and propagates tracing data throughout a request context. For example, when your application processes a request and creates a server span to represent that request, OpenTelemetry will store the server span in the OpenTelemetry Context so that when a child span is created, that child span can reference the span in the Context as its parent.
### Propagating trace context
Both X-Ray and OpenTelemetry use HTTP headers to propagate trace context across services. This allows you to link trace data generated by different services and maintain sampling decisions.
The X-Ray SDK automatically propagates trace context using the X-Ray trace header. When one service calls another, the trace header contains the context needed to maintain parent-child relationships between traces.
OpenTelemetry supports multiple trace header formats for context propagation, including:
+ W3C Trace Context (default)
+ X-Ray trace header
+ Other custom formats
**Note**
You can configure OpenTelemetry to use one or more header formats. For example, use the X-Ray Propagator to send trace context to AWS services that support X-Ray tracing.
Configure and use the X-Ray Propagator to enable tracing across AWS services. This allows you to propagate trace context to API Gateway endpoints and other services that support X-Ray.
+ For information about X-Ray trace headers, see [Tracing header](https://docs.aws.amazon.com/xray/latest/devguide/xray-concepts.html#xray-concepts-tracingheader) in the X-Ray Developer Guide
+ For information about OpenTelemetry context propagation, see [Context and Context Propagation](https://opentelemetry.io/docs/concepts/context-propagation/) in the OpenTelemetry documentation
### Using library instrumentation
Both X-Ray and OpenTelemetry provide library instrumentation that requires minimal code changes to add tracing to your applications.
X-Ray provides library instrumentation functionalities. This allows you to add pre-built X-Ray instrumentations with minimal application code changes. These instrumentations support specific libraries like the AWS SDK and HTTP Clients, as well as web frameworks like Spring Boot or Express.js.
OpenTelemetry's instrumentation libraries generate detailed spans for your libraries through library hooks or automatic code modification, requiring minimal code changes.
To determine if OpenTelemetry's Library Instrumentations supports your library, search for it in the OpenTelemetry Registry at [OpenTelemetry Registry](https://opentelemetry.io/ecosystem/registry/).
### Exporting traces
X-Ray and OpenTelemetry use different methods to export trace data.
#### X-Ray trace export
The X-Ray SDKs use an emitter to send trace data:
+ Sends segments and subsegments to the X-Ray Daemon
+ Uses UDP for non-blocking I/O
+ Configured by default in the SDK
#### OpenTelemetry trace export
OpenTelemetry uses configurable *Span Exporters* to send trace data:
+ Uses *http/protobuf* or *grpc* protocols
+ Exports spans to endpoints monitored by the OpenTelemetry Collector or CloudWatch Agent
+ Allows for custom exporter configurations
### Processing and forwarding traces
Both X-Ray and OpenTelemetry provide components to receive, process, and forward trace data.
#### X-Ray trace processing
The X-Ray Daemon handles trace processing:
+ Listens for UDP traffic from X-Ray SDKs
+ Batches segments and subsegments
+ Uploads batches to the X-Ray service
#### OpenTelemetry trace processing
The OpenTelemetry Collector handles trace processing:
+ Receives traces from instrumented services
+ Processes and optionally modifies trace data
+ Sends processed traces to various backends, including X-Ray
**Note**
The AWS CloudWatch Agent can also receive and send OpenTelemetry traces to X-Ray. For more information, see [Collect metrics and traces with OpenTelemetry](AmazonCloudWatch/latest/monitoring/CloudWatch-Agent-OpenTelemetry-metrics.html).
### Span processing (OpenTelemetry-specific concept)
OpenTelemetry uses Span Processors to modify spans as they're created:
+ Allows reading and modifying spans at creation or completion
+ Enables custom logic for span handling
### Baggage (OpenTelemetry-soecific concept)
OpenTelemetry's Baggage feature allows propagation of key-value data:
+ Enables passing arbitrary data alongside trace context
+ Useful for propagating application-specific information across service boundaries
For information about the OpenTelemetry Collector, see [OpenTelemetry Collector](https://opentelemetry.io/docs/collector/)
For information about X-Ray concepts, see [X-Ray concepts](https://docs.aws.amazon.com/xray/latest/devguide/xray-concepts.html) in the X-Ray Developer Guide
## Migration overview
This section provides an overview of the code changes required for migration. The list below are language-specific guidance and X-Ray Daemon migration steps.
**Important**
To fully migrate from X-Ray instrumentation to OpenTelemetry instrumentation, you need to:
Replace X-Ray SDK usage with an OpenTelemetry solution
Replace the X-Ray Daemon with the CloudWatch Agent or OpenTelemetry Collector (with X-Ray Exporter)
+ [Migrating to OpenTelemetry Java](xray-migration-opentelemetry.md)
+ [Migrate to OpenTelemetry Go](manual-instrumentation-go.md)
+ [Migrate to OpenTelemetry Node.js](migrate-xray-to-opentelemetry-nodejs.md)
+ [Migrate to OpenTelemetry .NET](introduction-dotnet.md)
+ [Migrate to OpenTelemetry Python](migrate-xray-to-opentelemetry-python.md)
+ [Migrate to OpenTelemetry Ruby](migrate-xray-to-opentelemetry-ruby.md)
### Recommendations for new and existing applications
For new and existing applications, it is recommended to use the following solutions to enable tracing in your applications:
Instrumentation
+ OpenTelemetry SDKs
+ AWS Distro for OpenTelemetry Instrumentation
Data Collection
+ OpenTelemetry Collector
+ CloudWatch Agent
After migrating to OpenTelemetry-based solutions, your CloudWatch experience will remain the same. You will still be able to view your traces in the same format in the CloudWatch console’s Traces and Trace Map pages, or retrieve your trace data through the [X-Ray APIs](https://docs.aws.amazon.com/xray/latest/devguide/xray-api.html).
### Tracing setup changes
You need to replace the X-Ray setup with an OpenTelemetry setup.
**Comparison of X-Ray and OpenTelemetry setup**
| Feature | X-Ray SDK | OpenTelemetry |
| --- | --- | --- |
| Default configurations | [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/xray/latest/devguide/xray-sdk-migration.html) | [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/xray/latest/devguide/xray-sdk-migration.html) |
| Manual configurations | [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/xray/latest/devguide/xray-sdk-migration.html) | [\[See the AWS documentation website for more details\]](http://docs.aws.amazon.com/xray/latest/devguide/xray-sdk-migration.html) |
### Library instrumentation changes
Update your code to use OpenTelemetry Library Instrumentation instead of X-Ray Library Instrumentation for AWS SDK, HTTP Clients, Web Frameworks, and other libraries. This generates OpenTelemetry Traces instead of X-Ray Traces.
**Note**
Code changes vary by language and library. Refer to the language-specific migration guides for detailed instructions.
### Lambda environment instrumentation changes
To use OpenTelemetry in your Lambda functions, choose one of these setup options:
1. Use an auto-instrumentation Lambda Layer:
+ (Recommended) [AWS Lambda Layer for OpenTelemetry](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-Application-Signals-Enable-LambdaMain.html)
**Note**
To use only tracing, set the Lambda environment variable `OTEL_AWS_APPLICATION_SIGNALS_ENABLED=false`.
+ [AWS managed Lambda Layer for ADOT](https://aws-otel.github.io/docs/getting-started/lambda)
1. Manually set up OpenTelemetry for your Lambda function:
+ Configure a Simple Span Processor with an X-Ray UDP Span Exporter
+ Set up an X-Ray Lambda propagator
### Manually creating trace data
Replace X-Ray segments and sub-segments with OpenTelemetry Spans:
+ Use an OpenTelemetry Tracer to create Spans
+ Add attributes to Spans (equivalent to X-Ray metadata and annotations)
**Important**
When sent to X-Ray:
Server Spans convert to X-Ray segments
Other Spans convert to X-Ray sub-segments
Attributes convert to metadata by default
To convert an attribute to an annotation, add its key to the `aws.xray.annotations` attribute list. For more information, see [Enable Customized X-Ray Annotations](https://aws-otel.github.io/docs/getting-started/x-ray#enable-the-customized-x-ray-annotations).
## Migrating from X-Ray Daemon to AWS CloudWatch agent or OpenTelemetry collector
You can use either the CloudWatch agent or OpenTelemetry collector to receive traces from your instrumented applications and send them to X-Ray.
**Note**
The CloudWatch agent version 1.300025.0 and later can collect OpenTelemetry traces. Using the CloudWatch agent instead of the X-Ray Daemon reduces the number of agents you need to manage. For more information, see [Collecting metrics, logs, and traces with the CloudWatch agent](AmazonCloudWatch/latest/monitoring/Install-CloudWatch-Agent.html).
**Topics**
+ [Migrating on Amazon EC2 or on-premises servers](#ec2-onprem-migration)
+ [Migrating on Amazon ECS](#ecs-migration)
+ [Migrating on Elastic Beanstalk](#beanstalk-migration)
### Migrating on Amazon EC2 or on-premises servers
**Important**
Stop the X-Ray Daemon process before using the CloudWatch agent or OpenTelemetry collector to prevent port conflicts.
#### Existing X-Ray Daemon setup
##### Installing the daemon
Your existing X-Ray Daemon usage was installed using one of these methods:
Manual installation
Download and run the executable file from the X-Ray daemon Amazon S3 bucket.
Automatic installation
Use this script to install the daemon when launching an instance:
```
#!/bin/bash
curl https://s3.us-east-2.amazonaws.com/aws-xray-assets.us-east-2/xray-daemon/aws-xray-daemon-3.x.rpm \
-o /home/ec2-user/xray.rpm
yum install -y /home/ec2-user/xray.rpm
```
##### Configuring the daemon
Your existing X-Ray Daemon usage was configured using either:
+ Command line arguments
+ Configuration file (`xray-daemon.yaml`)
**Example Using a configuration file**
```
./xray -c ~/xray-daemon.yaml
```
##### Running the daemon
Your existing X-Ray Daemon usage was started with the following command:
```
~/xray-daemon$ ./xray -o -n us-east-1
```
##### Removing the daemon
To remove the X-Ray Daemon from your Amazon EC2 instance:
1. Stop the daemon service:
```
systemctl stop xray
```
1. Delete the configuration file:
```
rm ~/path/to/xray-daemon.yaml
```
1. If configured, remove the log file:
**Note**
The log file location depends on your configuration:
Command line configuration: `/var/log/xray-daemon.log`
Configuration file: Check the `LogPath` setting
#### Setting up the CloudWatch agent
##### Installing the agent
For installation instructions, see [Installing the CloudWatch agent on an on-premises server](AmazonCloudWatch/latest/monitoring/install-CloudWatch-Agent-commandline-fleet.html#install-CloudWatch-Agent-iam_user-first).
##### Configuring the agent
1. Create a configuration file to enable trace collection. For more information, see [Creating the CloudWatch agent configuration file](AmazonCloudWatch/latest/monitoring/create-cloudwatch-agent-configuration-file.html).
1. Set up IAM permissions:
+ Attach an IAM role or specify credentials for the agent. For more information, see [Setting up IAM roles](AmazonCloudWatch/latest/monitoring/install-CloudWatch-Agent-commandline-fleet.html#install-CloudWatch-Agent-iam_permissions-first).
+ Make sure the role or credentials include the `xray:PutTraceSegments` permission.
##### Starting the agent
For instructions to start the agent, see [Starting the CloudWatch agent using the command line](AmazonCloudWatch/latest/monitoring/install-CloudWatch-Agent-commandline-fleet.html#start-CloudWatch-Agent-EC2-commands-fleet).
#### Setting up the OpenTelemetry collector
##### Installing the collector
Download and install the OpenTelemetry collector for your operating system. For instructions, see [Installing the collector](https://opentelemetry.io/docs/collector/installation/).
##### Configuring the collector
Configure the following components in your collector:
+ awsproxy extension
Required for X-Ray sampling
+ OTel receivers
Collects traces from your application
+ xray exporter
Sends traces to X-Ray
**Example Sample collector configuration — otel-collector-config.yaml**
```
extensions:
awsproxy:
endpoint: 127.0.0.1:2000
health_check:
receivers:
otlp:
protocols:
grpc:
endpoint: 127.0.0.1:4317
http:
endpoint: 127.0.0.1:4318
processors:
batch:
exporters:
awsxray:
region: 'us-east-1'
service:
pipelines:
traces:
receivers: [otlp]
exporters: [awsxray]
extensions: [awsproxy, health_check]
```
**Important**
Configure AWS credentials with the `xray:PutTraceSegments` permission. For more information, see [Specifying credentials](sdk-for-go/v1/developer-guide/configuring-sdk.html#specifying-credentials).
##### Starting the collector
Run the collector with your configuration file:
```
otelcol --config=otel-collector-config.yaml
```
### Migrating on Amazon ECS
**Important**
Your task role must have the `xray:PutTraceSegments` permission for any collector you use.
Stop any existing X-Ray Daemon container before running the CloudWatch agent or OpenTelemetry collector container on the same host to prevent port conflicts.
#### Using the CloudWatch agent
1. Get the Docker image from [Amazon ECR Public Gallery](https://gallery.ecr.aws/cloudwatch-agent/cloudwatch-agent).
1. Create a configuration file named `cw-agent-otel.json`:
```
{
"traces": {
"traces_collected": {
"xray": {
"tcp_proxy": {
"bind_address": "0.0.0.0:2000"
}
},
"otlp": {
"grpc_endpoint": "0.0.0.0:4317",
"http_endpoint": "0.0.0.0:4318"
}
}
}
}
```
1. Store the configuration in Systems Manager Parameter Store:
1. Open the [https://console.aws.amazon.com/systems-manager/](https://console.aws.amazon.com/systems-manager/)
1. Choose **Create parameter**
1. Enter the following values:
+ Name: `/ecs/cwagent/otel-config`
+ Tier: Standard
+ Type: String
+ Data type: Text
+ Value: [Paste the cw-agent-otel.json configuration here]
1. Create a task definition using bridge network mode:
In your task definition, the configuration depends on the networking mode that you use. Bridge networking is the default and can be used in your default VPC. In a bridge network, set the `OTEL_EXPORTER_OTLP_TRACES_ENDPOINT`environment variable to tell the OpenTelemetry SDK what the endpoint and port are for the CloudWatch agent. You should also create a link from your application container to the Collector container for traces to be sent from the OpenTelemetry SDK in your application to the Collector container.
**Example CloudWatch agent task definition**
```
{
"containerDefinitions": [
{
"name": "cwagent",
"image": "public.ecr.aws/cloudwatch-agent/cloudwatch-agent:latest",
"portMappings": [
{
"containerPort": 4318,
"hostPort": 4318,
"protocol": "tcp"
},
{
"containerPort": 4317,
"hostPort": 4317,
"protocol": "tcp"
},
{
"containerPort": 2000,
"hostPort": 2000,
"protocol": "tcp"
}
],
"secrets": [
{
"name": "CW_CONFIG_CONTENT",
"valueFrom": "/ecs/cwagent/otel-config"
}
]
},
{
"name": "application",
"image": "APPLICATION_IMAGE",
"links": ["cwagent"],
"environment": [
{
"name": "OTEL_EXPORTER_OTLP_TRACES_ENDPOINT",
"value": "http://cwagent:4318/v1/traces"
}
]
}
]
}
```
For more information, see [Deploying the CloudWatch agent to collect Amazon EC2 instance-level metrics on Amazon ECS ](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/deploy-container-insights-ECS-instancelevel.html).
#### Using the OpenTelemetry collector
1. Get the Docker image `otel/opentelemetry-collector-contrib` from [Docker Hub](https://hub.docker.com/r/otel/opentelemetry-collector-contrib).
1. Create a configuration file called `otel-collector-config.yaml` using the same content as shown in the **Amazon EC2 configuring the collector** section, but update the endpoints to use `0.0.0.0` instead of `127.0.0.1`.
1. To use this configuration in Amazon ECS, you can store the configuration in Systems Manager Parameter Store. First, go to Systems Manager Parameter Store console, and choose **Create new parameter **. Create a new parameter with the following information:
+ Name: /ecs/otel/config (this name will be referenced in the Task Definition for the Collector)
+ Tier: Standard
+ Type: String
+ Data type: Text
+ Value: [Paste the otel-collector-config.yaml configuration here]
1. Create a task definition to deploy the OpenTelemetry collector using the bridge network mode as an example.
In the task definition, the configuration depends on the networking mode that you use. Bridge networking is the default and can be used in your default VPC. In a bridge network, set the `OTEL_EXPORTER_OTLP_TRACES_ENDPOINT` environment variable to tell the OpenTelemetry SDK what the endpoint and port are for the OpenTelemetry Collector. You should also create a link from your application container to the Collector container for traces to be sent from the OpenTelemetry SDK in your application to the Collector container.
**Example OpenTelemetry collector task definition**
```
{
"containerDefinitions": [
{
"name": "otel-collector",
"image": "otel/opentelemetry-collector-contrib",
"portMappings": [
{
"containerPort": 2000,
"hostPort": 2000
},
{
"containerPort": 4317,
"hostPort": 4317
},
{
"containerPort": 4318,
"hostPort": 4318
}
],
"command": [
"--config",
"env:SSM_CONFIG"
],
"secrets": [
{
"name": "SSM_CONFIG",
"valueFrom": "/ecs/otel/config"
}
]
},
{
"name": "application",
"image": "APPLICATION_IMAGE",
"links": ["otel-collector"],
"environment": [
{
"name": "OTEL_EXPORTER_OTLP_TRACES_ENDPOINT",
"value": "http://otel-collector:4318/v1/traces"
}
]
}
]
}
```
### Migrating on Elastic Beanstalk
**Important**
Stop the X-Ray Daemon process before using the CloudWatch agent to prevent port conflicts.
Your existing X-Ray Daemon integration was turned on by using the Elastic Beanstalk console, or by configuring X-Ray Daemon in your application source code with a configuration file.
#### Using the CloudWatch agent
On the Amazon Linux 2 platform, configure the CloudWatch agent using an `.ebextensions` configuration file:
1. Create a directory named `.ebextensions` in your project root
1. Create a file named `cloudwatch.config` within the `.ebextensions` directory with the following content:
```
files:
"/opt/aws/amazon-cloudwatch-agent/etc/config.json":
mode: "0644"
owner: root
group: root
content: |
{
"traces": {
"traces_collected": {
"otlp": {
"grpc_endpoint": "12.0.0.1:4317",
"http_endpoint": "12.0.0.1:4318"
}
}
}
}
container_commands:
start_agent:
command: /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a append-config -c file:/opt/aws/amazon-cloudwatch-agent/etc/config.json -s
```
1. Include the `.ebextensions` directory in your application source bundle when you deploy
For more information about Elastic Beanstalk configuration files, see [Advanced environment customization with configuration files](elasticbeanstalk/latest/dg/ebextensions.html).
# Migrating to OpenTelemetry Java
This section provides guidance on migrating from the X-Ray SDK to the OpenTelemetry SDK for Java applications.
**Topics**
+ [Zero code automatic instrumentation solution](#xray-migration-zero-code)
+ [Manual instrumentation solutions with the SDK](#xray-migration-sdk)
+ [Tracing incoming requests (spring framework instrumentation)](#xray-migration-tracing-setup-otel)
+ [AWS SDK v2 instrumentation](#xray-migration-sdkv2)
+ [Instrumenting outgoing HTTP calls](#xray-migration-http)
+ [Instrumentation support for other libraries](#xray-migration-libraries)
+ [Manually creating trace data](#xray-migration-tracedata)
+ [Lambda instrumentation](#xray-migration-lambda)
## Zero code automatic instrumentation solution
------
#### [ With X-Ray Java agent ]
To enable the X-Ray Java agent, your application's JVM arguments were required to be modified.
```
-javaagent:/path-to-disco/disco-java-agent.jar=pluginPath=/path-to-disco/disco-plugins
```
------
#### [ With OpenTelemetry-based Java agent ]
To use OpenTelemetry-based Java agents.
+ Use the AWS Distro for OpenTelemetry (ADOT) Auto-Instrumentation Java agent for automatic instrumentation with the ADOT Java agent. For more information, see [Auto-Instrumentation for Traces and Metrics with the Java agent](https://aws-otel.github.io/docs/getting-started/java-sdk/auto-instr). If you only want tracing, disable the `OTEL_METRICS_EXPORTER=none ` environment variable. to export metrics from the Java agent.
(Optional) You can also enable CloudWatch Application Signals when automatically instrumenting your applications on AWS with the ADOT Java auto-instrumentation to monitor current application health and track long-term application performance. Application Signals provides an unified, application-centric view of your applications, services, and dependencies, and helps monitor and triage application health. For more information, see [Application Signals](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-Application-Monitoring-Sections.html).
+ Use the OpenTelemetry Java agent for automatic instrumentation. For more information, see [Zero-code instrumentation with the Java Agent](https://opentelemetry.io/docs/zero-code/java/agent/).
------
## Manual instrumentation solutions with the SDK
------
#### [ Tracing setup with X-Ray SDK ]
To instrument your code with the X-Ray SDK for Java, first, the `AWSXRay` class was required to be configured with service plug-ins and local sampling rules, then a provided recorder was used.
```
static {
AWSXRayRecorderBuilder builder = AWSXRayRecorderBuilder.standard().withPlugin(new EC2Plugin()).withPlugin(new ECSPlugin());
AWSXRay.setGlobalRecorder(builder.build());
}
```
------
#### [ Tracing setup with OpenTelemetry SDK ]
The following dependencies are required.
```
io.opentelemetry
opentelemetry-bom
1.49.0
pom
import
io.opentelemetry.instrumentation
opentelemetry-instrumentation-bom
2.15.0
pom
import
io.opentelemetry
opentelemetry-sdk
io.opentelemetry
opentelemetry-api
io.opentelemetry.semconv
opentelemetry-semconv
io.opentelemetry
opentelemetry-exporter-otlp
io.opentelemetry.contrib
opentelemetry-aws-xray
1.46.0
io.opentelemetry.contrib
opentelemetry-aws-xray-propagator
1.46.0-alpha
io.opentelemetry.contrib
opentelemetry-aws-resources
1.46.0-alpha
```
Configure the OpenTelemetry SDK by instantiating a `TracerProvider` and globally register an `OpenTelemetrySdk` object. Configure these components:
+ An OTLP Span Exporter (for example, OtlpGrpcSpanExporter) - Required for exporting traces to the CloudWatch agent or OpenTelemetry Collector
+ An AWS X-Ray Propagator – Required for propagating the Trace Context to AWS Services that are integrated with X-Ray
+ An AWS X-Ray Remote Sampler – Required if you need to sample requests using X-Ray Sampling Rules
+ Resource Detectors(for example, EcsResource or Ec2Resource) – Detect metadata of the host running your application
```
import io.opentelemetry.api.common.Attributes;
import io.opentelemetry.context.propagation.ContextPropagators;
import io.opentelemetry.contrib.aws.resource.Ec2Resource;
import io.opentelemetry.contrib.aws.resource.EcsResource;
import io.opentelemetry.contrib.awsxray.AwsXrayRemoteSampler;
import io.opentelemetry.contrib.awsxray.propagator.AwsXrayPropagator;
import io.opentelemetry.exporter.otlp.trace.OtlpGrpcSpanExporter;
import io.opentelemetry.sdk.OpenTelemetrySdk;
import io.opentelemetry.sdk.resources.Resource;
import io.opentelemetry.sdk.trace.SdkTracerProvider;
import io.opentelemetry.sdk.trace.export.BatchSpanProcessor;
import io.opentelemetry.sdk.trace.samplers.Sampler;
import static io.opentelemetry.semconv.ServiceAttributes.SERVICE_NAME;
// ...
private static final Resource otelResource =
Resource.create(Attributes.of(SERVICE_NAME, "YOUR_SERVICE_NAME"))
.merge(EcsResource.get())
.merge(Ec2Resource.get());
private static final SdkTracerProvider sdkTracerProvider =
SdkTracerProvider.builder()
.addSpanProcessor(BatchSpanProcessor.create(
OtlpGrpcSpanExporter.getDefault()
))
.addResource(otelResource)
.setSampler(Sampler.parentBased(
AwsXrayRemoteSampler.newBuilder(otelResource).build()
))
.build();
// Globally registering a TracerProvider makes it available throughout the application to create as many Tracers as needed.
private static final OpenTelemetrySdk openTelemetry =
OpenTelemetrySdk.builder()
.setTracerProvider(sdkTracerProvider)
.setPropagators(ContextPropagators.create(AwsXrayPropagator.getInstance()))
.buildAndRegisterGlobal();
```
------
## Tracing incoming requests (spring framework instrumentation)
------
#### [ With X-Ray SDK ]
For information on how to use the X-Ray SDK with the spring framework to instrument your application, see [AOP with Spring and the X-Ray SDK for Java](https://docs.aws.amazon.com/xray/latest/devguide/xray-sdk-java-aop-spring.html). To enable AOP in Spring, complete these steps.
1. [Configure Spring](https://docs.aws.amazon.com/xray/latest/devguide/xray-sdk-java-aop-spring.html#xray-sdk-java-aop-spring-configuration)
1. [Adding a tracing filter to your application ](https://docs.aws.amazon.com/xray/latest/devguide/xray-sdk-java-aop-spring.html#xray-sdk-java-aop-filters-spring)
1. [Annotate your code or implement an interface ](https://docs.aws.amazon.com/xray/latest/devguide/xray-sdk-java-aop-spring.html#xray-sdk-java-aop-annotate-or-implement)
1. [Activate X-Ray in your application](https://docs.aws.amazon.com/xray/latest/devguide/xray-sdk-java-aop-spring.html#xray-sdk-java-aop-activate-xray)
------
#### [ With OpenTelemetry SDK ]
OpenTelemetry provides instrumentation libraries to collect traces for incoming requests for Spring Boot applications. To enable Spring Boot instrumentation with minimal configuration, include the following dependency.
```
io.opentelemetry.instrumentation
opentelemetry-spring-boot-starter
```
For more information on how to enable and configure Spring Boot instrumentation for your OpenTelemetry setup, see OpenTelemetry's [Getting started](https://opentelemetry.io/docs/zero-code/java/spring-boot-starter/getting-started/).
------
#### [ Using OpenTelemetry-based Java agents ]
The default recommended method for instrumenting Spring Boot applications is by using the [OpenTelemetry Java agent](https://opentelemetry.io/docs/zero-code/java/agent/) with *bytecode* instrumentation, which also provides more out-of-the-box instrumentations and configurations when compared to directly using the SDK. For get started, see [Zero code automatic instrumentation solution](#xray-migration-zero-code).
------
## AWS SDK v2 instrumentation
------
#### [ With X-Ray SDK ]
The X-Ray SDK for Java can automatically instrument all AWS SDK v2 clients when you added the `aws-xray-recorder-sdk-aws-sdk-v2-instrumentor` sub-module in your build.
To instrument individual clients downstream client calls to AWS services with AWS SDK for Java 2.2 and later, the `aws-xray-recorder-sdk-aws-sdk-v2-instrumentor `module from your build configuration was excluded and the `aws-xray-recorder-sdk-aws-sdk-v2` module was included. Individual clients were instrumented by configuring them with a `TracingInterceptor`.
```
import com.amazonaws.xray.interceptors.TracingInterceptor;
import software.amazon.awssdk.core.client.config.ClientOverrideConfiguration
import software.amazon.awssdk.services.dynamodb.DynamoDbClient;
//...
public class MyModel {
private DynamoDbClient client = DynamoDbClient.builder()
.region(Region.US_WEST_2)
.overrideConfiguration(
ClientOverrideConfiguration.builder()
.addExecutionInterceptor(new TracingInterceptor())
.build()
)
.build();
//...
```
------
#### [ With OpenTelemetry SDK ]
To automatically instrument all AWS SDK clients, add the `opentelemetry-aws-sdk-2.2-autoconfigure` sub-module.
```
io.opentelemetry.instrumentation
opentelemetry-aws-sdk-2.2-autoconfigure
2.15.0-alpha
runtime
```
To instrument individual AWSSDK clients, add the `opentelemetry-aws-sdk-2.2` sub-module.
```
io.opentelemetry.instrumentation
opentelemetry-aws-sdk-2.2
2.15.0-alpha
compile
```
Then, register an interceptor when creating an AWS SDK Client.
```
import io.opentelemetry.instrumentation.awssdk.v2_2.AwsSdkTelemetry;
// ...
AwsSdkTelemetry telemetry = AwsSdkTelemetry.create(openTelemetry);
private final S3Client S3_CLIENT = S3Client.builder()
.overrideConfiguration(ClientOverrideConfiguration.builder()
.addExecutionInterceptor(telemetry.newExecutionInterceptor())
.build())
.build();
```
------
## Instrumenting outgoing HTTP calls
------
#### [ With X-Ray SDK ]
To instrument outgoing HTTP requests with X-Ray, the X-Ray SDK for Java’s version of the Apache HttpClient was required.
```
import com.amazonaws.xray.proxies.apache.http.HttpClientBuilder;
...
public String randomName() throws IOException {
CloseableHttpClient httpclient = HttpClientBuilder.create().build();
```
------
#### [ With OpenTelemetry SDK ]
Similarly to the X-Ray Java SDK, OpenTelemetry provides an `ApacheHttpClientTelemetry` class that has a builder method that allows creation of an instance of an the `HttpClientBuilder` to provide OpenTelemetry-based spans and context propagation for Apache HttpClient.
```
io.opentelemetry.instrumentation
opentelemetry-apache-httpclient-5.2
2.15.0-alpha
compile
```
The following is a code example from the [opentelemetry-java-instrumentation ](https://github.com/open-telemetry/opentelemetry-java-instrumentation/tree/main/instrumentation/apache-httpclient/apache-httpclient-5.2/library). The HTTP Client provided by newHttpClient() will generate traces for executed requests.
```
import io.opentelemetry.api.OpenTelemetry;
import io.opentelemetry.instrumentation.apachehttpclient.v5_2.ApacheHttpClientTelemetry;
import org.apache.hc.client5.http.classic.HttpClient;
import org.apache.hc.client5.http.impl.classic.HttpClientBuilder;
public class ApacheHttpClientConfiguration {
private OpenTelemetry openTelemetry;
public ApacheHttpClientConfiguration(OpenTelemetry openTelemetry) {
this.openTelemetry = openTelemetry;
}
// creates a new http client builder for constructing http clients with open telemetry instrumentation
public HttpClientBuilder createBuilder() {
return ApacheHttpClientTelemetry.builder(openTelemetry).build().newHttpClientBuilder();
}
// creates a new http client with open telemetry instrumentation
public HttpClient newHttpClient() {
return ApacheHttpClientTelemetry.builder(openTelemetry).build().newHttpClient();
}
}
```
------
## Instrumentation support for other libraries
Find the full list of supported Library instrumentations for OpenTelemetry Java in its respective instrumentation GitHub repository , under [Supported libraries, frameworks, application servers, and JVMs ](https://github.com/open-telemetry/opentelemetry-java-instrumentation/blob/main/docs/supported-libraries.md).
Alternatively, you can search the OpenTelemetry Registry to find out if OpenTelemetry supports instrumentation. To start searching, see [Registry](https://opentelemetry.io/ecosystem/registry/).
## Manually creating trace data
------
#### [ With X-Ray SDK ]
With the X-Ray SDK, the `beginSegment` and `beginSubsegment` methods are needed to manually create X-Ray segments and sub-segments.
```
Segment segment = xrayRecorder.beginSegment("ManualSegment");
segment.putAnnotation("annotationKey", "annotationValue");
segment.putMetadata("metadataKey", "metadataValue");
try {
Subsegment subsegment = xrayRecorder.beginSubsegment("ManualSubsegment");
subsegment.putAnnotation("key", "value");
// Do something here
} catch (Exception e) {
subsegment.addException(e);
} finally {
xrayRecorder.endSegment();
}
```
------
#### [ With OpenTelemetry SDK ]
You can use custom spans to monitor the performance of internal activities that are not captured by instrumentation libraries. Note that only span kind server are converted into X-Ray segments, all other spans are converted into X-Ray sub-segments.
First, you will need to create a *Tracer* in order to generate spans, which you can obtain through the `openTelemetry.getTracer` method. This will provide a Tracer instance from the `TracerProvider` that was registered globally in the [Manual instrumentation solutions with the SDK](#xray-migration-sdk) example. You can create as many Tracer instances as needed, but it is common to have one Tracer for an entire application.
```
Tracer tracer = openTelemetry.getTracer("my-app");
```
You can use the Tracer to create spans.
```
import io.opentelemetry.api.common.AttributeKey;
import io.opentelemetry.api.trace.Span;
import io.opentelemetry.api.trace.SpanKind;
import io.opentelemetry.api.trace.Tracer;
import io.opentelemetry.context.Scope;
...
// SERVER span will become an X-Ray segment
Span span = tracer.spanBuilder("get-token")
.setKind(SpanKind.SERVER)
.setAttribute("key", "value")
.startSpan();
try (Scope ignored = span.makeCurrent()) {
span.setAttribute("metadataKey", "metadataValue");
span.setAttribute("annotationKey", "annotationValue");
// The following ensures that "annotationKey: annotationValue" is an annotation in X-Ray raw data.
span.setAttribute(AttributeKey.stringArrayKey("aws.xray.annotations"), List.of("annotationKey"));
// Do something here
}
span.end();
```
Spans have a default type of *INTERNAL*.
```
// Default span of type INTERNAL will become an X-Ray subsegment
Span span = tracer.spanBuilder("process-header")
.startSpan();
try (Scope ignored = span.makeCurrent()) {
doProcessHeader();
}
```
**Adding annotations and metadata to traces with OpenTelemetry SDK**
In the above example, the `setAttribute` method is used to add attributes to each span. By default, all the span attributes will be converted into metadata in X-Ray raw data. To ensure that an attribute is converted into an annotation and not metadata, the above example adds that attribute’s key to the list of the `aws.xray.annotations` attribute. For more information, see [Enable the Customized X-Ray Annotations ](https://aws-otel.github.io/docs/getting-started/x-ray#enable-the-customized-x-ray-annotations) and [Annotations and metadata](https://docs.aws.amazon.com/xray/latest/devguide/xray-concepts.html#xray-concepts-annotations).
**With OpenTelemetry-based Java agents**
If you are using the Java agent to automatically instrument your application, you need to perform manual instrumentation in your application. For example, to instrument code within the application for sections that are not covered by any auto-instrumentation library.
To perform manual instrumentation with the agent, you need to use the `opentelemetry-api ` artifact. The artifact version cannot be newer than the agent version.
```
import io.opentelemetry.api.GlobalOpenTelemetry;
import io.opentelemetry.api.trace.Span;
// ...
Span parentSpan = Span.current();
Tracer tracer = GlobalOpenTelemetry.getTracer("my-app");
Span span = tracer.spanBuilder("my-span-name")
.setParent(io.opentelemetry.context.Context.current().with(parentSpan))
.startSpan();
span.end();
```
------
## Lambda instrumentation
------
#### [ With X-Ray SDK ]
Using the X-Ray SDK, after your Lambda has *Active Tracing* enabled, there is no additional configuration required to use the X-Ray SDK. Lambda will create a segment representing the Lambda handler invocation, and you can create sub-segments or instrument libraries using the X-Ray SDK without any additional configuration.
------
#### [ With OpenTelemetry-based solutions ]
Auto-instrumentation Lambda layers – You can automatically instrument your Lambda with AWS vended Lambda layers using the following solutions:
+ AWS Lambda Layer for OpenTelemetry (Recommended)
**Note**
This Lambda layer has CloudWatch Application Signals enabled by default, which enables performance and health monitoring for your Lambda application by collecting both metrics and traces. For just tracing, set the Lambda environment variable ` OTEL_AWS_APPLICATION_SIGNALS_ENABLED=false`.
+ Enables performance and health monitoring for your Lambda application
+ Collects both metrics and traces by default
+ AWS managed Lambda layer for ADOT Java. For more information, see [AWS Distro for OpenTelemetry Lambda Support For Java](https://aws-otel.github.io/docs/getting-started/lambda/lambda-java).
To use manual instrumentation along with auto-instrumentation layer, see [Manual instrumentation solutions with the SDK](#xray-migration-sdk). For reduced cold starts, consider using OpenTelemetry manual instrumentation to generate OpenTelemetry traces for your Lambda function.
------
**OpenTelemetry manual instrumentation for AWS Lambda**
Consider the following Lambda function code that makes an Amazon S3 ListBuckets call.
```
package example;
import com.amazonaws.services.lambda.runtime.Context;
import com.amazonaws.services.lambda.runtime.RequestHandler;
import software.amazon.awssdk.services.s3.S3Client;
import software.amazon.awssdk.services.s3.model.ListBucketsRequest;
import software.amazon.awssdk.services.s3.model.ListBucketsResponse;
import software.amazon.awssdk.services.s3.model.S3Exception;
public class ListBucketsLambda implements RequestHandler {
private final S3Client S3_CLIENT = S3Client.builder()
.build();
@Override
public String handleRequest(String input, Context context) {
try {
ListBucketsResponse response = makeListBucketsCall();
context.getLogger().log("response: " + response.toString());
return "Success";
} catch (Exception e) {
throw new RuntimeException(e);
}
}
private ListBucketsResponse makeListBucketsCall() {
try {
ListBucketsRequest listBucketsRequest = ListBucketsRequest.builder()
.build();
ListBucketsResponse response = S3_CLIENT.listBuckets(listBucketsRequest);
return response;
} catch (S3Exception e) {
throw new RuntimeException("Failed to call S3 listBuckets" + e.awsErrorDetails().errorMessage(), e);
}
}
}
```
Here are the dependencies.
```
dependencies {
implementation('com.amazonaws:aws-lambda-java-core:1.2.3')
implementation('software.amazon.awssdk:s3:2.28.29')
implementation('org.slf4j:slf4j-nop:2.0.16')
}
```
To manually instrument your Lambda handler and the Amazon S3 client, do the following.
1. Replace your function classes that implement `RequestHandler` (or RequestStreamHandler) with those that extend `TracingRequestHandler` (or TracingRequestStreamHandler).
1. Instantiate a TracerProvider and globally register an OpenTelemetrySdk object. The TracerProvider is recommended to be configured with:
1. A Simple Span Processor with an X-Ray UDP span exporter to send Traces to Lambda’s UDP X-Ray endpoint
1. A ParentBased always on sampler (Default if not configured)
1. A Resource with service.name set to the Lambda function name
1. An X-Ray Lambda propagator
1. Change the `handleRequest` method to `doHandleRequest` and pass the `OpenTelemetrySdk` object to the base class.
1. Instrument the Amazon S3 client with the OpenTemetry AWS SDK instrumentation by registering the interceptor when building the client.
You need the following OpenTelemetry-related dependencies.
```
dependencies {
...
implementation("software.amazon.distro.opentelemetry:aws-distro-opentelemetry-xray-udp-span-exporter:0.1.0")
implementation(platform('io.opentelemetry.instrumentation:opentelemetry-instrumentation-bom:2.14.0'))
implementation(platform('io.opentelemetry:opentelemetry-bom:1.48.0'))
implementation('io.opentelemetry:opentelemetry-sdk')
implementation('io.opentelemetry:opentelemetry-api')
implementation('io.opentelemetry.contrib:opentelemetry-aws-xray-propagator:1.45.0-alpha')
implementation('io.opentelemetry.contrib:opentelemetry-aws-resources:1.45.0-alpha')
implementation('io.opentelemetry.instrumentation:opentelemetry-aws-lambda-core-1.0:2.14.0-alpha')
implementation('io.opentelemetry.instrumentation:opentelemetry-aws-sdk-2.2:2.14.0-alpha')
}
```
The following code demonstrates the Lambda function after the required changes. You can create additional custom spans to complement the automatically provided spans.
```
package example;
import java.time.Duration;
import com.amazonaws.services.lambda.runtime.Context;
import io.opentelemetry.api.common.Attributes;
import io.opentelemetry.context.propagation.ContextPropagators;
import io.opentelemetry.contrib.aws.resource.LambdaResource;
import io.opentelemetry.contrib.awsxray.propagator.AwsXrayLambdaPropagator;
import io.opentelemetry.instrumentation.awslambdacore.v1_0.TracingRequestHandler;
import io.opentelemetry.instrumentation.awssdk.v2_2.AwsSdkTelemetry;
import io.opentelemetry.sdk.OpenTelemetrySdk;
import io.opentelemetry.sdk.resources.Resource;
import io.opentelemetry.sdk.trace.SdkTracerProvider;
import io.opentelemetry.sdk.trace.export.SimpleSpanProcessor;
import io.opentelemetry.sdk.trace.samplers.Sampler;
import static io.opentelemetry.semconv.ServiceAttributes.SERVICE_NAME;
import software.amazon.awssdk.core.client.config.ClientOverrideConfiguration;
import software.amazon.awssdk.services.s3.S3Client;
import software.amazon.awssdk.services.s3.model.ListBucketsRequest;
import software.amazon.awssdk.services.s3.model.ListBucketsResponse;
import software.amazon.awssdk.services.s3.model.S3Exception;
import software.amazon.distro.opentelemetry.exporter.xray.udp.trace.AwsXrayUdpSpanExporterBuilder;
public class ListBucketsLambda extends TracingRequestHandler {
private static final Resource lambdaResource = LambdaResource.get();
private static final SdkTracerProvider sdkTracerProvider =
SdkTracerProvider.builder()
.addSpanProcessor(SimpleSpanProcessor.create(
new AwsXrayUdpSpanExporterBuilder().build()
))
.addResource(
lambdaResource
.merge(Resource.create(Attributes.of(SERVICE_NAME, System.getenv("AWS_LAMBDA_FUNCTION_NAME"))))
)
.setSampler(Sampler.parentBased(Sampler.alwaysOn()))
.build();
private static final OpenTelemetrySdk openTelemetry =
OpenTelemetrySdk.builder()
.setTracerProvider(sdkTracerProvider)
.setPropagators(ContextPropagators.create(AwsXrayLambdaPropagator.getInstance()))
.buildAndRegisterGlobal();
private static final AwsSdkTelemetry telemetry = AwsSdkTelemetry.create(openTelemetry);
private final S3Client S3_CLIENT = S3Client.builder()
.overrideConfiguration(ClientOverrideConfiguration.builder()
.addExecutionInterceptor(telemetry.newExecutionInterceptor())
.build())
.build();
public ListBucketsLambda() {
super(openTelemetry, Duration.ofMillis(0));
}
@Override
public String doHandleRequest(String input, Context context) {
try {
ListBucketsResponse response = makeListBucketsCall();
context.getLogger().log("response: " + response.toString());
return "Success";
} catch (Exception e) {
throw new RuntimeException(e);
}
}
private ListBucketsResponse makeListBucketsCall() {
try {
ListBucketsRequest listBucketsRequest = ListBucketsRequest.builder()
.build();
ListBucketsResponse response = S3_CLIENT.listBuckets(listBucketsRequest);
return response;
} catch (S3Exception e) {
throw new RuntimeException("Failed to call S3 listBuckets" + e.awsErrorDetails().errorMessage(), e);
}
}
}
```
When invoking the Lambda function, you will see the following trace under *Trace Map* in the CloudWatch console.
![\[Trace map in CloudWatch console.\]](http://docs.aws.amazon.com/xray/latest/devguide/images/SDKDeprecation_Java.png)
# Migrate to OpenTelemetry Go
Use the following code examples to manually instrument your Go applications with the OpenTelemetry SDK when migrating from X-Ray.
## Manual instrumentation with the SDK
------
#### [ Tracing setup with X-Ray SDK ]
When using the X-Ray SDK for Go, service plugins or local sampling rules were required to be configured before instrumenting your code.
```
func init() {
if os.Getenv("ENVIRONMENT") == "production" {
ec2.Init()
}
xray.Configure(xray.Config{
DaemonAddr: "127.0.0.1:2000",
ServiceVersion: "1.2.3",
})
}
```
------
#### [ Set up tracing with OpenTelemetry SDK ]
Configure the OpenTelemetry SDK by instantiating a TracerProvider and registering it as the global tracer provider. We recommend configuring the following components:
+ OTLP Trace Exporter – Required for exporting traces to the CloudWatch Agent or OpenTelemetry Collector
+ X-Ray Propagator – Required for propagating the trace context to AWS services integrated with X-Ray
+ X-Ray Remote Sampler – Required for sampling requests using X-Ray sampling rules
+ Resource detectors – To detect metadata of the host running your application
```
import (
"go.opentelemetry.io/contrib/detectors/aws/ec2"
"go.opentelemetry.io/contrib/propagators/aws/xray"
"go.opentelemetry.io/contrib/samplers/aws/xray"
"go.opentelemetry.io/otel"
"go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc"
"go.opentelemetry.io/otel/sdk/trace"
)
func setupTracing() error {
ctx := context.Background()
exporterEndpoint := os.Getenv("OTEL_EXPORTER_OTLP_ENDPOINT")
if exporterEndpoint == "" {
exporterEndpoint = "localhost:4317"
}
traceExporter, err := otlptracegrpc.New(ctx,
otlptracegrpc.WithInsecure(),
otlptracegrpc.WithEndpoint(exporterEndpoint))
if err != nil {
return fmt.Errorf("failed to create OTLP trace exporter: %v", err)
}
remoteSampler, err := xray.NewRemoteSampler(ctx, "my-service-name", "ec2")
if err != nil {
return fmt.Errorf("failed to create X-Ray Remote Sampler: %v", err)
}
ec2Resource, err := ec2.NewResourceDetector().Detect(ctx)
if err != nil {
return fmt.Errorf("failed to detect EC2 resource: %v", err)
}
tp := trace.NewTracerProvider(
trace.WithSampler(remoteSampler),
trace.WithBatcher(traceExporter),
trace.WithResource(ec2Resource),
)
otel.SetTracerProvider(tp)
otel.SetTextMapPropagator(xray.Propagator{})
return nil
}
```
------
## Tracing incoming requests (HTTP handler instrumentation)
------
#### [ With X-Ray SDK ]
To instrument an HTTP handler with X-Ray, the X-Ray handler method was used to generate segments using NewFixedSegmentNamer.
```
func main() {
http.Handle("/", xray.Handler(xray.NewFixedSegmentNamer("myApp"), http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Write([]byte("Hello!"))
})))
http.ListenAndServe(":8000", nil)
}
```
------
#### [ With OpenTelemetry SDK ]
To instrument an HTTP handler with OpenTelemetry, use the OpenTelemetry's newHandler method to wrap your original handler code.
```
import (
"go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp"
)
helloHandler := func(w http.ResponseWriter, req *http.Request) {
ctx := req.Context()
span := trace.SpanFromContext(ctx)
span.SetAttributes(attribute.Bool("isHelloHandlerSpan", true), attribute.String("attrKey", "attrValue"))
_, _ = io.WriteString(w, "Hello World!\n")
}
otelHandler := otelhttp.NewHandler(http.HandlerFunc(helloHandler), "Hello")
http.Handle("/hello", otelHandler)
err = http.ListenAndServe(":8080", nil)
if err != nil {
log.Fatal(err)
}
```
------
## AWS SDK for Go v2 instrumentation
------
#### [ With X-Ray SDK ]
To instrument outgoing AWS requests from AWS SDK, your clients were instrumented as follows:
```
// Create a segment
ctx, root := xray.BeginSegment(context.TODO(), "AWSSDKV2_Dynamodb")
defer root.Close(nil)
cfg, err := config.LoadDefaultConfig(ctx, config.WithRegion("us-west-2"))
if err != nil {
log.Fatalf("unable to load SDK config, %v", err)
}
// Instrumenting AWS SDK v2
awsv2.AWSV2Instrumentor(&cfg.APIOptions)
// Using the Config value, create the DynamoDB client
svc := dynamodb.NewFromConfig(cfg)
// Build the request with its input parameters
_, err = svc.ListTables(ctx, &dynamodb.ListTablesInput{
Limit: aws.Int32(5),
})
if err != nil {
log.Fatalf("failed to list tables, %v", err)
}
```
------
#### [ With OpenTelemetry SDK ]
Tracing support for downstream AWS SDK calls is provided by OpenTelemetry's AWS SDK for Go v2 Instrumentation. Here's an example of tracing an S3 client call:
```
import (
...
"github.com/aws/aws-sdk-go-v2/config"
"github.com/aws/aws-sdk-go-v2/service/s3"
"go.opentelemetry.io/otel"
oteltrace "go.opentelemetry.io/otel/trace"
awsConfig "github.com/aws/aws-sdk-go-v2/config"
"go.opentelemetry.io/contrib/instrumentation/github.com/aws/aws-sdk-go-v2/otelaws"
)
...
// init aws config
cfg, err := awsConfig.LoadDefaultConfig(ctx)
if err != nil {
panic("configuration error, " + err.Error())
}
// instrument all aws clients
otelaws.AppendMiddlewares(&.APIOptions)
// Call to S3
s3Client := s3.NewFromConfig(cfg)
input := &s3.ListBucketsInput{}
result, err := s3Client.ListBuckets(ctx, input)
if err != nil {
fmt.Printf("Got an error retrieving buckets, %v", err)
return
}
```
------
## Instrumenting outgoing HTTP calls
------
#### [ With X-Ray SDK ]
To instrument outgoing HTTP calls with X-Ray, the xray.Client was used to create a copy of a provided HTTP client.
```
myClient := xray.Client(http-client)
resp, err := ctxhttp.Get(ctx, xray.Client(nil), url)
```
------
#### [ With OpenTelemetry SDK ]
To instrument HTTP clients with OpenTelemetry, use OpenTelemetry's otelhttp.NewTransport method to wrap the http.DefaultTransport.
```
import (
"go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp"
)
// Create an instrumented HTTP client.
httpClient := &http.Client{
Transport: otelhttp.NewTransport(
http.DefaultTransport,
),
}
req, err := http.NewRequestWithContext(ctx, http.MethodGet, "https://api.github.com/repos/aws-observability/aws-otel-go/releases/latest", nil)
if err != nil {
fmt.Printf("failed to create http request, %v\n", err)
}
res, err := httpClient.Do(req)
if err != nil {
fmt.Printf("failed to make http request, %v\n", err)
}
// Request body must be closed
defer func(Body io.ReadCloser) {
err := Body.Close()
if err != nil {
fmt.Printf("failed to close http response body, %v\n", err)
}
}(res.Body)
```
------
## Instrumentation support for other libraries
You can find the full list of supported library instrumentations for OpenTelemetry Go under [Instrumentation packages ](https://github.com/open-telemetry/opentelemetry-go-contrib/tree/main/instrumentation#instrumentation-packages).
Alternatively, you can search the OpenTelemetry registry to find out if OpenTelemetry supports instrumentation for your library under [Registry](https://opentelemetry.io/ecosystem/registry/).
## Manually creating trace data
------
#### [ With X-Ray SDK ]
With the X-Ray SDK, the BeginSegment and BeginSubsegment methods was required to manually create X-Ray segments and sub-segments.
```
// Start a segment
ctx, seg := xray.BeginSegment(context.Background(), "service-name")
// Start a subsegment
subCtx, subSeg := xray.BeginSubsegment(ctx, "subsegment-name")
// Add metadata or annotation here if necessary
xray.AddAnnotation(subCtx, "annotationKey", "annotationValue")
xray.AddMetadata(subCtx, "metadataKey", "metadataValue")
subSeg.Close(nil)
// Close the segment
seg.Close(nil)
```
------
#### [ With OpenTelemetry SDK ]
Use custom spans to monitor the performance of internal activities that are not captured by instrumentation libraries. Note that only spans of kind Server are converted into X-Ray segments, all other spans are converted into X-Ray sub-segments.
First, you will need to create a Tracer to generate spans, which you can obtain through the `otel.Tracer` method. This will provide a Tracer instance from the TracerProvider that was registered globally in the Tracing Setup example. You can create as many Tracer instances as needed, but it is common to have one Tracer for an entire application.
```
tracer := otel.Tracer("application-tracer")
```
```
import (
...
oteltrace "go.opentelemetry.io/otel/trace"
)
...
var attributes = []attribute.KeyValue{
attribute.KeyValue{Key: "metadataKey", Value: attribute.StringValue("metadataValue")},
attribute.KeyValue{Key: "annotationKey", Value: attribute.StringValue("annotationValue")},
attribute.KeyValue{Key: "aws.xray.annotations", Value: attribute.StringSliceValue([]string{"annotationKey"})},
}
ctx := context.Background()
parentSpanContext, parentSpan := tracer.Start(ctx, "ParentSpan", oteltrace.WithSpanKind(oteltrace.SpanKindServer), oteltrace.WithAttributes(attributes...))
_, childSpan := tracer.Start(parentSpanContext, "ChildSpan", oteltrace.WithSpanKind(oteltrace.SpanKindInternal))
// ...
childSpan.End()
parentSpan.End()
```
**Adding annotations and metadata to traces with OpenTelemetry SDK**
In the above example, the `WithAttributes` method is used to add attributes to each span. Note that by default, all the span attributes are converted into metadata in X-Ray raw data. To ensure that an attribute is converted into an annotation and not metadata, add the attribute's key to the list of the `aws.xray.annotations` attribute. For more information, see [Enable The Customized X-Ray Annotations ](https://aws-otel.github.io/docs/getting-started/x-ray#enable-the-customized-x-ray-annotations).
------
## Lambda manual instrumentation
------
#### [ With X-Ray SDK ]
With the X-Ray SDK, after your Lambda has *Active Tracing* was enabled, there were no additional configurations required to use the X-Ray SDK. Lambda created a segment representing the Lambda handler invocation, and you created sub-segments using the X-Ray SDK without any additional configuration.
------
#### [ With OpenTelemetry SDK ]
The following Lambda function code (without instrumentation) makes an Amazon S3 ListBuckets call and outgoing HTTP request.
```
package main
import (
"context"
"encoding/json"
"fmt"
"io"
"net/http"
"os"
"github.com/aws/aws-lambda-go/events"
"github.com/aws/aws-lambda-go/lambda"
awsconfig "github.com/aws/aws-sdk-go-v2/config"
"github.com/aws/aws-sdk-go-v2/service/s3"
)
func lambdaHandler(ctx context.Context) (interface{}, error) {
// Initialize AWS config.
cfg, err := awsconfig.LoadDefaultConfig(ctx)
if err != nil {
panic("configuration error, " + err.Error())
}
s3Client := s3.NewFromConfig(cfg)
// Create an HTTP client.
httpClient := &http.Client{
Transport: http.DefaultTransport,
}
input := &s3.ListBucketsInput{}
result, err := s3Client.ListBuckets(ctx, input)
if err != nil {
fmt.Printf("Got an error retrieving buckets, %v", err)
}
fmt.Println("Buckets:")
for _, bucket := range result.Buckets {
fmt.Println(*bucket.Name + ": " + bucket.CreationDate.Format("2006-01-02 15:04:05 Monday"))
}
fmt.Println("End Buckets.")
req, err := http.NewRequestWithContext(ctx, http.MethodGet, "https://api.github.com/repos/aws-observability/aws-otel-go/releases/latest", nil)
if err != nil {
fmt.Printf("failed to create http request, %v\n", err)
}
res, err := httpClient.Do(req)
if err != nil {
fmt.Printf("failed to make http request, %v\n", err)
}
defer func(Body io.ReadCloser) {
err := Body.Close()
if err != nil {
fmt.Printf("failed to close http response body, %v\n", err)
}
}(res.Body)
var data map[string]interface{}
err = json.NewDecoder(res.Body).Decode(&data)
if err != nil {
fmt.Printf("failed to read http response body, %v\n", err)
}
fmt.Printf("Latest ADOT Go Release is '%s'\n", data["name"])
return events.APIGatewayProxyResponse{
StatusCode: http.StatusOK,
Body: os.Getenv("_X_AMZN_TRACE_ID"),
}, nil
}
func main() {
lambda.Start(lambdaHandler)
}
```
To manually instrument your Lambda handler and the Amazon S3 client, do the following:
1. In *main()*, instantiate a TracerProvider (tp) and register it as the global tracer provider. The TracerProvider is recommended to be configured with:
1. Simple Span Processor with an X-Ray UDP span exporter to send Traces to Lambda's UDP X-Ray endpoint
1. Resource with *service.name* set to the Lambda function name
1. Change the usage of `lambda.Start(lambdaHandler)` to `lambda.Start(otellambda.InstrumentHandler(lambdaHandler, xrayconfig.WithRecommendedOptions(tp)...))`.
1. Instrument the Amazon S3 client with the OpenTemetry AWS SDK instrumentation by appending OpenTelemetry middleware for `aws-sdk-go-v2` into the Amazon S3 client configuration.
1. Instrument the http client by using OpenTelemetry's `otelhttp.NewTransport` method to wrap the `http.DefaultTransport`.
The following code is an example of how the Lambda Function will look like after the changes. You may manually create additional custom spans in addition to the spans provided automatically.
```
package main
import (
"context"
"encoding/json"
"fmt"
"io"
"net/http"
"os"
"github.com/aws-observability/aws-otel-go/exporters/xrayudp"
"github.com/aws/aws-lambda-go/events"
"github.com/aws/aws-lambda-go/lambda"
awsconfig "github.com/aws/aws-sdk-go-v2/config"
"github.com/aws/aws-sdk-go-v2/service/s3"
lambdadetector "go.opentelemetry.io/contrib/detectors/aws/lambda"
"go.opentelemetry.io/contrib/instrumentation/github.com/aws/aws-lambda-go/otellambda"
"go.opentelemetry.io/contrib/instrumentation/github.com/aws/aws-lambda-go/otellambda/xrayconfig"
"go.opentelemetry.io/contrib/instrumentation/github.com/aws/aws-sdk-go-v2/otelaws"
"go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp"
"go.opentelemetry.io/contrib/propagators/aws/xray"
"go.opentelemetry.io/otel"
"go.opentelemetry.io/otel/attribute"
"go.opentelemetry.io/otel/sdk/resource"
"go.opentelemetry.io/otel/sdk/trace"
semconv "go.opentelemetry.io/otel/semconv/v1.26.0"
)
func lambdaHandler(ctx context.Context) (interface{}, error) {
// Initialize AWS config.
cfg, err := awsconfig.LoadDefaultConfig(ctx)
if err != nil {
panic("configuration error, " + err.Error())
}
// Instrument all AWS clients.
otelaws.AppendMiddlewares(&cfg.APIOptions)
// Create an instrumented S3 client from the config.
s3Client := s3.NewFromConfig(cfg)
// Create an instrumented HTTP client.
httpClient := &http.Client{
Transport: otelhttp.NewTransport(
http.DefaultTransport,
),
}
// return func(ctx context.Context) (interface{}, error) {
input := &s3.ListBucketsInput{}
result, err := s3Client.ListBuckets(ctx, input)
if err != nil {
fmt.Printf("Got an error retrieving buckets, %v", err)
}
fmt.Println("Buckets:")
for _, bucket := range result.Buckets {
fmt.Println(*bucket.Name + ": " + bucket.CreationDate.Format("2006-01-02 15:04:05 Monday"))
}
fmt.Println("End Buckets.")
req, err := http.NewRequestWithContext(ctx, http.MethodGet, "https://api.github.com/repos/aws-observability/aws-otel-go/releases/latest", nil)
if err != nil {
fmt.Printf("failed to create http request, %v\n", err)
}
res, err := httpClient.Do(req)
if err != nil {
fmt.Printf("failed to make http request, %v\n", err)
}
defer func(Body io.ReadCloser) {
err := Body.Close()
if err != nil {
fmt.Printf("failed to close http response body, %v\n", err)
}
}(res.Body)
var data map[string]interface{}
err = json.NewDecoder(res.Body).Decode(&data)
if err != nil {
fmt.Printf("failed to read http response body, %v\n", err)
}
fmt.Printf("Latest ADOT Go Release is '%s'\n", data["name"])
return events.APIGatewayProxyResponse{
StatusCode: http.StatusOK,
Body: os.Getenv("_X_AMZN_TRACE_ID"),
}, nil
}
func main() {
ctx := context.Background()
detector := lambdadetector.NewResourceDetector()
lambdaResource, err := detector.Detect(context.Background())
if err != nil {
fmt.Printf("failed to detect lambda resources: %v\n", err)
}
var attributes = []attribute.KeyValue{
attribute.KeyValue{Key: semconv.ServiceNameKey, Value: attribute.StringValue(os.Getenv("AWS_LAMBDA_FUNCTION_NAME"))},
}
customResource := resource.NewWithAttributes(semconv.SchemaURL, attributes...)
mergedResource, _ := resource.Merge(lambdaResource, customResource)
xrayUdpExporter, _ := xrayudp.NewSpanExporter(ctx)
tp := trace.NewTracerProvider(
trace.WithSpanProcessor(trace.NewSimpleSpanProcessor(xrayUdpExporter)),
trace.WithResource(mergedResource),
)
defer func(ctx context.Context) {
err := tp.Shutdown(ctx)
if err != nil {
fmt.Printf("error shutting down tracer provider: %v", err)
}
}(ctx)
otel.SetTracerProvider(tp)
otel.SetTextMapPropagator(xray.Propagator{})
lambda.Start(otellambda.InstrumentHandler(lambdaHandler, xrayconfig.WithRecommendedOptions(tp)...))
}
```
------
When invoking Lambda, you will see the following trace in the `Trace Map` in the CloudWatch console:
![\[Trace map in CloudWatch console for Golang.\]](http://docs.aws.amazon.com/xray/latest/devguide/images/deprecation_golang.png)
# Migrate to OpenTelemetry Node.js
This section explains how to migrate your Node.js applications from X-Ray SDK to OpenTelemetry. It covers both automatic and manual instrumentation approaches, and provides specific examples for common use cases.
The X-Ray Node.js SDK helps you manually instrument your Node.js applications for tracing. This section provides code examples for migrating from X-Ray to OpenTelemetry instrumentation.
**Topics**
+ [Zero code automatic instrumentation solutions](#zero-code-instrumentation)
+ [Manual instrumentation solutions](#manual-instrumentation)
+ [Tracing incoming requests](#tracing-incoming-requests)
+ [AWS SDK JavaScript V3 instrumentation](#aws-sdk-instrumentation)
+ [Instrumenting outgoing HTTP calls](#http-instrumentation)
+ [Instrumentation support for other libraries](#other-libraries)
+ [Manually creating trace data](#manual-trace-creation)
+ [Lambda instrumentation](#lambda-instrumentation)
## Zero code automatic instrumentation solutions
To trace requests with the X-Ray SDK for Node.js, you must modify your application code. With OpenTelemetry, you can use zero-code auto-instrumentation solutions to trace requests.
**Zero code automatic instrumentation with OpenTelemetry-based automatic instrumentations.**
1. Using the AWS Distro for OpenTelemetry (ADOT) auto-instrumentation for Node.js – For automatic instrumentation for Node.js application, see [Tracing and Metrics with the AWS Distro for OpenTelemetry JavaScript Auto-Instrumentation](https://aws-otel.github.io/docs/getting-started/js-sdk/trace-metric-auto-instr).
(Optional) You can also enable CloudWatch Application Signals when automatically instrumenting your applications on AWS with the ADOT JavaScript auto-instrumentation to monitor current application health and track long-term application performance against your business objectives. Application Signals provides you with a unified, application-centric view of your applications, services, and dependencies, and helps you monitor and triage application health. For more information, see [Application Signals](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-Application-Monitoring-Sections.html).
1. Using the OpenTelemetry JavaScript zero-code automatic instrumentation – For automatic instrumentation with the OpenTelemetry JavaScript, see [JavaScript zero-code instrumentation ](https://opentelemetry.io/docs/zero-code/js/).
## Manual instrumentation solutions
------
#### [ Tracing setup with X-Ray SDK ]
When the X-Ray SDK for Node.js was used, the `aws-xray-sdk` package was required to configure the X-Ray SDK with service plug-ins or local sampling rules before using the SDK to instrument your code.
```
var AWSXRay = require('aws-xray-sdk');
AWSXRay.config([AWSXRay.plugins.EC2Plugin,AWSXRay.plugins.ElasticBeanstalkPlugin]);
AWSXRay.middleware.setSamplingRules();
```
------
#### [ Tracing setup with OpenTelemetry SDK ]
**Note**
AWS X-Ray Remote Sampling is currently not available to be configured for OpenTelemetry JS. However, support for X-Ray Remote Sampling is currently available through the ADOT Auto-Instrumentation for Node.js.
For the code example below, you will need the following dependencies:
```
npm install --save \
@opentelemetry/api \
@opentelemetry/sdk-node \
@opentelemetry/exporter-trace-otlp-proto \
@opentelemetry/propagator-aws-xray \
@opentelemetry/resource-detector-aws
```
You must set up and configure the OpenTelemetry SDK before running your application code. This can be done by using the [–-require ](https://nodejs.org/api/cli.html#-r---require-module) flag. Create a file named *instrumentation.js*, which will contain your OpenTelemetry instrumentation configuration and setup.
It is recommend that you configure the following components:
+ OTLPTraceExporter – Required for exporting traces to the CloudWatch Agent/OpenTelemetry Collector
+ AWSXRayPropagator – Required for propagating the Trace Context to AWS Services that are integrated with X-Ray
+ Resource Detectors (for example, Amazon EC2 Resource Detector) - To detect metadata of the host running your application
```
/*instrumentation.js*/
// Require dependencies
const { NodeSDK } = require('@opentelemetry/sdk-node');
const { OTLPTraceExporter } = require('@opentelemetry/exporter-trace-otlp-proto');
const { AWSXRayPropagator } = require("@opentelemetry/propagator-aws-xray");
const { detectResources } = require('@opentelemetry/resources');
const { awsEc2Detector } = require('@opentelemetry/resource-detector-aws');
const resource = detectResources({
detectors: [awsEc2Detector],
});
const _traceExporter = new OTLPTraceExporter({
url: 'http://localhost:4318/v1/traces'
});
const sdk = new NodeSDK({
resource: resource,
textMapPropagator: new AWSXRayPropagator(),
traceExporter: _traceExporter
});
sdk.start();
```
Then, you can run your application with your OpenTelemetry setup like:
```
node --require ./instrumentation.js app.js
```
You can use OpenTelemetry SDK library instrumentations to automatically create spans for libraries such as the AWS SDK. Enabling these will automatically create spans for modules such as the AWS SDK for JavaScript v3. OpenTelemetry provides the option to enable all library instrumentations or specify which library instrumentations to enable.
To enable all instrumentations, install the `@opentelemetry/auto-instrumentations-node` package:
```
npm install @opentelemetry/auto-instrumentations-node
```
Next, update the configuration to enable all library instrumentations as shown below.
```
const { getNodeAutoInstrumentations } = require('@opentelemetry/auto-instrumentations-node');
...
const sdk = new NodeSDK({
resource: resource,
instrumentations: [getNodeAutoInstrumentations()],
textMapPropagator: new AWSXRayPropagator(),
traceExporter: _traceExporter
});
```
------
#### [ Tracing setup with ADOT auto-instrumentation for Node.js ]
You can use the ADOT auto-instrumentation for Node.js to automatically configure OpenTelemetry for your Node.js applications. By using ADOT Auto-Instrumentation, you do not need to make manual code changes to trace incoming requests, or to trace libraries such as the AWS SDK or HTTP clients. For more information, see [Tracing and metrics with the AWS Distro for OpenTelemetry JavaScript Auto-Instrumentation](https://aws-otel.github.io/docs/getting-started/js-sdk/trace-metric-auto-instr).
ADOT auto-instrumentation for Node.js supports:
+ X-Ray remote sampling through environment variable – `export OTEL_TRACES_SAMPLER=xray`
+ X-Ray trace context propagation (enabled by default)
+ Resource detection (resource detection for Amazon EC2, Amazon ECS, and Amazon EKS environments are enabled by default)
+ Automatic library instrumentations for all supported OpenTelemetry instrumentations, which can be disabled/enabled selectively through `OTEL_NODE_ENABLED_INSTRUMENTATIONS` and `OTEL_NODE_DISABLED_INSTRUMENTATIONS` environment variables
+ Manual creation of Spans
------
## Tracing incoming requests
------
#### [ With X-Ray SDK ]
**Express.js**
With the X-Ray SDK to trace incoming HTTP requests received by *Express.js* applications, the two middlewares `AWSXRay.express.openSegment()` and `AWSXRay.express.closeSegment()` were required to wrap all of your defined routes in order to trace them.
```
app.use(xrayExpress.openSegment('defaultName'));
...
app.use(xrayExpress.closeSegment());
```
**Restify**
To trace incoming HTTP requests received by `Restify` applications, the middleware from the X-Ray SDK was used by running enable from the `aws-xray-sdk-restify `module on the Restify Server:
```
var AWSXRay = require('aws-xray-sdk');
var AWSXRayRestify = require('aws-xray-sdk-restify');
var restify = require('restify');
var server = restify.createServer();
AWSXRayRestify.enable(server, 'MyApp'));
```
------
#### [ With OpenTelemetry SDK ]
**Express.js**
Tracing support for incoming requests for `Express.js` is provided by the [OpenTelemetry HTTP instrumentation](https://github.com/open-telemetry/opentelemetry-js/tree/main/experimental/packages/opentelemetry-instrumentation-http) and [OpenTelemetry express instrumentation](https://github.com/open-telemetry/opentelemetry-js-contrib/tree/main/plugins/node/opentelemetry-instrumentation-express). Install the following dependencies with `npm`:
```
npm install --save @opentelemetry/instrumentation-http @opentelemetry/instrumentation-express
```
Update the OpenTelemetry SDK Configuration to enable instrumentation for the express module:
```
const { HttpInstrumentation } = require('@opentelemetry/instrumentation-http');
const { ExpressInstrumentation } = require('@opentelemetry/instrumentation-express');
...
const sdk = new NodeSDK({
...
instrumentations: [
...
// Express instrumentation requires HTTP instrumentation
new HttpInstrumentation(),
new ExpressInstrumentation(),
],
});
```
**Restify**
For Restify applications, you will need the [OpenTelemetry Restify instrumentation](https://github.com/open-telemetry/opentelemetry-js-contrib/tree/main/plugins/node/opentelemetry-instrumentation-restify). Install the following dependency:
```
npm install --save @opentelemetry/instrumentation-restify
```
Update the OpenTelemetry SDK Configuration to enable instrumentation for the restify module:
```
const { RestifyInstrumentation } = require('@opentelemetry/instrumentation-restify');
...
const sdk = new NodeSDK({
...
instrumentations: [
...
new RestifyInstrumentation(),
],
});
```
------
## AWS SDK JavaScript V3 instrumentation
------
#### [ With X-Ray SDK ]
To instrument outgoing AWS requests from AWS SDK, you instrumented clients like the following example:
```
import { S3, PutObjectCommand } from '@aws-sdk/client-s3';
const s3 = AWSXRay.captureAWSv3Client(new S3({}));
await s3.send(new PutObjectCommand({
Bucket: bucketName,
Key: keyName,
Body: 'Hello!',
}));
```
------
#### [ With OpenTelemetry SDK ]
Tracing support for downstream AWS SDK calls to DynamoDB, Amazon S3, and others is provided by the OpenTelemetry AWS SDK instrumentation. Install the following dependency with `npm`:
```
npm install --save @opentelemetry/instrumentation-aws-sdk
```
Update the OpenTelemetry SDK Configuration with the AWS SDK instrumentation.
```
import { AwsInstrumentation } from '@opentelemetry/instrumentation-aws-sdk';
...
const sdk = new NodeSDK({
...
instrumentations: [
...
new AwsInstrumentation()
],
});
```
------
## Instrumenting outgoing HTTP calls
------
#### [ With X-Ray SDK ]
To instrument outgoing HTTP requests with X-Ray, it was required to instrument clients. For example, see below.
Individual HTTP clients
```
var AWSXRay = require('aws-xray-sdk');
var http = AWSXRay.captureHTTPs(require('http'));
```
All HTTP clients (global)
```
var AWSXRay = require('aws-xray-sdk');
AWSXRay.captureHTTPsGlobal(require('http'));
var http = require('http');
```
------
#### [ With OpenTelemetry SDK ]
Tracing support for Node.js HTTP clients is provided by the OpenTelemetry HTTP Instrumentation. Install the following dependency with `npm`:
```
npm install --save @opentelemetry/instrumentation-http
```
Update the OpenTelemetry SDK Configuration as follows:
```
const { HttpInstrumentation } = require('@opentelemetry/instrumentation-http');
...
const sdk = new NodeSDK({
...
instrumentations: [
...
new HttpInstrumentation(),
],
});
```
------
## Instrumentation support for other libraries
You can find the full list of supported library instrumentations for OpenTelemetry JavaScript under [Supported instrumentations ](https://github.com/open-telemetry/opentelemetry-js-contrib/tree/main/metapackages/auto-instrumentations-node#supported-instrumentations).
Alternatively, you can search the OpenTelemetry registry to find out if OpenTelemetry supports instrumentation for your library under [Registry](https://opentelemetry.io/ecosystem/registry/).
## Manually creating trace data
------
#### [ With X-Ray SDK ]
Using X-Ray, the `aws-xray-sdk` package code was required to manually create segments and their child sub-segments to trace your application.
```
var AWSXRay = require('aws-xray-sdk');
AWSXRay.enableManualMode();
var segment = new AWSXRay.Segment('myApplication');
captureFunc('1', function(subsegment1) {
captureFunc('2', function(subsegment2) {
}, subsegment1);
}, segment);
segment.close();
segment.flush();
```
------
#### [ With OpenTelemetry SDK ]
You can create and use custom spans to monitor the performance of internal activities that are not captured by instrumentation libraries. Note that only spans of kind Server are converted into X-Ray segments, all other spans are converted into X-Ray sub-segments. For more information, see [Segments](https://docs.aws.amazon.com/xray/latest/devguide/xray-concepts.html#xray-concepts-segments).
You will need a Tracer instance after you have configured the OpenTelemetry SDK in Tracing Setup to create Spans. You can create as many Tracer instances as needed, but it is common to have one Tracer for an entire application.
```
const { trace, SpanKind } = require('@opentelemetry/api');
// Get a tracer instance
const tracer = trace.getTracer('your-tracer-name');
...
// This span will appear as a segment in X-Ray
tracer.startActiveSpan('server', { kind: SpanKind.SERVER }, span => {
// Do work here
// This span will appear as a subsegment in X-Ray
tracer.startActiveSpan('operation2', { kind: SpanKind.INTERNAL }, innerSpan => {
// Do more work here
innerSpan.end();
});
span.end();
});
```
**Adding annotations and metadata to traces with OpenTelemetry SDK**
You can also add custom key-value pairs as attributes in your spans. Note that by default, all these span attributes will be converted into metadata in X-Ray raw data. To ensure that an attribute is converted into an annotation and not metadata, add the attribute's key to the list of the `aws.xray.annotations` attribute. For more information, see [Enable The Customized X-Ray Annotations](https://aws-otel.github.io/docs/getting-started/x-ray#enable-the-customized-x-ray-annotations).
```
tracer.startActiveSpan('server', { kind: SpanKind.SERVER }, span => {
span.setAttribute('metadataKey', 'metadataValue');
span.setAttribute('annotationKey', 'annotationValue');
// The following ensures that "annotationKey: annotationValue" is an annotation in X-Ray raw data.
span.setAttribute('aws.xray.annotations', ['annotationKey']);
// Do work here
span.end();
});
```
------
## Lambda instrumentation
------
#### [ With X-Ray SDK ]
After you enable *Active Tracing* for your Lambda function, the X-Ray SDK was required without additional configuration. Lambda creates a segment representing the Lambda handler invocation, and you created sub-segments or instrument libraries using the X-Ray SDK without any additional configuration.
------
#### [ With OpenTelemetry SDK ]
You can automatically instrument your Lambda with AWS vended Lambda layers. There are two solutions:
+ (Recommended) AWS Lambda Layer for OpenTelemetry
**Note**
This Lambda layer has CloudWatch Application Signals enabled by default, which enables performance and health monitoring for your Lambda application by collecting both metrics and traces. If you only want tracing, you should set the Lambda environment variable `OTEL_AWS_APPLICATION_SIGNALS_ENABLED=false`. For more information, see [Enable your applications on Lambda ](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-Application-Signals-Enable-LambdaMain.html).
+ AWS managed Lambda layer for ADOT JS. For more information, see [AWS Distro for OpenTelemetry Lambda Support For JavaScript](https://aws-otel.github.io/docs/getting-started/lambda/lambda-js).
**Manually creating Spans with Lambda instrumentation**
While the ADOT JavaScript Lambda Layer provides automatic instrumentation for your Lambda function, you might find the need to perform manual instrumentation in your Lambda, for example, to provide custom data or to instrument code within the Lambda function itself that is not covered by library instrumentations.
To perform manual instrumentation alongside the automatic instrumentation, you will need to add `@opentelemetry/api` as a dependency. The version of this dependency is recommended to be the same version of the same dependency that is used by the ADOT JavaScript SDK. You can use the OpenTelemetry API to manually create spans in your Lambda function.
To add the `@opentelemetry/api` dependency using NPM:
```
npm install @opentelemetry/api
```
------
# Migrate to OpenTelemetry .NET
When using X-Ray Tracing in your .NET applications, the X-Ray .NET SDK with manual efforts is used for instrumentation.
This section provides code examples in the [Manual instrumentation solutions with the SDK](#manual-instrumentation-dotnet) section for migrating from the X-Ray manual instrumentation solution to OpenTelemetry manual Instrumentation solutions for .NET. Alternatively, you can migrate from X-Ray manual instrumentation to OpenTelemetry automatic instrumentation solutions to instrument .NET applications without having to modify application source code in the [Zero code automatic instrumentation solutions](#zero-code-instrumentation-dotnet) section.
**Topics**
+ [Zero code automatic instrumentation solutions](#zero-code-instrumentation-dotnet)
+ [Manual instrumentation solutions with the SDK](#manual-instrumentation-dotnet)
+ [Manually creating trace data](#manual-trace-creation-dotnet)
+ [Tracing incoming requests (ASP.NET and ASP.NET core instrumentation)](#tracing-incoming-requests-dotnet)
+ [AWS SDK instrumentation](#aws-sdk-instrumentation-dotnet)
+ [Instrumenting outgoing HTTP calls](#http-instrumentation-dotnet)
+ [Instrumentation support for other libraries](#other-libraries-dotnet)
+ [Lambda instrumentation](#lambda-instrumentation)
## Zero code automatic instrumentation solutions
OpenTelemetry provides zero-code auto-instrumentation solutions. These solutions trace requests without requiring changes to your application code.
**OpenTelemetry-based automatic instrumentation options**
1. Using the AWS Distro for OpenTelemetry (ADOT) auto-Instrumentation for .NET – To automatically instrument .NET applications, see [Tracing and Metrics with the AWS Distro for OpenTelemetry .NET Auto-Instrumentation](https://aws-otel.github.io/docs/getting-started/dotnet-sdk/auto-instr).
(Optional) Enable CloudWatch Application Signals when automatically instrumenting your applications on AWS with the ADOT .NET auto-instrumentation to:
+ Monitor current application health
+ Track long-term application performance against business objectives
+ Get a unified, application-centric view of your applications, services, and dependencies
+ Monitor and triage application health
For more information, see [Application Signals](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-Application-Monitoring-Sections.html).
1. Using the OpenTelemetry .Net zero-code automatic instrumentation – To automatically instrument with OpenTelemetry .Net, see [Tracing and Metrics with the AWS Distro for OpenTelemetry .NET Auto-Instrumentation](https://aws-otel.github.io/docs/getting-started/dotnet-sdk/auto-instr).
## Manual instrumentation solutions with the SDK
------
#### [ Tracing configuration with X-Ray SDK ]
For .NET web applications, the X-Ray SDK is configured in the appSettings section of the `Web.config` file.
Example Web.config
```
```
For .NET Core, a file named `appsettings.json` with a top-level key named `XRay` is used, and then a configuration object is built o initialize the X-Ray recorder.
Example for .NET `appsettings.json`
```
{
"XRay": {
"AWSXRayPlugins": "EC2Plugin"
}
}
```
Example for .NET Core Program.cs – Recorder configuration
```
using Amazon.XRay.Recorder.Core;
...
AWSXRayRecorder.InitializeInstance(configuration);
```
------
#### [ Tracing configuration with OpenTelemetry SDK ]
****
Add these dependencies:
```
dotnet add package OpenTelemetry
dotnet add package OpenTelemetry.Contrib.Extensions.AWSXRay
dotnet add package OpenTelemetry.Sampler.AWS --prerelease
dotnet add package OpenTelemetry.Resources.AWS
dotnet add package OpenTelemetry.Exporter.OpenTelemetryProtocol
dotnet add package OpenTelemetry.Extensions.Hosting
dotnet add package OpenTelemetry.Instrumentation.AspNetCore
```
For your .NET application, configure the OpenTelemetry SDK by setting up the Global TracerProvider. The following example configuration also enables instrumentation for `ASP.NET Core`. To instrument `ASP.NET`, see [Tracing incoming requests (ASP.NET and ASP.NET core instrumentation)](#tracing-incoming-requests-dotnet). To use OpenTelemetry with other frameworks, see [Registry ](https://opentelemetry.io/ecosystem/registry/) for more libraries for supported frameworks.
It is recommend that you configure the following components:
+ `An OTLP Exporter` – Required for exporting traces to the CloudWatch Agent/OpenTelemetry Collector
+ An AWS X-Ray Propagator – Required for propagating the Trace Context to [AWS Services that are integrated with X-Ray](https://docs.aws.amazon.com/xray/latest/devguide/xray-services.html)
+ An AWS X-Ray Remote Sampler – Required if you need to [sample requests using X-Ray sampling rules](https://docs.aws.amazon.com/xray/latest/devguide/xray-console-sampling.html)
+ `Resource Detectors` (for example, Amazon EC2 Resource Detector) - To detect metadata of the host running your application
```
using OpenTelemetry;
using OpenTelemetry.Contrib.Extensions.AWSXRay.Trace;
using OpenTelemetry.Sampler.AWS;
using OpenTelemetry.Trace;
using OpenTelemetry.Resources;
var builder = WebApplication.CreateBuilder(args);
var serviceName = "MyServiceName";
var serviceVersion = "1.0.0";
var resourceBuilder = ResourceBuilder
.CreateDefault()
.AddService(serviceName: serviceName)
.AddAWSEC2Detector();
builder.Services.AddOpenTelemetry()
.ConfigureResource(resource => resource
.AddAWSEC2Detector()
.AddService(
serviceName: serviceName,
serviceVersion: serviceVersion))
.WithTracing(tracing => tracing
.AddSource(serviceName)
.AddAspNetCoreInstrumentation()
.AddOtlpExporter()
.SetSampler(AWSXRayRemoteSampler.Builder(resourceBuilder.Build())
.SetEndpoint("http://localhost:2000")
.Build()));
Sdk.SetDefaultTextMapPropagator(new AWSXRayPropagator()); // configure X-Ray propagator
```
To use OpenTelemetry for a console app, add the following OpenTelemetry configuration at the startup of your program.
```
using OpenTelemetry;
using OpenTelemetry.Contrib.Extensions.AWSXRay.Trace;
using OpenTelemetry.Trace;
using OpenTelemetry.Resources;
var serviceName = "MyServiceName";
var resourceBuilder = ResourceBuilder
.CreateDefault()
.AddService(serviceName: serviceName)
.AddAWSEC2Detector();
var tracerProvider = Sdk.CreateTracerProviderBuilder()
.AddSource(serviceName)
.ConfigureResource(resource =>
resource
.AddAWSEC2Detector()
.AddService(
serviceName: serviceName,
serviceVersion: serviceVersion
)
)
.AddOtlpExporter() // default address localhost:4317
.SetSampler(new TraceIdRatioBasedSampler(1.00))
.Build();
Sdk.SetDefaultTextMapPropagator(new AWSXRayPropagator()); // configure X-Ray propagator
```
------
## Manually creating trace data
------
#### [ With X-Ray SDK ]
With the X-Ray SDK, the `BeginSegment` and `BeginSubsegment` methods were needed to manually create X-Ray segments and sub-segments.
```
using Amazon.XRay.Recorder.Core;
AWSXRayRecorder.Instance.BeginSegment("segment name"); // generates `TraceId` for you
try
{
// Do something here
// can create custom subsegments
AWSXRayRecorder.Instance.BeginSubsegment("subsegment name");
try
{
DoSometing();
}
catch (Exception e)
{
AWSXRayRecorder.Instance.AddException(e);
}
finally
{
AWSXRayRecorder.Instance.EndSubsegment();
}
}
catch (Exception e)
{
AWSXRayRecorder.Instance.AddException(e);
}
finally
{
AWSXRayRecorder.Instance.EndSegment();
}
```
------
#### [ With OpenTelemetry SDK ]
In .NET, you can use the activity API to create custom spans to monitor the performance of internal activities that are not captured by instrumentation libraries. Note that only spans of kind Server are converted into X-Ray segments, all other spans are converted into X-Ray sub-egments.
You can create as many `ActivitySource` instances as needed, but it is recommended to have only one for an entire application/service.
```
using System.Diagnostics;
ActivitySource activitySource = new ActivitySource("ActivitySourceName", "ActivitySourceVersion");
...
using (var activity = activitySource.StartActivity("ActivityName", ActivityKind.Server)) // this will be translated to a X-Ray Segment
{
// Do something here
using (var internalActivity = activitySource.StartActivity("ActivityName", ActivityKind.Internal)) // this will be translated to an X-Ray Subsegment
{
// Do something here
}
}
```
**Adding annotations and metadata to traces with OpenTelemetry SDK**
You can also add custom key-value pairs as attributes onto your spans by using the `SetTag` method on an activity. Note that by default, all the span attributes will be converted into metadata in X-Ray raw data. To ensure that an attribute is converted into an annotation and not metadata, you can add that attribute's key to the list of `aws.xray.annotations` attribute.
```
using (var activity = activitySource.StartActivity("ActivityName", ActivityKind.Server)) // this will be translated to a X-Ray Segment
{
activity.SetTag("metadataKey", "metadataValue");
activity.SetTag("annotationKey", "annotationValue");
string[] annotationKeys = {"annotationKey"};
activity.SetTag("aws.xray.annotations", annotationKeys);
// Do something here
using (var internalActivity = activitySource.StartActivity("ActivityName", ActivityKind.Internal)) // this will be translated to an X-Ray Subsegment
{
// Do something here
}
}
```
**With OpenTelemetry automatic instrumentation**
If you are using an OpenTelemetry automatic instrumentation solution for .NET, and if you need to perform manual instrumentation in your application, for example, to instrument code within the application itself for sections that are not covered by any auto-instrumentation library.
Since there can only be one global `TracerProvider`, manual instrumentation should not instantiate its own `TracerProvider` if used together alongside auto-instrumentation. When `TracerProvider` is used, custom manual tracing works the same way when using automatic instrumentation or manual instrumentation through the OpenTelemetry SDK.
------
## Tracing incoming requests (ASP.NET and ASP.NET core instrumentation)
------
#### [ With X-Ray SDK ]
To instrument requests served by the ASP.NET application, see [https://docs.aws.amazon.com/xray/latest/devguide/xray-sdk-dotnet-messagehandler.html](https://docs.aws.amazon.com/xray/latest/devguide/xray-sdk-dotnet-messagehandler.html) for information on how to call `RegisterXRay` in the `Init` method of your `global.asax` file.
```
AWSXRayASPNET.RegisterXRay(this, "MyApp");
```
To instrument requests served by your ASP.NET core application, the `UseXRay` method is called before any other middleware in the `Configure` method of your Startup class.
```
app.UseXRay("MyApp");
```
------
#### [ With OpenTelemetry SDK ]
OpenTelemetry also provides instrumentation libraries to collect traces for incoming web requests for ASP.NET and ASP.NET core. The following section lists the steps needed to add and enable these library instrumentations for your OpenTelemetry configuration, including how to add [ASP.NET](https://learn.microsoft.com/en-us/aspnet/overview) or [ASP.NET](https://learn.microsoft.com/en-us/aspnet/core/?view=aspnetcore-9.0) core instrumentation when creating the Tracer Provider.
For information on how to enable OpenTelemetry.Instrumentation.AspNet, see [Steps to enable OpenTelemetry.Instrumentation.AspNet ](https://github.com/open-telemetry/opentelemetry-dotnet-contrib/tree/main/src/OpenTelemetry.Instrumentation.AspNet#steps-to-enable-opentelemetryinstrumentationaspnet) and for information on how to enable OpenTelemetry.Instrumentation.AspNetCore, see [Steps to enable OpenTelemetry.Instrumentation.AspNetCore ](https://github.com/open-telemetry/opentelemetry-dotnet-contrib/tree/main/src/OpenTelemetry.Instrumentation.AspNetCore#steps-to-enable-opentelemetryinstrumentationaspnetcore).
------
## AWS SDK instrumentation
------
#### [ With X-Ray SDK ]
Install all AWS SDK clients by calling `RegisterXRayForAllServices()`.
```
using Amazon.XRay.Recorder.Handlers.AwsSdk;
AWSSDKHandler.RegisterXRayForAllServices(); //place this before any instantiation of AmazonServiceClient
AmazonDynamoDBClient client = new AmazonDynamoDBClient(RegionEndpoint.USWest2); // AmazonDynamoDBClient is automatically registered with X-Ray
```
Use one of the following methods for specific AWS service client instrumentation.
```
AWSSDKHandler.RegisterXRay(); // Registers specific type of AmazonServiceClient : All instances of IAmazonDynamoDB created after this line are registered
AWSSDKHandler.RegisterXRayManifest(String path); // To configure custom AWS Service Manifest file. This is optional, if you have followed "Configuration" section
```
------
#### [ With OpenTelemetry SDK ]
For the following code example, you will need the following dependency:
```
dotnet add package OpenTelemetry.Instrumentation.AWS
```
To instrument the AWS SDK, update the OpenTelemetry SDK configuration where the Global TracerProvider is setup.
```
builder.Services.AddOpenTelemetry()
...
.WithTracing(tracing => tracing
.AddAWSInstrumentation()
...
```
------
## Instrumenting outgoing HTTP calls
------
#### [ With X-Ray SDK ]
The X-Ray .NET SDK traces outgoing HTTP calls through the extension methods `GetResponseTraced()` or `GetAsyncResponseTraced()` when using `System.Net.HttpWebRequest`, or by using the `HttpClientXRayTracingHandler` handler when using `System.Net.Http.HttpClient`.
------
#### [ With OpenTelemetry SDK ]
For the following code example, you will need the following dependency:
```
dotnet add package OpenTelemetry.Instrumentation.Http
```
To instrument `System.Net.Http.HttpClient` and `System.Net.HttpWebRequest`, update the OpenTelemetry SDK configuration where the Global TracerProvider is setup.
```
builder.Services.AddOpenTelemetry()
...
.WithTracing(tracing => tracing
.AddHttpClientInstrumentation()
...
```
------
## Instrumentation support for other libraries
You can search and filter the OpenTelemetry Registry for .NET Instrumentation Libraries to find out if OpenTelemetry supports instrumentation for your Library. See the [Registry ](https://opentelemetry.io/ecosystem/registry/) to start searching.
## Lambda instrumentation
------
#### [ With X-Ray SDK ]
The following procedure was required to use the X-Ray SDK with Lambda:
1. Enable *Active Tracing* on your Lambda function
1. The Lambda service creates a segment that represents your handler's invocation
1. Create sub-segments or instrument libraries using the X-Ray SDK
------
#### [ With OpenTelemetry-based solutions ]
You can automatically instrument your Lambda with AWS vended Lambda layers. There are two solutions:
+ (Recommended) [AWS Lambda Layer for OpenTelemetry](https://docs.aws.amazon.com/lambda/latest/dg/monitoring-application-signals.html)
+ For better performance, you may want to consider using `OpenTelemetry Manual Instrumentation` to generate OpenTelemetry traces for your Lambda function.
------
**OpenTelemetry manual instrumentation for AWS Lambda**
The following is the Lambda function code (without instrumentation) example.
```
using System;
using System.Text;
using System.Threading.Tasks;
using Amazon.Lambda.Core;
using Amazon.S3;
using Amazon.S3.Model;
// Assembly attribute to enable Lambda function logging
[assembly: LambdaSerializer(typeof(Amazon.Lambda.Serialization.SystemTextJson.DefaultLambdaJsonSerializer))]
namespace ExampleLambda;
public class ListBucketsHandler
{
private static readonly AmazonS3Client s3Client = new();
// new Lambda function handler passed in
public async Task HandleRequest(object input, ILambdaContext context)
{
try
{
var DoListBucketsAsyncResponse = await DoListBucketsAsync();
context.Logger.LogInformation($"Results: {DoListBucketsAsyncResponse.Buckets}");
context.Logger.LogInformation($"Successfully called ListBucketsAsync");
return "Success!";
}
catch (Exception ex)
{
context.Logger.LogError($"Failed to call ListBucketsAsync: {ex.Message}");
throw;
}
}
private async Task DoListBucketsAsync()
{
try
{
var putRequest = new ListBucketsRequest
{
};
var response = await s3Client.ListBucketsAsync(putRequest);
return response;
}
catch (AmazonS3Exception ex)
{
throw new Exception($"Failed to call ListBucketsAsync: {ex.Message}", ex);
}
}
}
```
To manually instrument your Lambda handler and the Amazon S3 client, do the following.
1. Instantiate a TracerProvider – The TracerProvider is recommended to be configured with an `XrayUdpSpanExporter`, a ParentBased Always On Sampler, and a `Resource` with `service.name` set to the Lambda function name.
1. Instrument the Amazon S3 client with the OpenTemetry AWS SDK instrumentation by calling `AddAWSInstrumentation()` to add AWS SDK client instrumentation to `TracerProvider`
1. Create a wrapper function with the same signature as the original Lambda function. Call `AWSLambdaWrapper.Trace()` API and pass `TracerProvider`, the original Lambda function, and its inputs as parameters. Set the wrapper function as the Lambda handler input.
For the following code example, you will need the following dependencies:
```
dotnet add package OpenTelemetry.Instrumentation.AWSLambda
dotnet add package OpenTelemetry.Instrumentation.AWS
dotnet add package OpenTelemetry.Resources.AWS
dotnet add package AWS.Distro.OpenTelemetry.Exporter.Xray.Udp
```
The following code demonstrates the Lambda function after the required changes. You can create additional custom spans to complement the automatically provided spans.
```
using Amazon.Lambda.Core;
using Amazon.S3;
using Amazon.S3.Model;
using OpenTelemetry;
using OpenTelemetry.Instrumentation.AWSLambda;
using OpenTelemetry.Trace;
using AWS.Distro.OpenTelemetry.Exporter.Xray.Udp;
using OpenTelemetry.Resources;
// Assembly attribute to enable Lambda function logging
[assembly: LambdaSerializer(typeof(Amazon.Lambda.Serialization.SystemTextJson.DefaultLambdaJsonSerializer))]
namespace ExampleLambda;
public class ListBucketsHandler
{
private static readonly AmazonS3Client s3Client = new();
TracerProvider tracerProvider = Sdk.CreateTracerProviderBuilder()
.AddAWSLambdaConfigurations()
.AddProcessor(
new SimpleActivityExportProcessor(
// AWS_LAMBDA_FUNCTION_NAME Environment Variable will be defined in AWS Lambda Environment
new XrayUdpExporter(ResourceBuilder.CreateDefault().AddService(Environment.GetEnvironmentVariable("AWS_LAMBDA_FUNCTION_NAME")).Build())
)
)
.AddAWSInstrumentation()
.SetSampler(new ParentBasedSampler(new AlwaysOnSampler()))
.Build();
// new Lambda function handler passed in
public async Task HandleRequest(object input, ILambdaContext context)
=> await AWSLambdaWrapper.Trace(tracerProvider, OriginalHandleRequest, input, context);
public async Task OriginalHandleRequest(object input, ILambdaContext context)
{
try
{
var DoListBucketsAsyncResponse = await DoListBucketsAsync();
context.Logger.LogInformation($"Results: {DoListBucketsAsyncResponse.Buckets}");
context.Logger.LogInformation($"Successfully called ListBucketsAsync");
return "Success!";
}
catch (Exception ex)
{
context.Logger.LogError($"Failed to call ListBucketsAsync: {ex.Message}");
throw;
}
}
private async Task DoListBucketsAsync()
{
try
{
var putRequest = new ListBucketsRequest
{
};
var response = await s3Client.ListBucketsAsync(putRequest);
return response;
}
catch (AmazonS3Exception ex)
{
throw new Exception($"Failed to call ListBucketsAsync: {ex.Message}", ex);
}
}
}
```
When invoking this Lambda, you will see the following trace in the Trace Map in the CloudWatch console:
![\[Trace map in CloudWatch console for .Net\]](http://docs.aws.amazon.com/xray/latest/devguide/images/deprecation_dotnet.png)
# Migrate to OpenTelemetry Python
This guide helps you migrate Python applications from X-Ray SDK to OpenTelemetry instrumentation. It covers both automatic and manual instrumentation approaches, with code examples for common scenarios.
**Topics**
+ [Zero code automatic instrumentation solutions](#zero-code-python)
+ [Manually instrument your applications](#manual-instrumentation-python)
+ [Tracing setup initialization](#manual-instrumentation-python-tracing)
+ [Tracing incoming requests](#tracing-incoming-requests-python)
+ [AWS SDK instrumentation](#aws-sdk-instrumentation-python)
+ [Instrumenting outgoing HTTP calls through requests](#http-instrumentation-python)
+ [Instrumentation support for other libraries](#xray-migration-libraries-python)
+ [Manually creating trace data](#manual-trace-creation-python)
+ [Lambda instrumentation](#lambda-instrumentation-python)
## Zero code automatic instrumentation solutions
With X-Ray SDK, you had to modify your application code to trace requests. OpenTelemetry offers zero-code auto-instrumentation solutions to trace requests. With OpenTelemetry, you have the option of using zero-code auto-instrumentation solutions to trace requests.
**Zero code with OpenTelemetry-based automatic instrumentations**
1. Using the AWS Distro for OpenTelemetry (ADOT) auto-Instrumentation for Python – For automatic instrumentation for Python applications, see [Tracing and Metrics with the AWS Distro for OpenTelemetry Python Auto-Instrumentation](https://aws-otel.github.io/docs/getting-started/python-sdk/auto-instr).
(Optional) You can also enable CloudWatch Application Signals when automatically instrumenting your applications on AWS with the ADOT Python auto-instrumentation to monitor current application health and track long-term application performance against your business objectives. Application Signals provides you with a unified, application-centric view of your applications, services, and dependencies, and helps you monitor and triage application health.
1. Using the OpenTelemetry Python zero-code automatic instrumentation – For automatic instrumentation with the OpenTelemetry Python, see [Python zero-code instrumentation](https://opentelemetry.io/docs/zero-code/python/).
## Manually instrument your applications
You can manually instrument your applications using the `pip` command.
------
#### [ With X-Ray SDK ]
```
pip install aws-xray-sdk
```
------
#### [ With OpenTelemetry SDK ]
```
pip install opentelemetry-api opentelemetry-sdk opentelemetry-exporter-otlp opentelemetry-propagator-aws-xray
```
------
## Tracing setup initialization
------
#### [ With X-Ray SDK ]
In X-Ray, the global `xray_recorder` is initialized and used it to generate segments and sub-segments.
------
#### [ With OpenTelemetry SDK ]
**Note**
X-Ray Remote Sampling is currently not available to be configured for OpenTelemetry Python. However, support for X-Ray Remote Sampling is currently available through the ADOT Auto-Instrumentation for Python.
In OpenTelemetry, you need to initialize a global `TracerProvider`. Using this `TracerProvider`, you can acquire a [Tracer](https://opentelemetry.io/docs/concepts/signals/traces/#tracer) that you can use to generate spans anywhere in your application. It is recommend that you configure the following components:
+ `OTLPSpanExporter` – Required for exporting traces to the CloudWatch Agent/OpenTelemetry Collector
+ An AWS X-Ray Propagator – Required for propagating the Trace Context to AWS Services that are integrated with X-Ray
```
from opentelemetry import (
trace,
propagate
)
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter
from opentelemetry.sdk.trace.export import BatchSpanProcessor
from opentelemetry.propagators.aws import AwsXRayPropagator
# Sends generated traces in the OTLP format to an OTel Collector running on port 4318
otlp_exporter = OTLPSpanExporter(endpoint="http://localhost:4318/v1/traces")
# Processes traces in batches as opposed to immediately one after the other
span_processor = BatchSpanProcessor(otlp_exporter)
# More configurations can be done here. We will visit them later.
# Sets the global default tracer provider
provider = TracerProvider(active_span_processor=span_processor)
trace.set_tracer_provider(provider)
# Configures the global propagator to use the X-Ray Propagator
propagate.set_global_textmap(AwsXRayPropagator())
# Creates a tracer from the global tracer provider
tracer = trace.get_tracer("my.tracer.name")
# Use this tracer to create Spans
```
------
**With ADOT auto-Instrumentation for Python**
You can use the ADOT auto-instrumentation for Python to automatically configure OpenTelemetry for your Python applications. By using ADOT auto-instrumentation, you do not need to make manual code changes to trace incoming requests, or to trace libraries such as the AWS SDK or HTTP clients. For more information, see [Tracing and Metrics with the AWS Distro for OpenTelemetry Python Auto-Instrumentation](https://aws-otel.github.io/docs/getting-started/python-sdk/auto-instr).
ADOT auto-instrumentation for Python supports:
+ X-Ray remote sampling through the environment variable `export OTEL_TRACES_SAMPLER=xray`
+ X-Ray trace context propagation (enabled by default)
+ Resource detection (resource detection for Amazon EC2, Amazon ECS, and Amazon EKS environments are enabled by default)
+ Automatic library instrumentations for all supported OpenTelemetry instrumentations are enabled by default. You can disable selectively through the `OTEL_PYTHON_DISABLED_INSTRUMENTATIONS `environment variable. (all are enabled by default)
+ Manual creation of Spans
**From X-Ray service plug-ins to OpenTelemetry AWS resource providers**
The X-Ray SDK provides plug-ins that you could add to the `xray_recorder` to capture the platform specific information from the hosted service like Amazon EC2, Amazon ECS, and Elastic Beanstalk. It's similar to the Resource Providers in OpenTelemetry that captures the information as Resource attributes. There are multiple Resource Providers available for different AWS platforms.
+ Start by installing the AWS extension package, `pip install opentelemetry-sdk-extension-aws`
+ Configure the desired resource detector. The following example shows how to configure the Amazon EC2 resource provider in OpenTelemetry SDK
```
from opentelemetry import trace
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.extension.aws.resource.ec2 import (
AwsEc2ResourceDetector,
)
from opentelemetry.sdk.resources import get_aggregated_resources
provider = TracerProvider(
active_span_processor=span_processor,
resource=get_aggregated_resources([
AwsEc2ResourceDetector(),
]))
trace.set_tracer_provider(provider)
```
## Tracing incoming requests
------
#### [ With X-Ray SDK ]
The X-Ray Python SDK supports application frameworks like Django, Flask, and Bottle in tracing the incoming requests for Python applications running on them. This is done by adding `XRayMiddleware` to the application for each framework.
------
#### [ With OpenTelemetry SDK ]
OpenTelemetry provides instrumentations for [Django](https://opentelemetry-python-contrib.readthedocs.io/en/latest/instrumentation/django/django.html) and [Flask](https://opentelemetry-python-contrib.readthedocs.io/en/latest/instrumentation/flask/flask.html) through the specific instrumentation libraries. There is no instrumentation for Bottle available in OpenTelemetry, applications can still be traced by using the [OpenTelemetry WSGI Instrumentation ](https://opentelemetry-python-contrib.readthedocs.io/en/latest/instrumentation/wsgi/wsgi.html).
For the following code example, you need the following dependency:
```
pip install opentelemetry-instrumentation-flask
```
You must initialize the OpenTelemetry SDK and register the global TracerProvider before adding instrumentations for your application framework. Without it, the trace operations will be `no-ops`. Once you have configured the global `TracerProvider`, you can use the instrumentor for your application framework. The following example demonstrates a Flask application.
```
from flask import Flask
from opentelemetry import trace
from opentelemetry.instrumentation.flask import FlaskInstrumentor
from opentelemetry.sdk.extension.aws.resource import AwsEc2ResourceDetector
from opentelemetry.sdk.resources import get_aggregated_resources
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.trace.export import BatchSpanProcessor, ConsoleSpanExporter
provider = TracerProvider(resource=get_aggregated_resources(
[
AwsEc2ResourceDetector(),
]))
processor = BatchSpanProcessor(ConsoleSpanExporter())
provider.add_span_processor(processor)
trace.set_tracer_provider(provider)
# Creates a tracer from the global tracer provider
tracer = trace.get_tracer("my.tracer.name")
app = Flask(__name__)
# Instrument the Flask app
FlaskInstrumentor().instrument_app(app)
@app.route('/')
def hello_world():
return 'Hello World!'
if __name__ == '__main__':
app.run()
```
------
## AWS SDK instrumentation
------
#### [ With X-Ray SDK ]
The X-Ray Python SDK traces the AWS SDK client request by patching the `botocore` library. For more information, see [Tracing AWS SDK calls with the X-Ray SDK for Python ](https://docs.aws.amazon.com/xray/latest/devguide/xray-sdk-python-awssdkclients.html). In your application, the `patch_all()` method is used to instrument all the libraries or patch selectively using the `botocore` or `boto3`libraries using `patch((['botocore']))`. Any of the chosen method instruments all the Boto3 clients in your application and generates a sub-segment for any call made using these clients.
------
#### [ With OpenTelemetry SDK ]
For the following code example, you will need the following dependency:
```
pip install opentelemetry-instrumentation-botocore
```
Use the [OpenTelemetry Botocore Instrumentation ](https://opentelemetry-python-contrib.readthedocs.io/en/latest/instrumentation/botocore/botocore.html) programmatically to instrument all the Boto3 clients in your application. The following example demonstrates the `botocore` instrumentation.
```
import boto3
import opentelemetry.trace as trace
from botocore.exceptions import ClientError
from opentelemetry.sdk.trace import TracerProvider
from opentelemetry.sdk.resources import get_aggregated_resources
from opentelemetry.sdk.trace.export import (
BatchSpanProcessor,
ConsoleSpanExporter,
)
from opentelemetry.instrumentation.botocore import BotocoreInstrumentor
provider = TracerProvider()
processor = BatchSpanProcessor(ConsoleSpanExporter())
provider.add_span_processor(processor)
trace.set_tracer_provider(provider)
# Creates a tracer from the global tracer provider
tracer = trace.get_tracer("my.tracer.name")
# Instrument BotoCore
BotocoreInstrumentor().instrument()
# Initialize S3 client
s3 = boto3.client("s3", region_name="us-east-1")
# Your bucket name
bucket_name = "my-example-bucket"
# Get bucket location (as an example of describing it)
try:
response = s3.get_bucket_location(Bucket=bucket_name)
region = response.get("LocationConstraint") or "us-east-1"
print(f"Bucket '{bucket_name}' is in region: {region}")
# Optionally, get bucket's creation date via list_buckets
buckets = s3.list_buckets()
for bucket in buckets["Buckets"]:
if bucket["Name"] == bucket_name:
print(f"Bucket created on: {bucket['CreationDate']}")
break
except ClientError as e:
print(f"Failed to describe bucket: {e}")
```
------
## Instrumenting outgoing HTTP calls through requests
------
#### [ With X-Ray SDK ]
The X-Ray Python SDK traces outgoing HTTP calls through requests by patching the requests library. For more information, see [ Tracing calls to downstream HTTP web services using the X-Ray SDK for Python](https://docs.aws.amazon.com/xray/latest/devguide/xray-sdk-python-httpclients.html). In your application, you can use the `patch_all()` method to instrument all the libraries or by selectively patching the requests libraries by using `patch((['requests']))`. Any of the option instruments the `requests` library, generating a sub-segment for any call made through `requests`.
------
#### [ With OpenTelemetry SDK ]
For the following code example, you will need the following dependency:
```
pip install opentelemetry-instrumentation-requests
```
Use the OpenTelemetry Requests Instrumentation programmatically to instrument the requests library to generate traces for HTTP requests made by it in your application. For more information, see [OpenTelemetry requests Instrumentation ](https://opentelemetry-python-contrib.readthedocs.io/en/latest/instrumentation/requests/requests.html). The following example demonstrates the `requests` library instrumentation.
```
from opentelemetry.instrumentation.requests import RequestsInstrumentor
# Instrument Requests
RequestsInstrumentor().instrument()
...
example_session = requests.Session()
example_session.get(url="https://example.com")
```
Alternatively, you can also instrument the underlying `urllib3` library to trace HTTP requests:
```
# pip install opentelemetry-instrumentation-urllib3
from opentelemetry.instrumentation.urllib3 import URLLib3Instrumentor
# Instrument urllib3
URLLib3Instrumentor().instrument()
...
example_session = requests.Session()
example_session.get(url="https://example.com")
```
------
## Instrumentation support for other libraries
You can find the full list of supported Library instrumentations for OpenTelemetry Python under [Supported libraries, frameworks, application servers, and JVMs ](https://github.com/open-telemetry/opentelemetry-java-instrumentation/blob/main/docs/supported-libraries.md).
Alternatively, you can search the OpenTelemetry Registry to find out if OpenTelemetry supports instrumentation. See the [Registry](https://opentelemetry.io/ecosystem/registry/) to start searching.
## Manually creating trace data
You can create segments and sub-segments using the `xray_recorder` in your Python application. For more information, see [Instrumenting Python code manually ](https://docs.aws.amazon.com/xray/latest/devguide/xray-sdk-python-middleware.html#xray-sdk-python-middleware-manual). You can also manually add annotations and metadata to the trace data.
**Creating spans With OpenTelemetry SDK**
Use the `start_as_current_span` API to start a span and set it for creating spans. For examples on creating spans, see [Creating spans](https://opentelemetry.io/docs/languages/python/instrumentation/#creating-spans). Once a span is started and is in the current scope, you can add more information to it by adding attributes, events, exceptions, links, and so on. Like how we have segments and sub-segments in X-Ray, there are different kinds of spans in OpenTelemetry. Only the `SERVER` kind spans are converted to X-Ray segments while others are converted to X-Ray sub-segments.
```
from opentelemetry import trace
from opentelemetry.trace import SpanKind
import time
tracer = trace.get_tracer("my.tracer.name")
# Create a new span to track some work
with tracer.start_as_current_span("parent", kind=SpanKind.SERVER) as parent_span:
time.sleep(1)
# Create a nested span to track nested work
with tracer.start_as_current_span("child", kind=SpanKind.CLIENT) as child_span:
time.sleep(2)
# the nested span is closed when it's out of scope
# Now the parent span is the current span again
time.sleep(1)
# This span is also closed when it goes out of scope
```
**Adding annotations and metadata to traces with OpenTelemetry SDK**
The X-Ray Python SDK provides separate APIs, `put_annotation` and `put_metadata` for adding annotations and metadata to a trace. In OpenTelemetry SDK, the annotations and metadata are simply attributes on a span, added through the `set_attribute` API.
Span attributes that you want them to be annotations on a trace are added under the reserved key `aws.xray.annotations` whose value is a list of key-value pairs of annotations. All the other span attributes become metadata on the converted segment or sub-segment.
Additionally, if you are using the ADOT collector you can configure which span attributes should be converted to X-Ray annotations by specifying the `indexed_attributes` in the collector configuration.
The below example demonstrates how to add annotations and metadata to a trace using OpenTelemetry SDK.
```
with tracer.start_as_current_span("parent", kind=SpanKind.SERVER) as parent_span:
parent_span.set_attribute("TransactionId", "qwerty12345")
parent_span.set_attribute("AccountId", "1234567890")
# This will convert the TransactionId and AccountId to be searchable X-Ray annotations
parent_span.set_attribute("aws.xray.annotations", ["TransactionId", "AccountId"])
with tracer.start_as_current_span("child", kind=SpanKind.CLIENT) as child_span:
# The MicroTransactionId will be converted to X-Ray metadata for the child subsegment
child_span.set_attribute("MicroTransactionId", "micro12345")
```
## Lambda instrumentation
To monitor your lambda functions on X-Ray, you enable X-Ray and added appropriate permissions to the function invocation role. Additionally, if you are tracing downstream requests from your function, you would be instrumenting the code with X-Ray Python SDK.
With OpenTelemetry for X-Ray, it is recommended to use the AWS Lambda Layer for OpenTelemetry with [Application Signals](https://docs.aws.amazon.com/lambda/latest/dg/monitoring-application-signals.html) turned off. This will auto-instrument your function and will generate spans for the function invocation and any downstream request from your function. Besides tracing, if you are interested in using Application Signals to monitor the health of your function, see [Enable your applications on Lambda ](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-Application-Signals-Enable-LambdaMain.html).
+ Find the required Lambda layer ARN for your function from [AWS Lambda Layer for OpenTelemetry ARNs ](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-Application-Signals-Enable-LambdaMain.html#Enable-Lambda-Layers) and add it.
+ Set the following environment variables for your function.
+ `AWS_LAMBDA_EXEC_WRAPPER=/opt/otel-instrument` – This loads the auto-instrumentation for the function
+ `OTEL_AWS_APPLICATION_SIGNALS_ENABLED=false` – This will disable Application Signals monitoring
**Manually creating spans with Lambda instrumentation**
Additionally, you can generate custom spans within your function to track work. You can do by using only the `opentelemetry-api` package in conjunction with the AWS Lambda Layer for OpenTelemetry auto-instrumentation.
1. Include the `opentelemetry-api` as a dependency in your function
1. The following code snippet is a sample to generate custom spans
```
from opentelemetry import trace
# Get the tracer (auto‑configured by the AWS Lambda Layer for OpenTelemetry)
tracer = trace.get_tracer(__name__)
def handler(event, context):
# This span is a child of the layer's root span
with tracer.start_as_current_span("my-custom-span") as span:
span.set_attribute("key1", "value1")
span.add_event("custom-event", {"detail": "something happened"})
# Any logic you want to trace
result = some_internal_logic()
return {
"statusCode": 200,
"body": result
}
```
# Migrate to OpenTelemetry Ruby
To migrate your Ruby applications from X-Ray SDK to OpenTelemetry instrumentation, use the following code examples and guidance for manual instrumentation.
**Topics**
+ [Manually instrument your solutions with the SDK](#manual-instrumentation-ruby)
+ [Tracing incoming requests (Rails instrumentation)](#tracing-incoming-requests-ruby)
+ [AWS SDK instrumentation](#aws-sdk-instrumentation-ruby)
+ [Instrumenting outgoing HTTP calls](#http-instrumentation-ruby)
+ [Instrumentation support for other libraries](#xray-migration-libraries-ruby)
+ [Manually creating trace data](#manual-trace-creation-ruby)
+ [Lambda manual instrumentation](#lambda-instrumentation-ruby)
## Manually instrument your solutions with the SDK
------
#### [ Tracing setup with X-Ray SDK ]
X-Ray SDK for Ruby required you to configure your code with service plug-ins.
```
require 'aws-xray-sdk'
XRay.recorder.configure(plugins: [:ec2, :elastic_beanstalk])
```
------
#### [ Tracing setup with OpenTelemetry SDK ]
**Note**
X-Ray remote sampling is currently not available to be configured for OpenTelemetry Ruby.
For a Ruby on Rails application, place your configuration code in a Rails initializer. For more information, see [Getting Started](https://opentelemetry.io/docs/languages/ruby/getting-started/#initialization). For all manually instrumented Ruby programs, you must use the `OpenTelemetry::SDK.configure` method to configure the OpenTelemetry Ruby SDK.
First, install the following packages:
```
bundle add opentelemetry-sdk opentelemetry-exporter-otlp opentelemetry-propagator-xray
```
Next, configure the OpenTelemetry SDK through the configuration code that runs when your program initializes. It is recommend that you configure the following components:
+ `OTLP Exporter` – Required for exporting traces to the CloudWatch agent and OpenTelemetry collector
+ `An AWSX-Ray Propagator` – Required for propagating the trace context to AWS services that are integrated with X-Ray
```
require 'opentelemetry-sdk'
require 'opentelemetry-exporter-otlp'
# Import the gem containing the AWS X-Ray for OTel Ruby ID Generator and propagator
require 'opentelemetry-propagator-xray'
OpenTelemetry::SDK.configure do |c|
c.service_name = 'my-service-name'
c.add_span_processor(
# Use the BatchSpanProcessor to send traces in groups instead of one at a time
OpenTelemetry::SDK::Trace::Export::BatchSpanProcessor.new(
# Use the default OLTP Exporter to send traces to the ADOT Collector
OpenTelemetry::Exporter::OTLP::Exporter.new(
# The OpenTelemetry Collector is running as a sidecar and listening on port 4318
endpoint:"http://127.0.0.1:4318/v1/traces"
)
)
)
# The X-Ray Propagator injects the X-Ray Tracing Header into downstream calls
c.propagators = [OpenTelemetry::Propagator::XRay::TextMapPropagator.new]
end
```
OpenTelemetry SDKs also have the concept of library instrumentations. Enabling these will automatically create spans for libraries such as the AWS SDK. OpenTelemetry provides the option to enable all library instrumentations or specify which library instrumentations to enable.
To enable all instrumentations, first install the `opentelemetry-instrumentation-all` package:
```
bundle add opentelemetry-instrumentation-all
```
Next, update the configuration to enable all library instrumentations as shown below:
```
require 'opentelemetry/instrumentation/all'
...
OpenTelemetry::SDK.configure do |c|
...
c.use_all() # Enable all instrumentations
end
```
OpenTelemetry SDKs also have the concept of library instrumentations. Enabling these will automatically create spans for libraries such as the AWS SDK. OpenTelemetry provides the option to enable all library instrumentations or specify which library instrumentations to enable.
To enable all instrumentations, first install the `opentelemetry-instrumentation-all` package:
```
bundle add opentelemetry-instrumentation-all
```
Next, update the configuration to enable all library instrumentations as shown below:
```
require 'opentelemetry/instrumentation/all'
...
OpenTelemetry::SDK.configure do |c|
...
c.use_all() # Enable all instrumentations
end
```
------
## Tracing incoming requests (Rails instrumentation)
------
#### [ With X-Ray SDK ]
With X-Ray SDK, X-Ray tracing is configured for the Rails framework upon initialization.
**Example** – config/initializers/aws\$1xray.rb
```
Rails.application.config.xray = {
name: 'my app',
patch: %I[net_http aws_sdk],
active_record: true
}
```
------
#### [ With OpenTelemetry SDK ]
First, install the following packages:
```
bundle add opentelemetry-instrumentation-rack opentelemetry-instrumentation-rails opentelemetry-instrumentation-action_pack opentelemetry-instrumentation-active_record opentelemetry-instrumentation-action_view
```
Next, update the configuration to enable instrumentation for your Rails application as shown below:
```
# During SDK configuration
OpenTelemetry::SDK.configure do |c|
...
c.use 'OpenTelemetry::Instrumentation::Rails'
c.use 'OpenTelemetry::Instrumentation::Rack'
c.use 'OpenTelemetry::Instrumentation::ActionPack'
c.use 'OpenTelemetry::Instrumentation::ActiveSupport'
c.use 'OpenTelemetry::Instrumentation::ActionView'
...
end
```
------
## AWS SDK instrumentation
------
#### [ With X-Ray SDK ]
To instrument outgoing AWS requests from AWS SDK, the AWS SDK clients are patched with X-Ray like the following example:
```
require 'aws-xray-sdk'
require 'aws-sdk-s3'
# Patch AWS SDK clients
XRay.recorder.configure(plugins: [:aws_sdk])
# Use the instrumented client
s3 = Aws::S3::Client.new
s3.list_buckets
```
------
#### [ With OpenTelemetry SDK ]
AWS SDK for Ruby V3 provides support for recording and emitting OpenTelemetry traces. For information on how to configure OpenTelemetry for a service client, see [Configuring observability features in the AWS SDK for Ruby ](sdk-for-ruby/v3/developer-guide/observability.html).
------
## Instrumenting outgoing HTTP calls
When making HTTP calls to external services, you might need to manually instrument the calls if automatic instrumentation isn’t available or doesn’t provide enough detail.
------
#### [ With X-Ray SDK ]
To instrument downstream calls, the X-Ray SDK for Ruby was used to patch the `net/http` library that your application uses:
```
require 'aws-xray-sdk'
config = {
name: 'my app',
patch: %I[net_http]
}
XRay.recorder.configure(config)
```
------
#### [ With OpenTelemetry SDK ]
To enable the `net/http` instrumentation using OpenTelemetry, first install the `opentelemetry-instrumentation-net_http` package:
```
bundle add opentelemetry-instrumentation-net_http
```
Next, update the configuration to enable the `net/http` instrumentation as shown below:
```
OpenTelemetry::SDK.configure do |c|
...
c.use 'OpenTelemetry::Instrumentation::Net::HTTP'
...
end
```
------
## Instrumentation support for other libraries
You can find the full list of supported Library instrumentations for OpenTelemetry Ruby under [opentelemetry-ruby-contrib ](https://github.com/open-telemetry/opentelemetry-ruby-contrib/tree/main/instrumentation).
Alternatively, you can search the OpenTelemetry Registry to find out if OpenTelemetry supports instrumentation. For more information, see [Registry](https://opentelemetry.io/ecosystem/registry/).
## Manually creating trace data
------
#### [ With X-Ray SDK ]
Using X-Ray, the `aws-xray-sdk` package required you to manually create segments and their child sub-segments to trace your application. You may have also added X-Ray annotations and metadata to your segments or sub-segments:
```
require 'aws-xray-sdk'
...
# Start a segment
segment = XRay.recorder.begin_segment('my-service')
# Add annotations (indexed key-value pairs)
segment.annotations[:user_id] = 'user-123'
segment.annotations[:payment_status] = 'completed'
# Add metadata (non-indexed data)
segment.metadata[:order] = {
id: 'order-456',
items: [
{ product_id: 'prod-1', quantity: 2 },
{ product_id: 'prod-2', quantity: 1 }
],
total: 67.99
}
# Add metadata to a specific namespace
segment.metadata(namespace: 'payment') do |metadata|
metadata[:transaction_id] = 'tx-789'
metadata[:payment_method] = 'credit_card'
end
# Create a subsegment with annotations and metadata
segment.subsegment('payment-processing') do |subsegment1|
subsegment1.annotations[:payment_id] = 'pay-123'
subsegment1.metadata[:details] = { amount: 67.99, currency: 'USD' }
# Create a nested subsegment
subsegment1.subsegment('operation-2') do |subsegment2|
# Do more work...
end
end
# Close the segment
segment.close
```
------
#### [ With OpenTelemetry SDK ]
You can use custom spans to monitor the performance of internal activities that are not captured by instrumentation libraries. Note that only spans of kind server are converted into X-Ray segments, all other spans are converted into X-Ray sub-segments. By default, spans are `INTERNAL`.
First, create a Tracer in order to generate spans, which you can obtain through the `OpenTelemetry.tracer_provider.tracer('')` method. This will provide a Tracer instance that is registered globally in you application's OpenTelemetry configuration. It is common to have a single Tracer for an entire application. Create an OpenTelemetry tracer and use it to create spans:
```
require 'opentelemetry-sdk'
...
# Get a tracer
tracer = OpenTelemetry.tracer_provider.tracer('my-application')
# Create a server span (equivalent to X-Ray segment)
tracer.in_span('my-application', kind: OpenTelemetry::Trace::SpanKind::SERVER) do |span|
# Do work...
# Create nested spans of default kind INTERNAL will become an X-Ray subsegment
tracer.in_span('operation-1') do |child_span1|
# Set attributes (equivalent to X-Ray annotations and metadata)
child_span1.set_attribute('key', 'value')
# Do more work...
tracer.in_span('operation-2') do |child_span2|
# Do more work...
end
end
end
```
**Adding annotations and metadata to traces with OpenTelemetry SDK**
Use the `set_attribute` method to add attributes to each span. Note that by default, all these span attributes will be converted into metadata in X-Ray raw data. To ensure that an attribute is converted into an annotation and not metadata, you can add that attributes key to the list of `aws.xray.annotations` attribute. For more information, see [Enable The Customized X-Ray Annotations ](https://aws-otel.github.io/docs/getting-started/x-ray#enable-the-customized-x-ray-annotations).
```
# SERVER span will become an X-Ray segment
tracer.in_span('my-server-operation', kind: OpenTelemetry::Trace::SpanKind::SERVER) do |span|
# Your server logic here
span.set_attribute('attribute.key', 'attribute.value')
span.set_attribute("metadataKey", "metadataValue")
span.set_attribute("annotationKey1", "annotationValue")
# Create X-Ray annotations
span.set_attribute("aws.xray.annotations", ["annotationKey1"])
end
```
------
## Lambda manual instrumentation
------
#### [ With X-Ray SDK ]
After *Active Tracing* was enabled on Lambda, there are no additional configurations required to use the X-Ray SDK. Lambda will create a segment representing the Lambda handler invocation, and you can create sub-segments or instrument libraries using the X-Ray SDK without any additional configuration.
------
#### [ With OpenTelemetry SDK ]
Consider the following sample Lambda function code (without instrumentation):
```
require 'json'
def lambda_handler(event:, context:)
# TODO implement
{ statusCode: 200, body: JSON.generate('Hello from Lambda!') }
end
```
To manually instrument your Lambda, you will need to:
1. Add the following gems for your Lambda
```
gem 'opentelemetry-sdk'
gem 'opentelemetry-exporter-otlp'
gem 'opentelemetry-propagator-xray'
gem 'aws-distro-opentelemetry-exporter-xray-udp'
gem 'opentelemetry-instrumentation-aws_lambda'
gem 'opentelemetry-propagator-xray', '~> 0.24.0' # Requires version v0.24.0 or higher
```
1. Initialize OpenTelemetry SDK outside your Lambda Handler. The OpenTelemetry SDK is recommended to be configured with:
1. A simple span processor with an X-Ray UDP span exporter to send Traces to Lambda's UDP X-Ray endpoint
1. An X-Ray Lambda propagator
1. `service_name` configuration to be set to the Lambda function name
1. In your Lambda handler class, add the following lines to instrument your Lambda Handler:
```
class Handler
extend OpenTelemetry::Instrumentation::AwsLambda::Wrap
...
instrument_handler :process
end
```
The following code demonstrates the Lambda function after the required changes. You can create additional custom spans to complement the automatically provided spans.
```
require 'json'
require 'opentelemetry-sdk'
require 'aws/distro/opentelemetry/exporter/xray/udp'
require 'opentelemetry/propagator/xray'
require 'opentelemetry/instrumentation/aws_lambda'
# Initialize OpenTelemetry SDK outside handler
OpenTelemetry::SDK.configure do |c|
# Configure the AWS Distro for OpenTelemetry X-Ray Lambda exporter
c.add_span_processor(
OpenTelemetry::SDK::Trace::Export::SimpleSpanProcessor.new(
AWS::Distro::OpenTelemetry::Exporter::XRay::UDP::AWSXRayUDPSpanExporter.new
)
)
# Configure X-Ray Lambda propagator
c.propagators = [OpenTelemetry::Propagator::XRay.lambda_text_map_propagator]
# Set minimal resource information
c.resource = OpenTelemetry::SDK::Resources::Resource.create({
OpenTelemetry::SemanticConventions::Resource::SERVICE_NAME => ENV['AWS_LAMBDA_FUNCTION_NAME']
})
c.use 'OpenTelemetry::Instrumentation::AwsLambda'
end
module LambdaFunctions
class Handler
extend OpenTelemetry::Instrumentation::AwsLambda::Wrap
def self.process(event:, context:)
"Hello!"
end
instrument_handler :process
end
end
```
------
The following is an example trace map of an instrumented Lambda function written in Ruby.
![\[Trace map in CloudWatch console for Ruby.\]](http://docs.aws.amazon.com/xray/latest/devguide/images/deprecation_ruby.png)
You can also use Lambda layers to configure OpenTelemetry for your Lambda. For more information, see [OpenTelemetry AWS-Lambda Instrumentation ](https://github.com/open-telemetry/opentelemetry-ruby-contrib/tree/main/instrumentation/aws_lambda#usage).