Observability in Amazon OpenSearch Service

The default installation of OpenSearch Dashboards for Amazon OpenSearch Service includes the Observability plugin, which you can use to visualize data-driven events using Piped Processing Language (PPL) in order to explore, discover, and query data stored in OpenSearch. The plugin requires OpenSearch 1.2 or later.

The Observability plugin provides a unified experience for collecting and monitoring metrics, logs, and traces from common data sources. Data collection and monitoring in one place enables full-stack, end-to-end observability of your entire infrastructure.

Everyone's process for exploring data is different. If you’re new to exploring data and creating visualizations, we recommend trying a workflow like the following.

Explore your data with event analytics

To start, let's say that you're collecting flight data in your OpenSearch Service domain and you want to find out which airline had the most flights arriving in Pittsburgh International Airport last month. You write the following PPL query:

source=opensearch_dashboards_sample_data_flights |
    stats count() by Dest, Carrier |
    where Dest = "Pittsburgh International Airport"

This query pulls data from the index named opensearch_dashboards_sample_data_flights. It then uses the stats command to get a total count of flights and groups it according to destination airport and carrier. Finally, it uses the where clause to filter the results to flights arriving in Pittsburgh International Airport.

Here's what the data looks like when displayed over the last month:

You can choose the PPL button in the query editor to get usage information and examples for each PPL command:

Let's look at a more complex example, which queries for information about flight delays:

source=opensearch_dashboards_sample_data_flights |
    where FlightDelayMin > 0 |
    stats sum(FlightDelayMin) as minimum_delay, count() as total_delayed by Carrier, Dest |
    eval avg_delay=minimum_delay / total_delayed | 
    sort - avg_delay

Each command in the query impacts the final output:

With this query, you can determine that OpenSearch Dashboards Airlines has, on average, fewer delays.

You can find more sample PPL queries under Queries and Visualizations on the Event analytics page.

Create visualizations

Once you correctly query the data that you're interested in, you can save those queries as visualizations:

Then add those visualizations to operational panels to compare different pieces of data. Leverage notebooks to combine different visualizations and code blocks that you can share with team members.

Dive deeper with Trace Analytics

Trace Analytics provides a way to visualize the flow of events in your OpenSearch data to identify and fix performance problems in distributed applications.