Transformations
This documentation topic is designed for Grafana workspaces that support Grafana version 8.x.
For Grafana workspaces that support Grafana version 10.x, see Working in Grafana version 10.
For Grafana workspaces that support Grafana version 9.x, see Working in Grafana version 9.
Transformations process the result set before it’s passed to the visualization. You access transformations in the Transform tab of the Amazon Managed Grafana panel editor.
You can use transformations to rename fields, join separate time series together, do math across queries, and more. If you have large dashboards or heavy queries, being able to reuse the query result from one panel to another panel can provide a huge performance gain.
Note
Transformations sometimes result in data that cannot be graphed. When that happens, Amazon Managed Grafana displays a suggestion on the visualization. Choose the suggestion to switch to table visualization. This often helps you better understand what the transformation is doing to your data.
Amazon Managed Grafana applies transformations in the sequence that they are listed on the screen. Every transformation creates a new result set that is passed to the next transformation in the pipeline.
The order can make a huge difference in how your results look. For example, if you use reduce transformation to condense all the results of one column to a single value, you can apply transformations only to that single value.
Prerequisites
Before you apply transformations, all of the following must be true:
-
You have entered a query and returned data from a data source. For more information about queries, see Queries.
-
You have applied a visualization that supports queries, such as one of the following visualizations:
-
Bar gauge
-
Gauge
-
Graph
-
Heatmap
-
Logs
-
Stat
-
Table
-
Applying a transformation
Transformations are available from the Transform tab in the bottom pane of the panel editor, next to the Queries tab.
To apply a transformation
-
On the panel that you want to add transformations to, choose the panel title, and then choose Edit.
-
Choose the Transform tab.
-
Select a transformation.
In the transformation row that appears, you can configure the transformation options.
-
To apply another transformation, choose Add transformation. Keep in mind that the next transformation acts on the result set returned by the previous transformation.
If you have trouble, choose the bug icon to debug your transformations.
To remove a transformation, choose the trash can icon.
Transformation types and options
Grafana workspaces include the following transformations.
Topics
Reduce
Apply a Reduce transformation when you want to simplify your results down to one value. Reduce basically removes the time component. If visualized as a table, it reduces a column down to one row (value).
In the Calculations field, enter one or more calculation types. Choose to see a list of calculation choices. For information about available calculations, see Calculations list.
After you select at least one calculation, Amazon Managed Grafana displays one value using the calculation you selected. If you select more than one calculation, more than one value is displayed.
Merge
Use this transformation to combine the results from multiple queries into one single result. This is helpful when using the table panel visualization. Values that can be merged are combined into the same row. Values can be merged if the shared fields contain the same data.
In the following example, two queries return table data. The data are visualized as two separate tables before applying the transformation.
Query A
Time | Job | Uptime |
---|---|---|
2020-07-07 11:34:20 | node | 25260122 |
2020-07-07 11:24:20 | postgre | 123001233 |
Query B
Time | Job | Errors |
---|---|---|
2020-07-07 11:34:20 | node | 15 |
2020-07-07 11:24:20 | postgre | 5 |
Here is the result after applying the Merge transformation.
Time | Job | Errors | Uptime |
---|---|---|---|
2020-07-07 11:34:20 | node | 15 | 25260122 |
2020-07-07 11:24:20 | postgre | 5 | 123001233 |
Filter by name
Use this transformation to remove portions of the query results.
Amazon Managed Grafana displays the Identifier field, followed by the fields returned by your query.
You can apply filters in one of two ways:
-
Enter a regex expression.
-
Choose a field to toggle filtering on that field. Filtered fields are displayed with dark gray text, unfiltered fields have white text.
Filter data by query
Use this transformation in panels that have multiple queries, if you want to hide one or more of the queries.
Amazon Managed Grafana displays the query identification letters in dark gray text. To toggle filtering, choose a query identifier. If the query letter is white, the results are displayed. If the query letter is dark, the results are hidden.
Organize fields
Use this transformation to rename, reorder, or hide fields returned by the query.
Note
This transformation works only in panels that have a single query. If your panel has multiple queries, you must either apply a Join by field (outer join) transformation or remove the extra queries.
Amazon Managed Grafana displays a list of fields returned by the query. You can make any of the following changes:
-
Change the field order by pausing over a field. The cursor turns into a hand, and then you can drag the field to its new place.
-
Hide or show a field by choosing the eye icon next to the field name.
-
Rename fields by typing a new name in the Rename box.
Join by field (outer join)
Use this transformation to join multiple time series from a result set by field.
This transformation is useful if you want to combine queries so that you can calculate results from the fields.
Add field from calculation
Use this transformation to add a new field calculated from two other fields. Each transformation allows you to add one new field.
-
Mode – Select a mode:
-
Reduce row – Apply selected calculation on each row of selected fields independently.
-
Binary option – Apply a basic math operation (
sum
,multiply
, and so on) on values in a single row from two selected fields.
-
-
Field name – Select the names of fields that you want to use in the calculation for the new field.
-
Calculation - Select a calculation to use when Amazon Managed Grafana creates the new field. Choose the field to see a list of calculation choices. For information about available calculations, see Calculations list.
-
Alias – (Optional) Enter the name of your new field. If you leave this blank, the field will be named to match the calculation.
-
Replace all fields – (Optional) Use this option if you want to hide all other fields and display only your calculated field in the visualization.
Labels to fields
Note
To apply this transformation, your query needs to return labeled fields.
When you select this transformation, Amazon Managed Grafana automatically transforms all labeled data into fields.
For example, consider a query result of two time series.
1: labels Server=Server A, Datacenter=EU 2: labels Server=Server B, Datacenter=EU
This transformation would result in the following table.
Time | Server | Datacenter | Value |
---|---|---|---|
2020-07-07 11:34:20 | Server A | EU | 1 |
2020-07-07 11:34:20 | Server B | EU | 2 |
Value field name; If you selected Server
as
the Value field name, you would get one field for every
value of the Server
label.
Time | Datacenter | Server A | Server B |
---|---|---|---|
2020-07-07 11:34:20 | EU | 1 | 2 |
Group By
This transformation sorts each frame by the configured field. When
reverse
is checked, the values are returned in the
opposite order.
Group By
This transformation groups the data by a specified field (column) value and processes calculations on each group. The available calculations are the same as for the Reduce transformation.
Here’s an example of original data.
Time | Server ID | CPU Temperature | Server Status |
---|---|---|---|
2020-07-07 11:34:20 | server 1 | 80 | Shutdown |
2020-07-07 11:34:20 | server 3 | 62 | OK |
2020-07-07 10:32:20 | server 2 | 90 | Overload |
2020-07-07 10:31:22 | server 3 | 55 | OK |
2020-07-07 09:30:57 | server 3 | 62 | Rebooting |
2020-07-07 09:30:05 | server 2 | 88 | OK |
2020-07-07 09:28:06 | server 1 | 80 | OK |
2020-07-07 09:25:05 | server 2 | 88 | OK |
2020-07-07 09:23:07 | server 1 | 86 | OK |
This transformation takes two steps. First, you specify one or multiple
fields to group the data by. This will group all the same values of those fields
together, as if you sorted them. For instance, if you Group
By the Server ID
field, it will group the data this
way:
Time | Server ID | CPU Temperature | Server Status |
---|---|---|---|
2020-07-07 11:34:20 | server 1 | 80 | Shutdown |
2020-07-07 09:28:06 | server 1 | 80 | OK |
2020-07-07 09:23:07 | server 1 | 86 | OK |
2020-07-07 10:32:20 | server 2 | 90 | Overload 2020-07-07 09:30:05 | server 2 | 88 | OK 2020-07-07 09:25:05 | server 2 | 88 | OK 2020-07-07 11:34:20 | server 3 | 62 | OK 2020-07-07 10:31:22 | server 3 | 55 | OK 2020-07-07 09:30:57 | server 3 | 62 | Rebooting
All rows with the same value of Server ID
are grouped together.
After choosing which field you want to group your data by, you can add various
calculations on the other fields, and the calculation will be applied on each
group of rows. For instance, you might want to calculate the average CPU
temperature
for each of those servers. You can add the
mean calculation applied on the CPU
Temperature
field to get the following.
Server ID | CPU Temperature (mean) |
---|---|
server 1 | 82 |
server 2 | 88.6 |
server 3 | 59.6 |
And you can add more than one of those calculations. For instance, you can use the following calculations.
-
For field
Time
, you can calculate the Last value, to know when the last data point was received for each server. -
For field
Server Status
, you can calculate the Last value to know the last state value for each server. -
For field
Temperature
, you can also calculate the Last value to know the latest monitored temperature for each server.
The Group By transformation produces the following results.
Server ID | CPU Temperature (mean) | CPU Temperature (last) | Time (last) | Server Status (last) |
---|---|---|---|---|
server 1 | 82 | 80 | 2020-07-07 11:34:20 | Shutdown |
server 2 | 88.6 | 90 | 2020-07-07 10:32:20 | Overload |
server 3 | 59.6 | 62 | 2020-07-07 11:34:20 | OK |
Using this transformation, you can extract some key information out of your time series and display it in a convenient way.
Series to rows
Use this transformation to combine the results from multiple time series data queries into one single result. This is helpful when using the table panel visualization.
The result from this transformation will contain three columns:
Time
, Metric
, and Value
. The
Metric
column is added so that you can see which query the
metric originates from. Customize this value by defining Label
on
the source query.
In the example below, two queries return time series data. It is visualized as two separate tables before the transformation is applied.
Query A
Time | Temperature |
---|---|
2020-07-07 11:34:20 | 25 |
2020-07-07 10:31:22 | 22 |
2020-07-07 09:30:05 | 19 |
Query B
Time | Humidity |
---|---|
2020-07-07 11:34:20 | 24 |
2020-07-07 10:32:20 | 29 |
2020-07-07 09:30:57 | 33 |
Applying the Series to rows
transformation produces the
following results.
Time | Metric | Value |
---|---|---|
2020-07-07 11:34:20 | Temperature | 25 |
2020-07-07 11:34:20 | Humidity | 22 |
2020-07-07 10:32:20 | Humidity | 29 |
2020-07-07 10:31:22 | Temperature | 22 |
2020-07-07 09:30:57 | Humidity | 33 |
2020-07-07 09:30:05 | Temperature | 19 |
Filter data by value
This transformation allows you to filter your data directly in the Grafana workspace and remove some data points from your query result. You have the option to include or exclude data that match one or more conditions you define. The conditions are applied on a selected field.
This transformation is useful if your data source does not natively filter by values. You might also use this to narrow values to display if you are using a shared query.
The available conditions for all fields are as follows:
-
Regex – Match a regex expression.
-
Is Null – Match if the value is null.
-
Is Not Null – Match if the value is not null.
-
Equal – Match if the value is equal to the specified value.
-
Different – Match if the value is different than the specified value.
The available conditions for number fields are as follows:
-
Greater – Match if the value is greater than the specified value.
-
Lower – Match if the value is lower than the specified value.
-
Greater or equal – Match if the value is greater than or equal to the specified value.
-
Lower or equal – Match if the value is lower than or equal to the specified value.
-
Range – Match a range between a specified minimum and maximum. The minimum and maximum are included in the range.
You can add more than one condition to the filter. When you have more than one condition, you can choose if you want the include or exclude action to be applied on rows that Match all or any of the conditions you added.
Conditions that are not valid or incompletely configured are ignored.
Debug transformations
To see the input and the output result sets of the transformation, choose the bug icon on the right side of the transformation row.
Amazon Managed Grafana displays the transformation debug view below the transformation row.