Add and manage variables - Amazon Managed Grafana

Add and manage variables

This documentation topic is designed for Grafana workspaces that support Grafana version 10.x.

For Grafana workspaces that support Grafana version 9.x, see Working in Grafana version 9.

For Grafana workspaces that support Grafana version 8.x, see Working in Grafana version 8.

The following table lists the types of variables in Grafana.

Variable type Description

Query

Query-generated list of values such as metric names, server names, sensor IDs, data centers, and so on.

Custom

Define the variable options manually using a comma-separated list.

Text box

Display a free text input field with an optional default value.

Constant

Define a hidden constant.

Data source

Quickly change the data source for an entire dashboard.

Interval

Interval variables represent time spans.

Ad hoc filters

Key-value filters that are automatically added to all metric queries for a data source (Prometheus, Loki, InfluxDB, and Elasticsearch only).

Global variables

Built-in variables that can be used in expressions in the query editor.

Chained variables

Variable queries can contain other variables.

Entering General options

You must enter general options for any type of variable that you create.

To enter general options
  1. Navigate to the dashboard you want to make a variable for and select the Dashboard settings (gear) icon at the top of the page.

  2. On the Variables tab, select New variable.

  3. Enter a Name for the variable.

  4. In the Type list, select Query.

  5. (Optional) In Label, enter the display name of the variable dropdown.

    If you don’t enter a display name, then the dropdown label is the variable name.

  6. Choose a Hide option:

    • No selection (blank) – The variable dropdown displays the variable Name or Label value.

    • Label – The variable dropdown only displays the selected variable value and a down arrow.

    • Variable – No variable dropdown is displayed on the dashboard.

Adding a query variable

Query variables enable you to write a data source query that can return a list of metric names, tag values, or keys. For example, a query variable might return a list of server names, sensor IDs, or data centers. The variable values change as they dynamically fetch options with a data source query.

Query variables are generally only supported for strings. If your query returns numbers or any other data type, you might need to convert them to strings in order to use them as variables. For the Azure data source, for example, you can use the tostring function for this purpose.

Query expressions can contain references to other variables and in effect create linked variables. Grafana detects this and automatically refreshes a variable when one of its linked variables change.

Note

Query expressions are different for each data source. For more information, refer to the documentation for your data source.

To add a query variable
  1. Enter general options, as above.

  2. In the Data source list, select the target data source for the query.

  3. In the Refresh list, select when the variable should update options.

    • On Dashboard Load – Queries the data source every time the dashboard loads. This slows down dashboard loading, because the variable query needs to be completed before dashboard can be initialized.

    • On Time Range Change – Queries the data source when the dashboard time range changes. Only use this option if your variable options query contains a time range filter or is dependent on the dashboard time range.

  4. In the Query field, enter a query.

    • The query field varies according to your data source. Some data sources have custom query editors.

    • The query must return values named __text and __value. For example, in SQL, you can use a query such as SELECT hostname AS __text, id AS __value from MyTable. Queries for other languages will vary depending on syntax.

    • If you need more room in a single input field query editor, then hover your cursor over the lines in the lower right corner of the field and drag downward to expand.

  5. (Optional) In the Regex field, type a regex expression to filter or capture specific parts of the names returned by your data source query. To see examples, refer to Filter variables with regex.

  6. In the Sort list, select the sort order for values to be displayed in the dropdown list. The default option, Disabled, means that the order of options returned by your data source query will be used.

  7. (Optional) Enter Selection Options.

  8. In Preview of values, Grafana displays a list of the current variable values. Review them to ensure they match what you expect.

  9. Select Add to add the variable to the dashboard.

Adding a custom variable

Use a custom variable for a value that does not change, such as a number or a string.

For example, if you have server names or Region names that never change, then you might want to create them as custom variables rather than query variables. Because they do not change, you might use them in chained variables rather than other query variables. That would reduce the number of queries Grafana must send when chained variables are updated.

To add a custom variable
  1. Enter general options, as above.

  2. In the Values separated by comma list, enter the values for this variable in a comma-separated list. You can include numbers, strings, or key-value pairs separated by a space and a colon. For example, key1 : value1,key2 : value2.

  3. (Optional) Enter Selection Options.

  4. In Preview of values, Grafana displays a list of the current variable values. Review them to ensure they match what you expect.

  5. Select Add to add the variable to the dashboard.

Adding a text box variable

Text box variables display a free text input field with an optional default value. This is the most flexible variable, because you can enter any value. Use this type of variable if you have metrics with high cardinality or if you want to update multiple panels in a dashboard at the same time.

To add a text box variable
  1. Enter general options, as above.

  2. (Optional) In the Default value field, select the default value for the variable. If you do not enter anything in this field, then Grafana displays an empty text box for users to type text into.

  3. In Preview of values, Grafana displays a list of the current variable values. Review them to ensure they match what you expect.

  4. Select Add to add the variable to the dashboard.

Adding a constant variable

Constant variables enable you to define a hidden constant. This is useful for metric path prefixes for dashboards you want to share. When you export a dashboard, constant variables are converted to import options.

Constant variables are not flexible. Each constant variable only holds one value, and it cannot be updated unless you update the variable settings.

Constant variables are useful when you have complex values that you need to include in queries but don’t want to retype in every query. For example, if you had a server path called i-0b6a61efe2ab843gg, then you could replace it with a variable called $path_gg.

To add a constant variable
  1. Enter general options, as above.

  2. In the Value field, enter the variable value. You can enter letters, numbers, and symbols. You can even use wildcards if you use raw format.

  3. In Preview of values, Grafana displays a list of the current variable values. Review them to ensure they match what you expect.

  4. Select Add to add the variable to the dashboard.

Adding a data source variable

Data source variables enable you to quickly change the data source for an entire dashboard. They are useful if you have multiple instances of a data source, perhaps in different environments.

To add a data source variable
  1. Enter general options, as above.

  2. In the Type list, select the target data source for the variable.

    You can also choose Open advanced data source picker to see more options, including adding a data source (Admins only). For more information, see Connect to data sources.

  3. (Optional) In Instance name filter, enter a regex filter for which data source instances to choose from in the variable value dropdown list. Leave this field empty to display all instances.

  4. (Optional) Enter Selection Options.

  5. In Preview of values, Grafana displays a list of the current variable values. Review them to ensure they match what you expect.

  6. Select Add to add the variable to the dashboard.

Adding an interval variable

Use an interval variable to represents time spans such as 1m,1h, or 1d. You can think of them as a dashboard-wide group by time command. Interval variables change how the data is grouped in the visualization. You can also use the Auto Option to return a set number of data points per time span.

You can use an interval variable as a parameter to group by time (for InfluxDB), date histogram interval (for Elasticsearch), or as a summarize function parameter (for Graphite).

To add an interval variable
  1. Enter general options, as above.

  2. In the Values field, enter the time range intervals that you want to appear in the variable dropdown list. The following time units are supported: s (seconds), m (minutes), h (hours), d (days), w (weeks), M (months), and y (years). You can also accept or edit the default values: 1m,10m,30m,1h,6h,12h,1d,7d,14d,30d.

  3. (Optional) Turn on the Auto Option if you want to add the auto option to the list. This option allows you to specify how many times the current time range should be divided to calculate the current auto time span. If you turn it on, then two more options appear:

    • Step count – Select the number of times the current time range will be divided to calculate the value, similar to the Max data points query option. For example, if the current visible time range is 30 minutes, then the auto interval groups the data into 30 one-minute increments. The default value is 30 steps.

    • Min Interval – The minimum threshold below which the step count intervals will not divide the time. To continue the 30 minute example, if the minimum interval is set to 2m, then Grafana would group the data into 15 two-minute increments.

  4. In Preview of values, Grafana displays a list of the current variable values. Review them to ensure they match what you expect.

  5. Select Add to add the variable to the dashboard.

Interval variable examples

The following example shows a template variable myinterval in a Graphite function:

summarize($myinterval, sum, false)

Adding ad hoc filters

Ad hoc filters enable you to add key-value filters that are automatically added to all metric queries that use the specified data source. Unlike other variables, you do not use ad hoc filters in queries. Instead, you use ad hoc filters to write filters for existing queries.

Note

Ad hoc filter variables only work with Prometheus, Loki, InfluxDB, and Elasticsearch data sources.

  1. Enter general options, as above.

  2. In the Data source list, select the target data source.

    You can also choose Open advanced data source picker to see more options, including adding a data source (Admins only). For more information, see Connect to data sources.

  3. Select Add to add the variable to the dashboard.

Create ad hoc filters

Ad hoc filters are one of the most complex and flexible variable options available. Instead of a regular list of variable options, this variable allows you to build a dashboard-wide ad hoc query. Filters you apply in this manner are applied to all panels on the dashboard.

Configure variable selection options

Selection Options are a feature you can use to manage variable option selections. All selection options are optional, and they are off by default.

Multi-value variables

Interpolating a variable with multiple values selected is tricky as it is not straight forward how to format the multiple values into a string that is valid in the given context where the variable is used. Grafana tries to solve this by allowing each data source plugin to inform the templating interpolation engine what format to use for multiple values.

Note

The Custom all value option on the variable must be blank for Grafana to format all values into a single string. If it is left blank, then Grafana concatenates (adds together) all the values in the query. For example, value1,value2,value3. If a custom all value is used, then instead the value will be * or all.

Multi-value variables with a Graphite data source

Graphite uses glob expressions. A variable with multiple values is, in this case, be interpolated as {host1,host2,host3} if the current variable value iss host1, host2, and host3.

Multi-value variables with a Prometheus or InfluxDB datasource

InfluxDB and Prometheus use regex expressions, so the same variable is interpolated as (host1|host2|host3). Every value is also regex escaped. If it were not, a value with a regex control character would break the regex expression.

Multi-value variables with an Elastic data source

Elasticsearch uses lucene query syntax, so the same variable is formatted as ("host1" OR "host2" OR "host3"). In this case, every value is escaped so that the value only contains lucene control words and quotation marks.

Troubleshoot multi-value variables

Automatic escaping and formatting can cause problems and it can be tricky to grasp the logic behind it. Especially for InfluxDB and Prometheus where the use of regex syntax requires that the variable is used in regex operator context.

If you do not want Grafana to do this automatic regex escaping and formatting, then you must do one of the following:

  • Turn off the Multi-value or Include All option options.

  • Use the raw format.

Include All option

Grafana adds an All option to the variable dropdown list. If a user selects this option, then all variable options are selected.

Custom all value

This option is only visible if the Include All option is selected.

Enter regex, globs, or lucene syntax in the Custom all value field to define the value of the All option.

By default the All value includes all options in a combined expression. This can become very long and can have performance problems. Sometimes it can be better to specify a custom all value, like a wildcard regex.

To have custom regex, globs, or lucene syntax in the Custom all value option, it is never escaped so you will have to think about what is a valid value for your data source.

Global variables

Grafana has global built-in variables that can be used in expressions in the query editor. This topic lists them in alphabetical order and defines them. These variables are useful in queries, dashboard links, panel links, and data links.

$__dashboard

This variable is the name of the current dashboard.

$__from and $__to

Grafana has two built-in time range variables: $__from and $__to. They are currently always interpolated as epoch milliseconds by default, but you can control date formatting.

Syntax Example result Description

${__from}

1594671549254

Unix millisecond epoch

${__from:date}

2020-07-13T20:19:09.254Z

No args, defaults to ISO 8601/RFC 3339

${__from:date:iso}

2020-07-13T20:19:09.254Z

ISO 8601/RFC 3339

${__from:date:seconds}

1594671549

Unix seconds epoch

${__from:date:YYYY-MM}

2020-07

Any custom date format that does not include the : character

The syntax above also works with ${__to}.

$__interval

You can use the $__interval variable as a parameter to group by time (for InfluxDB, MySQL, Postgres, MSSQL), Date histogram interval (for Elasticsearch), or as a summarize function parameter (for Graphite).

Grafana automatically calculates an interval that can be used to group by time in queries. When there are more data points than can be shown on a graph, the queries can be made more efficient by grouping by a larger interval. For example, if you are looking at a graph of 3 months worth of data, you might not be able to see detail at the minute level. Grouping by the hour or day makes the query more efficient without affecting what the graph shows. The $__interval is calculated using the time range and the width of the graph (the number of pixels).

Approximate Calculation: (to - from) / resolution

For example, when the time range is 1 hour and the graph is full screen, then the interval might be calculated to 2m - points are grouped in 2 minute intervals. If the time range is 6 months and the graph is full screen, then the interval might be 1d (1 day) - points are grouped by day.

In the InfluxDB data source, the legacy variable $interval is the same variable. $__interval should be used instead.

The InfluxDB and Elasticsearch data sources have Group by time interval fields that are used to hard code the interval or to set the minimum limit for the $__interval variable (by using the > syntax, for example >10m).

$__interval_ms

This variable is the $__interval variable in milliseconds, not a time interval formatted string. For example, if the $__interval is 20m then the $__interval_ms is 1200000.

$__org

This variable is the ID of the current organization. ${__org.name} is the name of the current organization.

$__user

${__user.id} is the ID of the current user. ${__user.login} is the login handle of the current user. ${__user.email} is the email for the current user.

$__range

Only supported for Prometheus and Loki data sources. This variable represents the range for the current dashboard. It is calculated by to - from. It has a millisecond and a second representation called $__range_ms and $__range_s.

$__rate_interval

Only supported for Prometheus data sources. The $__rate_interval variable is meant to be used in the rate function.

$timeFilter or $__timeFilter

The $timeFilter variable returns the currently selected time range as an expression. For example, the time range interval Last 7 days expression is time > now() - 7d.

This is used in several places, including:

  • The WHERE clause for the InfluxDB data source. Grafana adds it automatically to InfluxDB queries when in Query Editor mode. You can add it manually in Text Editor mode: WHERE $timeFilter.

  • Log Analytics queries in the Azure Monitor data source.

  • SQL queries in MySQL, Postgres, and MSSQL.

  • The $__timeFilter variable is used in the MySQL data source.

$__timezone

The $__timezone variable returns the currently selected time zone, either utc or an entry of the IANA time zone data base (for example, America/New_York).

If the currently selected time zone is Browser Time, Grafana will try to determine your browser time zone.

Chained variables

Chained variables, also called linked variables or nested variables, are query variables with one or more other variables in their variable query.

Chained variable queries are different for every data source, but the premise is the same for all. You can use chained variable queries in any data source that allows them.

Extremely complex linked templated dashboards are possible, 5 or 10 levels deep. Technically, there is no limit to how deep or complex you can go, but the more links you have, the greater the query load.

Best practices and tips

The following practices will make your dashboards and variables easier to use.

Creating new linked variables

  • Chaining variables create parent/child dependencies. You can envision them as a ladder or a tree.

  • The easiest way to create a new chained variable is to copy the variable that you want to base the new one on. In the variable list, click the Duplicate variable icon to the right of the variable entry to create a copy. You can then add on to the query for the parent variable.

  • New variables created this way appear at the bottom of the list. You might need to drag it to a different position in the list to get it into a logical order.

Variable order

You can change the orders of variables in the dashboard variable list by clicking the up and down arrows on the right side of each entry. Grafana lists variable dropdowns left to right according to this list, with the variable at the top on the far left.

  • List variables that do not have dependencies at the top, before their child variables.

  • Each variable should follow the one it is dependent on.

  • Remember there is no indication in the UI of which variables have dependency relationships. List the variables in a logical order to make it easy on other users (and yourself).

Complexity consideration

The more layers of dependency you have in variables, the longer it will take to update dashboards after you change variables.

For example, if you have a series of four linked variables (country, Region, server, metric) and you change a root variable value (country), then Grafana must run queries for all the dependent variables before it updates the visualizations in the dashboard.

Manage variables

The variables page lets you add variables and manage existing variables. It also allows you to inspect variables and identify whether a variable is being referenced (or used) in other variables or dashboard.

Move – You can move a variable up or down the list using drag and drop.

Clone – To clone a variable, click the clone icon from the set of icons on the right. This creates a copy of the variable with the name of the original variable prefixed with copy_of_.

Delete – To delete a variable, click the trash icon from the set of icons on the right.

Filter variables with regex

Using the Regex Query option, you filter the list of options returned by the variable query or modify the options returned.

This page shows how to use regex to filter/modify values in the variable dropdown.

Using the Regex Query Option, you filter the list of options returned by the Variable query or modify the options returned. For more information, refer to the Mozilla guide on Regular expressions.

The following examples show filtering on the following list of options

backend_01 backend_02 backend_03 backend_04

Filter so that only the options that end with 01 or 02 are returned

Regex:

/ ( 01|02 ) $/

Result:

backend_01 backend_02

Filter and modify the options using a regex capture group to return part of the text

Regex:

/.* ( 01|02 ) /

Result:

01 02

Filter and modify - Prometheus Example

For this list of options:

up{instance="demo.robustperception.io:9090",job="prometheus"} 1 1521630638000 up{instance="demo.robustperception.io:9093",job="alertmanager"} 1 1521630638000 up{instance="demo.robustperception.io:9100",job="node"} 1 1521630638000

This regex:

/. *instance=" ( [^"]* ) .*/

Returns these results:

demo.robustperception.io:9090 demo.robustperception.io:9093 demo.robustperception.io:9100

Filter and modify using named text and value capture groups

Using named capture groups, you can capture separate ’text’ and ‘value’ parts from the options returned by the variable query. This allows the variable dropdown list to contain a friendly name for each value that can be selected.

For example, when querying the node_hwmon_chip_names Prometheus metric, the chip_name is a lot friendlier that the chip value. So the following variable query result:

node_hwmon_chip_names{chip="0000:d7:00_0_0000:d8:00_0",chip_name="enp216s0f0np0"} 1 node_hwmon_chip_names{chip="0000:d7:00_0_0000:d8:00_1",chip_name="enp216s0f0np1"} 1 node_hwmon_chip_names{chip="0000:d7:00_0_0000:d8:00_2",chip_name="enp216s0f0np2"} 1 node_hwmon_chip_names{chip="0000:d7:00_0_0000:d8:00_3",chip_name="enp216s0f0np3"} 1

Passed through the following Regex:

/chip_name="(?<text>[ ^ " ] + ) |chip=" (?<value >[ ^ " ] + )/g

Would produce the following dropdown list:

Display Name Value ------------ ------------------------- enp216s0f0np0 0000:d7:00_0_0000:d8:00_0 enp216s0f0np1 0000:d7:00_0_0000:d8:00_1 enp216s0f0np2 0000:d7:00_0_0000:d8:00_2 enp216s0f0np3 0000:d7:00_0_0000:d8:00_3

Only text and value capture group names are supported.