# Monitoring Neptune Using Amazon CloudWatch
Amazon Neptune and Amazon CloudWatch are integrated so that you can gather and analyze performance metrics. You can monitor these metrics using the CloudWatch console, the AWS Command Line Interface (AWS CLI), or the CloudWatch API.
CloudWatch also lets you set alarms so that you can be notified if a metric value breaches a threshold that you specify. You can even set up CloudWatch Events to take corrective action if a breach occurs. For more information about using CloudWatch and alarms, see the [CloudWatch Documentation](https://aws.amazon.com/documentation/cloudwatch).
**Topics**
+ [Viewing CloudWatch Data (Console)](#Console_Neptune)
+ [Viewing CloudWatch Data (AWS CLI)](#CloudwatchCLI_Neptune)
+ [Viewing CloudWatch Data (API)](#CloudwatchAPI_Neptune)
+ [Using CloudWatch to monitor DB instance performance in Neptune](cloudwatch-monitoring-instances.md)
+ [Neptune CloudWatch Metrics](cw-metrics.md)
+ [Neptune CloudWatch Dimensions](cw-dimensions.md)
## Viewing CloudWatch Data (Console)
**To view CloudWatch data for a Neptune cluster (console)**
1. Sign in to the AWS Management Console and open the CloudWatch console at [https://console.aws.amazon.com/cloudwatch/](https://console.aws.amazon.com/cloudwatch/).
1. In the navigation pane, choose **Metrics**.
1. In the **All Metrics** pane, choose **Neptune**, and then choose **DBClusterIdentifier**.
1. In the upper pane, scroll down to view the full list of metrics for your cluster. The available Neptune metric options appear in the **Viewing** list.
To select or deselect an individual metric, in the results pane, select the check box next to the resource name and metric. Graphs showing the metrics for the selected items appear at the bottom of the console. To learn more about CloudWatch graphs, see [Graph Metrics](https://docs.aws.amazon.com/AmazonCloudWatch/latest/DeveloperGuide/graph_metrics.html) in the *Amazon CloudWatch User Guide*.
## Viewing CloudWatch Data (AWS CLI)
**To view CloudWatch data for a Neptune cluster (AWS CLI)**
1. Install the AWS CLI. For instructions, see the [AWS Command Line Interface User Guide](https://docs.aws.amazon.com/cli/latest/userguide/).
1. Use the AWS CLI to fetch information. The relevant CloudWatch parameters for Neptune are listed in [Neptune CloudWatch Metrics](cw-metrics.md).
The following example retrieves CloudWatch metrics for the number of Gremlin requests per second for the `gremlin-cluster` cluster.
```
```
## Viewing CloudWatch Data (API)
CloudWatch also supports a `Query` action so that you can request information programmatically. For more information, see the [CloudWatch Query API documentation](https://docs.aws.amazon.com/AmazonCloudWatch/latest/DeveloperGuide/Using_Query_API.html) and [Amazon CloudWatch API Reference](https://docs.aws.amazon.com/AmazonCloudWatch/latest/APIReference/).
When a CloudWatch action requires a parameter that is specific to Neptune monitoring, such as `MetricName`, use the values listed in [Neptune CloudWatch Metrics](cw-metrics.md).
The following example shows a low-level CloudWatch request, using the following parameters:
+ `Statistics.member.1` = `Average`
+ `Dimensions.member.1` = `DBClusterIdentifier=gremlin-cluster`
+ `Namespace ` = `AWS/Neptune`
+ `StartTime ` = `2013-11-14T00:00:00Z`
+ `EndTime ` = `2013-11-16T00:00:00Z`
+ `Period ` = `60`
+ `MetricName ` = `GremlinRequestsPerSec`
Here is what the CloudWatch request looks like. However, this is just to show the form of the request; you must construct your own request based on your metrics and timeframe.
```
1. https://monitoring.amazonaws.com/
2. ?SignatureVersion=2
3. &Action=GremlinRequestsPerSec
4. &Version=2010-08-01
5. &StartTime=2018-03-03T00:00:00
6. &EndTime=2018-03-04T00:00:00
7. &Period=60
8. &Statistics.member.1=Average
9. &Dimensions.member.1=DBClusterIdentifier=gremlin-cluster
10. &Namespace=AWS/Neptune
11. &MetricName=GremlinRequests
12. &Timestamp=2018-03-04T17%3A48%3A21.746Z
13. &AWSAccessKeyId=AWS Access Key ID;
14. &Signature=signature
```
# Using CloudWatch to monitor DB instance performance in Neptune
You can use CloudWatch metrics in Neptune to monitor what is happening on your DB instances and keep track of the query queue length as observed by the database. The following metrics are particularly useful:
+ **`CPUUtilization`** – Shows the percentage of CPU utilization.
+ **`VolumeWriteIOPs`** – Shows the average number of disk I/O writes to the cluster volume, reported at 5-minute intervals.
+ **`MainRequestQueuePendingRequests`** – Shows the number of requests waiting in the input queue pending execution.
You can also find out how many requests are pending on the server by using the [Gremlin query status endpoint](gremlin-api-status.md) with the `includeWaiting` parameter. This will give you the status of all waiting queries.
The following indicators can help you adjust your Neptune provisioning and query strategies to improve efficiency and performance:
+ Consistent latency, high `CPUUtilization`, high `VolumeWriteIOPs` and low `MainRequestQueuePendingRequests` together show that the server is actively engaged processing concurrent write requests at a sustainable rate, with little I/O wait.
+ Consistent latency, low `CPUUtilization`, low `VolumeWriteIOPs` and no `MainRequestQueuePendingRequests` together show that you have excess capacity on the primary DB instance for processing write requests.
+ High `CPUUtilization` and high `VolumeWriteIOPs` but variable latency and `MainRequestQueuePendingRequests` together show that you are sending more work than the server can process in a given interval. Consider creating or resizing batch requests so as to do the same amount of work with less transactional overhead and/or scaling the primary instance up to increase the number of query threads capable of processing write requests concurrently.
+ Low `CPUUtilization` with high `VolumeWriteIOPs` mean that query threads are waiting for I/O operations to the storage layer to complete. If you see variable latencies and some increase in `MainRequestQueuePendingRequests`, consider creating or resizing batch requests so as to do the same amount of work with less transactional overhead.
# Neptune CloudWatch Metrics
**Note**
Amazon Neptune sends metrics to CloudWatch only when they have a non-zero value.
For all Neptune metrics, the aggregation granularity is 5 minutes.
**Topics**
+ [Neptune CloudWatch metrics](#cw-metrics-available)
+ [CloudWatch Metrics That Are Now Deprecated in Neptune](#cw-metrics-deprecated)
## Neptune CloudWatch metrics
The following table lists the CloudWatch metrics that Neptune supports.
**Note**
All cumulative metrics are reset to zero whenever the server restarts, whether for maintenance, a reboot, or recovering from a crash.
**Neptune CloudWatch metrics**
| Metric | Description |
| --- | --- |
| `BackupRetentionPeriodStorageUsed` | The total amount of backup storage, in bytes, used to support from the Neptune DB cluster's backup retention window. Included in the total reported by the `TotalBackupStorageBilled` metric. |
| `BufferCacheHitRatio` | The percentage of requests that are served by the buffer cache. This metric can be useful in diagnosing query latency, because cache misses induce significant latency. If the cache hit ratio is below 99.9, consider upgrading the instance type to cache more data in memory. |
| `ClusterReplicaLag` | For a read replica, the amount of lag when replicating updates from the primary instance, in milliseconds. |
| `ClusterReplicaLagMaximum` | The maximum amount of lag between the primary instance and each Neptune DB instance in the DB cluster, in milliseconds. |
| `ClusterReplicaLagMinimum` | The minimum amount of lag between the primary instance and each Neptune DB instance in the DB cluster, in milliseconds. |
| `CPUCreditBalance` | The number of CPU credits that an instance has accumulated, reported at 5-minute intervals. You can use this metric to determine how long a DB instance can burst beyond its baseline performance level at a given rate. |
| `CPUCreditUsage` | The number of CPU credits consumed during the specified period, reported at 5-minute intervals. This metric measures the amount of time during which physical CPUs have been used for processing instructions by virtual CPUs allocated to the DB instance. |
| `CPUSurplusCreditBalance` | The number of surplus credits that have been spent by an unlimited instance when its `CPUCreditBalance` value is zero. The `CPUSurplusCreditBalance` value is paid down by earned CPU credits. If the number of surplus credits exceeds the maximum number of credits that the instance can earn in a 24-hour period, the spent surplus credits above the maximum incur an additional charge. CPU credit metrics are available at a 5-minute frequency only. |
| `CPUSurplusCreditsCharged` | The number of spent surplus credits that are not paid down by earned CPU credits, and incur an additional charge. |
| `CPUUtilization` | The percentage of CPU utilization. |
| `EngineUptime` | The amount of time that the instance has been running, in seconds. |
| `FreeableMemory` | The amount of available random access memory, in bytes. |
| `GlobalDbDataTransferBytes` | The number of bytes of redo log data transferred from the primary AWS Region to a secondary AWS Region in a Neptune global database. |
| `GlobalDbReplicatedWriteIO` | The number of write I/O operations replicated from the primary AWS Region in the global database to the cluster volume in a secondary AWS Region. The billing calculations for each DB cluster in a Neptune global database use the `VolumeWriteIOPS` metric to account for writes performed within that cluster. For the primary DB cluster, the billing calculations use `GlobalDbReplicatedWriteIO` to account for the cross-region replication to secondary DB clusters. |
| `GlobalDbProgressLag` | The number of milliseconds that a secondary cluster is behind the primary cluster for both user transactions and system transactions. |
| `GremlinClientErrorsPerSec` | Number of client-side errors per second in Gremlin traversals. |
| `GremlinServerErrorsPerSec` | Number of server-side errors per second in Gremlin traversals. |
| `GremlinRequestsPerSec` | Number of requests per second to the Gremlin engine. |
| `GremlinWebSocketOpenConnections` | The number of open WebSocket connections to Neptune. |
| `LoaderClientErrorsPerSec` | Number of client-side errors per second from loader requests. |
| `LoaderRequestsPerSec` | Number of loader requests per second. |
| `LoaderServerErrorsPerSec` | Number of loader server-side errors per second. |
| `MainRequestQueuePendingRequests` | The number of requests waiting in the input queue pending execution. Neptune starts throttling requests when they exceed the maximum queue capacity. |
| `NCUUtilization``` | Only applicable to a [Neptune Serverless](neptune-serverless.md) DB instance or DB cluster. At an instance level, reports a percentage calculated as the number of Neptune capacity units (NCUs) currently being used by the instance in question, divided by the maximum NCU capacity setting for the cluster. An NCU, or Neptune capacity unit, consists of 2 GiB (gibibyte) of memory (RAM), along with associated virtual processor capacity (vCPU) and networking. At a cluster level, `NCUUtilization` reports the percentage of maximum capacity being used by the cluster as a whole. |
| `NetworkThroughput` | The amount of network throughput both received from and transmitted to clients by each instance in the Neptune DB cluster, in bytes per second. This throughput does **not** include network traffic between instances in the DB cluster and the cluster volume. |
| `NetworkTransmitThroughput` | The amount of outgoing network throughput transmitted to clients by each instance in the Neptune DB cluster, in bytes per second. This throughput does **not** include network traffic between instances in the DB cluster and the cluster volume. |
| NumIndexDeletesPerSec | Number of deletes from individual indexes. Deletes from each index are counted individually. This includes the deletes that may get rolled back if a query encounters an error. |
| NumIndexInsertsPerSec | Number of inserts to individual indexes. Inserts to each index are counted separately. This includes the inserts that may get rolled back if a query encounters an error. |
| NumIndexReadsPerSec | Number of statements scanned from any index. Any access pattern starts with a search on an index and reads of all matching statements. An increase in this metric can cause an increase in query latencies or CPU utilization. |
| `NumOpenCypherClientErrorsPerSec` | The number of OpenCypher client errors per second. |
| `NumOpenCypherRequestsPerSec` | The number of OpenCypher requests per second. |
| `NumOpenCypherServerErrorsPerSec` | The number of OpenCypher server errors per second. |
| `NumQueuedRequestsPerSec` | The number of requests queued per second. |
| `NumResultCacheHit` | Number of Gremlin result cache hits. |
| `NumResultCacheMiss` | Number of Gremlin result cache misses. |
| `NumTxCommitted` | The number of transactions successfully committed per second. |
| `NumTxOpened` | The number of transactions opened on the server per second. |
| `NumTxRolledBack` | For write queries, the number of transactions per second rolled back on the server because of errors. For read-only queries, this metric is equal to the number of completed read-only transactions per second. |
| NumUndoPagesPurged | This metric indicates the number of batches purged. This metric is indicator of progress in purging. The value is 0 for reader instances, and the metric only applies to the writer instance. |
| `OpenCypherRequestsPerSec` | Number of requests per second (both HTTPS and Bolt) to the openCypher engine. |
| `OpenCypherBoltOpenConnections` | The number of open Bolt connections to Neptune. |
| `ResultCacheSizeInBytes` | Total estimated size (in bytes) of all cached items in the Gremlin result cache. |
| `ResultCacheItemCount` | Number of items in the Gremlin result cache. |
| `ResultCacheOldestItemTimestamp` | The timestamp of the oldest item cached in the Gremlin result cache. |
| `ResultCacheNewestItemTimestamp` | The timestamp of the newest item cached in the Gremlin result cache. |
| `ServerlessDatabaseCapacity` | As an instance-level metric, `ServerlessDatabaseCapacity` reports the current instance capacity of a given [Neptune serverless](neptune-serverless.md) instance, in NCUs. An NCU, or Neptune capacity unit, consists of 2 GiB (gibibyte) of memory (RAM), along with associated virtual processor capacity (vCPU) and networking. At a cluster-level, `ServerlessDatabaseCapacity` reports the average of all the `ServerlessDatabaseCapacity` values of the DB instances in the cluster. |
| `SnapshotStorageUsed` | The total amount of backup storage consumed by all snapshots for a Neptune DB cluster outside its backup retention window, in bytes. Included in the total reported by the `TotalBackupStorageBilled` metric. |
| `SparqlClientErrorsPerSec` | The number of client-side errors per second in SPARQL queries. |
| `SparqlRequestsPerSec` | The number of requests per second to the SPARQL engine. |
| `SparqlServerErrorsPerSec` | The number of SPARQL server errors per second. |
| `StatsNumStatementsScanned` | The total number of statements scanned for [DFE statistics](neptune-dfe-statistics.md) since the server started. Every time statistics computation is triggered, this number increases, but when no computation is happening, it remains static. As a result, if you graph it over time, you can tell when computation happened and when it didn't: ![\[Graph of StatsNumStatementsScanned values over time\]](http://docs.aws.amazon.com/neptune/latest/userguide/images/StatsNumStatementsScanned-graph.png) By looking at the slope of the graph in periods where the metric is increasing, you can also tell how quickly the computation was going. If there is no such metric, it means that the statistics feature is disabled on your DB cluster, or that the engine version you're running doesn't have the statistics feature. If the metric value is zero, it means that no statistics computation has occurred. |
| `StorageNetworkReceiveThroughput` | The amount of network throughput received from the storage subsystem by each instance in the Neptune DB cluster. |
| StorageNetworkThroughput | The amount of network throughput received from and sent to the storage subsystem by each instance in the Neptune DB cluster. |
| `StorageNetworkTransmitThroughput` | The amount of network throughput sent to the storage subsystem by each instance in the Neptune DB cluster. |
| `SwapUsage` | The amount of swap space used. |
| `TempStorageIOPS` | The number of IOPS for both read and writes on local storage attached to the Neptune DB instance. This metric represents a count and is measured once per second. |
| `TempStorageThroughput` | The amount of data transferred to and from local storage associated with the Neptune DB instance. This metric represents bytes and is measured once per second. |
| `TotalBackupStorageBilled` | The total amount of backup storage for which you are billed for a given Neptune DB cluster, in bytes. Includes the backup storage measured by the `BackupRetentionPeriodStorageUsed` and `SnapshotStorageUsed` metrics. |
| `TotalRequestsPerSec` | The total number of requests per second to the server from all sources. |
| `TotalClientErrorsPerSec` | The total number per second of requests that errored out because of client-side issues. |
| `TotalServerErrorsPerSec` | The total number per second of requests that errored out on the server because of internal failures. |
| `UndoLogListSize` | The count of undo logs in the undo log list. Undo logs contain records of committed transactions that expire when all active transactions are more recent than the commit time. The expired records are periodically purged. Records for delete operations can take longer to purge than records for other types of transaction. Purging is done exclusively by the DB cluster's writer instance, so the rate of purging is dependent on the writer instance type. If the `UndoLogListSize` is high and growing in your DB cluster, upgrade the writer instance to increase the purge rate. Also, if you are upgrading to engine version `1.2.0.0` or higher from a version earlier than `1.2.0.0`, first make sure that the `UndoLogListSize` value is under a certain threshold. Otherwise, the patch will roll back and fail. The thresholds are based on instance type: the default limit is 40k for 4xlarge or larger instances, and 10k for instances smaller than 4xlarge. If you attempt to upgrade a cluster with the `UndoLogListSize` metric above the limit, the patch process will roll back, the upgrade will be canceled, and an event with the reason will be visible on the cluster event page. These limits can change for operational reasons without prior warning. Because engine versions `1.2.0.0` and higher use a different format for undo logs, the upgrade can only begin after your previous undo logs have been fully purged below the applicable threshold. See [Upgrading to 1.2.0.0 or above](engine-updates-1200-changes.md) for more information. |
| `VolumeBytesUsed` | The total amount of storage allocated to your Neptune DB cluster, in bytes. This is the amount of storage for which you are billed. It is the maximum amount of storage allocated to your DB cluster at any point in its existence, not the amount you are currently using (see [Neptune storage billing](feature-overview-storage.md#feature-overview-storage-billing)). |
| `VolumeReadIOPs` | The total number of billed read I/O operations from a cluster volume, reported a 5-minute intervals. Billed read operations are calculated at the cluster volume level, aggregated from all instances in the Neptune DB cluster, and then reported at 5-minute intervals. |
| VolumeWriteIOPs | The total number of write disk I/O operations to the cluster volume, reported at 5-minute intervals. |
## CloudWatch Metrics That Are Now Deprecated in Neptune
Use of these Neptune metrics has now been deprecated. They are still supported, but may be eliminated in the future as new and better metrics become available.
| Metric | Description |
| --- | --- |
| `GremlinHttp1xx` | Number of HTTP 1xx responses for the Gremlin endpoint per second. We recommend that you use the new `Http1xx` combined metric instead. |
| `GremlinHttp2xx` | Number of HTTP 2xx responses for the Gremlin endpoint per second. We recommend that you use the new `Http2xx` combined metric instead. |
| `GremlinHttp4xx` | Number of HTTP 4xx errors for the Gremlin endpoint per second. We recommend that you use the new `Http4xx` combined metric instead. |
| `GremlinHttp5xx` | Number of HTTP 5xx errors for the Gremlin endpoint per second. We recommend that you use the new `Http5xx` combined metric instead. |
| `GremlinErrors` | Number of errors in Gremlin traversals. |
| `GremlinRequests` | Number of requests to Gremlin engine. |
| `GremlinWebSocketSuccess` | Number of successful WebSocket connections to the Gremlin endpoint per second. |
| `GremlinWebSocketClientErrors` | Number of WebSocket client errors on the Gremlin endpoint per second. |
| `GremlinWebSocketServerErrors` | Number of WebSocket server errors on the Gremlin endpoint per second. |
| `GremlinWebSocketAvailableConnections` | Number of potential WebSocket connections currently available. |
| `Http100` | Number of HTTP 100 responses for the endpoint per second. We recommend that you use the new `Http1xx` combined metric instead. |
| `Http101` | Number of HTTP 101 responses for the endpoint per second. We recommend that you use the new `Http1xx` combined metric instead. |
| `Http1xx` | Number of HTTP 1xx responses for the endpoint per second. |
| `Http200` | Number of HTTP 200 responses for the endpoint per second. We recommend that you use the new `Http2xx` combined metric instead. |
| `Http2xx` | Number of HTTP 2xx responses for the endpoint per second. |
| `Http400` | Number of HTTP 400 errors for the endpoint per second. We recommend that you use the new `Http4xx` combined metric instead. |
| `Http403` | Number of HTTP 403 errors for the endpoint per second. We recommend that you use the new `Http4xx` combined metric instead. |
| `Http405` | Number of HTTP 405 errors for the endpoint per second. We recommend that you use the new `Http4xx` combined metric instead. |
| `Http413` | Number of HTTP 413 errors for the endpoint per second. We recommend that you use the new `Http4xx` combined metric instead. |
| `Http429` | Number of HTTP 429 errors for the endpoint per second. We recommend that you use the new `Http4xx` combined metric instead. |
| `Http4xx` | Number of HTTP 4xx errors for the endpoint per second. |
| `Http500` | Number of HTTP 500 errors for the endpoint per second. We recommend that you use the new `Http5xx` combined metric instead. |
| `Http501` | Number of HTTP 501 errors for the endpoint per second. We recommend that you use the new `Http5xx` combined metric instead. |
| `Http5xx` | Number of HTTP 5xx errors for the endpoint per second. |
| `LoaderErrors` | Number of errors from Loader requests. |
| `LoaderRequests` | Number of Loader Requests. |
| `SparqlHttp1xx` | Number of HTTP 1xx responses for the SPARQL endpoint per second. We recommend that you use the new `Http1xx` combined metric instead. |
| `SparqlHttp2xx` | Number of HTTP 2xx responses for the SPARQL endpoint per second. We recommend that you use the new `Http2xx` combined metric instead. |
| `SparqlHttp4xx` | Number of HTTP 4xx errors for the SPARQL endpoint per second. We recommend that you use the new `Http4xx` combined metric instead. |
| `SparqlHttp5xx` | Number of HTTP 5xx errors for the SPARQL endpoint per second. We recommend that you use the new `Http5xx` combined metric instead. |
| `SparqlErrors` | Number of errors in the SPARQL queries. |
| `SparqlRequests` | Number of requests to the SPARQL engine. |
| `StatusErrors` | Number of errors from the status endpoint. |
| `StatusRequests` | Number of requests to the status endpoint. |
# Neptune CloudWatch Dimensions
The metrics for Amazon Neptune are qualified by the values for the account, graph name, or operation. You can use the Amazon CloudWatch console to retrieve Neptune data along with any of the dimensions in the following table.
| Dimension | Description |
| --- | --- |
| DBInstanceIdentifier | Filters the data you request for a specific database instance within a cluster. |
| DBClusterIdentifier | Filters the data you request for a specific Neptune DB cluster. |
| DBClusterIdentifier, EngineName | Filters the data by the cluster. The engine name for all Neptune instances is neptune. |
| DBClusterIdentifier, Role | Filters the data you request for a specific Neptune DB cluster, aggregating the metric by instance role (WRITER/READER). For example, you can aggregate metrics for all READER instances that belong to a cluster. |
| DBClusterIdentifier, SourceRegion | Filters the data by the primary cluster in a global database primary region. |
| DatabaseClass | Filters the data you request for all instances in a database class. For example, you can aggregate metrics for all instances that belong to the database class db.r4.large |
| EngineName | The engine name for all Neptune instances is neptune. |
| GlobalDbDBClusterIdentifier, SecondaryRegion | Filters the data by the secondary cluster of a specified global database in a secondary region. |