# Managing clusters
Most MemoryDB operations are performed at the cluster level. You can set up a cluster with a specific number of nodes and a parameter group that controls the properties for each node. All nodes within a cluster are designed to be of the same node type and have the same parameter and security group settings.
Every cluster must have a cluster identifier. The cluster identifier is a customer-supplied name for the cluster. This identifier specifies a particular cluster when interacting with the MemoryDB API and AWS CLI commands. The cluster identifier must be unique for that customer in an AWS Region.
MemoryDB clusters are designed to be accessed using an Amazon EC2 instance. You can only launch your MemoryDB cluster in a virtual private cloud (VPC) based on the Amazon VPC service, but you can access it from outside AWS. For more information, see [Accessing MemoryDB resources from outside AWS](accessing-memorydb.md#access-from-outside-aws).
# Data tiering
Clusters that use a node type from the r6gd family have their data tiered between memory and local SSD (solid state drives) storage. Data tiering provides a new price-performance option for Valkey and Redis OSS workloads by utilizing lower-cost solid state drives (SSDs) in each cluster node in addition to storing data in memory. Similar to other node types, the data written to r6gd nodes is durably stored in a multi-AZ transaction log. Data tiering is ideal for workloads that access up to 20 percent of their overall dataset regularly, and for applications that can tolerate additional latency when accessing data on SSD.
On clusters with data tiering, MemoryDB monitors the last access time of every item it stores. When available memory (DRAM) is fully consumed, MemoryDB uses a least-recently used (LRU) algorithm to automatically move infrequently accessed items from memory to SSD. When data on SSD is subsequently accessed, MemoryDB automatically and asynchronously moves it back to memory before processing the request. If you have a workload that accesses only a subset of its data regularly, data tiering is an optimal way to scale your capacity cost-effectively.
Note that when using data tiering, keys themselves always remain in memory, while the LRU governs the placement of values on memory vs. disk. In general, we recommend that your key sizes are smaller than your value sizes when using data tiering.
Data tiering is designed to have minimal performance impact to application workloads. For example, assuming 500-byte String values, you can typically expect an additional 450 microseconds of latency for read requests to data stored on SSD compared to read requests to data in memory.
With the largest data tiering node size (db.r6gd.8xlarge), you can store up to \$1500 TBs in a single 500-node cluster (250 TB when using 1 read replica). For Data tiering, MemoryDB reserves 19% of (DRAM) memory per node for non-data use. Data tiering is compatible with all Valkey and Redis OSS commands and data structures supported in MemoryDB. You don't need any client-side changes to use this feature.
**Topics**
+ [Best practices](data-tiering-best-practices.md)
+ [Data tiering limitations](data-tiering-prerequisites.md)
+ [Data tiering pricing](data-tiering-pricing.md)
+ [Data tiering monitoring](data-tiering-monitoring.md)
+ [Using data tiering](data-tiering-enabling.md)
+ [Restoring data from a snapshot into clusters](data-tiering-enabling-snapshots.md)
# Best practices
We recommend the following best practices:
+ Data tiering is ideal for workloads that access up to 20 percent of their overall dataset regularly, and for applications that can tolerate additional latency when accessing data on SSD.
+ When using SSD capacity available on data-tiered nodes, we recommend that value size be larger than the key size. Value size cannot be greater than 128MB; else it will not be moved to disk. When items are moved between DRAM and SSD, keys will always remain in memory and only the values are moved to the SSD tier.
# Data tiering limitations
Data tiering has the following limitations:
+ The node type you use must be from the r6gd family, which is available in the following regions: `us-east-2`, `us-east-1`, `us-west-2`, `us-west-1`, `eu-west-1`, `eu-west-3`, `eu-central-1`, `ap-northeast-1`, `ap-southeast-1`, `ap-southeast-2`, `ap-south-1`, `ca-central-1` and `sa-east-1`.
+ You cannot restore a snapshot of an r6gd cluster into another cluster unless it also uses r6gd.
+ You cannot export a snapshot to Amazon S3 for data-tiering clusters.
+ Forkless save is not supported.
+ Scaling is not supported from a data tiering cluster (for example, a cluster using an r6gd node type) to a cluster that does not use data tiering (for example, a cluster using an r6g node type).
+ Data tiering only supports `volatile-lru`, `allkeys-lru` and `noeviction` maxmemory policies.
+ Items larger than 128 MiB are not moved to SSD.
# Data tiering pricing
R6gd nodes have 5x more total capacity (memory \$1 SSD) and can help you achieve over 60 percent storage cost savings when running at maximum utilization compared to R6g nodes (memory only). For more information, see [MemoryDB pricing](https://aws.amazon.com/memorydb/pricing/).
# Data tiering monitoring
MemoryDB offers metrics designed specifically to monitor the performance clusters that use data tiering. To monitor the ratio of items in DRAM compared to SSD, you can use the `CurrItems` metric at [Metrics for MemoryDB](metrics.memorydb.md). You can calculate the percentage as: `(CurrItems with Dimension: Tier = Memory * 100) / (CurrItems with no dimension filter)`.
If the configured eviction policy allows, then MemoryDB will start evicting items when the percentage of items in memory decreases below 5 percent. On nodes configured with noeviction policy, write operations will receive an out of memory error.
It is still recommended that you consider [Scaling MemoryDB clusters](scaling-cluster.md) when the percentage of items in memory decreases below 5 percent. For more information, see *Metrics for MemoryDB clusters that use data tiering* at [Metrics for MemoryDB](metrics.memorydb.md).
# Using data tiering
## Using data tiering using the AWS Management Console
When creating a cluster, you use data tiering by selecting a node type from the r6gd family, such as *db.r6gd.xlarge*. Selecting that node type automatically enables data tiering.
For more information on creating a cluster, see [Step 2: Create a cluster](getting-started.md#getting-started.createcluster).
## Enabling data tiering using the AWS CLI
When creating a cluster using the AWS CLI, you use data tiering by selecting a node type from the r6gd family, such as *db.r6gd.xlarge* and setting the `--data-tiering` parameter.
You cannot opt out of data tiering when selecting a node type from the r6gd family. If you set the `--no-data-tiering` parameter, the operation will fail.
For Linux, macOS, or Unix:
```
aws memorydb create-cluster \
--cluster-name my-cluster \
--node-type db.r6gd.xlarge \
--engine valkey \
--acl-name my-acl \
--subnet-group my-sg \
--data-tiering
```
For Windows:
```
aws memorydb create-cluster ^
--cluster-name my-cluster ^
--node-type db.r6gd.xlarge ^
--engine valkey ^
--acl-name my-acl ^
--subnet-group my-sg
--data-tiering
```
After running this operation, you will see a response similar to the following:
```
{
"Cluster": {
"Name": "my-cluster",
"Status": "creating",
"NumberOfShards": 1,
"AvailabilityMode": "MultiAZ",
"ClusterEndpoint": {
"Port": 6379
},
"NodeType": "db.r6gd.xlarge",
"EngineVersion": "7.2",
"EnginePatchVersion": "7.2.6",
"Engine": "valkey"
"ParameterGroupName": "default.memorydb-valkey7",
"ParameterGroupStatus": "in-sync",
"SubnetGroupName": "my-sg",
"TLSEnabled": true,
"ARN": "arn:aws:memorydb:us-east-1:xxxxxxxxxxxxxx:cluster/my-cluster",
"SnapshotRetentionLimit": 0,
"MaintenanceWindow": "wed:03:00-wed:04:00",
"SnapshotWindow": "04:30-05:30",
"ACLName": "my-acl",
"DataTiering":"true",
"AutoMinorVersionUpgrade": true
}
}
```
# Restoring data from a snapshot into clusters
You can restore a snapshot to a new cluster with data tiering enabled using the (Console), (AWS CLI) or (MemoryDB API). When you create a cluster using node types in the r6gd family, data tiering is enabled.
## Restoring data from a snapshot into clusters with data tiering enabled (console)
To restore a snapshot to a new cluster with data tiering enabled (console), follow the steps at [Restoring from a snapshot (Console)](snapshots-restoring.md#snapshots-restoring-CON)
Note that to enable data-tiering, you need to select a node type from the r6gd family.
## Restoring data from a snapshot into clusters with data tiering enabled (AWS CLI)
When creating a cluster using the AWS CLI, data tiering is by default used by selecting a node type from the r6gd family, such as *db.r6gd.xlarge* and setting the `--data-tiering` parameter.
You cannot opt out of data tiering when selecting a node type from the r6gd family. If you set the `--no-data-tiering` parameter, the operation will fail.
For Linux, macOS, or Unix:
```
aws memorydb create-cluster \
--cluster-name my-cluster \
--node-type db.r6gd.xlarge \
--engine valkey
--acl-name my-acl \
--subnet-group my-sg \
--data-tiering \
--snapshot-name my-snapshot
```
For Windows:
```
aws memorydb create-cluster ^
--cluster-name my-cluster ^
--node-type db.r6gd.xlarge ^
--engine valkey ^
--acl-name my-acl ^
--subnet-group my-sg ^
--data-tiering ^
--snapshot-name my-snapshot
```
After running this operation, you will see a response similar to the following:
```
{
"Cluster": {
"Name": "my-cluster",
"Status": "creating",
"NumberOfShards": 1,
"AvailabilityMode": "MultiAZ",
"ClusterEndpoint": {
"Port": 6379
},
"NodeType": "db.r6gd.xlarge",
"EngineVersion": "7.2",
"EnginePatchVersion": "7.2.6",
"Engine": "valkey"
"ParameterGroupName": "default.memorydb-valkey7",
"ParameterGroupStatus": "in-sync",
"SubnetGroupName": "my-sg",
"TLSEnabled": true,
"ARN": "arn:aws:memorydb:us-east-1:xxxxxxxxxxxxxx:cluster/my-cluster",
"SnapshotRetentionLimit": 0,
"MaintenanceWindow": "wed:03:00-wed:04:00",
"SnapshotWindow": "04:30-05:30",
"ACLName": "my-acl",
"DataTiering": "true"
}
```
# Preparing a cluster
Following, you can find instructions on creating a cluster using the MemoryDB console, the AWS CLI, or the MemoryDB API.
Whenever you create a cluster, it is a good idea to do some preparatory work so you won't need to upgrade or make changes right away.
**Topics**
+ [Determining your requirements](cluster-create-determine-requirements.md)
# Determining your requirements
**Preparation**
Knowing the answers to the following questions helps make creating your cluster go smoother:
+ Make sure to create a subnet group in the same VPC before you start creating a cluster. Alternatively, you can use the default subnet group provided. For more information, see [Subnets and subnet groups](subnetgroups.md).
MemoryDB is designed to be accessed from within AWS using Amazon EC2. However, if you launch in a VPC based on Amazon VPC, you can provide access from outside AWS. For more information, see [Accessing MemoryDB resources from outside AWS](accessing-memorydb.md#access-from-outside-aws).
+ Do you need to customize any parameter values?
If you do, create a custom parameter group. For more information, see [Creating a parameter group](parametergroups.creating.md).
+ Do you need to create a VPC security group?
For more information, see [Security in Your VPC](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_Security.html).
+ How do you intend to implement fault tolerance?
For more information, see [Mitigating Failures](faulttolerance.md).
**Topics**
+ [Memory and processor requirements](#cluster-create-determine-requirements-memory)
+ [MemoryDB cluster configuration](#cluster-configuration)
+ [Enhanced I/O Multiplexing](#cluster-create-determine-requirements-multiplexing)
+ [Scaling requirements](#cluster-create-determine-requirements-scaling)
+ [Access requirements](#cluster-create-determine-requirements-access)
+ [Region and Availability Zones](#cluster-create-determine-requirements-region)
## Memory and processor requirements
The basic building block of MemoryDB is the node. Nodes are configured in shards to form clusters. When determining the node type to use for your cluster, take the cluster’s node configuration and the amount of data you have to store into consideration.
## MemoryDB cluster configuration
MemoryDB clusters are comprised of from 1 to 500 shards. The data in a MemoryDB cluster is partitioned across the shards in the cluster. Your application connects with a MemoryDB cluster using a network address called an Endpoint. In addition to the node endpoints, the MemoryDB cluster itself has an endpoint called the *cluster endpoint*. Your application can use this endpoint to read from or write to the cluster, leaving the determination of which node to read from or write to up to MemoryDB.
## Enhanced I/O Multiplexing
If you are running Valkey or Redis OSS version 7.0 or higher, you will get additional acceleration with enhanced I/O multiplexing, where each dedicated network IO thread pipelines commands from multiple clients into the engine, taking advantage of the ability to efficiently process commands in batches. For more information, see [Ultra-fast performance](https://aws.amazon.com/memorydb/features/#Ultra-fast_performance) and [Supported node types](nodes.supportedtypes.md).
## Scaling requirements
All clusters can be scaled up a larger node type. When you scale up a MemoryDB cluster, you can do it online so the cluster remains available or you can seed a new cluster from a snapshot and avoid having the new cluster start out empty.
For more information, see [Scaling](scaling.md) in this guide.
## Access requirements
By design, MemoryDB clusters are accessed from Amazon EC2 instances. Network access to a MemoryDB cluster is limited to the account that created the cluster. Therefore, before you can access a cluster from an Amazon EC2 instance, you must authorize ingress to the cluster. For detailed instructions, see [Step 3: Authorize access to the cluster](getting-started.md#getting-started.authorizeaccess) in this guide.
## Region and Availability Zones
By locating your MemoryDB clusters in an AWS Region close to your application you can reduce latency. If your cluster has multiple nodes, locating your nodes in different Availability Zones can reduce the impact of failures on your cluster.
For more information, see the following:
+ [Choosing Regions and Availability Zones](regionsandazs.md)
+ [Mitigating Failures](faulttolerance.md)
# Creating a cluster
MemoryDB offers three ways to create a cluster. For more information, see [Step 2: Create a cluster](getting-started.md#getting-started.createcluster).
# Viewing a cluster's details
You can view detail information about one or more clusters using the MemoryDB console, AWS CLI, or MemoryDB API.
## Viewing details for a MemoryDB cluster (Console)
The following procedure details how to view the details of a MemoryDB cluster using the MemoryDB console.
****
1. Sign in to the AWS Management Console and open the MemoryDB console at [https://console.aws.amazon.com/memorydb/](https://console.aws.amazon.com/memorydb/).
1. To see details of a cluster, choose the radio button to the left of the cluster's name and then choose **View details**. You can also click directly on the cluster to view the cluster details page.
The **Cluster details** page displays details about the cluster, including the cluster endpoint. You can view more details using the multiple tabs available in the **Cluster details** page.
1. Choose the **Shards and nodes** tab to see a listing of the cluster's shards and the number of nodes in each shard.
1. To view specific information on a node, expand the shard in the table below. Alternatively you can also search for the shard using the search box.
Doing this displays information about each node, including its Availability Zone, slots/keyspaces and status.
1. Choose the **Metrics** tab to monitor their respective processes, such as **CPU Utilization** and **Engine CPU Utilization**. For more information, see [Metrics for MemoryDB](metrics.memorydb.md).
1. Choose the **Network and security** tab to see details of the subnet group and security groups.
1. In **Subnet group**, you can see the subnet group's name, a link to the VPC that subnet belongs to and the subnet group's Amazon Resource Name (ARN).
1. In **Security groups**, you can see the security group ID, name and description.
1. Choose the **Maintenace and snapshot** tab to see details of the snapshot settings.
1. In **Snapshot**, you can see whether Automated Snapshots are enabled, the snapshot retention period and the snapshot window.
1. In **Snapshots**, you will see a list of any snapshots to this cluster, including the snapshot name, size, number of shards and status.
For more information, see [Snapshot and restore](snapshots.md).
1. Choose the **Maintenace and snapshot** tab to see details of the Maintenance Window, along with any pending ACL, Resharding or Service updates. For more information, see [Managing maintenance](maintenance-window.md).
1. Choose the **Service Updates** tab to see details of the any service updates that are applicable to this cluster. For more information, see [Service updates in MemoryDB](service-updates.md).
1. Choose the **Tags** tab to see details of any resource or cost-allocation tags that are associated with this cluster. For more information, see [Tagging snapshots](snapshots-tagging.md).
## Viewing a cluster's details (AWS CLI)
You can view the details for a cluster using the AWS CLI `describe-clusters` command. If the `--cluster-name` parameter is omitted, details for multiple clusters, up to `--max-results`, are returned. If the `--cluster-name` parameter is included, details for the specified cluster are returned. You can limit the number of records returned with the `--max-results` parameter.
The following code lists the details for `my-cluster`.
```
aws memorydb describe-clusters --cluster-name my-cluster
```
The following code list the details for up to 25 clusters.
```
aws memorydb describe-clusters --max-results 25
```
**Example**
For Linux, macOS, or Unix:
```
aws memorydb describe-clusters \
--cluster-name my-cluster \
--show-shard-details
```
For Windows:
```
aws memorydb describe-clusters ^
--cluster-name my-cluster ^
--show-shard-details
```
The following JSON output shows the response:
```
{
"Clusters": [
{
"Name": "my-cluster",
"Description": "my cluster",
"Status": "available",
"NumberOfShards": 1,
"Shards": [
{
"Name": "0001",
"Status": "available",
"Slots": "0-16383",
"Nodes": [
{
"Name": "my-cluster-0001-001",
"Status": "available",
"AvailabilityZone": "us-east-1a",
"CreateTime": 1629230643.961,
"Endpoint": {
"Address": "my-cluster-0001-001.my-cluster.abcdef.memorydb.us-east-1.amazonaws.com",
"Port": 6379
}
},
{
"Name": "my-cluster-0001-002",
"Status": "available",
"CreateTime": 1629230644.025,
"Endpoint": {
"Address": "my-cluster-0001-002.my-cluster.abcdef.memorydb.us-east-1.amazonaws.com",
"Port": 6379
}
}
],
"NumberOfNodes": 2
}
],
"ClusterEndpoint": {
"Address": "clustercfg.my-cluster.abcdef.memorydb.us-east-1.amazonaws.com",
"Port": 6379
},
"NodeType": "db.r6g.large",
"EngineVersion": "6.2",
"EnginePatchVersion": "6.2.6",
"ParameterGroupName": "default.memorydb-redis6",
"ParameterGroupStatus": "in-sync",
"SubnetGroupName": "default",
"TLSEnabled": true,
"ARN": "arn:aws:memorydb:us-east-1:000000000:cluster/my-cluster",
"SnapshotRetentionLimit": 0,
"MaintenanceWindow": "sat:06:30-sat:07:30",
"SnapshotWindow": "04:00-05:00",
"ACLName": "open-access",
"DataTiering": "false",
"AutoMinorVersionUpgrade": true,
}
```
For more information, see the AWS CLI for MemoryDB topic [https://docs.aws.amazon.com/cli/latest/reference/memorydb/describe-clusters.html](https://docs.aws.amazon.com/cli/latest/reference/memorydb/describe-clusters.html).
## Viewing a cluster's details (MemoryDB API)
You can view the details for a cluster using the MemoryDB API `DescribeClusters` action. If the `ClusterName` parameter is included, details for the specified cluster are returned. If the `ClusterName` parameter is omitted, details for up to `MaxResults` (default 100) clusters are returned. The value for `MaxResults` cannot be less than 20 or greater than 100.
The following code lists the details for `my-cluster`.
```
https://memory-db.us-east-1.amazonaws.com/
?Action=DescribeClusters
&ClusterName=my-cluster
&Version=2021-01-01
&SignatureVersion=4
&SignatureMethod=HmacSHA256
&Timestamp=20210802T192317Z
&X-Amz-Credential=
```
The following code list the details for up to 25 clusters.
```
https://memory-db.us-east-1.amazonaws.com/
?Action=DescribeClusters
&MaxResults=25
&Version=2021-02-02
&SignatureVersion=4
&SignatureMethod=HmacSHA256
&Timestamp=20210802T192317Z
&X-Amz-Credential=
```
For more information, see the MemoryDB API reference topic [https://docs.aws.amazon.com/memorydb/latest/APIReference/API_DescribeClusters.html](https://docs.aws.amazon.com/memorydb/latest/APIReference/API_DescribeClusters.html).
# Modifying a MemoryDB cluster
In addition to adding or removing nodes from a cluster, there can be times where you need to make other changes to an existing cluster, such as adding a security group, changing the maintenance window or a parameter group.
We recommend that you have your maintenance window fall at the time of lowest usage. Thus it might need modification from time to time.
When you change a cluster's parameters, the change is applied to the cluster immediately. This is true whether you change the cluster's parameter group itself or a parameter value within the cluster's parameter group.
You can also update your clusters' engine version. For example, you can select a new engine minor version and MemoryDB will start updating your cluster immediately.
## Using the AWS Management Console
**To modify a cluster**
1. Sign in to the AWS Management Console and open the MemoryDB console at [https://console.aws.amazon.com/memorydb/](https://console.aws.amazon.com/memorydb/).
1. From the list in the upper-right corner, choose the AWS Region where the cluster that you want to modify is located.
1. From the left navigation, go to **Clusters**. From **Clusters detail**, select the cluster using the radio button and go to **Actions** and then **Modify**.
1. The **Modify** page appears.
1. In the **Modify** window, make the modifications that you want. Options include:
+ Description
+ Subnet groups
+ VPC Security Group(s)
+ Node type
**Note**
If the cluster is using a node type from the r6gd family, you can only choose a different node size from within that family. If you choose a node type from the r6gd family, data tiering will automatically be enabled. For more information, see [Data tiering](data-tiering.md).
+ Valkey or Redis OSS version compatibility
+ Enable Automatic snapshots
+ Snapshot Retention Period
+ Snapshot Window
+ Maintenance window
+ Topic for SNS Notification
1. Choose **Save changes**.
You can also go to the **Cluster details** page and click on **modify** to make modifications to the cluster. If you want to modify specific sections of the cluster, you can go to the respective tab in the **Cluster details** page and click **Modify**.
## Using the AWS CLI
You can modify an existing cluster using the AWS CLI `update-cluster` operation. To modify a cluster's configuration value, specify the cluster's ID, the parameter to change and the parameter's new value. The following example changes the maintenance window for a cluster named `my-cluster` and applies the change immediately.
For Linux, macOS, or Unix:
```
aws memorydb update-cluster \
--cluster-name my-cluster \
--preferred-maintenance-window sun:23:00-mon:02:00
```
For Windows:
```
aws memorydb update-cluster ^
--cluster-name my-cluster ^
--preferred-maintenance-window sun:23:00-mon:02:00
```
For more information, see [update-cluster](https://docs.aws.amazon.com/cli/latest/reference/memorydb/update-cluster.html) in the AWS CLI Command Reference.
## Using the MemoryDB API
You can modify an existing cluster using the MemoryDB API [UpdateCluster](https://docs.aws.amazon.com/memorydb/latest/APIReference/API_UpdateCluster.html) operation. To modify a cluster's configuration value, specify the cluster's ID, the parameter to change and the parameter's new value. The following example changes the maintenance window for a cluster named `my-cluster` and applies the change immediately.
```
https://memory-db.us-east-1.amazonaws.com/
?Action=UpdateCluster
&ClusterName=my-cluster
&PreferredMaintenanceWindow=sun:23:00-mon:02:00
&SignatureVersion=4
&SignatureMethod=HmacSHA256
&Timestamp=20210801T220302Z
&X-Amz-Algorithm=Amazon4-HMAC-SHA256
&X-Amz-Date=20210802T220302Z
&X-Amz-SignedHeaders=Host
&X-Amz-Expires=20210801T220302Z
&X-Amz-Credential=
&X-Amz-Signature=
```
## How to trigger a cross-engine upgrade from Redis OSS to Valkey
You can upgrade an existing Redis OSS cluster to the Valkey engine using Console, API or CLI.
If you have an existing Redis OSS cluster that is using the default parameter group, you can upgrade to Valkey by specifying the new engine and engine version with update-cluster API.
For Linux, macOS, or Unix:
```
aws memorydb update-cluster \
--cluster-name myCluster \
--engine valkey \
--engine-version 7.2
```
For Windows:
```
aws memorydb update-cluster ^
--cluster-name myCluster ^
--engine valkey ^
--engine-version 7.2
```
If you have a custom parameter group applied to the existing Redis OSS cluster you wish to upgrade, you will need to pass a custom Valkey parameter group in the request as well. The input Valkey custom parameter group must have the same Redis OSS static parameter values as the existing Redis OSS custom parameter group.
For Linux, macOS, or Unix:
```
aws memorydb update-cluster \
--cluster-name myCluster \
--engine valkey \
--engine-version 7.2 \
--parameter-group-name myParamGroup
```
For Windows:
```
aws memorydb update-cluster ^
--cluster-name myCluster ^
--engine valkey ^
--engine-version 7.2 ^
--parameter-group-name myParamGroup
```
# Adding / Removing nodes from a cluster
You can add or remove nodes from a cluster using the AWS Management Console, the AWS CLI, or the MemoryDB API.
## Using the AWS Management Console
****
1. Sign in to the AWS Management Console and open the MemoryDB console at [https://console.aws.amazon.com/memorydb/](https://console.aws.amazon.com/memorydb/).
1. From the list of clusters, choose the cluster name from which you want to add or remove a node.
1. Under the **Shards and nodes** tab, choose **Add/Delete nodes**
1. In **New number of nodes**, enter the the number of nodes you want.
1. Choose **Confirm**.
**Important**
If you set the number of nodes to 1, you will no longer be Multi-AZ enabled. You can also to choose to enable **Auto failover**.
## Using the AWS CLI
1. Identify the names of the nodes that you want to remove. For more information, see [Viewing a cluster's details](clusters.viewdetails.md).
1. Use the `update-cluster` CLI operation with a list of the nodes to remove, as in the following example.
To remove nodes from a cluster using the command-line interface, use the command `update-cluster` with the following parameters:
+ `--cluster-name` The ID of the cluster that you want to remove nodes from.
+ `--replica-configuration` – Allows you to set the number of replicas:
+ `ReplicaCount` – Set this property to specify the number of replica nodes you want.
+ `--region` Specifies the AWS Region of the cluster that you want to remove nodes from.
For Linux, macOS, or Unix:
```
aws memorydb update-cluster \
--cluster-name my-cluster \
--replica-configuration \
ReplicaCount=1 \
--region us-east-1
```
For Windows:
```
aws memorydb update-cluster ^
--cluster-name my-cluster ^
--replica-configuration ^
ReplicaCount=1 ^
--region us-east-1
```
For more information, see the AWS CLI topics [https://docs.aws.amazon.com/cli/latest/reference/memorydb/update-cluster.html](https://docs.aws.amazon.com/cli/latest/reference/memorydb/update-cluster.html).
## Using the MemoryDB API
To remove nodes using the MemoryDB API, call the `UpdateCluster` API operation with the cluster name and a list of nodes to remove, as shown:
+ `ClusterName` The ID of the cluster that you want to remove nodes from.
+ `ReplicaConfiguration` – Allows you to set the number of replicas:
+ `ReplicaCount` – Set this property to specify the number of replica nodes you want.
+ `Region` Specifies the AWS Region of the cluster that you want to remove a node from.
For more information, see [UpdateCluster](https://docs.aws.amazon.com/memorydb/latest/APIReference/API_UpdateCluster.html).
# Accessing your cluster
Your MemoryDB instances are designed to be accessed through an Amazon EC2 instance.
You can access your MemoryDB node from an Amazon EC2 instance in the same Amazon VPC. Or, by using VPC peering, you can access your MemoryDB node from an Amazon EC2 in a different Amazon VPC.
**Topics**
+ [Grant access to your cluster](#grant-access)
+ [Accessing MemoryDB resources from outside AWS](#access-from-outside-aws)
## Grant access to your cluster
You can connect to your MemoryDB cluster only from an Amazon EC2 instance that is running in the same Amazon VPC. In this case, you will need to grant network ingress to the cluster.
**To grant network ingress from an Amazon VPC security group to a cluster**
1. Sign in to the AWS Management Console and open the Amazon EC2 console at [https://console.aws.amazon.com/ec2/](https://console.aws.amazon.com/ec2/).
1. In the left navigation pane, under **Network & Security**, choose **Security Groups**.
1. From the list of security groups, choose the security group for your Amazon VPC. Unless you created a security group for MemoryDB use, this security group will be named *default*.
1. Choose the **Inbound** tab, and then do the following:
1. Choose **Edit**.
1. Choose **Add rule**.
1. In the **Type** column, choose **Custom TCP rule**.
1. In the **Port range** box, type the port number for your cluster node. This number must be the same one that you specified when you launched the cluster. The default port for both Valkey and Redis OSS is **6379**.
1. In the **Source** box, choose **Anywhere** which has the port range (0.0.0.0/0) so that any Amazon EC2 instance that you launch within your Amazon VPC can connect to your MemoryDB nodes.
**Important**
Opening up the MemoryDB cluster to 0.0.0.0/0 does not expose the cluster to the Internet because it has no public IP address and therefore cannot be accessed from outside the VPC. However, the default security group may be applied to other Amazon EC2 instances in the customer’s account, and those instances may have a public IP address. If they happen to be running something on the default port, then that service could be exposed unintentionally. Therefore, we recommend creating a VPC Security Group that will be used exclusively by MemoryDB. For more information, see [Custom Security Groups](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-network-security.html#creating-your-own-security-groups).
1. Choose **Save**.
When you launch an Amazon EC2 instance into your Amazon VPC, that instance will be able to connect to your MemoryDB cluster.
## Accessing MemoryDB resources from outside AWS
MemoryDB is a service designed to be used internally to your VPC. External access is discouraged due to the latency of Internet traffic and security concerns. However, if external access to MemoryDB is required for test or development purposes, it can be done through a VPN.
Using the AWS Client VPN, you allow external access to your MemoryDB nodes with the following benefits:
+ Restricted access to approved users or authentication keys;
+ Encrypted traffic between the VPN Client and the AWS VPN endpoint;
+ Limited access to specific subnets or nodes;
+ Easy revocation of access from users or authentication keys;
+ Audit connections;
The following procedures demonstrate how to:
**Topics**
+ [Create a certificate authority](#create-cert)
+ [Configuring AWS client VPN components](#configure-vpn-components)
+ [Configure the VPN client](#configure-vpn-client)
### Create a certificate authority
It is possible to create a Certificate Authority (CA) using different techniques or tools. We suggest the easy-rsa utility, provided by the [OpenVPN](https://openvpn.net/community-resources/openvpn-project/) project. Regardless of the option you choose, make sure to keep the keys secure. The following procedure downloads the easy-rsa scripts, creates the Certificate Authority and the keys to authenticate the first VPN client:
+ To create the initial certificates, open a terminal and do the following:
+ `git clone` [https://github.com/OpenVPN/easy-rsa](https://github.com/OpenVPN/easy-rsa)
+ `cd easy-rsa`
+ `./easyrsa3/easyrsa init-pki`
+ `./easyrsa3/easyrsa build-ca nopass`
+ `./easyrsa3/easyrsa build-server-full server nopass`
+ `./easyrsa3/easyrsa build-client-full client1.domain.tld nopass`
A **pki** subdirectory containing the certificates will be created under **easy-rsa**.
+ Submit the server certificate to the AWS Certificate manager (ACM):
+ On the ACM console, select **Certificate Manager**.
+ Select **Import Certificate**.
+ Enter the public key certificate available in the `easy-rsa/pki/issued/server.crt` file in the **Certificate body** field.
+ Paste the private key available in the `easy-rsa/pki/private/server.key` in the **Certificate private key** field. Make sure to select all the lines between `BEGIN AND END PRIVATE KEY` (including the `BEGIN` and `END` lines).
+ Paste the CA public key available on the `easy-rsa/pki/ca.crt` file in the **Certificate chain** field.
+ Select **Review and import**.
+ Select **Import**.
To submit the server's certificates to ACM using the AWS CLI, run the following command: `aws acm import-certificate --certificate fileb://easy-rsa/pki/issued/server.crt --private-key file://easy-rsa/pki/private/server.key --certificate-chain file://easy-rsa/pki/ca.crt --region region`
Note the Certificate ARN for future use.
### Configuring AWS client VPN components
**Using the AWS Console**
On the AWS console, select **Services** and then **VPC**.
Under **Virtual Private Network**, select **Client VPN Endpoints** and do the following:
**Configuring AWS Client VPN components**
+ Select **Create Client VPN Endpoint**.
+ Specify the following options:
+ **Client IPv4 CIDR**: use a private network with a netmask of at least /22 range. Make sure that the selected subnet does not conflict with the VPC networks' addresses. Example: 10.0.0.0/22.
+ In **Server certificate ARN**, select the ARN of the certificate previously imported.
+ Select **Use mutual authentication**.
+ In **Client certificate ARN**, select the ARN of the certificate previously imported.
+ Select **Create Client VPN Endpoint**.
**Using the AWS CLI**
Run the following command:
`aws ec2 create-client-vpn-endpoint --client-cidr-block "10.0.0.0/22" --server-certificate-arn arn:aws:acm:us-east-1:012345678912:certificate/0123abcd-ab12-01a0-123a-123456abcdef --authentication-options Type=certificate-authentication,,MutualAuthentication={ClientRootCertificateChainArn=arn:aws:acm:us-east-1:012345678912:certificate/123abcd-ab12-01a0-123a-123456abcdef} --connection-log-options Enabled=false `
Example output:
`"ClientVpnEndpointId": "cvpn-endpoint-0123456789abcdefg", "Status": { "Code": "pending-associate" }, "DnsName": "cvpn-endpoint-0123456789abcdefg.prod.clientvpn.us-east-1.amazonaws.com" } `
**Associate the target networks to the VPN endpoint**
+ Select the new VPN endpoint, and then select the **Associations** tab.
+ Select **Associate** and specify the following options.
+ **VPC**: Select the MemoryDB Cluster's VPC.
+ Select one of the MemoryDB cluster's networks. If in doubt, review the networks in the **Subnet Groups** on the MemoryDB dashboard.
+ Select **Associate**. If necessary, repeat the steps for the remaining networks.
**Using the AWS CLI**
Run the following command:
`aws ec2 associate-client-vpn-target-network --client-vpn-endpoint-id cvpn-endpoint-0123456789abcdefg --subnet-id subnet-0123456789abdcdef`
Example output:
`"Status": { "Code": "associating" }, "AssociationId": "cvpn-assoc-0123456789abdcdef" } `
**Review the VPN security group**
The VPN Enpoint will automatically adopt the VPC's default security group. Check the inbound and outbound rules and confirm if the security group allows the traffic from the VPN network (defined on the VPN Endpoint settings) to the MemoryDB networks on the service ports (by default, 6379 for Redis).
If you need to change the security group assigned to the VPN Endpoint, proceed as follows:
+ Select the current security group.
+ Select **Apply Security Group**.
+ Select the new Security Group.
**Using the AWS CLI**
Run the following command:
`aws ec2 apply-security-groups-to-client-vpn-target-network --client-vpn-endpoint-id cvpn-endpoint-0123456789abcdefga --vpc-id vpc-0123456789abdcdef --security-group-ids sg-0123456789abdcdef`
Example output:
`"SecurityGroupIds": [ "sg-0123456789abdcdef" ] } `
**Note**
The MemoryDB security group also needs to allow traffic coming from the VPN clients. The clients' addresses will be masked with the VPN Endpoint address, according to the VPC Network. Therefore, consider the VPC network (not the VPN Clients' network) when creating the inbound rule on the MemoryDB security group.
**Authorize the VPN access to the destination networks**
On the **Authorization** tab, select **Authorize Ingress** and specify the following:
+ Destination network to enable access: Either use 0.0.0.0/0 to allow access to any network (including the Internet) or restrict the the MemoryDB networks/hosts.
+ Under **Grant access to:**, select **Allow access to all users**.
+ Select **Add Authorization Rules**.
**Using the AWS CLI**
Run the following command:
`aws ec2 authorize-client-vpn-ingress --client-vpn-endpoint-id cvpn-endpoint-0123456789abcdefg --target-network-cidr 0.0.0.0/0 --authorize-all-groups`
Example output:
`{ "Status": { "Code": "authorizing" } }`
**Allowing access to the Internet from the VPN clients**
If you need to browse the Internet through the VPN, you need to create an additional route. Select the **Route Table** tab and then select **Create Route**:
+ Route destination: 0.0.0.0/0
+ **Target VPC Subnet ID**: Select one of the associated subnets with access to the Internet.
+ Select **Create Route**.
**Using the AWS CLI**
Run the following command:
`aws ec2 create-client-vpn-route --client-vpn-endpoint-id cvpn-endpoint-0123456789abcdefg --destination-cidr-block 0.0.0.0/0 --target-vpc-subnet-id subnet-0123456789abdcdef`
Example output:
`{ "Status": { "Code": "creating" } } `
### Configure the VPN client
On the AWS Client VPN Dashboard, select the VPN endpoint recently created and select **Download Client Configuration**. Copy the configuration file, and the files `easy-rsa/pki/issued/client1.domain.tld.crt` and `easy-rsa/pki/private/client1.domain.tld.key`. Edit the configuration file and change or add the following parameters:
+ cert: add a new line with the parameter cert pointing to the `client1.domain.tld.crt` file. Use the full path to the file. Example: `cert /home/user/.cert/client1.domain.tld.crt`
+ cert: key: add a new line with the parameter key pointing to the `client1.domain.tld.key` file. Use the full path to the file. Example: `key /home/user/.cert/client1.domain.tld.key`
Establish the VPN connection with the command: `sudo openvpn --config downloaded-client-config.ovpn`
**Revoking access**
If you need to invalidate the access from a particular client key, the key needs to be revoked in the CA. Then submit the revocation list to AWS Client VPN.
Revoking the key with easy-rsa:
+ `cd easy-rsa`
+ `./easyrsa3/easyrsa revoke client1.domain.tld`
+ Enter "yes" to continue, or any other input to abort.
`Continue with revocation: `yes` ... * `./easyrsa3/easyrsa gen-crl`
+ An updated CRL has been created. CRL file: `/home/user/easy-rsa/pki/crl.pem`
Importing the revocation list to the AWS Client VPN:
+ On the AWS Management Console, select **Services** and then **VPC**.
+ Select **Client VPN Endpoints**.
+ Select the Client VPN Endpoint and then select **Actions** -> **Import Client Certificate CRL**.
+ Paste the contents of the `crl.pem` file.
**Using the AWS CLI**
Run the following command:
`aws ec2 import-client-vpn-client-certificate-revocation-list --certificate-revocation-list file://./easy-rsa/pki/crl.pem --client-vpn-endpoint-id cvpn-endpoint-0123456789abcdefg `
Example output:
`Example output: { "Return": true } `
# Finding connection endpoints
Your application connects to your cluster using the endpoint. An endpoint is a cluster's unique address. Use the cluster's *Cluster Endpoint* for all operations.
The following sections guide you through discovering the endpoint you'll need.
## Finding the Endpoint for a MemoryDB Cluster (AWS Management Console)
**To find a MemoryDB cluster's endpoint**
1. Sign in to the AWS Management Console and open the MemoryDB console at [https://console.aws.amazon.com/memorydb/](https://console.aws.amazon.com/memorydb/).
1. From the navigation pane, choose **Clusters**.
The clusters screen will appear with a list of clusters. Choose the cluster you wish to connect to.
1. To find the cluster's endpoint, choose the cluster's name (not the radio button).
1. The **Cluster endpoint** is displayed under **Cluster details**. To copy it, choose the *copy* icon to the left of the endpoint.
## Finding the Endpoint for a MemoryDB Cluster (AWS CLI)
You can use the `describe-clusters` command to discover the endpoint for a cluster. The command returns the cluster's endpoint.
The following operation retrieves the endpoint, which in this example is represented as a *sample*, for the cluster `mycluster`.
It returns the following JSON response:
```
aws memorydb describe-clusters \
--cluster-name mycluster
```
For Windows:
```
aws memorydb describe-clusters ^
--cluster-name mycluster
```
```
{
"Clusters": [
{
"Name": "my-cluster",
"Status": "available",
"NumberOfShards": 1,
"ClusterEndpoint": {
"Address": "clustercfg.my-cluster.xxxxxx.memorydb.us-east-1.amazonaws.com",
"Port": 6379
},
"NodeType": "db.r6g.large",
"EngineVersion": "6.2",
"EnginePatchVersion": "6.2.4",
"ParameterGroupName": "default.memorydb-redis6",
"ParameterGroupStatus": "in-sync",
"SubnetGroupName": "my-sg",
"TLSEnabled": true,
"ARN": "arn:aws:memorydb:us-east-1:zzzexamplearn:cluster/my-cluster",
"SnapshotRetentionLimit": 0,
"MaintenanceWindow": "wed:03:00-wed:04:00",
"SnapshotWindow": "04:30-05:30",
"ACLName": "my-acl",
"AutoMinorVersionUpgrade": true
}
]
}
```
For more information, see [describe-clusters](https://docs.aws.amazon.com/cli/latest/reference/memorydb/describe-clusters.html).
## Finding the Endpoint for a MemoryDB Cluster (MemoryDB API)
You can use the MemoryDB API to discover the endpoint of a cluster.
### Finding the Endpoint for a MemoryDB Cluster (MemoryDB API)
You can use the MemoryDB API to discover the endpoint for a cluster with the `DescribeClusters` action. The action returns the cluster's endpoint.
The following operation retrieves the cluster endpoint for the cluster `mycluster`.
```
https://memory-db.us-east-1.amazonaws.com/
?Action=DescribeClusters
&ClusterName=mycluster
&SignatureVersion=4
&SignatureMethod=HmacSHA256
&Timestamp=20210802T192317Z
&Version=2021-01-01
&X-Amz-Credential=
```
For more information, see [DescribeClusters](https://docs.aws.amazon.com/memorydb/latest/APIReference/API_DescribeClusters.html).
# Working with shards
A shard is a collection of one to 6 nodes. You can create a cluster with higher number of shards and lower number of replicas totaling up to 500 nodes per cluster. This cluster configuration can range from 500 shards and 0 replicas to 100 shards and 4 replicas, which is the maximum number of replicas allowed. The cluster's data is partitioned across the cluster's shards. If there is more than one node in a shard, the shard implements replication with one node being the read/write primary node and the other nodes read-only replica nodes.
When you create a MemoryDB cluster using the AWS Management Console, you specify the number of shards in the cluster and the number of nodes in the shards. For more information, see [Creating a MemoryDB cluster](getting-started.md#clusters.create).
Each node in a shard has the same compute, storage and memory specifications. The MemoryDB API lets you control cluster-wide attributes, such as the number of nodes, security settings, and system maintenance windows.
For more information, see [Offline resharding for MemoryDB](cluster-resharding-offline.md) and [Online resharding for MemoryDB](cluster-resharding-online.md).
## Finding a shard's name
You can find a shard's name using the AWS Management Console, the AWS CLI or the MemoryDB API.
### Using the AWS Management Console
The following procedure uses the AWS Management Console to find a MemoryDB's cluster's shard names.
****
1. Sign in to the AWS Management Console and open the MemoryDB console at [https://console.aws.amazon.com/memorydb/](https://console.aws.amazon.com/memorydb/).
1. On the left navigation pane, choose **Clusters**.
1. Choose the cluster under **Name** whose shard names you want to find.
1. Under the **Shards and nodes** tab, view the list of shards under **Name**. You can also expand each one to view details of their nodes.
### Using the AWS CLI
To find shard (shard) names for MemoryDB clusters use the AWS CLI operation `describe-clusters` with the following optional parameter.
+ **`--cluster-name`**—An optional parameter which when used limits the output to the details of the specified cluster. If this parameter is omitted, the details of up to 100 clusters is returned.
+ **`--show-shard-details`**—Returns details of the shards, including their names.
This command returns the details for `my-cluster`.
For Linux, macOS, or Unix:
```
aws memorydb describe-clusters \
--cluster-name my-cluster
--show-shard-details
```
For Windows:
```
aws memorydb describe-clusters ^
--cluster-name my-cluster
--show-shard-details
```
It returns the following JSON response:
Line breaks are added for ease of reading.
```
{
"Clusters": [
{
"Name": "my-cluster",
"Status": "available",
"NumberOfShards": 1,
"Shards": [
{
"Name": "0001",
"Status": "available",
"Slots": "0-16383",
"Nodes": [
{
"Name": "my-cluster-0001-001",
"Status": "available",
"AvailabilityZone": "us-east-1a",
"CreateTime": "2021-08-21T20:22:12.405000-07:00",
"Endpoint": {
"Address": "clustercfg.my-cluster.xxxxx.memorydb.us-east-1.amazonaws.com",
"Port": 6379
}
},
{
"Name": "my-cluster-0001-002",
"Status": "available",
"AvailabilityZone": "us-east-1b",
"CreateTime": "2021-08-21T20:22:12.405000-07:00",
"Endpoint": {
"Address": "clustercfg.my-cluster.xxxxx.memorydb.us-east-1.amazonaws.com",
"Port": 6379
}
}
],
"NumberOfNodes": 2
}
],
"ClusterEndpoint": {
"Address": "clustercfg.my-cluster.xxxxx.memorydb.us-east-1.amazonaws.com",
"Port": 6379
},
"NodeType": "db.r6g.large",
"EngineVersion": "6.2",
"EnginePatchVersion": "6.2.6",
"ParameterGroupName": "default.memorydb-redis6",
"ParameterGroupStatus": "in-sync",
"SubnetGroupName": "my-sg",
"TLSEnabled": true,
"ARN": "arn:aws:memorydb:us-east-1:xxxxxexamplearn:cluster/my-cluster",
"SnapshotRetentionLimit": 0,
"MaintenanceWindow": "wed:03:00-wed:04:00",
"SnapshotWindow": "04:30-05:30",
"ACLName": "my-acl",
"DataTiering": "false",
"AutoMinorVersionUpgrade": true
}
]
}
```
### Using the MemoryDB API
To find shard ids for MemoryDB clusters use the API operation `DescribeClusters` with the following optional parameter.
+ **`ClusterName`**—An optional parameter which when used limits the output to the details of the specified cluster. If this parameter is omitted, the details of up to 100 clusters is returned.
+ **`ShowShardDetails`**—Returns details of the shards, including their names.
**Example**
This command returns the details for `my-cluster`.
For Linux, macOS, or Unix:
```
https://memory-db.us-east-1.amazonaws.com/
?Action=DescribeClusters
&ClusterName=sample-cluster
&ShowShardDetails=true
&Version=2021-01-01
&SignatureVersion=4
&SignatureMethod=HmacSHA256
&Timestamp=20210802T192317Z
&X-Amz-Credential=
```