Using a snapshot to migrate data - Amazon OpenSearch Service

Using a snapshot to migrate data

In-place upgrades are the easier, faster, and more reliable way to upgrade a domain to a later OpenSearch or Elasticsearch version. Snapshots are a good option if you need to migrate from a pre-5.1 version of Elasticsearch or want to migrate to an entirely new cluster.

The following table shows how to use snapshots to migrate data to a domain that uses a different OpenSearch or Elasticsearch version. For more information about taking and restoring snapshots, see Creating index snapshots in Amazon OpenSearch Service.

From version To version Migration process
OpenSearch 1.3 or 2.x OpenSearch 2.x

  1. Review breaking changes for OpenSearch 2.3 to see if you need to make adjustments to your indexes or applications.

  2. Create a manual snapshot of the 1.3 or 2.x domain.

  3. Create a 2.x domain that's a higher version than your original 1.3 or 2.x domain.

  4. Restore the snapshot from the original domain to the 2.x domain. During the operation, you might need to restore your .opensearch index under a new name:

    POST _snapshot/<repository-name>/<snapshot-name>/_restore
{
  "indices": "*",
  "ignore_unavailable": true,
  "rename_pattern": ".opensearch",
  "rename_replacement": ".backup-opensearch"
}

    Then you can reindex .backup-opensearch on the new domain and alias it to .opensearch. Note that the _restore REST call doesn't include include_global_state because the default in _restore is false. As a result, the test domain won't include any index templates and won't have the full state from the backup.

  5. If you no longer need your original domain, delete it. Otherwise, you continue to incur charges for the domain.
OpenSearch 1.x OpenSearch 1.x

  1. Create a manual snapshot of the 1.x domain.

  2. Create a 1.x domain that's a higher version than your original 1.x domain.

  3. Restore the snapshot from the original domain to the new 1.x domain. During the operation, you might need to restore your .opensearch index under a new name:

    POST _snapshot/<repository-name>/<snapshot-name>/_restore
{
  "indices": "*",
  "ignore_unavailable": true,
  "rename_pattern": ".opensearch",
  "rename_replacement": ".backup-opensearch"
}

    Then you can reindex .backup-opensearch on the new domain and alias it to .opensearch. Note that the _restore REST call doesn't include include_global_state because the default in _restore is false. As a result, the test domain won't include any index templates and won't have the full state from the backup.

  4. If you no longer need your original domain, delete it. Otherwise, you continue to incur charges for the domain.
Elasticsearch 6.x or 7.x OpenSearch 1.x

  1. Review breaking changes for OpenSearch 1.0 to see if you need to make adjustments to your indexes or applications.

  2. Create a manual snapshot of the Elasticsearch 7.x or 6.x domain.

  3. Create an OpenSearch 1.x domain.

  4. Restore the snapshot from the Elasticsearch domain to the OpenSearch domain. During the operation, you might need to restore your .elasticsearch index under a new name:

    POST _snapshot/<repository-name>/<snapshot-name>/_restore
{
  "indices": "*",
  "ignore_unavailable": true,
  "rename_pattern": ".elasticsearch",
  "rename_replacement": ".backup-opensearch"
}

    Then you can reindex .backup-opensearch on the new domain and alias it to .elasticsearch. Note that the _restore REST call doesn't include include_global_state because the default in _restore is false. As a result, the test domain won't include any index templates and won't have the full state from the backup.

  5. If you no longer need your original domain, delete it. Otherwise, you continue to incur charges for the domain.
Elasticsearch 6.x Elasticsearch 7.x

  1. Review breaking changes for 7.0 to see if you need to make adjustments to your indexes or applications.

  2. Create a manual snapshot of the 6.x domain.

  3. Create a 7.x domain.

  4. Restore the snapshot from the original domain to the 7.x domain. During the operation, you likely need to restore the .opensearch index under a new name:

    POST _snapshot/<repository-name>/<snapshot-name>/_restore
{
  "indices": "*",
  "ignore_unavailable": true,
  "rename_pattern": ".elasticsearch",
  "rename_replacement": ".backup-elasticsearch"
}

    Then you can reindex .backup-elasticsearch on the new domain and alias it to .elasticsearch. Note that the _restore REST call doesn't include include_global_state because the default in _restore is false. As a result, the test domain won't include any index templates and won't have the full state from the backup.

  5. If you no longer need your original domain, delete it. Otherwise, you continue to incur charges for the domain.
Elasticsearch 6.x Elasticsearch 6.8

  1. Create a manual snapshot of the 6.x domain.

  2. Create a 6.8 domain.

  3. Restore the snapshot from the original domain to the 6.8 domain.

  4. If you no longer need your original domain, delete it. Otherwise, you continue to incur charges for the domain.
Elasticsearch 5.x Elasticsearch 6.x

  1. Review breaking changes for 6.0 to see if you need to make adjustments to your indices or applications.

  2. Create a manual snapshot of the 5.x domain.

  3. Create a 6.x domain.

  4. Restore the snapshot from the original domain to the 6.x domain.

  5. If you no longer need your 5.x domain, delete it. Otherwise, you continue to incur charges for the domain.
Elasticsearch 5.x Elasticsearch 5.6

  1. Create a manual snapshot of the 5.x domain.

  2. Create a 5.6 domain.

  3. Restore the snapshot from the original domain to the 5.6 domain.

  4. If you no longer need your original domain, delete it. Otherwise, you continue to incur charges for the domain.
Elasticsearch 2.3 Elasticsearch 6.x

Elasticsearch 2.3 snapshots are not compatible with 6.x. To migrate your data directly from 2.3 to 6.x, you must manually recreate your indexes in the new domain.

Alternately, you can follow the 2.3 to 5.x steps in this table, perform _reindex operations in the new 5.x domain to convert your 2.3 indexes to 5.x indexes, and then follow the 5.x to 6.x steps.
Elasticsearch 2.3 Elasticsearch 5.x

  1. Review breaking changes for 5.0 to see if you need to make adjustments to your indexes or applications.

  2. Create a manual snapshot of the 2.3 domain.

  3. Create a 5.x domain.

  4. Restore the snapshot from the 2.3 domain to the 5.x domain.

  5. If you no longer need your 2.3 domain, delete it. Otherwise, you continue to incur charges for the domain.
Elasticsearch 1.5 Elasticsearch 5.x

Elasticsearch 1.5 snapshots are not compatible with 5.x. To migrate your data from 1.5 to 5.x, you must manually recreate your indexes in the new domain.

Important

1.5 snapshots are compatible with 2.3, but OpenSearch Service 2.3 domains do not support the _reindex operation. Because you cannot reindex them, indexes that originated in a 1.5 domain still fail to restore from 2.3 snapshots to 5.x domains.
Elasticsearch 1.5 Elasticsearch 2.3

  1. Use the migration plugin to find out if you can directly upgrade to version 2.3. You might need to make changes to your data before migration.

    1. In a web browser, open http://domain-endpoint/_plugin/migration/.

    2. Choose Run checks now.

    3. Review the results and, if needed, follow the instructions to make changes to your data.

  2. Create a manual snapshot of the 1.5 domain.

  3. Create a 2.3 domain.

  4. Restore the snapshot from the 1.5 domain to the 2.3 domain.

  5. If you no longer need your 1.5 domain, delete it. Otherwise, you continue to incur charges for the domain.