Migrate workloads to the VMware Cloud on AWS by using VMware HCX - AWS Prescriptive Guidance
SummaryPrerequisites and limitationsArchitectureToolsEpicsTroubleshootingRelated resources

Migrate workloads to the VMware Cloud on AWS by using VMware HCX

Created by Deepak Kumar (AWS), Derek Cox (AWS), and Himanshu Gupta (AWS)

Environment: Production

Source: On-premises VMware workloads

Target: VMware Cloud on AWS

R Type: Relocate

Workload: All other workloads

Technologies: Migration; Hybrid cloud

AWS services: VMware Cloud on AWS; Amazon VPC

Summary

Notice: As of April 30, 2024, VMware Cloud on AWS is no longer resold by AWS or its channel partners. The service will continue to be available through Broadcom. We encourage you to reach out to your AWS representative for details.

This pattern explains how you can use VMware Hybrid Cloud Extension (HCX) to migrate workloads from your on-premises VMware environment to VMware Cloud on AWS without changing the underlying platform. VMware HCX streamlines migration, helps rebalance workloads, helps protect data, and optimizes disaster recovery processes for both on-premises data centers and cloud servers. The pattern discusses the steps for installing, configuring, upgrading, and uninstalling HCX.

HCX supports the following:

  • Older versions of VMware vSphere – HCX helps you migrate virtual machines (VMs) from older versions of vSphere to VMware Cloud on AWS. The hosts are automatically updated and repaired to eliminate time-consuming updates in preparation for migration.

  • Bulk migrations – You can use HCX with a WAN optimization service to migrate a large number of VMs in one step without downtime, to expand your on-premises networks to the cloud.

  • Heterogeneous network environments – Your current network (such as vSphere, NSX, VXLAN, or NSX-T) determines the complexity of your migration. HCX extracts the fundamentals of your network application and extends your current network to the cloud without requiring any complicated procedures.

  • Slow network speeds – Migrations generally require connection speeds above 250 Mbps. HCX can migrate your workloads at much lower speeds, around 100 Mbps.

HCX supports three types of cloud migrations:

  • Hybridity (data center extension) – Extending an existing, on-premises VMware software-defined data center (SDDC) to AWS to provide footprint expansion, on-demand capacity, a testing/development environment, and virtual desktops.

  • Cloud evacuation (data center-wide infrastructure refresh) – Consolidating data centers and moving completely to the AWS Cloud (including handling data center co-location or end of lease).

  • Application-specific migration – Moving individual applications to the AWS Cloud to meet specific business needs. 

You can use HCX to migrate workloads bidirectionally between your on-premises environment and VMware Cloud on AWS. HCX offers multiple ways to migrate your workloads between source and target locations:

  • HCX cold migration migrates VMs that are offline. This method is suitable for VMs that are powered off because it requires significant downtime.

  • HCX vMotion uses the VMware vMotion protocol to move VMs. HCX vMotion offers zero downtime migration but can migrate only one VM at a time.

  • HCX Bulk Migration uses VMware vSphere replication protocols to move VMs to the destination. You can migrate multiple VMs in parallel and schedule a switchover. The downtime is equivalent to a server reboot, and switchover for all VMs happen in parallel.

  • HCX Replication Assisted vMotion (RAV) is a combination of HCX bulk migration and HCX vMotion. It provides parallel migrations, scheduling, and zero downtime.

  • HCX OS Assisted Migration helps you migrate multiple VMs in bulk when you’re using multiple hypervisors and non-vSphere VMs on premises. HCX OS Assisted Migration is free when you use it to migrate from on premises to VMware Cloud on AWS, but requires additional licenses when you want to migrate between two on-premises environments or from on premises to other cloud providers.

Prerequisites and limitations

Prerequisites

  • A VMware account for access to the VMware console at vmware.com.

    • The following firewall ports are required for HCX.

      Source

      Destination

      Port

      HCX Manager and appliances IP on premises

      HCX Manager and appliances IP on VMware Cloud on AWS

      UDP 500, UDP 4500, and ICMP

      HCX Manager and appliances IP on premises

      connect.hcx.vmware.com | hybridity-depot.vmware.com

      TCP 443

      HCX Manager and appliances IP on premises

      HCX cloud URL

      TCP 443

      If the on-premises network has internal firewalls, you will have to allow a few more ports locally within the data center. For a full list of port requirements for HCX, see the VMware HCX documentation.

  • To configure HCX, you need the Domain Name System (DNS) IP, the vCenter fully qualified domain name (FQDN), the NTP server FQDN, the single sign-on (SSO) user, and similar information. Gather these details in advance to avoid any delays in the deployment.

Limitations

You can use the Network Extension appliance to extend a maximum of eight networks between the on-premises environment and VMware Cloud on AWS. For a full list of HCX service limits, see the VMware HCX documentation.

Architecture

Source technology stack

  • On-premises VMware workloads

Target technology stack

  • VMware Cloud on AWS

Tools

Tools

  • VMware Cloud on AWS is a service jointly designed by AWS and VMware to help you migrate and extend your on-premises VMware vSphere-based environments to the AWS Cloud.

  • VMware Hybrid Cloud Extension (HCX) is a VMware utility for migrating workloads from your on-premises VMware environment to VMware Cloud on AWS without changing the underlying platform.

Epics

TaskDescriptionSkills required

Enable HCX service in VMware Cloud on AWS

  1. Log in to the VMware Cloud on AWS console.

  2. Navigate to your SDCC and choose View details.

  3. Choose the Add Ons tab.

  4. Choose Open HCX.

  5. Choose Deploy HCX and confirm. HCX deployment will begin.

Cloud administrator, Systems administrator

Generate the HCX activation key.

  1. On the VMware Cloud on AWS console.

  2. Navigate to your SDCC and choose View details.

  3. Choose the Add Ons tab.

  4. Choose Open HCX, and then choose Activation keys.

  5. Choose Create activation key and copy the key.

Cloud administrator, Systems administrator

Add firewall rules for HCX on cloud SDDC.

After the HCX Manager is deployed, you need to configure firewall rules to enable communications between the on-premises environment and the SDDC. You need to create two firewall rules: one for inbound and the other for outbound communications.

  1. On the VMware Cloud on AWS console, select your SDDC and navigate to Networking & Security.

  2. Choose Gateway Firewall, and then choose the Management Gateway tab.

  3. Choose Add rule and create an outbound rule:

    1. Provide the rule name.

    2. Edit the source and select HCX.

    3. Edit the destination and provide the on-premises IP and subnet where HCX can be accessed.

    4. For Services, choose Any.

    5. For Action, choose Allow.

    6. Choose Publish.

  4. Choose Add rule and create an inbound rule:

    1. Provide the rule name.

    2. Edit the source and provide the on-premises IP and subnet where HCX can be accessed.

    3. Edit the destination and select HCX.

    4. For Services, choose SSH, HTTPS, TCP (9443), and ICMP.

    5. For Action, choose Allow.

    6. Choose Publish.

Cloud administrator, Systems administrator

Install HCX Manager on premises.

  1. Log in to the cloud vCenter and navigate to HCX from the menu.

  2. On the HCX dashboard, choose Administration, System Updates.

  3. Request the download link for VMware HCX Connector, and download the on-premises OVA file.

  4. Log in to your on-premises vCenter and deploy the OVF template by using the downloaded OVA file.

  5. During template deployment, provide static IP, NTP, DNS, DNS search list, and other details when prompted.

  6. Verify all details to finish HCX Manager deployment.

Cloud administrator, Systems administrator

Configure HCX Manager on premises.

  1. Open HCX Manager in a browser: https://<HCX_Manager_IP>:9433.

  2. Log in by using the username and password provided during the deployment.

  3. Enter the activation key you created previously, and choose Activate to activate your HCX instance.

  4. Choose Confirm to go to the next step.

  5. Select the location of your on-premises data center, and then choose Continue.

  6. For System Name, enter the hostname, and then choose Continue to complete the activation.

  7. Enter the information to configure your vCenter connection.

  8. Enter the information to configure SSO/PSC details.

  9. Choose Restart for your changes to take effect.

Cloud administrator, Systems administrator

Configure site pairing.

After you have configured HCX in the cloud and on premises, follow these steps to configure site pairing between them.

  1. Log in to your on-premises vCenter, and navigate to the HCX dashboard.

  2. In the left navigation pane, choose Site pairing, and then choose Connect to Remote Site.

  3. In the Connect to Remote Site dialog box, add the HCX cloud URL and credentials, and then choose Connect.

When site pairing is complete, the site pairing dashboard shows the on-premises and cloud SDDC connected.

Cloud administrator, Systems administrator

Create a network profile.

A network profile is an abstraction of the Layer 3 components of a network. This profile is a prerequisite for creating a compute profile.

  1. Log in to your cloud vCenter, and navigate to the HCX dashboard.

  2. Choose Interconnect, choose the Network Profiles tab, and then choose Create network profile.

  3. Configure the network profile:

    1. Choose the vCenter server.

    2. Choose the network.

    3. Add a name for the profile.

    4. Provide the IP pool, prefix length, gateway, DND, and MTU.

    5. Choose Create.

  4. Follow the same process to create a network profile on premises.

Cloud administrator, Systems administrator

Create a compute profile.

The compute profile consists of network, storage and compute details for HCX.  HCX uses these settings when it creates HCX appliances during the creation of the service mesh.

  1. Log in to your on-premises vCenter, and navigate to the HCX dashboard.

  2. Choose Interconnect, choose the Compute Profiles tab, and then choose Create compute profile.

  3. Specify a name for the compute profile.

  4. Select the HCX services that you want to enable, and then choose Continue.

  5. Select the service resources. If there are multiple clusters, select each cluster that you want HCX services to be activated for, and then choose Continue.

  6. Select compute and storage resources for deploying HCX appliances, and then choose Continue.

  7. Select a management network profile that can be used to reach the management interface of vCenter and ESXi hosts, and then choose Continue.

  8. Select an uplink network profile that can be used to reach interconnect appliances on the remote site and that remote site appliances can use to reach the local interconnect appliances, and then choose Continue.

  9. Select the vMotion network profile, and then choose Continue.

  10. Select the vSphere replication network profile, and then choose Continue.

  11. Select the appropriate distributed switch for network extensions, and then choose Continue.

  12. Review all the ports that need to be opened in WAN and LAN connections, and then choose Continue.

  13. To create the compute profile, choose Finish.

  14. Follow the same steps to create a compute profile on the cloud site.

Cloud administrator, Systems administrator

Create a service mesh.

The service mesh provides HCX service configuration for both the on-premises site and the cloud site. Creating a service mesh initiates the deployment of HCX interconnect virtual appliances on both sites. The interconnect service must be created on the source site.

  1. Log in to your on-premises vCenter, and navigate to the HCX dashboard.

  2. Choose Interconnect, choose the Service Mesh tab, and then choose Create service mesh.

  3. Select the source and destination site the service mesh will be created between, and then choose Continue.

  4. Select the compute profile for the source and the destination sites that you created earlier, and then choose Continue.

  5. Select the HCX service that you want to enable, and then choose Continue.

  6. Select the uplink profile for both source and target sites, and then choose Continue.

  7. Review the resources and networks, and then choose Continue.

  8. Provide a name for the service mesh, and then choose Finish.

Service mesh deployment will start. You can follow the progress in the Tasks tab for the service mesh. When deployment is complete, the status of all HCX services that you enabled for the service mesh is displayed.

Cloud administrator, Systems administrator
TaskDescriptionSkills required

Create a network extension.

You can use HCX network extension capabilities to create a L2 network extension at the cloud SDDC HCX site and bridge the remote and source networks.

This allows you to migrate servers from on-premises to VMware Cloud on AWS while retaining the same IP addresses.

  1. Log in to your on-premises vCenter, and navigate to the HCX dashboard.

  2. Choose Services, Network Extension.

  3. Choose Extend networks or Create a network extension.

  4. Select the appropriate service mesh, distributed port group, or NSX logical switch.

  5. Provide the gateway IP address and then choose Submit.

When the network extension is complete, the system shows Extension complete.

Cloud administrator, Systems administrator
TaskDescriptionSkills required

Configure replication.

To replicate VMs by using HCX:

  1. Log in to your on-premises vCenter, and navigate to the HCX dashboard.

  2. Choose Migration, and then choose the Migrate tab.

  3. Provide a mobility group name, select the VM that you want to migrate, and then choose Add.

  4. Choose the target compute container, storage folder, migration type (cold, bulk, RAV, vMotion), and switchover schedule.

  5. Choose Validate, wait for validation to complete, and then choose Go to start the replication.

Cloud administrator, Systems administrator
TaskDescriptionSkills required

Review recommendations and steps.

A large migration project can last from six to eight months, sometimes longer, and VMware periodically publishes HCX updates that consist of software fixes, security updates, and bug fixes. We recommend that you keep HCX and your appliances up to date to eliminate any security vulnerabilities and to take advantage of new functionality.

Note: If your current HCX version is three versions behind the latest release or older, you cannot upgrade HCX and will have to redeploy it.

An HCX upgrade consists of three steps:

  1. Back up HCX Manager on premises and in the cloud.

  2. Upgrade HCX Manager on premises and in the cloud.

  3. Upgrade service mesh appliances on on premises and in the cloud.

The following stories discuss these steps in more detail.

Cloud administrator, Systems administrator

Back up HCX Cloud Manager.

HCX Cloud Manager for VMware Cloud on AWS is managed by VMware, so you cannot take snapshots. To back up HCX Cloud Manager, you must download a backup from the HCX console and use this backup to restore the HCX configuration in case the upgrade fails or you have to roll back to a previous stage.

  1. Log in to HCX Cloud Manager at https://<HCX_cloudmanager_ip_or_fqdn>:9433.

  2. Navigate to Administration, Troubleshooting, Backup & Restore.

  3. In the Backup section, choose Generate to create a backup file.

  4. Choose Download to save the backup file.

HCX service appliances such as HCX-IX, HCX-NE, and HCX-WO do not require individual backups.

Cloud administrator, Systems administrator

Back up HCX Manager on premises.

You can back up HCX Manager on-premises in two ways: by taking a VM snapshot or by backing up the configuration file.

To take a VM snapshot:

  1. Log in to your on-premises vCenter.

  2. Go to VM and templates, and navigate to HCX manager VM.

  3. Choose Actions, Snapshots, Take Snapshot.

To back up the configuration file:

  1. Log in to HCX Cloud Manager at https://<HCX_cloudmanager_ip_or_fqdn>:9433.

  2. Navigate to Administration, Troubleshooting, Backup & Restore.

  3. In the Backup section, choose Generate to create a backup file.

  4. Choose Download to save the backup file.

HCX service appliances such as HCX-IX, HCX-NE, and HCX-WO do not require individual backups.

Cloud administrator, Systems administrator

Upgrade HCX Manager on premises and in the cloud.

You must upgrade HCX Manager on-premises first, and then upgrade HCX Cloud Manager.

To upgrade HCX Manager on premises:

  1. Log in to vCenter and navigate to the HCX dashboard.

  2. Choose System, Administration.

  3. On the Administration page, choose the System Updates tab. The Available Service Update Versions column shows pending updates.

  4. Choose Select Service Update, Download to download the update for a later upgrade, or choose Download & Upgrade to download and deploy the update immediately. If you selected Download, choose Upgrade and confirm to initiate the upgrade when you’re ready.

  5. When the upgrade is complete:

    • On the HCX manager Administration page, validate that the latest HCX version is displayed.

    • On the HCX dashboard, check to confirm that site pairing is Up.

    • Choose Infrastructure, Service Mesh, and confirm that all HCX services are healthy.

Follow the same steps to upgrade HCX Cloud Manager.

Cloud administrator, Systems administrator

Upgrade service mesh appliances.

The service mesh is updated independently of HCX Manager at the source site. Service mesh appliances on the target site are updated automatically.

To upgrade service mesh appliances at the source site:

  1. Log in to vCenter, and navigate to the HCX dashboard.

  2. Choose Infrastructure, and then choose the Service mesh tab.

  3. If you see the banner "New version for service mesh appliances is available. Click on Update Appliances to upgrade to latest," choose Update appliances.

  4. In the dialog box that displays appliances, choose one or more appliances, and then choose OK to start the upgrade process. (We recommend that you update all service mesh appliances.)

  5. Choose View tasks for each service mesh to monitor the upgrade.

  6. When the upgrade is complete, a banner appears for each appliance and service to confirm successful completion.

  7. Validate the tunnel status after the upgrade:

    • Choose Infrastructure, Service mesh, View appliance.

    • The tunnel status column should show Up and the screen shouldn’t indicate any other available versions for the appliance.

Cloud administrator, Systems administrator
TaskDescriptionSkills required

Unextend network.

An earlier step explained how to use HCX network extension capabilities to create L2 network extensions and to keep existing IPs during migration from on premises to VMware cloud on AWS. When all the VMs from a particular VLAN have been moved to VMware Cloud on AWS, you must unextend the network between the on-premises site and the cloud SDDC, and make the network routable in the SDDC.

We recommend that you remove the extended network as soon as all VMs are migrated from on premises to VMware Cloud on AWS to avoid latency.

  1. Log in to your on-premises vCenter, and navigate to the HCX dashboard.

  2. On the HCX dashboard, choose Services, Network Extension.

  3. Select the network you want to unextend, and then choose Unextend network.

  4. Select Connect cloud network to cloud edge gateway after unextending. This activates the network on the cloud side.

Cloud administrator, Systems administrator

Route moved network in cloud SDDC.

  1. Log in to the VMC portal.

  2. Navigate to the SDCC, and then choose View details.

  3. Choose the Networking & Security tab.

  4. On the Networking & Security page:

    • Choose Network, Segments, and confirm that the recently unextended subnet is shown as routable.

    • Choose Inventory, Groups, and add that subnet to a group.

    • Choose Security, Distributed firewall, and confirm that the group is part of the intended firewall rule.

Cloud administrator, Systems administrator
TaskDescriptionSkills required

Check prerequisites.

In the case of a data center exit, we recommend that you uninstall HCX and remove its components at the end of your migration project. However, if you still retain an on-premises footprint, you might want to keep HCX running.

Before you uninstall HCX, make sure that:

  • There are no active migrations.

  • All network extensions have been removed.

Cloud administrator, Systems administrator

Uninstall HCX on premises.

  1. Log in to your on-premises vCenter and navigate to the HCX console.

  2. Choose Services, Migration, and confirm that you have no active migrations.

  3. Choose Services, Network extension, and confirm that there is no extended network.

  4. Choose Infrastructure, Site pairing, Service mesh.

  5. Identify the service mesh, and then choose Delete.

  6. In the confirmation prompt, choose Delete again. The banner “Removing Service Mesh” appears on the service mesh screen.

  7. Repeat steps 5-6 for any other service meshes you have.

  8. To remove site pairing, choose Infrastructure, Site pairing, and then disconnect all paired sites.

  9. Remove the HCX manager appliance:

    1. Log in to your on-premises vCenter and navigate to the HCX Manager appliance.

    2. Choose Actions, Power, Power Off.

    3. Choose Actions, Delete from Disk.

Cloud administrator, Systems administrator

Deregister HCX plugin from on-premises vCenter server.

  1. Log in to the vCenter MOB UI at https://<vc_fqdn>/mob.

  2. In the Properties section, choose the content in the Value column.

  3. On the content page, choose ExtensionManager to see all registered plugins.

  4. Note the extensions that start with com.vmware.hybridity, com.vmware.hcsp.alarm, and com.vmware.vca.marketing.ngc.ui.

  5. Remove the extensions:

    • In the Methods section, choose UnregisterExtension.

    • Enter the extension key noted in step 4, and then choose Invoke Method to remove the extension.

When all extensions have been removed, the HCX plugin will disappear from the vSphere Web Client.

Cloud administrator, Systems administrator

Uninstall HCX in the cloud.

To remove the HCX service mesh and site pairing in the cloud, repeat the steps described earlier in Uninstall HCX on premises. In VMware Cloud on AWS, HCX Manager is managed by VMware. You cannot delete it from vCenter, but you can undeploy it from the VMC management interface.

To undeploy HCX Manager:

  1. Log in to the VMC management interface.

  2. Choose your organization and SDDC.

  3. Choose Add Ons to display all SDDCs that have HCX deployed.

  4. Choose Undeploy HCX.

Cloud administrator, Systems administrator

Troubleshooting

IssueSolution

You’re unable to select the servers to migrate when you configure HCX bulk migration.

Cause: Migration for these servers were canceled, but the HCX database wasn’t updated during the cleanup. HCX views database migration as still being in progress, so it has locked the status at "Switchover in-progress."

Solution: Reach out to the VMware support team to clean up the HCX database.

Switchover fails but works with the Force Power Off option.

Cause: The version of VMware Tools didn’t meet the prerequisites for HCX bulk migration, so HCX could not shut down the source VM.

Solution: Update the VMware tool to the recommended version for your migration type.

HCX site pairing appliance upgrade fails with the error "Operation not allowed for ongoing bulk migration" while migration is in progress.

Cause: The HCX database didn't update after the switchover.

Solution: Make sure there are no ongoing migrations. Choose Force upgrade when you upgrade the site pairing appliance.

Cutover fails with error "Low resource availability."

Cause: Low storage on the host VM.

Solution: Check storage and compute resources before migration.

Related resources

References

Tools

Partners

Videos