You can use AWS DataSync to transfer data between FSx for Lustre file systems. DataSync is a data transfer service that simplifies, automates, and accelerates moving and replicating data between self-managed storage systems and AWS storage services over the internet or AWS Direct Connect. DataSync can transfer your file system data and metadata, such as ownership, timestamps, and access permissions.
How to migrate existing files to FSx for Lustre using
AWS DataSync
You can use DataSync with FSx for Lustre file systems to perform one-time data migrations, periodically ingest data for distributed workloads, and schedule replication for data protection and recovery. For information about specific transfer scenarios, see Where can I transfer my data with AWS DataSync? in the AWS DataSync User Guide.
Prerequisites
To migrate data into your FSx for Lustre setup, you need a server and network that meet the DataSync requirements. To learn more, see Setting up with AWS DataSync in the AWS DataSync User Guide.
You have created a destination FSx for Lustre file system. For more information, see Step 1: Create your FSx for Lustre file system.
The source and destination file systems are connected in the same virtual private cloud (VPC). The source file system can be located on-premises or in another Amazon VPC, AWS account, or AWS Region, but it must be in a network peered with that of the destination file system using Amazon VPC Peering, Transit Gateway, AWS Direct Connect, or AWS VPN. For more information, see What is VPC peering? in the Amazon VPC Peering Guide.
Note
DataSync can only transfer across AWS accounts to or from FSx for Lustre if the other transfer location is Amazon S3.
Basic steps for migrating files using DataSync
Transferring files from a source to a destination using DataSync involves the following basic steps:
-
Download and deploy an agent in your environment and activate it (not required if transferring between AWS services).
-
Create a source and destination location.
-
Create a task.
-
Run the task to transfer files from the source to the destination.
For more information, see the following topics in the AWS DataSync User Guide:
