Deployment options for FSx for Lustre file systems

Amazon FSx for Lustre provides two file system deployment options: persistent and scratch.

You choose the file system deployment type when you create a new file system, using the AWS Management Console, the AWS Command Line Interface (AWS CLI), or the Amazon FSx for Lustre API. For more information, see Step 1: Create your FSx for Lustre file system and CreateFileSystem in the Amazon FSx API Reference.

Encryption of data at rest is automatically enabled when you create an Amazon FSx for Lustre file system, regardless of the deployment type you use. Scratch 2 and persistent file systems automatically encrypt data in transit when they are accessed from Amazon EC2 instances that support encryption in transit. For more information on encryption, see Data encryption in Amazon FSx for Lustre.

Persistent file systems

Persistent file systems are designed for longer-term storage and workloads. The file servers are highly available, and data is automatically replicated within the same Availability Zone in which the file system is located. The data volumes attached to the file servers are replicated independently from the file servers to which they are attached.

Amazon FSx continuously monitors persistent file systems for hardware failures, and automatically replaces infrastructure components in the event of a failure. On a persistent file system, if a file server becomes unavailable, it's replaced automatically within minutes of failure. During that time, client requests for data on that server transparently retry and eventually succeed after the file server is replaced. Data on persistent file systems is replicated on disks, and any failed disks are automatically replaced transparently.

Use persistent file systems for longer-term storage and for throughput-focused workloads that run for extended periods or indefinitely, and that might be sensitive to disruptions in availability.

Persistent deployment types automatically encrypt data in transit when they are accessed from Amazon EC2 instances that support encryption in transit.

Amazon FSx for Lustre supports two persistent deployment types: Persistent 1 and Persistent 2.

Persistent 2 deployment type

Persistent 2 is the latest generation of Persistent deployment type, and is best-suited for use cases that require longer-term storage, and have latency-sensitive workloads that require the highest levels of IOPS and throughput. Persistent 2 deployment types support higher levels of throughput per unit storage (that is, 125, 250, 500, and 1000 MB/s/TiB), higher metadata IOPS (if you specify a metadata configuration), and higher throughput per client (if you enable EFA support), compared to Persistent 1 file systems.

You can create Persistent 2 file systems with a metadata configuration and EFA enabled using the Amazon FSx console, AWS Command Line Interface, and API.

Persistent 1 deployment type

The Persistent 1 deployment type is well-suited for use cases that require longer-term storage, and have throughput-focused workloads that aren't latency-sensitive. Persistent 1 deployment types support SSD (solid state drive) and HDD (hard disk drive) storage options.

For a Persistent 1 file system with SSD storage, the throughput per unit of storage is either 50, 100, or 200 MB/s per tebibyte (TiB). For HDD storage, Persistent 1 throughput per unit of storage is 12 or 40 MB/s per TiB.

You can create Persistent 1 deployment types only by using the AWS CLI and the Amazon FSx API.

Scratch file systems

Scratch file systems are designed for temporary storage and shorter-term processing of data. Data isn't replicated and doesn't persist if a file server fails. Scratch file systems provide high burst throughput of up to six times the baseline throughput of 200 MBps per TiB of storage capacity. For more information, see Aggregate file system performance.

Use scratch file systems when you need cost-optimized storage for short-term, processing-heavy workloads.

On a scratch file system, file servers aren't replaced if they fail and data isn't replicated. If a file server or a storage disk becomes unavailable on a scratch file system, files stored on other servers are still accessible. If clients try to access data that is on the unavailable server or disk, clients experience an immediate I/O error.

The following table illustrates the availability or durability that scratch file systems of example sizes are designed for, over the course of a day and a week. Because larger file systems have more file servers and more disks, the probabilities of failure are increased.

Deployment type availability

Scratch 2, Persistent 1, and Persistent 2 deployment types are available in the following AWS Regions: