Directory buckets overview
There are two types of Amazon S3 buckets, general purpose buckets and directory buckets. Choose the bucket type that best fits your application and performance requirements:
-
General purpose buckets are the original S3 bucket type and are recommended for most use cases and access patterns. General purpose buckets also allow objects that are stored across all storage classes, except S3 Express One Zone.
-
Directory buckets use the S3 Express One Zone storage class, which is recommended if your application is performance sensitive and benefits from single-digit millisecond
PUTandGETlatencies.
Directory buckets are used for workloads or performance-critical applications that require consistent single-digit millisecond latency. Directory buckets organize data hierarchically into directories as opposed to the flat storage structure of general purpose buckets. There aren't prefix limits for directory buckets, and individual directories can scale horizontally.
Directory buckets use the S3 Express One Zone storage class, which stores data across multiple devices within a single Availability Zone but doesn't store data redundantly across Availability Zones. When you create a directory bucket, we recommend that you specify an AWS Region and an Availability Zone that's local to your Amazon EC2, Amazon Elastic Kubernetes Service, or Amazon Elastic Container Service (Amazon ECS) compute instances to optimize performance.
You can create up to 10 directory buckets in each of your AWS accounts, with no limit on the number of objects that you can store in a bucket. Your bucket quota is applied to each Region in your AWS account. If your application requires increasing this limit, contact AWS Support.
Important
Directory buckets that have no request activity for a period of at least 90 days
transition to an inactive state. While in an inactive state, a directory bucket is
temporarily inaccessible for reads and writes. Inactive buckets retain all storage,
object metadata, and bucket metadata. Existing storage charges apply to inactive
buckets. If you make an access request to an inactive bucket, the bucket transitions to
an active state, typically within a few minutes. During this transition period, reads
and writes return an HTTP 503 (Service Unavailable) error code.
The following topics provide information about directory buckets. For more information about general purpose buckets, see Buckets overview.
Topics
- Availability Zones
- Directory bucket names
- Directories
- Key names
- Access management
- Working with directory buckets
- Directory bucket naming rules
- Creating a directory bucket
- Viewing directory bucket properties
- Managing bucket policies for directory buckets
- Emptying a directory bucket
- Deleting a directory bucket
- Listing directory buckets
- Using HeadBucket to determine whether you can access a directory bucket
Availability Zones
When you create a directory bucket, you choose the Availability Zone and AWS Region.
Directory buckets use the S3 Express One Zone storage class, which is built to be used by performance-sensitive applications. S3 Express One Zone is the first S3 storage class where you can select a single Availability Zone with the option to co-locate your object storage with your compute resources, which provides the highest possible access speed.
With
S3 Express One Zone, your data is redundantly stored on multiple devices within a single
Availability Zone. S3 Express One Zone is designed for 99.95 percent availability within a single Availability Zone and is backed
by the Amazon S3 Service Level Agreement
Directory bucket names
A directory bucket name consists of a base name that you provide and a suffix that contains the ID of the Availability Zone that your bucket is located in. Directory bucket names must use the following format and follow the naming rules for directory buckets:
bucket-base-name--azid--x-s3
For example, the following directory bucket name contains the Availability Zone ID usw2-az1:
bucket-base-name--usw2-az1--x-s3
For more information, see Directory bucket naming rules.
Directories
Directory buckets organize data hierarchically into directories as opposed to the flat sorting structure of general purpose buckets. Each S3 directory bucket can support hundreds of thousands of transactions per second (TPS), independent of the number of directories within the bucket.
With a hierarchical namespace, the delimiter in the object key is important. The only
supported delimiter is a forward slash (/). Directories are determined by
delimiter boundaries. For example, the object key
dir1/dir2/file1.txt results in the directories
dir1/ and dir2/ being automatically
created, and the object file1.txt being added to the
/dir2 directory in the path
dir1/dir2/file1.txt.
The directory bucket indexing model returns unsorted results for the
ListObjectsV2 API operation. If you need to limit your results to a
subsection of your bucket, you can specify a subdirectory path in the
prefix parameter, for example, prefix=dir1/.
Key names
For directory buckets, subdirectories that are common to multiple object keys are created with the first object key. Additional object keys for the same subdirectory use the previously created subdirectory. This model gives you flexibility in choosing object keys that are best suited to the application, with equal support for sparse and dense directories.
Access management
Directory buckets have all S3 Block Public Access settings enabled by default at the bucket level. S3 Object Ownership is set to bucket owner enforced and access control lists (ACLs) are disabled. These settings can't be modified.
By default, users don't have permissions for directory buckets and S3 Express One Zone operations. To grant access permissions for directory buckets, you can use IAM to create users, groups, or roles and attach permissions to those identities. For more information, see AWS Identity and Access Management (IAM) for S3 Express One Zone.
Working with directory buckets
For more information about working with directory buckets, see the following topics.
Topics
