Data repository event logs - FSx for Lustre

Data repository event logs

You can turn on logging to CloudWatch Logs to log information about any failures experienced while importing or exporting files using automatic import, automatic export, and data repository tasks. For more information, see Logging with Amazon CloudWatch Logs.

Note

When a data repository task fails, Amazon FSx also writes failure information to the task completion report. For more information about failure information in completion reports, see Troubleshooting data repository task failures.

Automatic import, automatic export, and data repository tasks can fail for several reasons, including those listed below. For information on viewing these logs, see Viewing logs.

Import events
Error code Log level Log message Root cause Error code in completion report
S3ImportListObjectError ERROR Failed to list S3 objects in S3 bucket bucket_name with prefix prefix. Amazon FSx failed to list S3 objects in the S3 bucket. This can happen if the S3 bucket policy does not provide sufficient permissions to Amazon FSx. N/A
S3ImportUnsupportedTierWarning WARN Failed to import S3 object with key key_value in S3 bucket bucket_name due to an S3 object in an unsupported tier S3_tier_name.

Amazon FSx was unable to import an S3 object because it's in an Amazon S3 storage class that is not supported, such as S3 Glacier Flexible Retrieval or S3 Glacier Deep Archive storage class.

 S3ObjectInUnsupportedTier
S3ImportSymlinkInUnsupportedTierWarning WARN Failed to import S3 object with key key_value in S3 bucket bucket_name due to an S3 symlink object in an unsupported tier S3_tier_name.

Amazon FSx was unable to import a symlink object because it's in an Amazon S3 storage class that is not supported, such as S3 Glacier Flexible Retrieval or S3 Glacier Deep Archive storage class.

 S3SymlinkInUnsupportedTier
S3ImportAccessDenied ERROR Failed to import S3 object with key key_value in S3 bucket bucket_name because access to the S3 object was denied.

Access was denied to Amazon S3 for a data repository export import task.

For import tasks, the Amazon FSx file system must have permission to perform the s3:HeadObject and s3:GetObject operations to import from a linked data repository on S3.

For import tasks, if your S3 bucket uses server-side encryption with customer managed keys stored in AWS Key Management Service (SSE-KMS), you must follow the policy configurations in Working with server-side encrypted Amazon S3 buckets.

 S3AccessDenied
S3ImportDeleteAccessDenied ERROR Failed to delete local file for S3 object with key key_value in S3 bucket bucket_name because access to the S3 object was denied. Automatic import was denied access to an S3 object. N/A
S3ImportObjectPathNotPosixCompliant ERROR Failed to import S3 object with key key_value in S3 bucket bucket_name because S3 object is not POSIX compliant.

The Amazon S3 object exists but can't be imported because it isn't a POSIX-compliant object. For information about supported POSIX metadata, see POSIX metadata support for data repositories.

 S3ObjectPathNotPosixCompliant
S3ImportObjectTypeMismatch ERROR Failed to import S3 object with key key_value in S3 bucket bucket_name because an S3 object with the same name has already been imported into the file system. The S3 object being imported is of a different type (file or directory) than an existing object with the same name in the file system. S3ObjectTypeMismatch
S3ImportDirectoryMetadataUpdateError ERROR Failed to update local directory metadata due to an internal error. Directory metadata could not be imported due to an internal error. N/A
S3ImportObjectDeleted ERROR Failed to import S3 object with key key_value because it was not found in S3 bucket bucket_name.

Amazon FSx was unable to import file metadata because the corresponding object doesn't exist in the data repository.

 S3FileDeleted
S3ImportBucketDoesNotExist ERROR Failed to import S3 object with key key_value in S3 bucket bucket_name due to bucket does not exist. Amazon FSx cannot automatically import an S3 object to the file system because the S3 bucket no longer exists. N/A
S3ImportDeleteBucketDoesNotExist ERROR Failed to delete local file for S3 object with key key_value in S3 bucket bucket_name due to bucket does not exist. Amazon FSx cannot delete a file linked to an S3 object on the file system because the S3 bucket no longer exists. N/A
S3ImportDirectoryCreateError ERROR Failed to create local directory due to an internal error. Amazon FSx failed to automatically import a directory creation on the file system due to an internal error. N/A
NoDiskSpace ERROR Failed to import S3 object with key key_value in S3 bucket bucket_name because the file system is full. File system ran out of disk space on the metadata server(s) while creating file or directory. N/A
Export events
Error code Log level Log message Root cause Error code in completion report
S3ExportInternalError ERROR Failed to export S3 object with key key_value in S3 bucket bucket_name due to an internal error. Object was not exported due to an internal error. INTERNAL_ERROR
S3ExportAccessDenied ERROR Failed to export file because access was denied to S3 object with key key_value in S3 bucket bucket_name.

Access was denied to Amazon S3 for a data repository export task.

For export tasks, the Amazon FSx file system must have permission to perform the s3:PutObject operation to export to a linked data repository on S3. This permission is granted in the AWSServiceRoleForFSxS3Access_fs-0123456789abcdef0 service-linked role. For more information, see Using service-linked roles for Amazon FSx.

Because the export task requires data to flow outside a file system's VPC, this error can occur if the target repository has a bucket policy that contains one of the aws:SourceVpc or aws:SourceVpce IAM global condition keys.

If your S3 bucket contains objects uploaded from a different AWS account than your file system linked S3 bucket account, you can ensure that your data repository tasks can modify S3 metadata or overwrite S3 objects regardless of which account uploaded them. We recommend that you enable the S3 Object Ownership feature for your S3 bucket. This feature enables you to take ownership of new objects that other AWS accounts upload to your bucket, by forcing uploads to provide the --acl bucket-owner-full-control canned ACL. You enable S3 Object Ownership by choosing the Bucket owner preferred option in your S3 bucket. For more information, see Controlling ownership of uploaded objects using S3 Object Ownership in the Amazon S3 User Guide.

 S3AccessDenied
S3ExportPathSizeTooLong ERROR Failed to export file because the local file path size exceeds the maximum object key length supported by S3.

The export path is too long. The maximum object key length supported by S3 is 1,024 characters.

 PathSizeTooLong
S3ExportFileSizeTooLarge ERROR Failed to export file because the file size exceeds the maximum supported S3 objects size. The maximum object size supported by Amazon S3 is 5 TiB. FileSizeTooLarge
S3ExportKMSKeyNotFound ERROR Failed to export file for S3 object with key key_value in S3 bucket bucket_name because the bucket's KMS key was not found. Amazon FSx was unable to export the file because the AWS KMS key couldn't be found. Be sure to use a key that's in the same AWS Region as the S3 bucket. For more information on creating KMS keys, see Creating keys in the AWS Key Management Service Developer Guide. N/A
S3ExportResourceBusy ERROR Failed to export file because it is being used by another process.

Amazon FSx was unable to export the file because it was being modified by another client on the file system. You can retry the task after your workflow has finished writing to the file.

 ResourceBusy
S3ExportLocalObjectReleaseWithoutS3Source WARN Export skipped: Local file is in released state and a linked S3 object with key key_value was not found in bucket bucket_name. Amazon FSx was unable to export the file because it was in a released state on the file system. N/A
S3ExportLocalObjectNotMatchDra WARN Export skipped: local file does not belong to a data repository linked file system path. Amazon FSx was unable to export because the object doesn't belong to a file system path that is linked to a data repository. N/A
InternalAutoExportError ERROR Automatic export encountered an internal error while exporting a file system object The export failed because of an internal (auto-export- or lustre-level) error. N/A
S3CompletionReportUploadFailure ERROR Failed to upload data repository task completion report into bucket_name Amazon FSx was unable to upload the completion report. N/A
S3CompletionReportValidateFailure ERROR Failed to upload data repository task completion report into bucket bucket_name because the completion report path report_path does not belong to a data repository associated with this file system Amazon FSx was unable to upload the completion report because the customer-provided S3 path does not belong to a linked data repository. N/A