Supported formats for Amazon S3 manifest files

You use JSON manifest files to specify files in Amazon S3 to import into Amazon QuickSight. These JSON manifest files can use either the Amazon QuickSight format described following or the Amazon Redshift format described in Using a manifest to specify data files in the Amazon Redshift Database Developer Guide. You don't have to use Amazon Redshift to use the Amazon Redshift manifest file format.

If you use an Amazon QuickSight manifest file, it must have a .json extension, for example my_manifest.json. If you use an Amazon Redshift manifest file, it can have any extension.

If you use an Amazon Redshift manifest file, Amazon QuickSight processes the optional mandatory option as Amazon Redshift does. If the associated file isn't found, Amazon QuickSight ends the import process and returns an error.

Files that you select for import must be delimited text (for example, .csv or .tsv), log (.clf), or extended log (.elf) format, or JSON (.json). All files identified in one manifest file must use the same file format. Plus, they must have the same number and type of columns. Amazon QuickSight supports UTF-8 file encoding, but not UTF-8 with byte-order mark (BOM). If you are importing JSON files, then for globalUploadSettings specify format, but not delimiter, textqualifier, or containsHeader.

Make sure that any files that you specify are in Amazon S3 buckets that you have granted Amazon QuickSight access to. For information about granting Amazon QuickSight access to AWS resources, see Accessing data sources.

Manifest file format for Amazon QuickSight

Amazon QuickSight manifest files use the following JSON format.

{
    "fileLocations": [
        {
            "URIs": [
                "uri1",
                "uri2",
                "uri3"
            ]
        },
        {
            "URIPrefixes": [
                "prefix1",
                "prefix2",
                "prefix3"
            ]
        }
    ],
    "globalUploadSettings": {
        "format": "JSON",
        "delimiter": ",",
        "textqualifier": "'",
        "containsHeader": "true"
    }
}
                            

Use the fields in the fileLocations element to specify the files to import, and the fields in the globalUploadSettings element to specify import settings for those files, such as field delimiters.

The manifest file elements are described following:

Manifest file examples for Amazon QuickSight

The following are some examples of completed Amazon QuickSight manifest files.

The following example shows a manifest file that identifies two specific .csv files for import. These files use double quotes for text qualifiers. The format, delimiter, and containsHeader fields are skipped because the default values are acceptable.

{
    "fileLocations": [
        {
            "URIs": [
                "https://yourBucket.s3.amazonaws.com/data-file.csv",
                "https://yourBucket.s3.amazonaws.com/data-file-2.csv"
            ]
        }
    ],
    "globalUploadSettings": {
        "textqualifier": "\""
    }
}

The following example shows a manifest file that identifies one specific .tsv file for import. This file also includes a bucket in another AWS Region that contains additional .tsv files for import. The textqualifier and containsHeader fields are skipped because the default values are acceptable.

{
    "fileLocations": [
        {
            "URIs": [
                "https://s3.amazonaws.com/awsexamplebucket/data.tsv"
            ]
        },
        {
            "URIPrefixes": [
                "https://s3-us-east-1.amazonaws.com/awsexamplebucket/"
            ]
        }
    ],
    "globalUploadSettings": {
        "format": "TSV",
        "delimiter": "\t"
    }
}

The following example identifies two buckets that contain .clf files for import. One is in the same AWS Region as the Amazon QuickSight account, and one in a different AWS Region. The delimiter, textqualifier, and containsHeader fields are skipped because they are not applicable to log files.

{
    "fileLocations": [
        {
            "URIPrefixes": [
                "https://awsexamplebucket.your-s3-url.com",
                "s3://awsexamplebucket2/"
            ]
        }
    ],
    "globalUploadSettings": {
        "format": "CLF"
    }
}

The following example uses the Amazon Redshift format to identify a .csv file for import.

{
    "entries": [
        {
            "url": "https://awsexamplebucket.your-s3-url.com/myalias-test/file-to-import.csv",
            "mandatory": true
        }
    ]
}

The following example uses the Amazon Redshift format to identify two JSON files for import.

{
    "fileLocations": [
        {
            "URIs": [
                "https://yourBucket.s3.amazonaws.com/data-file.json",
                "https://yourBucket.s3.amazonaws.com/data-file-2.json"
            ]
        }
    ],
    "globalUploadSettings": {
        "format": "JSON"
    }
}