Native export for RDF data

The Neptune export API enables you to export data from your Neptune database to Amazon S3. You can export RDF data in N-Triples or N-Quads format.

How native export works

Native export runs on the writer instance of your Neptune cluster and writes exported data to Amazon S3 using multipart uploads. Because export uses the writer instance's compute resources, we strongly recommend running exports against a cloned cluster to avoid impacting production workloads. See Recommendations for more information.

Export throughput

Export throughput scales approximately linearly with instance size up to r7i.16xlarge. As a conservative planning estimate, expect approximately 50,000 statements per second per vCPU.

Use this formula to estimate export duration:

export_seconds = total_statements / (vCPUs × 50,000)

Actual throughput depends on dataset characteristics including predicate cardinality, statement complexity, and cluster size.

Prerequisites

Before using the export API, you must:

Recommendations

We strongly recommend running export operation on a cloned cluster without any read/write workloads to avoid impact on production performance.

For optimal price-to-performance ratio, we recommend using 16xlarge instances for export operations. This instance type provides:

IAM permissions

The export feature involves two separate IAM roles:

Caller permissions (Neptune data-access actions)

The IAM principal calling the export API must have the following Neptune data-access actions in its IAM policy:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AllowNeptuneExportActions",
      "Effect": "Allow",
      "Action": [
        "neptune-db:StartExportJob",
        "neptune-db:GetExportJobStatus",
        "neptune-db:ListExportJobs",
        "neptune-db:CancelExportJob"
      ],
      "Resource": "arn:aws:neptune-db:us-east-1:123456789012:cluster-resource-id/*"
    }
  ]
}

For more information, see Using IAM data-access policy statements.

S3 access role permissions

The IAM role passed in the iamRoleArn request parameter must be associated with your Neptune cluster and must grant Neptune permission to write to the target S3 bucket. For steps on creating an IAM role and associating it with your cluster, see Create an IAM role to allow Neptune to access Amazon S3.

Attach the following permissions policy to the S3 access role:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AllowS3WriteForNeptuneExport",
      "Effect": "Allow",
      "Action": [
        "s3:ListBucket",
        "s3:GetObject",
        "s3:PutObject",
        "s3:AbortMultipartUpload",
        "s3:GetBucketPublicAccessBlock"
      ],
      "Resource": [
        "arn:aws:s3:::amzn-s3-demo-bucket",
        "arn:aws:s3:::amzn-s3-demo-bucket/*"
      ]
    }
  ]
}

Optional KMS permissions

If you specify a kmsKeyIdentifier in the export request, add the following permissions to the S3 access role:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "AllowKMSForNeptuneExport",
      "Effect": "Allow",
      "Action": [
        "kms:Decrypt",
        "kms:Encrypt",
        "kms:GenerateDataKey"
      ],
      "Resource": "arn:aws:kms:us-east-1:123456789012:key/key-id"
    }
  ]
}

Export endpoint

To export data, you send HTTP requests to the https://your-neptune-endpoint:port/export endpoint.

Export request syntax

POST https://your-neptune-endpoint:port/export

Request headers

Request body

{
  "destination": "string",
  "format": "string",
  "iamRoleArn": "string",
  "region": "string",
  "compression": "string",
  "kmsKeyIdentifier": "string",
  "exportFilter": {
    "namedGraphUris": ["string"]
  }
}

Request parameters

Response syntax

{
  "status": "string",
  "payload": {
    "exportId": "string"
  }
}

Export status endpoint

To check the status of an export task, you send an HTTP GET request to the export endpoint with the export ID.

GET https://your-neptune-endpoint:port/export?exportId=export-id

Request parameters

Response syntax

{
  "status": "string",
  "payload": {
    "exportId": "string",
    "destination": "string",
    "status": "string",
    "statusReason": "string",
    "format": "string",
    "iamRoleArn": "string",
    "kmsKeyIdentifier": "string",
    "exportFilter": {
      "namedGraphUris": ["string"]
    },
    "exportTaskDetails": {
      "timeElapsedSeconds": number,
      "startTime": number,
      "numRecordsWritten": number,
      "progressPercentage": number
    }
  }
}

List exports endpoint

To list all export tasks, you send an HTTP GET request to the export endpoint.

GET https://your-neptune-endpoint:port/export

Response syntax

{
  "status": "string",
  "payload": [
    "string"
  ]
}

Cancel export endpoint

To cancel an export task, you send an HTTP DELETE request to the export endpoint with the export ID.

DELETE https://your-neptune-endpoint:port/export?exportId=export-id

Request parameters

Response syntax

{
  "status": "string",
  "payload": {
    "message": "string"
  }
}

Export output format

S3 directory structure

Exported data is organized in your S3 bucket as follows:

s3://your-bucket/export-id/
├── data/
│   ├── part-00000.nt
│   ├── part-00001.nt
│   └── ...
└── export_status.json

Error responses

Common error codes

Example error response

{
  "code": "BadRequestException",
  "requestId": "request-id", 
  "message": "Export already in progress with ID 'existing-id'. Please wait or cancel it first.",
  "detailedMessage": "Detailed error description"
}

Examples

Start an export

curl -X POST https://your-cluster-endpoint:8182/export \
  -H "Content-Type: application/json" \
  -d '{
    "destination": "s3://my-bucket/exports/",
    "format": "ntriples", 
    "iamRoleArn": "arn:aws:iam::123456789012:role/neptune-export-role",
    "region": "us-west-2"
  }'

Start a filtered export (named graphs)

curl -X POST https://your-cluster-endpoint:8182/export \
  -H "Content-Type: application/json" \
  -d '{
    "destination": "s3://my-bucket/exports/",
    "format": "nquads",
    "iamRoleArn": "arn:aws:iam::123456789012:role/neptune-export-role",
    "region": "us-west-2",
    "exportFilter": {
      "namedGraphUris": [
        "http://example.com/graph1",
        "http://example.com/graph2"
      ]
    }
  }'

Check export status

curl -X GET "https://your-cluster-endpoint:8182/export?exportId=<id>"

Cancel an export

curl -X DELETE "https://your-cluster-endpoint:8182/export?exportId=<id>"

Limitations

Native export has the following limitations: