select_object_content¶
Operation¶
async
¶
select_object_content(input: SelectObjectContentInput, plugins: list[Plugin] | None = None) -> AsyncOutputEventStream[SelectObjectContentEventStream, SelectObjectContentOutput]
Note
This operation is not supported for directory buckets.
This action filters the contents of an Amazon S3 object based on a simple structured query language (SQL) statement. In the request, along with the SQL expression, you must also specify a data serialization format (JSON, CSV, or Apache Parquet) of the object. Amazon S3 uses this format to parse object data into records, and returns only records that match the specified SQL expression. You must also specify the data serialization format for the response.
This functionality is not supported for Amazon S3 on Outposts.
For more information about Amazon S3 Select, see Selecting Content from Objects and SELECT Command in the Amazon S3 User Guide.
- Permissions
-
You must have the
s3:GetObjectpermission for this operation. Amazon S3 Select does not support anonymous access. For more information about permissions, see Specifying Permissions in a Policy in the Amazon S3 User Guide. - Object Data Formats
-
You can use Amazon S3 Select to query objects that have the following format properties:
-
CSV, JSON, and Parquet - Objects must be in CSV, JSON, or Parquet format.
-
UTF-8 - UTF-8 is the only encoding type Amazon S3 Select supports.
-
GZIP or BZIP2 - CSV and JSON files can be compressed using GZIP or BZIP2. GZIP and BZIP2 are the only compression formats that Amazon S3 Select supports for CSV and JSON files. Amazon S3 Select supports columnar compression for Parquet using GZIP or Snappy. Amazon S3 Select does not support whole-object compression for Parquet objects.
-
Server-side encryption - Amazon S3 Select supports querying objects that are protected with server-side encryption.
For objects that are encrypted with customer-provided encryption keys (SSE-C), you must use HTTPS, and you must use the headers that are documented in the GetObject. For more information about SSE-C, see Server-Side Encryption (Using Customer-Provided Encryption Keys) in the Amazon S3 User Guide.
For objects that are encrypted with Amazon S3 managed keys (SSE-S3) and Amazon Web Services KMS keys (SSE-KMS), server-side encryption is handled transparently, so you don't need to specify anything. For more information about server-side encryption, including SSE-S3 and SSE-KMS, see Protecting Data Using Server-Side Encryption in the Amazon S3 User Guide.
- Working with the Response Body
-
Given the response size is unknown, Amazon S3 Select streams the response as a series of messages and includes a
Transfer-Encodingheader withchunkedas its value in the response. For more information, see Appendix: SelectObjectContent Response. - GetObject Support
-
The
SelectObjectContentaction does not support the followingGetObjectfunctionality. For more information, see GetObject.
-
Range: Although you can specify a scan range for an Amazon S3 Select request (see SelectObjectContentRequest - ScanRange in the request parameters), you cannot specify the range of bytes of an object to return. -
The
GLACIER,DEEP_ARCHIVE, andREDUCED_REDUNDANCYstorage classes, or theARCHIVE_ACCESSandDEEP_ARCHIVE_ACCESSaccess tiers of theINTELLIGENT_TIERINGstorage class: You cannot query objects in theGLACIER,DEEP_ARCHIVE, orREDUCED_REDUNDANCYstorage classes, nor objects in theARCHIVE_ACCESSorDEEP_ARCHIVE_ACCESSaccess tiers of theINTELLIGENT_TIERINGstorage class. For more information about storage classes, see Using Amazon S3 storage classes in the Amazon S3 User Guide.
- Special Errors
-
For a list of special errors for this operation, see List of SELECT Object Content Error Codes
The following operations are related to SelectObjectContent:
Warning
You must URL encode any signed header values that contain spaces. For
example, if your header value is my file.txt, containing two spaces
after my, you must URL encode this value to my%20%20file.txt.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
input
|
SelectObjectContentInput
|
An instance of |
required |
plugins
|
list[Plugin] | None
|
A list of callables that modify the configuration dynamically. Changes made by these plugins only apply for the duration of the operation execution and will not affect any other operation invocations. |
None
|
Returns:
| Type | Description |
|---|---|
AsyncOutputEventStream[SelectObjectContentEventStream, SelectObjectContentOutput]
|
An |
Input¶
dataclass
¶
Bases: _SerdeShape
Note
Learn Amazon S3 Select is no longer available to new customers. Existing customers of Amazon S3 Select can continue to use the feature as usual. Learn more
Request to filter the contents of an Amazon S3 object based on a simple Structured Query Language (SQL) statement. In the request, along with the SQL expression, you must specify a data serialization format (JSON or CSV) of the object. Amazon S3 uses this to parse object data into records. It returns only records that match the specified SQL expression. You must also specify the data serialization format for the response. For more information, see S3Select API Documentation.
Attributes¶
class-attribute
instance-attribute
¶
expected_bucket_owner: str | None = ...
The account ID of the expected bucket owner. If the account ID that you
provide does not match the actual owner of the bucket, the request fails
with the HTTP status code 403 Forbidden (access denied).
class-attribute
instance-attribute
¶
expression: str | None = ...
The expression that is used to query the object.
class-attribute
instance-attribute
¶
expression_type: str | None = ...
The type of the provided expression (for example, SQL).
class-attribute
instance-attribute
¶
Describes the format of the data in the object that is being queried.
class-attribute
instance-attribute
¶
Describes the format of the data that you want Amazon S3 to return in response.
class-attribute
instance-attribute
¶
Specifies if periodic request progress information should be enabled.
class-attribute
instance-attribute
¶
Specifies the byte range of the object to get the records from. A record is processed when its first byte is contained by the range. This parameter is optional, but when specified, it must not be empty. See RFC 2616, Section 14.35.1 about how to specify the start and end of the range.
ScanRangemay be used in the following ways:
-
<scanrange><start>50``</start><end>100``</end></scanrange>- process only the records starting between the bytes 50 and 100 (inclusive, counting from zero) -
<scanrange><start>50``</start></scanrange>- process only the records starting after the byte 50 -
<scanrange><end>50``</end></scanrange>- process only the records within the last 50 bytes of the file.
class-attribute
instance-attribute
¶
sse_customer_algorithm: str | None = ...
The server-side encryption (SSE) algorithm used to encrypt the object. This parameter is needed only when the object was created using a checksum algorithm. For more information, see Protecting data using SSE-C keys in the Amazon S3 User Guide.
class-attribute
instance-attribute
¶
sse_customer_key: str | None = ...
The server-side encryption (SSE) customer managed key. This parameter is needed only when the object was created using a checksum algorithm. For more information, see Protecting data using SSE-C keys in the Amazon S3 User Guide.
class-attribute
instance-attribute
¶
sse_customer_key_md5: str | None = ...
The MD5 server-side encryption (SSE) customer managed key. This parameter is needed only when the object was created using a checksum algorithm. For more information, see Protecting data using SSE-C keys in the Amazon S3 User Guide.
Output¶
This operation returns an AsyncOutputEventStream for server-to-client streaming.
Event Stream Structure¶
Output Event Type¶
SelectObjectContentEventStream
Initial Response Structure¶
dataclass
¶
Bases: _SerdeShape
Dataclass for SelectObjectContentOutput structure.