選取您的 Cookie 偏好設定

我們使用提供自身網站和服務所需的基本 Cookie 和類似工具。我們使用效能 Cookie 收集匿名統計資料，以便了解客戶如何使用我們的網站並進行改進。基本 Cookie 無法停用，但可以按一下「自訂」或「拒絕」以拒絕效能 Cookie。

如果您同意，AWS 與經核准的第三方也會使用 Cookie 提供實用的網站功能、記住您的偏好設定，並顯示相關內容，包括相關廣告。若要接受或拒絕所有非必要 Cookie，請按一下「接受」或「拒絕」。若要進行更詳細的選擇，請按一下「自訂」。

偏好設定
聯絡我們
意見回饋
AWS Documentation
建立 AWS 帳戶

Parquet SerDe

RSS
焦點模式
Parquet SerDe - Amazon Athena
序列化程式庫名稱忽略 Parquet 統計資料

本文為英文版的機器翻譯版本，如內容有任何歧義或不一致之處，概以英文版為準。

本文為英文版的機器翻譯版本，如內容有任何歧義或不一致之處，概以英文版為準。

使用 Parquet SerDe 從 Parquet 資料建立 Athena 資料表。

Parquet SerDe 用於以 Parquet 格式存放的資料。要將資料轉換為 Parquet 格式，您可以使用 CREATE TABLE AS SELECT (CTAS) 查詢。如需詳細資訊，請參閱從查詢結果建立資料表 (CTAS)、CTAS 查詢的範例及使用 CTAS 和 INSERT INTO 進行 ETL 和資料分析。

序列化程式庫名稱

Parquet SerDe 的序列化程式庫名稱為 org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe。如需原始程式碼資訊，請參閱 Apache 文件中的類別 ParquetHiveSerDe。

注意

在 s3://athena-examples-myregion/path/to/data/ 中，以您執行 Athena 之處的區域識別符 (例如 s3://athena-examples-us-west-1/path/to/data/) 取代 myregion。

使用下列CREATE TABLE陳述式，從存放在 Amazon S3 中 Parquet 格式的基礎資料建立 Athena 資料表：

CREATE EXTERNAL TABLE flight_delays_pq (
    yr INT,
    quarter INT,
    month INT,
    dayofmonth INT,
    dayofweek INT,
    flightdate STRING,
    uniquecarrier STRING,
    airlineid INT,
    carrier STRING,
    tailnum STRING,
    flightnum STRING,
    originairportid INT,
    originairportseqid INT,
    origincitymarketid INT,
    origin STRING,
    origincityname STRING,
    originstate STRING,
    originstatefips STRING,
    originstatename STRING,
    originwac INT,
    destairportid INT,
    destairportseqid INT,
    destcitymarketid INT,
    dest STRING,
    destcityname STRING,
    deststate STRING,
    deststatefips STRING,
    deststatename STRING,
    destwac INT,
    crsdeptime STRING,
    deptime STRING,
    depdelay INT,
    depdelayminutes INT,
    depdel15 INT,
    departuredelaygroups INT,
    deptimeblk STRING,
    taxiout INT,
    wheelsoff STRING,
    wheelson STRING,
    taxiin INT,
    crsarrtime INT,
    arrtime STRING,
    arrdelay INT,
    arrdelayminutes INT,
    arrdel15 INT,
    arrivaldelaygroups INT,
    arrtimeblk STRING,
    cancelled INT,
    cancellationcode STRING,
    diverted INT,
    crselapsedtime INT,
    actualelapsedtime INT,
    airtime INT,
    flights INT,
    distance INT,
    distancegroup INT,
    carrierdelay INT,
    weatherdelay INT,
    nasdelay INT,
    securitydelay INT,
    lateaircraftdelay INT,
    firstdeptime STRING,
    totaladdgtime INT,
    longestaddgtime INT,
    divairportlandings INT,
    divreacheddest INT,
    divactualelapsedtime INT,
    divarrdelay INT,
    divdistance INT,
    div1airport STRING,
    div1airportid INT,
    div1airportseqid INT,
    div1wheelson STRING,
    div1totalgtime INT,
    div1longestgtime INT,
    div1wheelsoff STRING,
    div1tailnum STRING,
    div2airport STRING,
    div2airportid INT,
    div2airportseqid INT,
    div2wheelson STRING,
    div2totalgtime INT,
    div2longestgtime INT,
    div2wheelsoff STRING,
    div2tailnum STRING,
    div3airport STRING,
    div3airportid INT,
    div3airportseqid INT,
    div3wheelson STRING,
    div3totalgtime INT,
    div3longestgtime INT,
    div3wheelsoff STRING,
    div3tailnum STRING,
    div4airport STRING,
    div4airportid INT,
    div4airportseqid INT,
    div4wheelson STRING,
    div4totalgtime INT,
    div4longestgtime INT,
    div4wheelsoff STRING,
    div4tailnum STRING,
    div5airport STRING,
    div5airportid INT,
    div5airportseqid INT,
    div5wheelson STRING,
    div5totalgtime INT,
    div5longestgtime INT,
    div5wheelsoff STRING,
    div5tailnum STRING
)
PARTITIONED BY (year STRING)
STORED AS PARQUET
LOCATION 's3://athena-examples-myregion/flight/parquet/'
tblproperties ("parquet.compression"="SNAPPY");

在資料表上執行 MSCK REPAIR TABLE 陳述式，以重新整理分割區中繼資料：

MSCK REPAIR TABLE flight_delays_pq;

查詢延誤超過 1 小時的前 10 個航線：

SELECT origin, dest, count(*) as delays
FROM flight_delays_pq
WHERE depdelayminutes > 60
GROUP BY origin, dest
ORDER BY 3 DESC
LIMIT 10;
注意

航班資料表資料來自美國運輸部交通統計局提供的航班。從原本資料去掉飽和度。

注意

在 s3://athena-examples-myregion/path/to/data/ 中，以您執行 Athena 之處的區域識別符 (例如 s3://athena-examples-us-west-1/path/to/data/) 取代 myregion。

使用下列CREATE TABLE陳述式，從存放在 Amazon S3 中 Parquet 格式的基礎資料建立 Athena 資料表：

CREATE EXTERNAL TABLE flight_delays_pq (
    yr INT,
    quarter INT,
    month INT,
    dayofmonth INT,
    dayofweek INT,
    flightdate STRING,
    uniquecarrier STRING,
    airlineid INT,
    carrier STRING,
    tailnum STRING,
    flightnum STRING,
    originairportid INT,
    originairportseqid INT,
    origincitymarketid INT,
    origin STRING,
    origincityname STRING,
    originstate STRING,
    originstatefips STRING,
    originstatename STRING,
    originwac INT,
    destairportid INT,
    destairportseqid INT,
    destcitymarketid INT,
    dest STRING,
    destcityname STRING,
    deststate STRING,
    deststatefips STRING,
    deststatename STRING,
    destwac INT,
    crsdeptime STRING,
    deptime STRING,
    depdelay INT,
    depdelayminutes INT,
    depdel15 INT,
    departuredelaygroups INT,
    deptimeblk STRING,
    taxiout INT,
    wheelsoff STRING,
    wheelson STRING,
    taxiin INT,
    crsarrtime INT,
    arrtime STRING,
    arrdelay INT,
    arrdelayminutes INT,
    arrdel15 INT,
    arrivaldelaygroups INT,
    arrtimeblk STRING,
    cancelled INT,
    cancellationcode STRING,
    diverted INT,
    crselapsedtime INT,
    actualelapsedtime INT,
    airtime INT,
    flights INT,
    distance INT,
    distancegroup INT,
    carrierdelay INT,
    weatherdelay INT,
    nasdelay INT,
    securitydelay INT,
    lateaircraftdelay INT,
    firstdeptime STRING,
    totaladdgtime INT,
    longestaddgtime INT,
    divairportlandings INT,
    divreacheddest INT,
    divactualelapsedtime INT,
    divarrdelay INT,
    divdistance INT,
    div1airport STRING,
    div1airportid INT,
    div1airportseqid INT,
    div1wheelson STRING,
    div1totalgtime INT,
    div1longestgtime INT,
    div1wheelsoff STRING,
    div1tailnum STRING,
    div2airport STRING,
    div2airportid INT,
    div2airportseqid INT,
    div2wheelson STRING,
    div2totalgtime INT,
    div2longestgtime INT,
    div2wheelsoff STRING,
    div2tailnum STRING,
    div3airport STRING,
    div3airportid INT,
    div3airportseqid INT,
    div3wheelson STRING,
    div3totalgtime INT,
    div3longestgtime INT,
    div3wheelsoff STRING,
    div3tailnum STRING,
    div4airport STRING,
    div4airportid INT,
    div4airportseqid INT,
    div4wheelson STRING,
    div4totalgtime INT,
    div4longestgtime INT,
    div4wheelsoff STRING,
    div4tailnum STRING,
    div5airport STRING,
    div5airportid INT,
    div5airportseqid INT,
    div5wheelson STRING,
    div5totalgtime INT,
    div5longestgtime INT,
    div5wheelsoff STRING,
    div5tailnum STRING
)
PARTITIONED BY (year STRING)
STORED AS PARQUET
LOCATION 's3://athena-examples-myregion/flight/parquet/'
tblproperties ("parquet.compression"="SNAPPY");

在資料表上執行 MSCK REPAIR TABLE 陳述式，以重新整理分割區中繼資料：

MSCK REPAIR TABLE flight_delays_pq;

查詢延誤超過 1 小時的前 10 個航線：

SELECT origin, dest, count(*) as delays
FROM flight_delays_pq
WHERE depdelayminutes > 60
GROUP BY origin, dest
ORDER BY 3 DESC
LIMIT 10;
注意

航班資料表資料來自美國運輸部交通統計局提供的航班。從原本資料去掉飽和度。

忽略 Parquet 統計資料

當您讀取 Parquet 資料時，您可能會收到下列錯誤訊息：

HIVE_CANNOT_OPEN_SPLIT: Index x out of bounds for length y
HIVE_CURSOR_ERROR: Failed to read x bytes
HIVE_CURSOR_ERROR: FailureException at Malformed input: offset=x
HIVE_CURSOR_ERROR: FailureException at java.io.IOException: 
can not read class org.apache.parquet.format.PageHeader: Socket is closed by peer.

若要解決此問題，請使用 CREATE TABLE 或 ALTER TABLE SET TBLPROPERTIES 陳述式，將 Parquet SerDe parquet.ignore.statistics 屬性設定為 true，如下列範例所示。

CREATE TABLE 範例

...
ROW FORMAT SERDE  
'org.apache.hadoop.hive.ql.io.parquet.serde.ParquetHiveSerDe' 
WITH SERDEPROPERTIES (  
'parquet.ignore.statistics'='true')  
STORED AS PARQUET 
...

ALTER TABLE 範例

ALTER TABLE ... SET TBLPROPERTIES ('parquet.ignore.statistics'='true')

在本頁面

下一個主題：

Regex SerDe

上一個主題：

ORC SerDe

需要協助？

隱私權網站條款Cookie 偏好設定
© 2025, Amazon Web Services, Inc.或其附屬公司。保留所有權利。