監控資料載入 - Amazon Aurora

本文為英文版的機器翻譯版本,如內容有任何歧義或不一致之處,概以英文版為準。

監控資料載入

Aurora PostgreSQL 無限資料庫提供數種監控資料載入任務的方式:

列出資料載入任務

您可以連線至叢集端點,並使用 rds_aurora.limitless_data_load_jobs 檢視來列出資料載入任務。

postgres_limitless=> SELECT * FROM rds_aurora.limitless_data_load_jobs LIMIT 6; job_id | status | message | source_db_identifier | source_db_name | full_load_complete_time | progress_details | start_time | last_updated_time | streaming_mode | source_engine_type | ignore_primary_key_conflict | is_dryrun ---------------+-----------+---------+-------------------------------+----------------+-------------------------+-------------------------------------------------------------------------------------------------------------------------------------------------+------------------------+------------------------+-------------------+--------------------+-----------------------------+----------- 1725697520693 | COMPLETED | | persistent-kdm-auto-source-01 | postgres | 2024-09-07 08:48:15+00 | {"FULL_LOAD": {"STATUS": "COMPLETED", "DETAILS": "9 of 9 tables loaded", "COMPLETED_AT": "2024/09/07 08:48:15+00", "RECORDS_MIGRATED": 600003}} | 2024-09-07 08:47:13+00 | 2024-09-07 08:48:15+00 | full_load | aurora_postgresql | t | f 1725696114225 | COMPLETED | | persistent-kdm-auto-source-01 | postgres | 2024-09-07 08:24:20+00 | {"FULL_LOAD": {"STATUS": "COMPLETED", "DETAILS": "3 of 3 tables loaded", "COMPLETED_AT": "2024/09/07 08:24:20+00", "RECORDS_MIGRATED": 200001}} | 2024-09-07 08:23:56+00 | 2024-09-07 08:24:20+00 | full_load | aurora_postgresql | t | f 1725696067630 | COMPLETED | | persistent-kdm-auto-source-01 | postgres | 2024-09-07 08:23:45+00 | {"FULL_LOAD": {"STATUS": "COMPLETED", "DETAILS": "6 of 6 tables loaded", "COMPLETED_AT": "2024/09/07 08:23:45+00", "RECORDS_MIGRATED": 400002}} | 2024-09-07 08:23:10+00 | 2024-09-07 08:23:45+00 | full_load | aurora_postgresql | t | f 1725694221753 | CANCELED | | persistent-kdm-auto-source-01 | postgres | | {} | 2024-09-07 07:31:18+00 | 2024-09-07 07:51:49+00 | full_load_and_cdc | aurora_postgresql | t | f 1725691698210 | COMPLETED | | persistent-kdm-auto-source-01 | postgres | 2024-09-07 07:10:51+00 | {"FULL_LOAD": {"STATUS": "COMPLETED", "DETAILS": "1 of 1 tables loaded", "COMPLETED_AT": "2024/09/07 07:10:51+00", "RECORDS_MIGRATED": 100000}} | 2024-09-07 07:10:42+00 | 2024-09-07 07:10:52+00 | full_load | aurora_postgresql | t | f 1725691695049 | COMPLETED | | persistent-kdm-auto-source-01 | postgres | 2024-09-07 07:10:48+00 | {"FULL_LOAD": {"STATUS": "COMPLETED", "DETAILS": "1 of 1 tables loaded", "COMPLETED_AT": "2024/09/07 07:10:48+00", "RECORDS_MIGRATED": 100000}} | 2024-09-07 07:10:41+00 | 2024-09-07 07:10:48+00 | full_load | aurora_postgresql | t | f (6 rows)

任務記錄會在 90 天後刪除。

使用任務 ID 檢視資料載入任務的詳細資訊

如果您知道任務 ID,您可以連線至叢集端點,並使用 rds_aurora.limitless_data_load_job_details 檢視來查看該資料載入任務的詳細資訊,包括資料表名稱、任務狀態和載入的資料列數目。您可以在對資料載入開始函數的回應中,或從 rds_aurora.limitless_data_load_jobs 檢視取得任務 ID。

postgres_limitless=> SELECT * FROM rds_aurora.limitless_data_load_job_details WHERE job_id='1725696114225'; job_id | destination_table_name | destination_schema_name | start_time | status | full_load_rows | full_load_total_rows | full_load_complete_time | cdc_insert | cdc_update | cdc_delete --------------+------------------------+-------------------------+------------------------+-----------+----------------+----------------------+-------------------------+------------+------------+------------ 1725696114225 | standard_1 | public | 2024-09-07 08:23:57+00 | COMPLETED | 100000 | 100000 | 2024-09-07 08:24:08+00 | 0 | 0 | 0 1725696114225 | standard_2 | public | 2024-09-07 08:24:08+00 | COMPLETED | 100000 | 100000 | 2024-09-07 08:24:17+00 | 0 | 0 | 0 1725696114225 | standard_3 | public | 2024-09-07 08:24:18+00 | COMPLETED | 1 | 1 | 2024-09-07 08:24:20+00 | 0 | 0 | 0 1725696114225 | standard_4 | public | 2024-09-07 08:23:58+00 | PENDING | 0 | 0 | | 0 | 0 | 0 (4 rows)

任務記錄會在 90 天後刪除。

監控 Amazon CloudWatch 日誌群組

資料載入任務狀態變更為 後RUNNING,您可以使用 Amazon CloudWatch Logs 檢查執行時間進度。

監控 CloudWatch 日誌串流

登入 AWS Management Console 並在 開啟 CloudWatch 主控台https://console.aws.amazon.com/cloudwatch/

  1. 導覽至 日誌,然後 日誌群組

  2. 選擇 /aws/rds/aurora-limitless-database 日誌群組。

  3. job_id 搜尋資料載入任務的日誌串流。

    日誌串流具有 Data-Load-Job-job_id 模式。

  4. 選擇日誌串流以查看日誌事件。

每個日誌串流都會顯示事件,其中包含任務狀態和載入 Aurora PostgreSQL 無限資料庫目的地資料表的資料列數目。如果資料載入任務失敗,也會建立錯誤日誌,顯示失敗狀態和原因。

任務記錄會在 90 天後刪除。

監控RDS事件

資料載入任務也會發佈RDS事件,例如任務成功、失敗或取消。您可以從目的地資料庫檢視事件。

如需詳細資訊,請參閱資料庫碎片群組事件