本文為英文版的機器翻譯版本,如內容有任何歧義或不一致之處,概以英文版為準。
AWS Batch 客戶的常見工作流程是執行先決條件設定任務、針對大量輸入任務執行一系列命令,然後以彙總結果並將摘要資料寫入 Amazon S3、DynamoDB、Amazon Redshift 或 Aurora 的任務結束。
例如:
-
JobA
:一項標準、非陣列任務,對 Amazon S3 儲存貯體中的物件執行快速清單和中繼資料驗證BucketA
。SubmitJob JSON 語法如下所示。{ "jobName": "JobA", "jobQueue": "ProdQueue", "jobDefinition": "JobA-list-and-validate:1" }
-
JobB
:陣列任務具有 10,000 個複本,這些複本依賴JobA
於 中對每個物件執行 CPU 密集型命令,BucketA
並將結果上傳至BucketB
。SubmitJob JSON 語法如下所示。{ "jobName": "JobB", "jobQueue": "ProdQueue", "jobDefinition": "JobB-CPU-Intensive-Processing:1", "containerOverrides": { "resourceRequirements": [ { "type": "MEMORY", "value": "4096" }, { "type": "VCPU", "value": "32" } ] } "arrayProperties": { "size": 10000 }, "dependsOn": [ { "jobId": "
JobA_job_ID
" } ] } -
JobC
:另一個JobB
10,000 個複本陣列任務,其依賴N_TO_N
於相依性模型,針對 中的每個項目執行記憶體密集型命令BucketB
、將中繼資料寫入 DynamoDB,並將產生的輸出上傳到BucketC
。SubmitJob JSON 語法如下所示。{ "jobName": "JobC", "jobQueue": "ProdQueue", "jobDefinition": "JobC-Memory-Intensive-Processing:1", "containerOverrides": { "resourceRequirements": [ { "type": "MEMORY", "value": "32768" }, { "type": "VCPU", "value": "1" } ] } "arrayProperties": { "size": 10000 }, "dependsOn": [ { "jobId": "
JobB_job_ID
", "type": "N_TO_N" } ] } -
JobD
:執行 10 個驗證步驟的陣列任務,每個步驟都需要查詢 DynamoDB,並且可能與上述任何 Amazon S3 儲存貯體互動。中的每個步驟都會JobD
執行相同的命令。不過,行為會根據任務容器內AWS_BATCH_JOB_ARRAY_INDEX
環境變數的值而有所不同。這些驗證步驟會依序執行 (例如,JobD:0
然後是JobD:1
)。SubmitJob JSON 語法如下所示。{ "jobName": "JobD", "jobQueue": "ProdQueue", "jobDefinition": "JobD-Sequential-Validation:1", "containerOverrides": { "resourceRequirements": [ { "type": "MEMORY", "value": "32768" }, { "type": "VCPU", "value": "1" } ] } "arrayProperties": { "size": 10 }, "dependsOn": [ { "jobId": "
JobC_job_ID
" }, { "type": "SEQUENTIAL" }, ] } -
JobE
:執行一些簡單清除操作的最終非陣列任務,並傳送 Amazon SNS 通知,其中包含管道已完成的訊息和輸出 URL 的連結。SubmitJob JSON 語法如下所示。{ "jobName": "JobE", "jobQueue": "ProdQueue", "jobDefinition": "JobE-Cleanup-and-Notification:1", "parameters": { "SourceBucket": "s3://amzn-s3-demo-source-bucket", "Recipient": "pipeline-notifications@mycompany.com" }, "dependsOn": [ { "jobId": "
JobD_job_ID
" } ] }