View a markdown version of this page

任务完成登录 AWS 个 - AWS 个
先决条件设置任务完成日志如何查找任务完成日志作业完成日志字段任务完成日志示例

本文属于机器翻译版本。若本译文内容与英语原文存在差异，则一律以英文原文为准。

任务完成登录 AWS 个

任务完成日志可在 AWS 并行计算服务 (AWS PCS) 任务完成后为您提供有关任务的关键详细信息，无需额外付费。您可以使用其他 AWS 服务来访问和处理您的日志数据，例如亚马逊 CloudWatch 日志、亚马逊简单存储服务 (Amazon S3) 和亚马逊数据消防站； AWS PCS 记录有关您的任务的元数据，例如以下内容。

  • 任务 ID 和名称

  • 用户和群组信息

  • 作业状态（例如COMPLETED、FAILED、CANCELLED）

  • 使用的分区

  • 时间限制

  • 开始、结束、提交和符合条件的时间

  • 节点列表和数量

  • 处理器数量

  • 工作目录

  • 资源使用情况（CPU、内存）

  • 退出码

  • 节点详细信息（名称、实例 ID、实例类型）

先决条件

管理 AWS PCS 集群的 IAM 委托人必须允许该pcs:AllowVendedLogDeliveryForResource操作。

以下示例 IAM 策略授予所需的权限。

JSON
{
   "Version":"2012-10-17",
   "Statement": [
      {
         "Sid": "PcsAllowVendedLogsDelivery",
         "Effect": "Allow",
         "Action": ["pcs:AllowVendedLogDeliveryForResource"],
         "Resource": [
            "arn:aws:pcs:*::cluster/*"
         ]
      }
   ]
}

设置任务完成日志

您可以使用 AWS 管理控制台 或为 AWS PCS 集群设置任务完成日志 AWS CLI。

AWS 管理控制台
使用控制台设置任务完成日志

  1. 打开 AWS PCS 控制台。

  2. 在导航窗格中，选择集群。

  3. 选择要添加任务完成日志的集群。

  4. 在集群详细信息页面上，选择日志选项卡。

  5. 在 “任务完成日志” 下，选择 “添加”，从日志、Amazon S3 和 Firehose 中添加最多 3 个 CloudWatch 日志传输目的地。

  6. 选择 “更新日志交付” 。

AWS CLI
要设置任务完成日志，请使用 AWS CLI

  1. 创建日志传送目标：

    aws logs put-delivery-destination --region region \
  --name pcs-logs-destination \
  --delivery-destination-configuration \
  destinationResourceArn=resource-arn

    进行如下替换：

    • region— 您要创建目的地 AWS 区域 的位置，例如 us-east-1

    • pcs-logs-destination— 目的地的名称

    • resource-arn— CloudWatch 日志组、S3 存储桶或 Firehose 交付流的亚马逊资源名称 (ARN)。

    有关更多信息，请参阅PutDeliveryDestination《亚马逊 CloudWatch 日志 API 参考》。

  2. 将 PCS 集群设置为日志传输源：

    aws logs put-delivery-source --region region \
  --name cluster-logs-source-name \
  --resource-arn cluster-arn \
  --log-type PCS_JOBCOMP_LOGS

    进行如下替换：

    • region— 您的 AWS 区域 集群的，例如 us-east-1

    • cluster-logs-source-name— 来源的名称

    • cluster-arn— 您的 AWS PCS 集群的 ARN

    有关更多信息，请参阅PutDeliverySource《亚马逊 CloudWatch 日志 API 参考》。

  3. 将传送源连接到交付目的地：

    aws logs create-delivery --region region \
  --delivery-source-name cluster-logs-source \
  --delivery-destination-arn destination-arn

    进行如下替换：

    • region— 那个 AWS 区域，比如 us-east-1

    • cluster-logs-source— 您的交付来源的名称

    • destination-arn— 您的送货目的地的 ARN

    有关更多信息，请参阅CreateDelivery《亚马逊 CloudWatch 日志 API 参考》。

如何查找任务完成日志

您可以在日志和 Amazon S3 中配置 CloudWatch 日志目的地。 AWS PCS 使用以下结构化路径名和文件名。

CloudWatch 日志

AWS PCS 对 CloudWatch 日志流使用以下名称格式：

AWSLogs/PCS/cluster-id/jobcomp.log

例如：AWSLogs/PCS/pcs_abc123de45/jobcomp.log

Amazon S3

AWS PCS 对 S3 路径使用以下名称格式：

AWSLogs/account-id/PCS/region/cluster-id/jobcomp/year/month/day/hour/

例如：AWSLogs/111122223333/PCS/us-east-1/pcs_abc123de45/jobcomp/2025/06/19/11/

AWS PCS 对日志文件使用以下名称格式：

PCS_jobcomp_year-month-day-hour_cluster-id_random-id.log.gz

例如：PCS_jobcomp_2025-06-19-11_pcs_abc123de45_04be080b.log.gz

作业完成日志字段

AWS PCS 以 JSON 对象的形式写入任务完成日志数据。每个日志条目都包含顶级元数据字段和一个fields对象。顶级字段标识集群、事件时间和调度程序版本。该fields对象包含任务的详细信息。fields对象中的某些字段仅在数组作业或异构作业中出现。

下表描述了每个日志条目中的顶级字段。

Top-level 字段
Name 示例值 必填 注意
resource_id "pcs_22l8nzr3t9" 是 AWS PCS 集群 ID
resource_type "PCS_CLUSTER" 是 永远 "PCS_CLUSTER"
event_timestamp 1750370337 是 事件发生时的 Unix 纪元秒数
scheduler_type "slurm" 是 集群的调度器类型
scheduler_major_version "25.11" 是 调度程序的主要版本

下表描述了fields对象内部的字段。

fields 对象内的字段
Name 示例值 必填 注意
job_id 11 是 始终以价值呈现
user "root" 是 始终以价值呈现
user_id 0 是 始终以价值呈现
group "root" 是 始终以价值呈现
group_id 0 是 始终以价值呈现
name "wrap" 是 始终以价值呈现
job_state "COMPLETED" 是 始终以价值呈现
partition "MpiQueue-abcdef01-7" 是 始终以价值呈现
time_limit "UNLIMITED" 是 始终在场，但可能是 "UNLIMITED"
start_time "2025-06-19T10:58:57" 是 始终在场，但可能是 "Unknown"
end_time "2025-06-19T10:58:57" 是 始终在场，但可能是 "Unknown"
node_list "MpiNG-abcdef01-2345-1" 是 始终以价值呈现
node_cnt 1 是 始终以价值呈现
proc_cnt 1 是 始终以价值呈现
work_dir "/root" 是 始终在场，但可能是 "Unknown"
reservation_name "weekly_maintenance" 是 始终存在，但可能是一个空字符串 ""
tres.cpu 1 是 始终以价值呈现
tres.mem.val 600 是 始终以价值呈现
tres.mem.unit "M" 是 可以是"M"或 "bb"
tres.node 1 是 始终以价值呈现
tres.billing 1 是 始终以价值呈现
account "finance" 是 始终存在，但可能是一个空字符串 ""
qos "normal" 是 始终存在，但可能是一个空字符串 ""
wc_key "project_1" 是 始终存在，但可能是一个空字符串 ""
cluster "unknown" 是 始终在场，但可能是 "unknown"
submit_time "2025-06-19T10:55:46" 是 始终在场，但可能是 "Unknown"
eligible_time "2025-06-19T10:55:46" 是 始终在场，但可能是 "Unknown"
array_job_id 12 否 仅当任务是阵列作业时才出现
array_task_id 1 否 仅当任务是阵列作业时才出现
het_job_id 10 否 仅当作业是异构作业时才出现
het_job_offset 0 否 仅当作业是异构作业时才出现
derived_exit_code_status 0 是 始终以价值呈现
derived_exit_code_signal 0 是 始终以价值呈现
exit_code_status 0 是 始终以价值呈现
exit_code_signal 0 是 始终以价值呈现
node_details[0].name "MpiNG-abcdef01-2345-1" 否 始终在场，但node_details可能是 "[]"
node_details[0].instance_id "i-0abcdef01234567a" 否 始终在场，但node_details可能是 "[]"
node_details[0].instance_type "t4g.micro" 否 始终在场，但node_details可能是 "[]"

任务完成日志示例

以下示例显示了各种作业类型和状态的任务完成日志：

{ "resource_id": "pcs_abc123de45", "resource_type": "PCS_CLUSTER", "event_timestamp": 1750350777, "scheduler_type": "slurm", "scheduler_major_version": "25.11", "fields": { "job_id": 1, "user": "root", "user_id": 0, "group": "root", "group_id": 0, "name": "wrap", "job_state": "COMPLETED", "partition": "MpiQueue-abcdef01-7", "time_limit": "UNLIMITED", "start_time": "2025-06-19T16:32:57", "end_time": "2025-06-19T16:33:03", "node_list": "MpiNG-abcdef01-2345-[1-2]", "node_cnt": 2, "proc_cnt": 2, "work_dir": "/usr/bin", "reservation_name": "", "tres": { "cpu": 2, "mem": { "val": 1944, "unit": "M" }, "node": 2, "billing": 2 }, "account": "", "qos": "", "wc_key": "", "cluster": "unknown", "submit_time": "2025-06-19T16:29:40", "eligible_time": "2025-06-19T16:29:41", "derived_exit_code_status": 0, "derived_exit_code_signal": 0, "exit_code_status": 0, "exit_code_signal": 0, "node_details": [ { "name": "MpiNG-abcdef01-2345-1", "instance_id": "i-0abc123def45678", "instance_type": "t4g.micro" }, { "name": "MpiNG-abcdef01-2345-2", "instance_id": "i-0def456abc78901", "instance_type": "t4g.micro" } ] } }
{ "resource_id": "pcs_abc123de45", "resource_type": "PCS_CLUSTER", "event_timestamp": 1750350793, "scheduler_type": "slurm", "scheduler_major_version": "25.11", "fields": { "job_id": 2, "user": "root", "user_id": 0, "group": "root", "group_id": 0, "name": "wrap", "job_state": "COMPLETED", "partition": "MpiQueue-abcdef01-7", "time_limit": "UNLIMITED", "start_time": "2025-06-19T16:33:13", "end_time": "2025-06-19T16:33:14", "node_list": "MpiNG-abcdef01-2345-[1-2]", "node_cnt": 2, "proc_cnt": 2, "work_dir": "/usr/bin", "reservation_name": "", "tres": { "cpu": 2, "mem": { "val": 1944, "unit": "M" }, "node": 2, "billing": 2 }, "account": "", "qos": "", "wc_key": "", "cluster": "unknown", "submit_time": "2025-06-19T16:33:13", "eligible_time": "2025-06-19T16:33:13", "derived_exit_code_status": 0, "derived_exit_code_signal": 0, "exit_code_status": 0, "exit_code_signal": 0, "node_details": [ { "name": "MpiNG-abcdef01-2345-1", "instance_id": "i-0abc123def45678", "instance_type": "t4g.micro" }, { "name": "MpiNG-abcdef01-2345-2", "instance_id": "i-0def456abc78901", "instance_type": "t4g.micro" } ] } }
{ "resource_id": "pcs_abc123de45", "resource_type": "PCS_CLUSTER", "event_timestamp": 1750373937, "scheduler_type": "slurm", "scheduler_major_version": "25.11", "fields": { "job_id": 3, "user": "root", "user_id": 0, "group": "root", "group_id": 0, "name": "wrap", "job_state": "COMPLETED", "partition": "MpiQueue-abcdef01-7", "time_limit": "UNLIMITED", "start_time": "2025-06-19T22:58:57", "end_time": "2025-06-19T22:58:57", "node_list": "MpiNG-abcdef01-2345-1", "node_cnt": 1, "proc_cnt": 1, "work_dir": "/root", "reservation_name": "", "tres": { "cpu": 1, "mem": { "val": 972, "unit": "M" }, "node": 1, "billing": 1 }, "account": "", "qos": "", "wc_key": "", "cluster": "unknown", "submit_time": "2025-06-19T22:55:46", "eligible_time": "2025-06-19T22:55:46", "derived_exit_code_status": 0, "derived_exit_code_signal": 0, "exit_code_status": 0, "exit_code_signal": 0, "node_details": [ { "name": "MpiNG-abcdef01-2345-1", "instance_id": "i-0abc234def56789", "instance_type": "t4g.micro" } ] } }
{ "resource_id": "pcs_abc123de45", "resource_type": "PCS_CLUSTER", "event_timestamp": 1750374267, "scheduler_type": "slurm", "scheduler_major_version": "25.11", "fields": { "job_id": 4, "user": "root", "user_id": 0, "group": "root", "group_id": 0, "name": "wrap", "job_state": "COMPLETED", "partition": "MpiQueue-abcdef01-7", "time_limit": "525600", "start_time": "2025-06-19T23:04:27", "end_time": "2025-06-19T23:04:27", "node_list": "MpiNG-abcdef01-2345-[1-2]", "node_cnt": 2, "proc_cnt": 2, "work_dir": "/root", "reservation_name": "", "tres": { "cpu": 2, "mem": { "val": 1944, "unit": "M" }, "node": 2, "billing": 2 }, "account": "", "qos": "", "wc_key": "", "cluster": "unknown", "submit_time": "2025-06-19T23:01:38", "eligible_time": "2025-06-19T23:01:38", "derived_exit_code_status": 0, "derived_exit_code_signal": 0, "exit_code_status": 0, "exit_code_signal": 0, "node_details": [ { "name": "MpiNG-abcdef01-2345-1", "instance_id": "i-0abc234def56789", "instance_type": "t4g.micro" }, { "name": "MpiNG-abcdef01-2345-2", "instance_id": "i-0def345abc67890", "instance_type": "t4g.micro" } ] } }
{ "resource_id": "pcs_abc123de45", "resource_type": "PCS_CLUSTER", "event_timestamp": 1750374540, "scheduler_type": "slurm", "scheduler_major_version": "25.11", "fields": { "job_id": 5, "user": "root", "user_id": 0, "group": "root", "group_id": 0, "name": "wrap", "job_state": "FAILED", "partition": "MpiQueue-abcdef01-7", "time_limit": "UNLIMITED", "start_time": "2025-06-19T23:09:00", "end_time": "2025-06-19T23:09:00", "node_list": "(null)", "node_cnt": 0, "proc_cnt": 0, "work_dir": "/root", "reservation_name": "", "tres": { "cpu": 1, "mem": { "val": 1, "unit": "G" }, "node": 1, "billing": 1 }, "account": "", "qos": "", "wc_key": "", "cluster": "unknown", "submit_time": "2025-06-19T23:09:00", "eligible_time": "2025-06-19T23:09:00", "derived_exit_code_status": 0, "derived_exit_code_signal": 0, "exit_code_status": 0, "exit_code_signal": 1, "node_details": [] } }
{ "resource_id": "pcs_abc123de45", "resource_type": "PCS_CLUSTER", "event_timestamp": 1750374576, "scheduler_type": "slurm", "scheduler_major_version": "25.11", "fields": { "job_id": 6, "user": "root", "user_id": 0, "group": "root", "group_id": 0, "name": "wrap", "job_state": "CANCELLED", "partition": "MpiQueue-abcdef01-7", "time_limit": "UNLIMITED", "start_time": "2025-06-19T23:09:36", "end_time": "2025-06-19T23:09:36", "node_list": "(null)", "node_cnt": 0, "proc_cnt": 0, "work_dir": "/root", "reservation_name": "", "tres": { "cpu": 1, "mem": { "val": 400, "unit": "M" }, "node": 1, "billing": 1 }, "account": "", "qos": "", "wc_key": "", "cluster": "unknown", "submit_time": "2025-06-19T23:09:35", "eligible_time": "2025-06-19T23:09:36", "het_job_id": 6, "het_job_offset": 0, "derived_exit_code_status": 0, "derived_exit_code_signal": 0, "exit_code_status": 0, "exit_code_signal": 1, "node_details": [] } }
{ "resource_id": "pcs_abc123de45", "resource_type": "PCS_CLUSTER", "event_timestamp": 1750374603, "scheduler_type": "slurm", "scheduler_major_version": "25.11", "fields": { "job_id": 7, "user": "root", "user_id": 0, "group": "root", "group_id": 0, "name": "wrap", "job_state": "CANCELLED", "partition": "MpiQueue-abcdef01-7", "time_limit": "UNLIMITED", "start_time": "2025-06-19T23:10:03", "end_time": "2025-06-19T23:10:03", "node_list": "(null)", "node_cnt": 0, "proc_cnt": 0, "work_dir": "/root", "reservation_name": "", "tres": { "cpu": 1, "mem": { "val": 400, "unit": "M" }, "node": 1, "billing": 1 }, "account": "", "qos": "", "wc_key": "", "cluster": "unknown", "submit_time": "2025-06-19T23:10:03", "eligible_time": "2025-06-19T23:10:03", "het_job_id": 7, "het_job_offset": 0, "derived_exit_code_status": 0, "derived_exit_code_signal": 0, "exit_code_status": 0, "exit_code_signal": 1, "node_details": [] } }
{ "resource_id": "pcs_abc123de45", "resource_type": "PCS_CLUSTER", "event_timestamp": 1750374684, "scheduler_type": "slurm", "scheduler_major_version": "25.11", "fields": { "job_id": 8, "user": "root", "user_id": 0, "group": "root", "group_id": 0, "name": "wrap", "job_state": "COMPLETED", "partition": "MpiQueue-abcdef01-7", "time_limit": "UNLIMITED", "start_time": "2025-06-19T23:11:24", "end_time": "2025-06-19T23:11:24", "node_list": "MpiNG-abcdef01-2345-1", "node_cnt": 1, "proc_cnt": 1, "work_dir": "/root", "reservation_name": "", "tres": { "cpu": 1, "mem": { "val": 400, "unit": "M" }, "node": 1, "billing": 1 }, "account": "", "qos": "", "wc_key": "", "cluster": "unknown", "submit_time": "2025-06-19T23:11:23", "eligible_time": "2025-06-19T23:11:23", "het_job_id": 8, "het_job_offset": 0, "derived_exit_code_status": 0, "derived_exit_code_signal": 0, "exit_code_status": 0, "exit_code_signal": 0, "node_details": [ { "name": "MpiNG-abcdef01-2345-1", "instance_id": "i-0abc234def56789", "instance_type": "t4g.micro" } ] } }
{ "resource_id": "pcs_abc123de45", "resource_type": "PCS_CLUSTER", "event_timestamp": 1750374684, "scheduler_type": "slurm", "scheduler_major_version": "25.11", "fields": { "job_id": 9, "user": "root", "user_id": 0, "group": "root", "group_id": 0, "name": "wrap", "job_state": "COMPLETED", "partition": "MpiQueue-abcdef01-7", "time_limit": "UNLIMITED", "start_time": "2025-06-19T23:11:24", "end_time": "2025-06-19T23:11:24", "node_list": "MpiNG-abcdef01-2345-2", "node_cnt": 1, "proc_cnt": 1, "work_dir": "/root", "reservation_name": "", "tres": { "cpu": 1, "mem": { "val": 400, "unit": "M" }, "node": 1, "billing": 1 }, "account": "", "qos": "", "wc_key": "", "cluster": "unknown", "submit_time": "2025-06-19T23:11:23", "eligible_time": "2025-06-19T23:11:23", "het_job_id": 8, "het_job_offset": 1, "derived_exit_code_status": 0, "derived_exit_code_signal": 0, "exit_code_status": 0, "exit_code_signal": 0, "node_details": [ { "name": "MpiNG-abcdef01-2345-2", "instance_id": "i-0def345abc67890", "instance_type": "t4g.micro" } ] } }
{ "resource_id": "pcs_abc123de45", "resource_type": "PCS_CLUSTER", "event_timestamp": 1750374744, "scheduler_type": "slurm", "scheduler_major_version": "25.11", "fields": { "job_id": 10, "user": "root", "user_id": 0, "group": "root", "group_id": 0, "name": "wrap", "job_state": "COMPLETED", "partition": "MpiQueue-abcdef01-7", "time_limit": "UNLIMITED", "start_time": "2025-06-19T23:12:24", "end_time": "2025-06-19T23:12:24", "node_list": "MpiNG-abcdef01-2345-1", "node_cnt": 1, "proc_cnt": 1, "work_dir": "/root", "reservation_name": "", "tres": { "cpu": 1, "mem": { "val": 400, "unit": "M" }, "node": 1, "billing": 1 }, "account": "", "qos": "", "wc_key": "", "cluster": "unknown", "submit_time": "2025-06-19T23:12:14", "eligible_time": "2025-06-19T23:12:14", "het_job_id": 10, "het_job_offset": 0, "derived_exit_code_status": 0, "derived_exit_code_signal": 0, "exit_code_status": 0, "exit_code_signal": 0, "node_details": [ { "name": "MpiNG-abcdef01-2345-1", "instance_id": "i-0abc234def56789", "instance_type": "t4g.micro" } ] } }
{ "resource_id": "pcs_abc123de45", "resource_type": "PCS_CLUSTER", "event_timestamp": 1750374744, "scheduler_type": "slurm", "scheduler_major_version": "25.11", "fields": { "job_id": 11, "user": "root", "user_id": 0, "group": "root", "group_id": 0, "name": "wrap", "job_state": "COMPLETED", "partition": "MpiQueue-abcdef01-7", "time_limit": "UNLIMITED", "start_time": "2025-06-19T23:12:24", "end_time": "2025-06-19T23:12:24", "node_list": "MpiNG-abcdef01-2345-2", "node_cnt": 1, "proc_cnt": 1, "work_dir": "/root", "reservation_name": "", "tres": { "cpu": 1, "mem": { "val": 600, "unit": "M" }, "node": 1, "billing": 1 }, "account": "", "qos": "", "wc_key": "", "cluster": "unknown", "submit_time": "2025-06-19T23:12:14", "eligible_time": "2025-06-19T23:12:14", "het_job_id": 10, "het_job_offset": 1, "derived_exit_code_status": 0, "derived_exit_code_signal": 0, "exit_code_status": 0, "exit_code_signal": 0, "node_details": [ { "name": "MpiNG-abcdef01-2345-2", "instance_id": "i-0def345abc67890", "instance_type": "t4g.micro" } ] } }
{ "resource_id": "pcs_abc123de45", "resource_type": "PCS_CLUSTER", "event_timestamp": 1750376877, "scheduler_type": "slurm", "scheduler_major_version": "25.11", "fields": { "job_id": 13, "user": "root", "user_id": 0, "group": "root", "group_id": 0, "name": "wrap", "job_state": "COMPLETED", "partition": "MpiQueue-abcdef01-7", "time_limit": "UNLIMITED", "start_time": "2025-06-19T23:47:57", "end_time": "2025-06-19T23:47:58", "node_list": "MpiNG-abcdef01-2345-1", "node_cnt": 1, "proc_cnt": 1, "work_dir": "/root", "reservation_name": "", "tres": { "cpu": 1, "mem": { "val": 972, "unit": "M" }, "node": 1, "billing": 1 }, "account": "", "qos": "", "wc_key": "", "cluster": "unknown", "submit_time": "2025-06-19T23:43:56", "eligible_time": "2025-06-19T23:43:56", "array_job_id": 12, "array_task_id": 1, "derived_exit_code_status": 0, "derived_exit_code_signal": 0, "exit_code_status": 0, "exit_code_signal": 0, "node_details": [ { "name": "MpiNG-abcdef01-2345-1", "instance_id": "i-0abc345def67890", "instance_type": "t4g.micro" } ] } }
{ "resource_id": "pcs_abc123de45", "resource_type": "PCS_CLUSTER", "event_timestamp": 1750376878, "scheduler_type": "slurm", "scheduler_major_version": "25.11", "fields": { "job_id": 12, "user": "root", "user_id": 0, "group": "root", "group_id": 0, "name": "wrap", "job_state": "COMPLETED", "partition": "MpiQueue-abcdef01-7", "time_limit": "UNLIMITED", "start_time": "2025-06-19T23:47:58", "end_time": "2025-06-19T23:47:58", "node_list": "MpiNG-abcdef01-2345-1", "node_cnt": 1, "proc_cnt": 1, "work_dir": "/root", "reservation_name": "", "tres": { "cpu": 1, "mem": { "val": 972, "unit": "M" }, "node": 1, "billing": 1 }, "account": "", "qos": "", "wc_key": "", "cluster": "unknown", "submit_time": "2025-06-19T23:43:56", "eligible_time": "2025-06-19T23:43:56", "array_job_id": 12, "array_task_id": 2, "derived_exit_code_status": 0, "derived_exit_code_signal": 0, "exit_code_status": 0, "exit_code_signal": 0, "node_details": [ { "name": "MpiNG-abcdef01-2345-1", "instance_id": "i-0abc345def67890", "instance_type": "t4g.micro" } ] } }