本文為英文版的機器翻譯版本，如內容有任何歧義或不一致之處，概以英文版為準。

Neuron 任務定義範例

受管裝置配置範例

下列範例顯示使用 resourceRequirements 參數請求執行個體上所有 Neuron 裝置的任務定義。此方法僅適用於受管執行個體。

{
    "family": "ecs-neuron",
    "requiresCompatibilities": ["MANAGED_INSTANCES"],
    "networkMode": "awsvpc",
    "cpu": "8192",
    "memory": "16384",
    "executionRoleArn": "${YOUR_EXECUTION_ROLE}",
    "containerDefinitions": [
        {
            "name": "neuron-inference",
            "image": "763104351884.dkr.ecr.us-east-1.amazonaws.com/huggingface-vllm-inference-neuronx:0.11.0-optimum0.4.5-neuronx-py310-sdk2.26.1-ubuntu22.04",
            "essential": true,
            "command": [
                "--model", "${YOUR_HUGGING_FACE_MODEL_ID}",
                "--port", "8080",
                "--tensor-parallel-size", "2",
                "--allow-non-cached-model"
            ],
            "portMappings": [
                {
                    "containerPort": 8080,
                    "protocol": "tcp"
                }
            ],
            "resourceRequirements": [
                {
                    "type": "NeuronDevice",
                    "value": "ALL"
                }
            ]
        }
    ]
}

在此範例中，容器映像包含針對 AWS Neuron 最佳化的 vLLM 推論伺服器。影像的進入點會從 HuggingFace 下載模型、為 Neuron 編譯模型，並在連接埠 8080 上啟動與 OpenAI 相容的 API 伺服器。${YOUR_HUGGING_FACE_MODEL_ID} 將 取代為您的 HuggingFace 模型 ID。

手動裝置規格範例

下列範例顯示搭配 inf1.xlarge使用 EC2 啟動類型的 Linux 任務定義linuxParameters.devices，以指定 Neuron 裝置路徑。

{
    "family": "ecs-neuron",
    "requiresCompatibilities": ["EC2"],
    "placementConstraints": [
        {
            "type": "memberOf",
            "expression": "attribute:ecs.os-type == linux"
        },
        {
            "type": "memberOf",
            "expression": "attribute:ecs.instance-type == inf1.xlarge"
        }
    ],
    "executionRoleArn": "${YOUR_EXECUTION_ROLE}",
    "containerDefinitions": [
        {
            "entryPoint": [
                "/usr/local/bin/entrypoint.sh",
                "--port=8500",
                "--rest_api_port=9000",
                "--model_name=resnet50_neuron",
                "--model_base_path=s3://amzn-s3-demo-bucket/resnet50_neuron/"
            ],
            "portMappings": [
                {
                    "hostPort": 8500,
                    "protocol": "tcp",
                    "containerPort": 8500
                },
                {
                    "hostPort": 8501,
                    "protocol": "tcp",
                    "containerPort": 8501
                },
                {
                    "hostPort": 0,
                    "protocol": "tcp",
                    "containerPort": 80
                }
            ],
            "linuxParameters": {
                "devices": [
                    {
                        "containerPath": "/dev/neuron0",
                        "hostPath": "/dev/neuron0",
                        "permissions": [
                            "read",
                            "write"
                        ]
                    }
                ],
                "capabilities": {
                    "add": [
                        "IPC_LOCK"
                    ]
                }
            },
            "cpu": 0,
            "memoryReservation": 1000,
            "image": "763104351884.dkr.ecr.us-east-1.amazonaws.com/tensorflow-inference-neuron:1.15.4-neuron-py37-ubuntu18.04",
            "essential": true,
            "name": "resnet50"
        }
    ]
}