Compile a Model (Amazon SageMaker Console) - Amazon SageMaker

Compile a Model (Amazon SageMaker Console)

You can create an Amazon SageMaker Neo compilation job in the Amazon SageMaker console.

  1. In the Amazon SageMaker console, choose Compilation jobs, and then choose Create compilation job.

    Create a compilation job.

  2. On the Create compilation job page, under Job name, enter a name. Then select an IAM role.

    Create compilation job page.

  3. If you don’t have an IAM role, choose Create a new role.

    Create IAM role location.

  4. On the Create an IAM role page, choose Any S3 bucket, and choose Create role.

    Create IAM role page.
  5. Non PyTorch Frameworks

    Within the Input configuration section, enter the full path of the Amazon S3 bucket URI that contains your model artifacts in the Location of model artifacts input field. Your model artifacts must be in a compressed tarball file format (.tar.gz).

    For the Data input configuration field, enter the JSON string that specifies the shape of the input data.

    For Machine learning framework, choose the framework of your choice.

    Input configuration page.

    To find the JSON string examples of input data shapes depending on frameworks, see What input data shapes Neo expects.

    PyTorch Framework

    Similar instructions apply for compiling PyTorch models. However, if you trained with PyTorch and are trying to compile the model for ml_* (except ml_inf) target, you can optionally specify the version of PyTorch you used.

    Example Input configuration section showing where to choose the Framework version.

    To find the JSON string examples of input data shapes depending on frameworks, see What input data shapes Neo expects.

    Notes

    • If you saved your model by using PyTorch version 2.0 or later, the Data input configuration field is optional. SageMaker Neo gets the input configuration from the model definition file that you create with PyTorch. For more information about how to create the definition file, see the PyTorch section under Saving Models for SageMaker Neo.

    • When compiling for ml_* instances using PyTorch framework, use Compiler options field in Output Configuration to provide the correct data type (dtype) of the model’s input. The default is set to "float32".

    Example Output Configuration section.
    Warning

    If you specify a Amazon S3 bucket URI path that leads to .pth file, you will receive the following error after starting compilation: ClientError: InputConfiguration: Unable to untar input model.Please confirm the model is a tar.gz file

  6. Go to the Output configuration section. Choose where you want to deploy your model. You can deploy your model to a Target device or a Target platform. Target devices include cloud and edge devices. Target platforms refer to specific OS, architecture, and accelerators you want your model to run on.

    For S3 Output location, enter the path to the S3 bucket where you want to store the model. You can optionally add compiler options in JSON format under the Compiler options section.

    Output configuration page.

  7. Check the status of the compilation job when started. This status of the job can be found at the top of the Compilation Job page, as shown in the following screenshot. You can also check the status of it in the Status column.

    Compilation job status.

  8. Check the status of the compilation job when completed. You can check the status in the Status column as shown in the following screenshot.

    Compilation job status.