Blogs and Case Studies PyTorch example notebooks

Amazon SageMaker AI model parallelism library v2 examples

This page provides a list of blogs and Jupyter notebooks that present practical examples of implementing the SageMaker model parallelism (SMP) library v2 to run distributed training jobs on SageMaker AI.

Blogs and Case Studies

The following blogs discuss case studies about using SMP v2.

Amazon SageMaker AI model parallel library now accelerates PyTorch FSDP workloads by up to 20%

PyTorch example notebooks

Example notebooks are provided in the SageMaker AI examples GitHub repository. To download the examples, run the following command to clone the repository and go to training/distributed_training/pytorch/model_parallel_v2.

Note

Clone and run the example notebooks in the following SageMaker AI ML IDEs.

SageMaker JupyterLab (available in Studio created after December 2023)
SageMaker Code Editor (available in Studio created after December 2023)
Studio Classic (available as an application in Studio created after December 2023)
SageMaker Notebook Instances


git clone https://github.com/aws/amazon-sagemaker-examples.git
cd amazon-sagemaker-examples/training/distributed_training/pytorch/model_parallel_v2

SMP v2 example notebooks

Warning Javascript is disabled or is unavailable in your browser.

To use the Amazon Web Services Documentation, Javascript must be enabled. Please refer to your browser's Help pages for instructions.

Document Conventions

Checkpointing using SMP

Best practices