Optional: Configuring your device for ML qualification

SSH into your device.

Run the following scripts to verify that the dependencies are correctly installed.

sudo python3.7 -c "import mxnet; print(mxnet.__version__)"

sudo python3.7 -c "import numpy; print(numpy.__version__)"

The output prints the version number and the script should exit without error.

Make sure that device.json is configured for ML qualification. For more information, see Configure device.json for ML qualification.

Run the dependencies test for the framework.

devicetester_[linux | mac | win_x86-64] run-suite --group-id mldependencies --test-id mxnet_dependency_check

The test summary displays a PASSED result for mldependencies.

SSH into your device.

Run the following script to verify that the dependency is correctly installed.

sudo python3.7 -c "import tensorflow; print(tensorflow.__version__)"

The output prints the version number and the script should exit without error.

Make sure that device.json is configured for ML qualification. For more information, see Configure device.json for ML qualification.

Run the dependencies test for the framework.

devicetester_[linux | mac | win_x86-64] run-suite --group-id mldependencies --test-id tensorflow_dependency_check

The test summary displays a PASSED result for mldependencies.

SSH into your device.

Run the following scripts to verify that the dependencies are correctly installed.

sudo python3.7 -c "import dlr; print(dlr.__version__)"

sudo python3.7 -c "import numpy; print(numpy.__version__)"

The output prints the version number and the script should exit without error.

Make sure that device.json is configured for ML qualification. For more information, see Configure device.json for ML qualification.

Run the dependencies test for the framework.

devicetester_[linux | mac | win_x86-64] run-suite --group-id mldependencies --test-id dlr_dependency_check

The test summary displays a PASSED result for mldependencies.

Verify that your device type is supported by SageMaker. For more information, see the target device options the Amazon SageMaker API Reference. If your device type is not currently supported by SageMaker, follow the steps in Option 2: Use TVM to compile the DLR model.

Download the tarball file that contains the uncompiled, pretrained MXNet model for DLR:

Decompress the tarball. This command generates the following directory structure.

Move synset.txt out of the resnet18 directory. Make a note of the new location. You copy this file to compiled model directory later.

Compress the contents of the resnet18 directory.

tar cvfz model.tar.gz resnet18v1-symbol.json resnet18v1-0000.params

Upload the compressed file to an Amazon S3 bucket in your AWS account, and then follow the steps in Compile a Model (Console) to create a compilation job.

1. For Input configuration, use the following values:

   • For Data input configuration, enter {"data": [1, 3, 224, 224]}.

   • For Machine learning framework, choose MXNet.

2. For Output configuration, use the following values:

   • For S3 Output location, enter the path to the Amazon S3 bucket or folder where you want to store the compiled model.

   • For Target device, choose your device type.

Download the compiled model from the output location you specified, and then unzip the file.

Copy synset.txt into the compiled model directory.

Change the name of the compiled model directory to resnet18.

Your compiled model directory must have the following directory structure.

Download the tarball file that contains the uncompiled, pretrained MXNet model for DLR:

Decompress the tarball. This command generates the following directory structure.

Follow the instructions in the TVM documentation to build and install TVM from source for your platform.

After TVM is built, run the TVM compilation for the resnet18 model. The following steps are based on Quick Start Tutorial for Compiling Deep Learning Models in the TVM documentation.

1. Open the relay_quick_start.py file from the cloned TVM repository.

2. Update the code that defines a neural network in relay. You can use one of following options:

   • Option 1: Use mxnet.gluon.model_zoo.vision.get_model to get the relay module and parameters:

     from mxnet.gluon.model_zoo.vision import get_model
     block = get_model('resnet18_v1', pretrained=True)
     mod, params = relay.frontend.from_mxnet(block, {"data": data_shape})

   • Option 2: From the uncompiled model that you downloaded in step 1, copy the following files to the same directory as the relay_quick_start.py file. These files contain the relay module and parameters.

     • resnet18v1-symbol.json

     • resnet18v1-0000.params

3. Update the code that saves and loads the compiled module to use the following code.

   from tvm.contrib import util
   path_lib = "deploy_lib.so"
   #  Export the model library based on your device architecture
   lib.export_library("deploy_lib.so", cc="aarch64-linux-gnu-g++")
   with open("deploy_graph.json", "w") as fo:
       fo.write(graph)
   with open("deploy_param.params", "wb") as fo:
       fo.write(relay.save_param_dict(params))

4. Build the model:

   python3 tutorials/relay_quick_start.py --build-dir ./model

   This command generates the following files.

   • deploy_graph.json

   • deploy_lib.so

   • deploy_param.params

Copy the generated model files into a directory named resnet18. This is your compiled model directory.

Copy the compiled model directory to your host computer. Then copy synset.txt from the uncompiled model that you downloaded in step 1 into the compiled model directory.

Your compiled model directory must have the following directory structure.