Make Inferences on Your Device

Download your compiled model from Amazon S3 to your device and extract it from the compressed tarfile.

# Download compiled model locally to edge device
object_path = f'output/{model_name}-{target_device}.tar.gz'
neo_compiled_model = f'compiled-{model_name}.tar.gz'
s3_client.download_file(bucket_name, object_path, neo_compiled_model)

# Extract model from .tar.gz so DLR can use it
!mkdir ./dlr_model # make a directory to store your model (optional)
!tar -xzvf ./compiled-detect.tar.gz --directory ./dlr_model

Import DLR and an initialized DLRModel object.

import dlr

device = 'cpu'
model = dlr.DLRModel('./dlr_model', device)

Download an image for inferencing and format it based on how your model was trained.

For the coco_ssd_mobilenet example, you can download an image from the COCO dataset and then reform the image to 300x300:

from PIL import Image

# Download an image for model to make a prediction
input_image_filename = './input_image.jpg'
!curl https://farm9.staticflickr.com/8325/8077197378_79efb4805e_z.jpg --output {input_image_filename}

# Format image so model can make predictions
resized_image = image.resize((300, 300))

# Model is quantized, so convert the image to uint8
x = np.array(resized_image).astype('uint8')

Use DLR to make inferences.

Finally, you can use DLR to make a prediction on the image you just downloaded:

out = model.run(x)