Explainability report

Amazon SageMaker Autopilot provides an explainability report to help explain how a best model candidate makes predictions for image classification problems. This report can assist ML engineers, product managers, and other internal stakeholders in understanding the characteristics of the model. Both consumers and regulators rely on transparency in machine learning to trust and interpret decisions made on model predictions. You can use these explanations for auditing and meeting regulatory requirements, establishing trust in the model, supporting human decision-making, and debugging and improving model performance.

The Autopilot explanatory functionality for image classification uses a visual class activation map (CAM) approach that produces a heatmap where the distribution and intensity of each color highlights the areas of an image that contribute the most to a specific prediction. This approach relies on principal components derived from an implementation of Eigen-CAM.

Autopilot generates the explainability report as a JSON file. The report includes analysis details that are based on the validation dataset. Each image used to generate the report contains the following information:

You can find the Amazon S3 prefix to the explainability artifacts generated for the best candidate in the response to DescribeAutoMLJobV2 at BestCandidate.CandidateProperties.CandidateArtifactLocations.Explainability.

The following examples illustrates what the heatmaps look like on few samples from Oxford-IIIT Pet Dataset. The heatmap image displays color gradients that indicate the relative importance of different features within the image. The red color represents regions with greater importance in predicting the "predicted_label" of the input image compared to the features represented by the blue color.

Input Image	Heatmap Image