Concepts in uncertainty

Uncertainty represents the reliability of our inferences. Some statistics that proxy or approximate uncertainty include the softmax probability, predictive variance , and Shannon’s entropy of the softmax vector . This section introduces these statistics and explains what information about our predictions they provide. The section also outlines major pathologies of uncertainty that data scientists should be aware of.