The softmax function maps a real vector to a vector of positive
components that sum to 1. For
, its components are
Every component is positive and the components sum to 1. Adding the same constant to every component of
leaves the result unchanged. For
, the softmax function can be
expressed using the sigmoid function. It is widely
used to convert a collection of real-valued scores into probabilities for a categorical
distribution.