Fisher Information

The Fisher information in an observation about a scalar parameter is the variance of the score function. If has probability density function , it is

where denotes the expectation value of under the distribution with parameter . Under regularity conditions that permit differentiation under the integral sign, this can also be written as

Fisher information is additive for independent observations. Thus, independent observations from the same distribution have information . The reciprocal of the information gives the Cramér-Rao bound on the variance of an unbiased estimator under corresponding regularity conditions.

A sharp bound for Boolean encoders of correlated binary data is the Amari-Kobayashi conjecture, proved by Chen et al. (2026).