A biased estimator is an estimator which exhibits estimator bias.
Bias can arise when the observations included in an estimate are selected using the observed sequence. For example, consider three independent
fair coin tosses, with heads and tails denoted H
and T. Select each of the second and third tosses whose immediate predecessor is
H. Let
be the number selected and
the number of selected heads. When
, use the empirical heads proportion
.
Of the eight equally likely sequences, TTT and TTH have and are excluded. On the other six,
takes the value 0 three times, 1 twice, and
once. Its conditional
expectation is therefore
Thus its conditional estimator bias for estimating the underlying heads probability is
(Miller and Sanjurjo 2018, Numberphile 2018).
Pooling the selected outcomes over the complete table of eight sequences instead gives four heads among eight selected tosses, a proportion of . This weights the sequences by their numbers of selected
tosses, rather than giving each eligible sequence equal weight, and is a different
statistic.