Line metric for the entropy estimation

The problem of the estimation of the entropy of a stationary process $\mu$ is considered. A new metric is constructed for the nonparametric entropy estimator. It is shown that the estimator converges almost surely and its variance is upper-bounded by $\mathcal O(n^{-1})$ for a large class of stationary ergodic processes with a finite state space. For the class of the symmetric Bernoulli measures an explicit formula for the estimator bias is obtained.