Asymptotics of the Shannon and Renyi Entropies for Sums of Independent Random Variables

We investigate the asymptotics as $n\to\infty$ of the Shannon and Renyi entropies for sums $\zeta_n=\xi_1+\dots +\xi_n$, where all xi are independent identically distributed (i.i.d.) random variables. We consider the cases of discrete and absolutely continuous distributions of $\xi_i$. If $0<\mathbf D\xi_i<\infty$, then we find the dominant part of the asymptotics. Under the additional condition $\mathbf E|\xi_i|^N<\infty$ for an integer $N\geq 3$, we construct the expansion of the entropies in powers of $n$ with the remainder term $\overline{o}\biggl (n^{-\frac{N-2}{2}}\biggr)$. The coefficients of this expansion depend on the semi-invariants of $\xi_i$. Proofs are performed by using local limit theorems. As examples, we construct several first coefficients for Poisson, binomial, and geometric distributions. This paper improves the previous results [Fundamentalnaya Prikl. Mat., 2, No. 4, 1019–1028 (1996)].