Teoriya Veroyatnostei i ee Primeneniya
RUS  ENG    JOURNALS   PEOPLE   ORGANISATIONS   CONFERENCES   SEMINARS   VIDEO LIBRARY   PACKAGE AMSBIB  
General information
Latest issue
Archive
Impact factor
Guidelines for authors
Submit a manuscript

Search papers
Search references

RSS
Latest issue
Current issues
Archive issues
What is RSS



Teor. Veroyatnost. i Primenen.:
Year:
Volume:
Issue:
Page:
Find






Personal entry:
Login:
Password:
Save password
Enter
Forgotten password?
Register


Teoriya Veroyatnostei i ee Primeneniya, 2021, Volume 66, Issue 3, Pages 601–609
DOI: https://doi.org/10.4213/tvp5442
(Mi tvp5442)
 

This article is cited in 1 scientific paper (total in 1 paper)

Short Communications

On the maximum entropy of a sum of independent discrete random variables

M. Kovačević

Faculty of Technical Sciences, University of Novi Sad, Novi Sad, Serbia
Full-text PDF (348 kB) Citations (1)
References:
Abstract: Let $X_1, \dots, X_n$ be independent random variables taking values in the alphabet $\{0, 1, \dots, r\}$, and let $S_n=\sum_{i=1}^n X_i$. The Shepp–Olkin theorem states that in the binary case (${r=1}$), the Shannon entropy of $S_n$ is maximized when all the $X_i$'s are uniformly distributed, i.e., Bernoulli(1/2). In an attempt to generalize this theorem to arbitrary finite alphabets, we obtain a lower bound on the maximum entropy of $S_n$ and prove that it is tight in several special cases. In addition to these special cases, an argument is presented supporting the conjecture that the bound represents the optimal value for all $n$, $r$, i.e., that $H(S_n)$ is maximized when $X_1, \dots, X_{n-1}$ are uniformly distributed over $\{0, r\}$, while the probability mass function of $X_n$ is a mixture (with explicitly defined nonzero weights) of the uniform distributions over $\{0, r\}$ and $\{1, \dots, r-1\}$.
Keywords: maximum entropy, Bernoulli sum, binomial distribution, Shepp–Olkin theorem, ultra-log-concavity.
Funding agency Grant number
Ministry of Science and Technology of Republic of Srpska 451-03-68/2020-14/200156
EU Framework Programme for Research and Innovation 856967
This work was supported by the European Union's Horizon 2020 research and innovation programme under grant agreement 856967, and by the Ministry of Education, Science and Technological Development of the Republic of Serbia through the project 451-03-68/2020-14/200156.
Received: 15.08.2020
Revised: 04.02.2021
Accepted: 04.02.2021
English version:
Theory of Probability and its Applications, 2021, Volume 66, Issue 3, Pages 482–487
DOI: https://doi.org/10.1137/S0040585X97T99054X
Bibliographic databases:
Document Type: Article
Language: Russian
Citation: M. Kovačević, “On the maximum entropy of a sum of independent discrete random variables”, Teor. Veroyatnost. i Primenen., 66:3 (2021), 601–609; Theory Probab. Appl., 66:3 (2021), 482–487
Citation in format AMSBIB
\Bibitem{Kov21}
\by M.~Kova{\v c}evi\'c
\paper On the maximum entropy of a~sum of independent discrete random variables
\jour Teor. Veroyatnost. i Primenen.
\yr 2021
\vol 66
\issue 3
\pages 601--609
\mathnet{http://mi.mathnet.ru/tvp5442}
\crossref{https://doi.org/10.4213/tvp5442}
\mathscinet{http://mathscinet.ams.org/mathscinet-getitem?mr=4294343}
\zmath{https://zbmath.org/?q=an:1479.60029}
\transl
\jour Theory Probab. Appl.
\yr 2021
\vol 66
\issue 3
\pages 482--487
\crossref{https://doi.org/10.1137/S0040585X97T99054X}
\scopus{https://www.scopus.com/record/display.url?origin=inward&eid=2-s2.0-85129683571}
Linking options:
  • https://www.mathnet.ru/eng/tvp5442
  • https://doi.org/10.4213/tvp5442
  • https://www.mathnet.ru/eng/tvp/v66/i3/p601
  • This publication is cited in the following 1 articles:
    Citing articles in Google Scholar: Russian citations, English citations
    Related articles in Google Scholar: Russian articles, English articles
    Теория вероятностей и ее применения Theory of Probability and its Applications
     
      Contact us:
     Terms of Use  Registration to the website  Logotypes © Steklov Mathematical Institute RAS, 2024