Problemy Peredachi Informatsii
RUS  ENG    JOURNALS   PEOPLE   ORGANISATIONS   CONFERENCES   SEMINARS   VIDEO LIBRARY   PACKAGE AMSBIB  
General information
Latest issue
Archive
Impact factor
Guidelines for authors

Search papers
Search references

RSS
Latest issue
Current issues
Archive issues
What is RSS


Probl. Peredachi Inf.:
Year:
Volume:
Issue:
Page:
Find




Personal entry:
Login:
Password:
Save password
Enter
Forgotten password?
Register

Problemy Peredachi Informatsii, 2020, Volume 56, Issue 1, Pages 15–25
DOI: https://doi.org/10.31857/S0555292320010027 (Mi ppi2308) 		 
Information Theory

Entropy and compression: a simple proof of an inequality of Khinchin–Ornstein–Shields

R. Aragonaa, F. Marzia, F. Mignosiab, M. Spezialettic

a DISIM, University of L'Aquila, Coppito, L'Aquila, Italy
b ICAR-CNR, Palermo, Italy
c University of Naples “Federico II,” Napoli, Italy
Full-text PDF (171 kB)
References:
PDF
HTML
DOI: https://doi.org/10.31857/S0555292320010027
Abstract: This paper concerns the folklore statement that “entropy is a lower bound for compression.” More precisely, we derive from the entropy theorem a simple proof of a pointwise inequality first stated by Ornstein and Shields and which is the almost-sure version of an average inequality first stated by Khinchin in 1953. We further give an elementary proof of the original Khinchin inequality, which can be used as an exercise for information theory students, and we conclude by giving historical and technical notes of such inequality.
Keywords: ergodic sources, entropy, lossless data compression, one-to-one code sequence, Shannon–McMillan theorem.
Received: 12.12.2019
Revised: 08.01.2020
Accepted: 15.01.2020
English version:
Problems of Information Transmission, 2020, Volume 56, Issue 1, Pages 13–22
DOI: https://doi.org/10.1134/S0032946020010020
Bibliographic databases:
Document Type: Article
UDC: 621.391.1 : 519.72
Language: Russian
Citation: R. Aragona, F. Marzi, F. Mignosi, M. Spezialetti, “Entropy and compression: a simple proof of an inequality of Khinchin–Ornstein–Shields”, Probl. Peredachi Inf., 56:1 (2020), 15–25; Problems Inform. Transmission, 56:1 (2020), 13–22
Citation in format AMSBIB
\Bibitem{AraMarMig20}
\by R.~Aragona, F.~Marzi, F.~Mignosi, M.~Spezialetti
\paper Entropy and compression: a simple proof of an inequality of Khinchin--Ornstein--Shields
\jour Probl. Peredachi Inf.
\yr 2020
\vol 56
\issue 1
\pages 15--25
\mathnet{http://mi.mathnet.ru/ppi2308}
\crossref{https://doi.org/10.31857/S0555292320010027}
\transl
\jour Problems Inform. Transmission
\yr 2020
\vol 56
\issue 1
\pages 13--22
\crossref{https://doi.org/10.1134/S0032946020010020}
\isi{https://gateway.webofknowledge.com/gateway/Gateway.cgi?GWVersion=2&SrcApp=Publons&SrcAuth=Publons_CEL&DestLinkType=FullRecord&DestApp=WOS_CPL&KeyUT=000526343800002}
\scopus{https://www.scopus.com/record/display.url?origin=inward&eid=2-s2.0-85083504728}
Linking options:
  • https://www.mathnet.ru/eng/ppi2308
  • https://www.mathnet.ru/eng/ppi/v56/i1/p15
    • Citing articles in Google Scholar: Russian citations, English citations
    Related articles in Google Scholar: Russian articles, English articles
    		Проблемы передачи информации Problems of Information Transmission
    Statistics & downloads:
    Abstract page:168
    Full-text PDF :35
    References:29
    First page:14
     
      Contact us:
    		  Terms of Use  Registration to the website  Logotypes © Steklov Mathematical Institute RAS, 2024