|
Teoriya Veroyatnostei i ee Primeneniya, 1964, Volume 9, Issue 1, Pages 154–157
(Mi tvp355)
|
|
|
|
Short Communications
On Estimates for the Entropy of a Language According to Shannon
A. P. Savčuk Moscow
Abstract:
C. E. Shannon proposed the upper and lower estimates for the entropy of a language. It is proved in this paper that in order to attain the lower estimate, it is necessary and sufficient that some letters be equally probable, the probability of the rest being equal to zero after any combination $b_i^N$ of $N$ letters such that $p(b_i^{N-1})>0$. In order to attain the upper estimate, it is necessary and sufficient that the probability that the $k$-th letter appears after $b_i^N$ be dependent on $k$ and $N$, and independent of $i$, for a sequence of language letters arranged in descending order as to the probability that they appear after $b_i^N$ (note that the arrangement of letters in this sequence, however, depends on $i$). In the latter result it is supposed that no letter occurs with the probability $=1$, which is true for every real language.
Received: 20.02.1962
Citation:
A. P. Savčuk, “On Estimates for the Entropy of a Language According to Shannon”, Teor. Veroyatnost. i Primenen., 9:1 (1964), 154–157; Theory Probab. Appl., 9:1 (1964), 138–141
Linking options:
https://www.mathnet.ru/eng/tvp355 https://www.mathnet.ru/eng/tvp/v9/i1/p154
|
Statistics & downloads: |
Abstract page: | 369 | Full-text PDF : | 189 | First page: | 5 |
|