|
This article is cited in 1 scientific paper (total in 1 paper)
Information Theory
The $f$-divergence and coupling of probability distributions
V. V. Prelov Kharkevich Institute for Information Transmission Problems,
Russian Academy of Sciences, Moscow, Russia
Abstract:
We consider the problem of finding the minimum and maximum values of $f$-divergence for discrete probability distributions $P$ and $Q$ provided that one of these distributions and the value of their coupling are given. An explicit formula for the minimum value of the $f$-divergence under the above conditions is obtained, as well as a precise expression for its maximum value. This precise expression is not explicit in the general case, but in many special cases it allows us to write out both explicit formulas and simple upper bounds, which are sometimes optimal. Similar explicit formulas and upper bounds are also obtained for the Kullback–Leibler and $\chi^2$ divergences, which are the most important cases of the $f$-divergence.
Keywords:
$f$-divergence, Kullback–Leibler divergence, $\chi^2$ divergence, coupling of discrete probability distributions.
Received: 17.11.2020 Revised: 04.01.2021 Accepted: 11.01.2021
Citation:
V. V. Prelov, “The $f$-divergence and coupling of probability distributions”, Probl. Peredachi Inf., 57:1 (2021), 64–80; Problems Inform. Transmission, 57:1 (2021), 54–69
Linking options:
https://www.mathnet.ru/eng/ppi2335 https://www.mathnet.ru/eng/ppi/v57/i1/p64
|
Statistics & downloads: |
Abstract page: | 142 | Full-text PDF : | 6 | References: | 17 | First page: | 10 |
|