|
Applied mathematics
On the infuence of the cental trend on the nature of the density distribution of maximum entropy in machine learning
A. V. Kvasnova, A. A. Baranenkob, E. Yu. Butyrskyc, U. P. Zaranikc a Peter the Great St. Petersburg Polytechnic University, 29, Polytekhnicheskaya ul., St. Petersburg, 195251, Russian Federation
b Military Educational and Scientific Center of the Navy “Naval Medical Academy’’ named after N. G. Kuznetsov, 17/1, Ushakovskaia nab., St. Petersburg, 197025, Russian Federation
c St. Petersburg State University, 7–9, Universitetskaya nab., St. Petersburg, 199034, Russian Federation
Abstract:
The principle of maximum entropy (ME) has a number of advantages that allow it to be used in machine learning. The density distribution of maximum entropy (WEO) requires solving the problem of calculus of variations on the a priori distribution, where the central tendency can be used as a parameter. In Lebesgue space, the central tendency is described by the generalized Gelder average. The paper shows the evolution of the density of the ME distribution depending on the given norm of the average. The minimum Kulback — Leibler divergence between the WEO and the a prior density is achieved at the harmonic mean, which is effective in reducing the dimensionality of the training sample. At the same time, this leads to a deterioration in the function of loss in the conditions of machine learning by precedents.
Keywords:
maximum entropy principle, maximum entropy distribution, central trend, generalized average, machine learning.
Received: March 10, 2023 Accepted: April 25, 2023
Citation:
A. V. Kvasnov, A. A. Baranenko, E. Yu. Butyrsky, U. P. Zaranik, “On the infuence of the cental trend on the nature of the density distribution of maximum entropy in machine learning”, Vestnik S.-Petersburg Univ. Ser. 10. Prikl. Mat. Inform. Prots. Upr., 19:2 (2023), 176–184
Linking options:
https://www.mathnet.ru/eng/vspui575 https://www.mathnet.ru/eng/vspui/v19/i2/p176
|
|