Approximate estimation using the accelerated maximum entropy method. Part 1. Problem statement and implementation for the regression problem

The work is devoted to the development of an entropy estimation method with “soft” randomization for restoring the parameters of probabilistic mathematical models from the available observations. Soft randomization refers to the technique of adding regularization to the information entropy functional to simplify the optimization problem and speed up learning process compared to the traditional maximum entropy method. In this work, the concept of the soft randomization entropy estimation method was developed, including obtaining entropy-optimal PDF functions in general form. During the experiments, several types of model regularization were tested on the example of a classical regression analysis problem.