Feature selection method based on a probabilistic approach and cross-entropy metric for image recognition problem

The paper considers the problem of feature selection in the classification problem. A method for selecting informative features based on a probabilistic approach and cross-entropy metrics is proposed. Several variants of the information criterion for selecting features for a binary classification problem are considered, as well as its generalization to the case of a multiclass problem. Demonstration examples of the proposed method for the task of image recognition from the mnist collection are given.