Ordering the set of neural network parameters

This paper investigates a method for setting order on a set of the model parameters. It considers linear models and neural networks. The set is ordered by the covariance matrix of the gradients. It is proposed to use a given order to freeze the model parameters during the optimization procedure. It is assumed that, after few iterations of the optimization algorithm, most of the model parameters can be frozen without significant loss of the model quality. It reduces the dimensionality of the optimization problem. This method is analyzed in the computational experiment on the real data. The proposed order is compared with the random order on the set of the model parameters.