Distilling face recognition models trained using margin-based softmax function

The use of convolutional neural networks trained with the margin-based softmax function allows achieving the highest accuracy in the face recognition problem. The development of embedded systems such as smart intercoms has increased interest in lightweight neural networks. Thus, lightweight neural network models, trained using the margin-based softmax function, were proposed for the face identification problem. In the present paper, we propose a distillation method that allows obtaining greater accuracy than other methods for the face recognition problem on LFW, AgeDB-30, and Megaface datasets. The main idea of our approach is to use the class centers of the teacher network to initialize the student network. Then the student network is trained to produce biometric vectors the angles from which to the class centers are equal to the angles in the teacher network.