Fisher Loss: A More Discriminative Feature Learning Method in Classification

Abstract

Convolution neural networks(CNN) and softmax loss(cross entropy) function have been widely used in various classification tasks. It achieves better performance than MSE and other loss. However in most classification tasks, because of the high dimension, excessive noise of input samples, strong coupling of the extracted features, as well as the complicated output-input projection relationship, the features extracted by CNN have large intra-class variance and small inter-class distance, so different class of samples cannot be distinguished well, which will greatly reduce the accuracy of the classification. Aiming at these problems, we combine the Fisher discrimination criterion with the neural network and propose a more discriminative feature learning algorithm, fisher loss. In the experiment, the proposed algorithm is compared with the center loss, island loss, and other algorithms on MNIST datasets with two commonly used network architecture. It is found that our algorithm can better distinguish features and achieve higher classification accuracy.