Classification accuracy improvement of the optical diffractive deep neural network by employing a knowledge distillation and stochastic gradient descent β-Lasso joint training framework

Abstract

Optical diffractive deep neural network (OD2NN) is an optical machine learning framework that utilizes diffraction on the cascaded diffractive surfaces to perform an arbitrary function. Compared with the deep neural networks (DNNs) implemented in electronic domain, proof-of-principle demonstrations of OD2NNs show promising advantages in terms of speed and power efficiency. However, the classification accuracy of the demonstrated OD2NN has been limited by the absence of optical nonlinear operations, even in the hybrid OD2NNs which are integrated with electronic neural networks. Here, we propose a novel training framework to improve the classification accuracy of the OD2NNs without employing any nonlinear physical elements. In this framework, the hybrid OD2NN with a fully connected electronic layer integrated (hybrid) are preferred and knowledge distillation (KD) and stochastic gradient descent β-Lasso (SGD-β-Lasso) joint-training are used. A blind testing classification accuracy of 70.19% and 85.17% have been obtained for Cifar-10 and Cats vs. Dogs dataset, respectively, which is the state-of-the-art accuracy achieved by the hybrid OD2NN so far. In addition, the proposed framework can significantly reduce the complexity of hardware fabrication and layers alignment since the hybrid OD2NN only consists of 5 diffractive layers. This work take a big step forward the application of the OD2NN in realistic scenarios.