An Ordered Aggregation-Based Ensemble Selection Method of Lightweight Deep Neural Networks With Random Initialization

Abstract

Due to the popularity of 5G connectivity and The Internet of Things sensors, deep learning algorithms are being extended to edge devices. Compared with AI(Artificial Intelligence) cloud platforms, the deployment of deep neural networks on edge devices must focus on low power consumption, low latency, stability and reliability. In recent years, the development of lightweight deep neural network architecture has provided a basis for the deployment of deep neural networks on edge devices. However, the shortcomings of deep neural networks, such as overconfidence, vulnerability to adversarial attack, and easy over fitting when samples are insufficient, still limit their applications in many fields. One of the ways to compensate for these defects is to use deep ensemble. An ordered aggregation-based ensemble selection algorithm is proposed, which uses soft-margin as the importance assessment metric to take full advantage of the diversity and complementarity of lightweight deep neural networks obtained from different initialization training, so as to improve the overall performance of multiple edge devices. The experimental results show that this algorithm has a significant improvement in generalization performance compared with random ensemble and ordered aggregation algorithms based on accuracy or diversity, and provides a new complementary idea for the deployment of lightweight deep neural networks on edge devices.