NeuE: Automated Neural Network Ensembles for Edge Intelligence

Abstract

Artificial Intelligence (AI) applications have been established in the mobile industry and are decisively determining the progress in entrepreneurial value creation. This paper explores the potential of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Edge Computing</i> to enhance the performance of AI applications. In particular, a DNN ensemble formation (DEF) problem is studied which judiciously recruits members for DNN ensembles considering the device heterogeneity, computing resource limitation, and service deadline of edge computing systems, in an attempt to optimize the performance of edge AI services. We design a novel algorithm called Neural Ensemble (NeuE) to solve the DEF problem. NeuE involves an online learning process that learns the in-practice performance of DNN ensembles and adaptively forms DNN ensembles according to the features of admitted tasks. It leverages the framework of contextual multi-armed bandit and follows the constraints of computing resource limitation and service deadline. We also show theoretically that NeuE provides asymptotic optimality. However, NeuE suffers from poor scalability due to exponentially-growing ensemble decision space. We then propose a variant of NeuE, called NeuE-S, to expedite NeuE. NeuE-S identifies representative ensemble decisions using similarities of ensemble decisions and carries out learning with a reduced decision space. We show via theoretical analysis that NeuE-S drastically reduces the computation complexity with negligible performance loss. We implement our method on an edge computing testbed. The results show that our method dramatically improves the performance of edge AI services.