Real-time edge computing on multi-processes and multi-threading architectures for deep learning applications

Abstract

As the computing power of embedded system hardware devices continues to grow, more and more deep learning models have been gradually transplanted into edge devices. Accordingly, a variety of application scenarios have been developed with more complex inference models or multiple models that work together. Moreover, with consideration given to cost, a single edge computing device can integrate multiple input sources and simultaneously complete the application of multiple scenarios. To achieve good performance on edge computing devices, this paper probes into the software architecture design of multi-processes and multi-threading architectures on edge computing device, in order to realize real-time edge computing. In the experiment, multiple face-related deep learning models, namely, face detection, face recognition, age estimation, gender estimation, and emotion estimation, are used to demonstrate the differences between multi-processes and multi-threading on edge computing devices. According to the experimental results, it is known that if the number of central processing unit (CPU) cores and memory space are small, the multi-threading architecture can better improve efficiency; conversely, the multi-processes architecture can be used. The architecture proposed in this paper has reference value, and can improve the execution efficiency of deep learning technology on edge computing, and reduce the cost of deployment.