Embedded sensory data memory optimization for IoT edge inference with privacy, accuracy, and energy efficiency

Abstract

In recent years, deep learning is transforming many modern applications. For example, deep learning has demonstrated exceptional performance in disease diagnosis of brain disorders and various forms of cancers due to the availability of a large amount of patients' data. Meanwhile, with the advent of wearable technologies and Internet of Things (IoT), there is a rising interest in providing personalized experience with health recommender systems. Such learning-enabled benefit, however, does come with its own cost, such as associated serious privacy concerns. Sharing personal data carries inherent risks to individual privacy. Due to the substantial requirements for computation and storage resources, today's deep learning systems are typically built upon large, centralized data repositories. Based on this centralized-training paradigm, data owners need to upload their private data to the provider and do not have control over how their private data is being used. To protect privacy, one popular technique is differentially private deep learning algorithms, which add random noise to the computation so that the output does not significantly depend on any particular training sample. When introducing noise, the privacy-guarantee comes at the cost of compromising the inference accuracy of the systems. In addition to the privacy and accuracy design considerations, as the deep learning networks grow, extensive memory accesses consume over 50% of total system power and have become a major constraint to the resource-limited IoT devices. As shown in Fig. 1, during the IoT edge inference process, memory traffic mainly contains two components: (i) synaptic weight storage for neural network models and (ii) sensory data memory to store IoT sensory signals. To reduce the memory pressure, state-of-the art deep learning systems have adopted embedded memory (e.g., exploiting data reuse or designing approximate memories) to store synaptic weight to eliminate off-chip memory accesses. Accordingly, the sensory data storage become dominant for external memory traffic. Particularly, given the number of IoT devices will continue to rapidly increase, the resulting rapid explosion of collected sensory data brings huge pressure for data storage. Consequently, power-efficient embedded sensory data memory is one of the key design considerations for IoT edge inference. This work brings memory hardware optimization to meet the tight power budget in IoT edge devices by considering the privacy, accuracy, and power efficiency tradeoff in differentially efficient deep learning systems. Based on a detailed analysis on these characteristics, mathematical models are developed to optimize the sensory data quality, thereby enabling the optimal power efficiency and inference accuracy with privacy guarantee. Our simulation results show that the proposed technique can enable near-threshold energy-efficient memory operation for different privacy requirements, with less than 1% degradation in classification accuracy.