Deep Learning Models for Audio Processing Applications Under Resource-Constrained Devices: A Survey

Abstract

In the recent few years, integrating deep learning models into resource-constrained consumer electronic devices for use in applications ranging from simple personal assistants to clinical decision-making has gained more attention both in industry and academia. This is partially due to the advent of consumer electronic devices such as smartphones and the impressive capability of deep learning (DL) models in learning highly complex problems. However, the processing and memory capacity of consumer devices such as smartphones, smartwatches, and other lightweight embedded systems fall short in handling a computation involving millions of parameters of the DL models. As a result, devising techniques for squeezing DL models to reduce the computational complexity and fit into the capacity of resource-constrained devices (RCDs) has been a topic of active research. This work presented DL architectures commonly used for audio processing tasks on RCDs. Also, the model compression techniques proposed in the literature to squeeze the baseline models have been investigated. The reported results from the proposed architectures and model squeezing techniques indicate that a compressed model with fewer parameters can be achieved without or with minimal loss of accuracy compared to the baseline deep learning model. This survey work also investigated challenges in deploying DL models in RCDs. These challenges include the limitation for performing on-device model training, lack of sufficient datasets for model training, difficulty in model interpretability, and lack of robustness.