Lightweight Multilayer Random Forests for Monitoring Driver Emotional Status

Mira Jeong,Jaeyeal Nam,Byoung Chul Ko

doi:10.1109/access.2020.2983202

Mira Jeong, Jaeyeal Nam + Show 1 more

Open Access

https://doi.org/10.1109/access.2020.2983202

Copy DOI

Abstract

This study proposes a lightweight multilayer random forest (LMRF) model, which is a non-neural network style deep model consisting of layer-by-layer random forests. Although a deep neural network (DNN) is a powerful algorithm for facial expression recognition (FER), the requirement of too many parameters, careful parameter tuning, a huge amount of training data, black-box models, and a pre-trained architecture remain significant burdens for a current DNN, particularly for real-time processing. To overcome the limitations of a DNN, our FER system uses LMRF consisting of a two-layer structure and a small number of trees per layer for fast FER. The proposed LMRF achieves a similar performance as a DNN even with a small number of hyper-parameters, and a faster processing time using a CPU. We conducted experiments using a benchmark database captured indoors and a real driving database captured using a near-infrared (NIR) camera. Based on a performance evaluation against a few other state-of-the-art FER methods, the proposed method showed a more uniform performance than DNN-based methods, and required a reduced number of parameters and operations without a loss of accuracy when compared to DNN model compression algorithms. As a replacement for deeper and wider networks, the proposed model can be embedded in low-power and low-memory in vehicle systems for the monitoring of a driver's emotion.

Highlights

With the recent increase in interest regarding autonomous vehicles, advanced driver assistant systems (ADASs) have been researched as a core technology of autonomous vehicles
driver status monitoring (DSM)-related researches can be divided into three categories: 1) driving patterns based on moving records, 2) psycho-physiological states based on sensor information, and 3) in-vehicle image analysis based on a camera sensor
We introduced the short version of an lightweight multilayer random forest (LMRF) model [26] for facial expression recognition (FER) system

Summary

INTRODUCTION

With the recent increase in interest regarding autonomous vehicles, advanced driver assistant systems (ADASs) have been researched as a core technology of autonomous vehicles. The detail contributions of this paper are as follows; 1) we propose an LMRF that has a deep structure based on layerby-layer processing using ensemble RFs instead of a neural network, which requires backpropagation, 2) LMRF offers fewer hyper-parameters and a faster processing time than a DNN while maintaining a similar performance, 3) apart from a CNN, this study try to improve the performance by applying face landmark detection and facial area cropping as a preprocessing, 4) we conduct experiments using public benchmark FER datasets, and a new dataset captured using a near-infrared (NIR) in a real driving environment, 5) through enough experiments, we prove that the proposed approach can be embedded into low-power and low-memory systems in place of a deep and wide network. The extracted geometric features are applied to the proposed LMRF with a N-layer structure, and we classify final facial expression with the maximum average value of the output probabilities of each RF of the last LMRF layer

GEOMETRIC FEATURE EXTRACTION

EXPERIMENTAL RESULTS

CONCLUSIOIN AND FUTURE WORKS