Depressioner: Facial dynamic representation for automatic depression level prediction

Abstract

Physiological studies have shown that facial changes can be seen as a biomarker to analyze the severity of depression. Therefore, this study proposes a Depressioner model to predict the depression level by examining facial changes. Our method is mainly to solve two problems in the previous works: (1) each channel in the tensor obtained by the convolution layer can be regarded as a pattern extraction result related to depression. However, previous works rarely explore the relationship among channels, which is limited in integrating the advantages of various channels; (2) the average (or max) pooling is often used to vectorize the tensor, which is not conduction to capturing the depression cues from tensors with temporal attribute. To this end, this study designs two novel blocks namely Graph Convolution Embedding (GCE) block and Multi-Scale Vectorization (MSV) block. The GCE block treats each channel as a node in the graph and constructs the corresponding adjacency matrix. Furthermore, the GCE block adopts the graph convolution operation to examine the relationship among channels to take advantage of each channel and highlight useful elements. The MSV block combines the dilated convolution and attention mechanism to process each channel to extract the multi-scale representation of depression cues along temporal dimension. Moreover, it aggregates these representations into the vectorization result of tensor along channel dimension. Experimental results on AVEC 2013 (RMSE = 7.49, MAE = 6.12) and AVEC 2014 (RMSE = 7.56, MAE = 6.01) depression databases illustrate the effectiveness of our method, which may promote the auxiliary diagnosis of depression screening in the future. Meanwhile, these results also show that the proposed Depressioner model can capture the differences of facial changes among individuals with different depression levels.