Multi-task learning for hand heat trace time estimation and identity recognition

Abstract

The thermal imager can capture the invisible thermal trace, and the potential information contained in the thermal trace can be extracted through the identification and analysis of the infrared thermal trace, such as the identity information and time information of the trace. This technology has great application value in criminal investigation, military and other fields. However, with the passage of time, the heat traces will gradually fade, and the intra-class distances will gradually increase. For hand heat trace, the trace may exhibit significant variations at different time points or in different states. This will pose challenges for extracting potential information and present significant challenges for model recognition. Therefore, we propose a multi task framework using deep convolutional neural networks (MTLHand) to jointly handle the two tasks of heat trace identification and heat trace departure time estimation. It can learn invariant representations of thermal trace identities, better capturing the dynamic changes in palmprint data, to enhance the accuracy of identity recognition. Specifically, soft threshold is inserted as nonlinear transformation layers into an improved ResNet to extract deep mixed hand trace features. The mixed features are decomposed into identity related features and time related features through spatial attention and channel attention. Use the multi task training method to carry out the corresponding learning tasks. In addition, we collected and constructed an infrared hand heat trace dataset containing labels such as departure time, gender, hand posture, identity category, etc., to promote the development of heat trace research. Experiments on this dataset show that our method is superior to other deep learning methods for the specific task of infrared heat trace recognition, and the identification accuracy can reach 83.48 %. In the trace leaving time estimation task, the recognition error rate is 18 % if the residual value between the estimated time and the real time is less than 60 s, and 3.55 % if the residual value is less than 120 s.