Dual Visual-Thermal Correlator Combined with Audio-Assisted mmWave FEM for Smart Sensing and Environmental Awareness

Abstract

This paper proposes a Thermal-Visual-RF system Correlator embedding advanced Multi-Scale auto-correlation and cross-correlation signal-processing in time and frequency domains combined with convolutional Artificial-Intelligence (AI) and Machine-Learning (ML) algorithms. Three different signal processing methods: Fast Fourier Transform (FFT), Independent Component Analysis (ICA) and Principal Component Analysis (PCA) are applied on the color channels in video recordings for extracting the blood volume pulse (BVP) from the facial regions. The Thermal-Visual-RF system Correlator is integrated with a phased-array Smart-Audio (SA) system implementing background noise cancellation functionalities. The synchronization of the phased-array SA with mmWave FEM (Front-End-Module) beamforming capabilities opens new possibilities for cognitive environmental awareness and sensing.