Dual mode information fusion with pre-trained CNN models and transformer for video-based non-invasive anaemia detection

Abstract

Anaemia, caused by low haemoglobin levels, affects billions of people worldwide. India has one of the highest anaemia rates among developing nations, according to WHO data. Non-invasive, low-cost, and accurate anaemia screening technologies are in high demand because invasive methods are expensive and difficult to administer globally. The current effort aims to construct a reliable anaemia detection system by combining cutting-edge computational methods with the age-old practise of imprecise assessment of haemoglobin levels from pallor in palm. The proposed method utilises time-domain analysis to establish the connection between blood haemoglobin concentration and colour changes in the palm caused by pressure application and release. A smartphone camera sensor which records the full event of palm colour changes generated by a customised device is processed and analysed. Some key frames are selected from extracted frames and passed to the pre-trained deep learning models for feature extractions. These features are then passed through vision transformers for classification and regression tasks. This approach have the power of pre-trained models for feature extraction and utilises the flexibility of transformer encoders and MLP networks for efficient and effective representation learning, classification, and regression. Two different approaches are proposed i.e., decision level and feature level fusion. The suggested approach accurately estimates blood haemoglobin levels with RMSE and accuracy of 0.483 and 96.296%, respectively based on 531 video samples of palm evidence.