Thermography based skin allergic reaction recognition by convolutional neural networks

Łukasz Neumann,Jacek Stępień,Robert Nowak,Ewelina Chmielewska,Patryk Pankiewicz,Radosław Solan,Karina Jahnz-Różyk

doi:10.1038/s41598-022-06460-9

Łukasz Neumann, Jacek Stępień + Show 5 more

Open Access

https://doi.org/10.1038/s41598-022-06460-9

Copy DOI

Abstract

In this work we present an automated approach to allergy recognition based on neural networks. Allergic reaction classification is an important task in modern medicine. Currently it is done by humans, which has obvious drawbacks, such as subjectivity in the process. We propose an automated method to classify prick allergic reactions using correlated visible-spectrum and thermal images of a patient’s forearm. We test our model on a real-life dataset of 100 patients (1584 separate allergen injections). Our solution yields good results—0.98 ROC AUC; 0.97 AP; 93.6% accuracy. Additionally, we present a method to segment separate allergen injection areas from the image of the patient’s forearm (multiple injections per forearm). The proposed approach can possibly reduce the time of an examination, while taking into consideration more information than possible by human staff.

Highlights

In this work we present an automated approach to allergy recognition based on neural networks
A bubble is formed under the influence of fluid penetration from the skin’s microvessels dilated by the released histamine, but it may appear in other situations such as toxic dermatitis or even at the site of a negative test; a bubble is non-specific symptom
The instrument was developed for capturing allergic reactions, technical data of the thermovision system can be found in Supplementary Tables S1 and S2 online

Summary

Introduction

In this work we present an automated approach to allergy recognition based on neural networks. The second technique is an in vivo functional method in the form of skin tests, allowing for the diagnosis of both type I (skin prick test—SPT) and type IV (delayed - patch test) allergies[4] In this method tiny amounts of allergen are dropped into a small puncture made by a lancet. Because skin tests of various types allow mapping the actual response of the patient’s immune system to the allergen it is assumed that they are a clinically reliable predictor of individual hypersensitivity to the tested substances[5] Both methods have limitations, can lead to wrong diagnoses, are invasive and require trained professionals to perform the test and to interpret results. The skin test results are observed by the doctor and, based on a subjective visual assessment of the symptoms, associated with an allergic skin reaction. A bubble is formed under the influence of fluid penetration from the skin’s microvessels dilated by the released histamine, but it may appear in other situations such as toxic dermatitis or even at the site of a negative test; a bubble is non-specific symptom

Methods

Results

Conclusion