Assessing household damages using multi-model deep learning pipeline

Abstract

Since the beginning of the pandemic, the home insurance sector has suffered from various difficulties. One of the most important difficulties was assessing the damages in the insurance owners’ homes. Due to the current pandemic, letting the experts assess the damages in place is a life-threatening risk. Therefore, the idea of automatically assessing the damage is born. This study aims to create a full report for home damages using Convolutional Neural Network (CNN) and various large deep learning model architectures such as EfficientNet, ResNet, U-Net, or Feature Pyramid Network (FPN). Multiple models for tasks such as binary classification and instance segmentation were developed to create an end-to-end reporting pipeline. In more detail, the pipeline consists of two binary classification models and a segmentation model. Binary classification models are responsible for detecting if the picture is indoors and if there is a wall in the picture, whereas the instance segmentation model is responsible for segmenting the damaged parts of the wall class. The effectiveness of the pipeline was measured using different metrics for each task, including accuracy, F1, dice, and Intersection over Union (IoU) scores. The data for each task is labeled by hand and fed to models. The results show that the constructed pipeline can successfully classify and segment the given images according to the needs of our project. The project will affect the home insurance assessment procedure and time spent tremendously by automatizing these repetitive processes.