Automated classification of “cluttered” construction housekeeping images through supervised and self-supervised feature representation learning

Abstract

Construction housekeeping is crucial for safety, but frequent manual inspections are difficult to maintain. A computer vision approach to automatically monitor housekeeping can overcome these issues. However, it requires labelling large number of “cluttered” construction housekeeping images that are difficult to label, even by experts. Thus, this paper presents an alternative approach that evaluates the use of self-supervised learning feature extraction to classify “cluttered” construction housekeeping images. The most suitable (84% accuracy) backbone architecture for supervised classification of housekeeping images was found to be Swin-transformer. In addition, the experiments show that self-supervised learning approach can perform better (1–4% improvement in prediction accuracy, precision, and recall) than the supervised learning approach in a non-transfer learning context and when the number of training images is reduced.