NN2ViT: Neural Networks and Vision Transformers based approach for Visual Anomaly Detection in Industrial Images

Abstract

Listen

Translate article

Ensuring product quality through automated anomaly detection is crucial in manufacturing. Traditional methods often struggle to capture both local and global features effectively, relying heavily on predefined templates that limit their adaptability and accuracy. To address these challenges, this study propose NN2ViT, a novel approach that integrates a Single Shot Detector (SSD) for local feature detection and the Segment Anything Model (SAM) for global feature segmentation. This integration allows for a comprehensive analysis of anomalies in industrial images. Our method improves anomaly segmentation performance by fine-tuning SAM for precise segmentation in industrial product images. Experiments on the MVTec benchmark dataset demonstrate that NN2ViT outperforms traditional models and achieved the highest 95.54% and 96.23% Image AUROC and AP scores, respectively thus enhancing interpretability and adaptability to various anomaly patterns. This research presents a significant advancement in manufacturing quality control, contributing to improved product quality and operational efficiency.