Augmented Reality Maintenance Assistant Using YOLOv5

Ana Malta,Mateus Mendes,Torres Farinha

doi:10.3390/app11114758

Ana Malta, Mateus Mendes + Show 1 more

Open Access

https://doi.org/10.3390/app11114758

Copy DOI

Abstract

Maintenance professionals and other technical staff regularly need to learn to identify new parts in car engines and other equipment. The present work proposes a model of a task assistant based on a deep learning neural network. A YOLOv5 network is used for recognizing some of the constituent parts of an automobile. A dataset of car engine images was created and eight car parts were marked in the images. Then, the neural network was trained to detect each part. The results show that YOLOv5s is able to successfully detect the parts in real time video streams, with high accuracy, thus being useful as an aid to train professionals learning to deal with new equipment using augmented reality. The architecture of an object recognition system using augmented reality glasses is also designed.

Highlights

Maintenance technicians often have to perform a large number of different jobs, involving numerous parts, manufactured by numerous different companies
The main objective of the present work is to build a system to recognize mechanical parts in car engines, and give directions, which come from a work order, to the technician
The precision obtained for the two models is in line with that obtained by other authors for similar problems, namely the “Artificial Intelligence for Real Time Threat Detection and Monitoring” [4] that has a precision of 0.803, 0.89 for recall and 0.905 for mean average precision (mAP) 0.5

Summary

Introduction

Maintenance technicians often have to perform a large number of different jobs, involving numerous parts, manufactured by numerous different companies. The main objective of the present work is to build a system to recognize mechanical parts in car engines, and give directions, which come from a work order, to the technician. The technician wears augmented reality glasses during the procedure. Through the glasses, he sees the car parts and the instructions on how to proceed, which are added in real time. The field of object detection has garnered new attention with recent developments in deep neural network architectures, and it is constantly growing. Many neural network architectures can detect a large number of objects, with high accuracy, in real time

Objectives

Methods

Results

Discussion

Conclusion