Industrial Augmented Reality Research Articles

Industrial Augmented Reality: A Framework for Defining Requirements

Augmented Reality (AR) boasts a wide array of applications throughout the entire product lifecycle; however, its adoption in industrial settings is often impeded by factors such as high setup costs, poor system integration, and limited modality. Despite these challenges, current Industrial Augmented Reality (IAR) applications exhibit a significant overlap in components used for information gathering and visualization. This paper describes the current state of modern IAR architectures and introduces a novel framework for the definition of requirements specific to IAR applications. This draws upon the principles of human-cantered design to provide a structured approach for integrating IAR more effectively in industrial contexts. Finally, a case-study shows how the framework can be used to help implement an assistance system for assembly of a simple product.

Augmented Reality in Industry 4.0 Assistance and Training Areas: A Systematic Literature Review and Bibliometric Analysis

Augmented reality (AR) technology is making a strong appearance on the industrial landscape, driven by significant advances in technological tools and developments. Its application in areas such as training and assistance has attracted the attention of the research community, which sees AR as an opportunity to provide operators with a more visual, immersive and interactive environment. This article deals with an analysis of the integration of AR in the context of the fourth industrial revolution, commonly referred to as Industry 4.0. Starting with a systematic review, 60 relevant studies were identified from the Scopus and Web of Science databases. These findings were used to build bibliometric networks, providing a broad perspective on AR applications in training and assistance in the context of Industry 4.0. The article presents the current landscape, existing challenges and future directions of AR research applied to industrial training and assistance based on a systematic literature review and citation network analysis. The findings highlight a growing trend in AR research, with a particular focus on addressing and overcoming the challenges associated with its implementation in complex industrial environments.

From lab to industry: lessons learned from the evaluation of augmented and virtual reality use cases in the Austrian manufacturing industry

ABSTRACT For evaluation of industrial augmented and virtual reality (AR/VR) applications, a comprehensive knowledge of their impact on productivity, process metrics, and human factors is critical to facilitate the transition from lab-scale research to industrial settings. While laboratory-scale evaluations have been extensively researched, successful implementation in industrial settings has fallen short. This paper presents a multi-criteria evaluation model using a mixed methods approach to study thirteen industrial use cases of AR/VR applications in manufacturing over two years. Semi-structured interviews were conducted with companies about the impact of the application. A human factors questionnaire was completed by 69 participants, including in-house users, industry experts, and inexperienced novices. The results of the use cases show an average reduction of 23.61% in process and 25% in cycle time. Process quality indicators improved significantly compared to baseline. The evaluation found no significant differences in workload and ease of use between age groups.

Environment awareness, multimodal interaction, and intelligent assistance in industrial augmented reality solutions with deep learning

Environment awareness, multimodal interaction, and intelligent assistance in industrial augmented reality solutions with deep learning

A monocular-based tracking framework for industrial augmented reality applications

A monocular-based tracking framework for industrial augmented reality applications

Towards Universal Industrial Augmented Reality: Implementing a Modular IAR System to Support Assembly Processes

While Industrial Augmented Reality (IAR) has many applications across the whole product lifecycle, most IAR applications today are custom-built for specific use-cases in practice. This contribution builds upon a scoping literature review of IAR data representations to present a modern, modular IAR architecture. The individual modules of the presented approach are either responsible for user interface and user interaction or for data processing. They are use-case neutral and independent of each other, while communicating through a strictly separated application layer. To demonstrate the architecture, this contribution presents an assembly process that is supported once with a pick-to-light system and once using in situ projections. Both are implemented on top of the novel architecture, allowing most of the work on the individual models to be reused. This IAR architecture, based on clearly separated modules with defined interfaces, particularly allows small companies with limited personnel resources to adapt IAR for their specific use-cases more easily than developing single-use applications from scratch.

Mapping the Emergent Trends in Industrial Augmented Reality

Augmented reality (AR) is a rapidly developing technology with the potential to revolutionize various sectors of industry by integrating digital information with the real world. This paper presents an overview of the emergent trends in industrial augmented reality (IAR) over the past five years. The study utilizes a comprehensive literature review analysis of industrial studies (searched on two scientific databases: Scopus and Clarivate Web of Science) to map the evolution of IAR trends from 2018 to 2022. The results revealed ten trending topics of AR application: Industry 4.0, artificial intelligence, smart manufacturing, industrial robots, digital twin, assembly, Internet of Things, visualization, maintenance, and training. Each topic is discussed in detail, providing insight into existing applications and research trends for each application field.

Advanced visualization of ergonomic assessment data through industrial Augmented Reality

The industrial transition to the 4.0 paradigm defines new scenarios in which the operator plays a central role within the industrial ecosystem. Thanks to the enabling technologies of Industry 4.0, it is possible to effectively improve operators' working conditions by applying the Human-Centered approach. Nowadays, one of the main challenges is to reduce work-related musculoskeletal disorders resulting from ergonomically incorrect working conditions in order to prevent the occurrence of occupational diseases. To this end, we developed a software tool that leverages a low-cost D-RGB camera (Kinect v2) to track the human body and an Augmented Reality (AR) visualization system based on Microsoft HoloLens 2. The tool assesses postural ergonomic risk in real-time according to the Rapid Upper Limb Assessment (RULA) metric. The proposed AR application allows a three-dimensional visualization of postures, which can be observed directly superimposed on the operator's body in the real scene. This approach aims to optimize the understanding of postures by creating a link between real information (operator's body) and virtual information (virtual skeleton, RULA score, and angles) by providing a simple and immediate user interface for ergonomists.

Lessons Learned from Human Pose Interaction in an Industrial Spatial Augmented Reality Application

This paper is a technical description of a novel Augmented Reality application in the industrial domain of furniture production. In the presented case, workers suffered from high cognitive load in doing end-of-line quality inspection and individual handling of a high variety of products. The proposed solution consists of a Spatial Augmented Reality system, where a projector directly displays information on the product to assist the worker. At the same time results of an in-line quality inspections are shown which can be interactively modified through human gestures.The main contribution of this work is two-fold: (1) in contrast to other industrial augmented reality applications the described one technically builds upon a deep neural net based pose estimation. This allows a seamless interaction with the system and tracking of human actions rather than deriving them from the current state of the work piece, (2) the paper recapitulates experiences and results of the approach with a focus on lessons learned for using human pose estimation in smart production.

Impact of AR and VR in Industrial Revolution

Abstract: Augmented Reality (AR) and Virtual Reality (VR) are state-of-the-art technologies for superimposing information onto the real world, has recently started to have an effect on our everyday lives. There has been an increase in the maturity and versatility of applications based on AR and VR technologies in recent years. Consequently, this improves user productivity and accuracy and offers economic benefits to companies. A wide range of augmented reality applications are now possible thanks to recent advances in the internet of things, smart sensors, and sophisticated algorithms. Currently, many research pieces are being done to expand the application of augmented reality and enhance its effectiveness in industrial production processes. The purpose of this paper is to analyze some use cases and the prospects associated with industrial augmented reality in order to highlight its critical role. This article explores the main future directions for industrial augmented reality applications in industry 4.0 through a systematic analysis. Various applications of this technology and their impact on improving production conditions are discussed in the article. In conclusion, this technology's challenges and research opportunities are discussed.

Ultra Low-Latency Backscatter for Fast-Moving Location Tracking

This paper explores building an ultra-low latency and high-accuracy location tracking solution using battery-free tags. While there is rich prior work on location tracking with battery-free RFID tags and backscatter devices, these systems typically face tradeoffs with accuracy, power consumption, and latency. Such limitations make these existing solutions unsuitable for emerging applications like industrial augmented reality which requires tracking fast-moving machinery; monitoring indoor sports activities that require real-time tracking of fast-moving objects with high precision and under stringent latency constraints. We propose and demonstrate FastLoc, a precision tracking system that locates tiny, battery-free analog backscatter tags at sub-millisecond latency and sub-centimeter accuracy. FastLoc is a hybrid system that simultaneously uses RF and optical signals to track tiny tags that can be attached to everyday objects. FastLoc leverages the RF channel responses from tags for estimating the coarse region where the tags may be located. It simultaneously uses the sensed optical information modulated on the backscatter signals to enable fine-grained location estimation within the coarse region. To achieve this, we design and fabricate a custom analog tag that consumes less than 150 uW and instantaneously converts incident optical signals to one-shot wideband harmonic RF responses at nanosecond latency. We then develop a static high-density distributed-frequency structured light pattern that can localize tags in the area of interest at a sub-centimeter accuracy and microsecond-scale latency. A detailed experimental evaluation of FastLoc shows a median accuracy of 0.7 cm in tag localization with a 0.51 ms effective localization latency.

Lessons Learned from Industrial Augmented Reality Applications

Smart production concepts are on the way to revolutionize industrial processes. A good example is the application of Extended Reality techniques that have emerged in recent years. This paper presents six different use cases where Extended Reality is realized utilizing tablets and digital projectors. The use cases reach from virtual sticky notes to artificial intelligence guided projections. The main aim of this work is to not only tell the success stories, but also give a sight behind the curtain and highlight challenges and lessons learned for each application such that the community can profit from the presentations.

A Systematic Review of User Acceptance in Industrial Augmented Reality

In the industrial work context, Augmented Reality (AR) can support work processes and employees’ cognitive relief through the location-specific and context-related superimposition of real objects with virtual information. The AR-based support of industrial work processes ranges over product development, manufacturing, assembly, maintenance, and training. In all these areas, numerous location-based AR support functions are being prototypically implemented, aiming to improve work efficiency, communication in mobile work situations, or employee qualification in the work process. In contrast to the increasing number of developed AR solutions in recent years, there is no widespread use of these solutions in industrial practice. AR systems’ successful introduction is closely related to user acceptance, which has not been comprehensively considered over the system development process. In addition to improving AR hardware ergonomic features, usability or user interface design play an essential role in user acceptance. Particularly in the context of employee qualification, increasing employee engagement can be named as a success factor. Previous user studies of industrial AR systems only include individual user acceptance aspects. The use of game elements has not been widely addressed in connection with manual tasks in production environments, including AR-based assistance systems. This paper aims to examine user acceptance of industrial AR systems and the relevant factors for investigating user acceptance, e.g., ease of use or enjoyment, based on a systematic literature review. An analysis of existing review articles on industrial AR systems elaborates the current state of the art and identifies the research gap. This review of 109 scientific articles from 2011 to 2020 provides an overview of the current state of research on the inclusion of user acceptance in industrial AR systems. The identified papers from the scientific databases, Scopus, Web of Science, IEEE Xplore Digital Library, ACM Digital Library, and Science Direct, are evaluated for their relevance and selected for further analysis based on inclusion and exclusion criteria, e.g., year of publication. This review presents the current challenges regarding user acceptance of industrial AR systems and future possibilities for the comprehensive integration of user acceptance factors into the development, evaluation, and implementation process.

VR and AR for Industrial Augmented Reality

VR and AR for Industrial Augmented Reality

User Interface Pattern for AR in Industrial Applications

The background of the paper is that there are currently no specifications or guidelines for the design of a user interface for an augmented reality system in an industrial context. In this area, special requirements apply for the perception and recognition of content, which are given by the framework conditions of the industrial environment, the human–technology interaction, and the work task. This paper addresses the software-technical design of augmented reality surfaces in the industrial environment. The aim is to give first design examples for software tasks by means of sample solutions. For a user-oriented implementation, the methods of personas and an empirical investigation were used. Personas are a stereotypical representation of end users that reflect their characteristics and requirements. For the subsequent development of the pattern catalog, different prototypes with layout and interaction variants were tested in an empirical study with 50 participants. By observing the current realizations, these can be examined more closely in terms of their specific use in an industrial environment. The result is a pattern catalog with 26 solutions for layout and interaction variants. For the layout variants, no direct favorite of the testers could be ascertained; the existing solutions already offer a wide spectrum, which are chosen according to personal preferences. For interaction, on the other hand, it is important to enable fast input. During the study, gesture control revealed weaknesses in this regard. This can be supportive in the development of an industrial augmented reality system regarding a user-oriented representation of the interface.

Augmented Reality in Industry 4.0 and Future Innovation Programs

Augmented Reality (AR) is worldwide recognized as one of the leading technologies of the 21st century and one of the pillars of the new industrial revolution envisaged by the Industry 4.0 international program. Several papers describe, in detail, specific applications of Augmented Reality developed to test its potentiality in a variety of fields. However, there is a lack of sources detailing the current limits of this technology in the event of its introduction in a real working environment where everyday tasks could be carried out by operators using an AR-based approach. A literature analysis to detect AR strength and weakness has been carried out, and a set of case studies has been implemented by authors to find the limits of current AR technologies in industrial applications outside the laboratory-protected environment. The outcome of this paper is that, even though Augmented Reality is a well-consolidated computer graphic technique in research applications, several improvements both from a software and hardware point of view should be introduced before its introduction in industrial operations. The originality of this paper lies in the detection of guidelines to improve the Augmented Reality potentialities in factories and industries.

A Hybrid Approach to Industrial Augmented Reality Using Deep Learning-Based Facility Segmentation and Depth Prediction.

Typical AR methods have generic problems such as visual mismatching, incorrect occlusions, and limited augmentation due to the inability to estimate depth from AR images and attaching the AR markers onto physical objects, which prevents the industrial worker from conducting manufacturing tasks effectively. This paper proposes a hybrid approach to industrial AR for complementing existing AR methods using deep learning-based facility segmentation and depth prediction without AR markers and a depth camera. First, the outlines of physical objects are extracted by applying a deep learning-based instance segmentation method to the RGB image acquired from the AR camera. Simultaneously, a depth prediction method is applied to the AR image to estimate the depth map as a 3D point cloud for the detected object. Based on the segmented 3D point cloud data, 3D spatial relationships among the physical objects are calculated, which can assist in solving the visual mismatch and occlusion problems properly. In addition, it can deal with a dynamically operating or a moving facility, such as a robot—the conventional AR cannot do so. For these reasons, the proposed approach can be utilized as a hybrid or complementing function to existing AR methods, since it can be activated whenever the industrial worker requires handing of visual mismatches or occlusions. Quantitative and qualitative analyses verify the advantage of the proposed approach compared with existing AR methods. Some case studies also prove that the proposed method can be applied not only to manufacturing but also to other fields. These studies confirm the scalability, effectiveness, and originality of this proposed approach.

Creating Collaborative Augmented Reality Experiences for Industry 4.0 Training and Assistance Applications: Performance Evaluation in the Shipyard of the Future

Industrial Augmented Reality (IAR) is one of the key technologies pointed out by the Industry 4.0 paradigm as a tool for improving industrial processes and for maximizing worker efficiency. Training and assistance are two of the most popular IAR-enabled applications, since they may significantly facilitate, support, and optimize production and assembly tasks in industrial environments. This article presents an IAR collaborative application developed jointly by Navantia, one of the biggest European shipbuilders, and the University of A Coruña (Spain). The analysis, design, and implementation of such an IAR application are described thoroughly so as to enable future developers to create similar IAR applications. The IAR application is based on the Microsoft HoloLens smart glasses and is able to assist and to guide shipyard operators during their training and in assembly tasks. The proposed IAR application embeds a novel collaborative protocol that allows operators to visualize and interact in a synchronized way with the same virtual content. Thus, all operators that share an IAR experience see each virtual object positioned at the same physical spot and in the same state. The collaborative application is first evaluated and optimized in terms of packet communications delay and anchor transmission latency, and then, its validation in a shipyard workshop by Navantia’s operators is presented. The performance results show fast response times for regular packets (less than 5 ms), low interference rates in the 5 GHz band, and an anchor transmission latency of up to 30 s. Regarding the validation tests, they allow for obtaining useful insights and feedback from the industrial operators, as well as clear guidelines that will help future developers to face the challenges that will arise when creating the next generation of IAR applications.

What, How, and Why are Visual Assets Used in Industrial Augmented Reality? A Systematic Review and Classification in Maintenance, Assembly, and Training (From 1997 to 2019).

Industrial Augmented Reality (iAR) has demonstrated its advantages to communicate technical information in the fields of maintenance, assembly, and training. However, literature is scattered among different visual assets (i.e., AR visual user interface elements associated with a real scene). In this work, we present a systematic literature review of visual assets used in these industrial fields. We searched five databases, initially finding 1757 papers. Then, we selected 122 iAR papers from 1997 to 2019 and extracted 348 visual assets. We propose a classification for visual assets according to (i) what is displayed, (ii) how it conveys information (frame of reference, color coding, animation), and, (iii) why it is used. Our review shows that product models, text and auxiliary models are, in order, the most common, with each most often used to support operating, checking and locating tasks respectively. Other visual assets are scarcely used. Product and auxiliary models are commonly rendered world-fixed, color coding is not used as often as expected, while animations are limited to product and auxiliary model. This survey provides a snapshot of over 20 years of literature in iAR, useful to understand established practices to orientate in iAR interface design and to present future research directions.

Industrial Augmented Reality as a Support Tool in Maintenance of Electrical Panels

Maintenances of electrical panels are necessary to ensure the proper functioning of industrial machinery and equipment. This sector has some concerns related to the safety of professionals and the correct execution of maintenance protocols. There are often standardized documents containing the steps to be followed in the process that end up being neglected or not followed correctly by the professionals. This study aims to demonstrate the creation of a minimum viable product of an Augmented Reality application to assist in maintenance tasks for electrical control panels of industrial machinery and equipment, displaying information about the panels and the sequence of steps that must be taken and correctly followed so that maintenance is carried out correctly, thus facilitating training and increasing the safety of professionals involved in these processes. To create this application, a method is used that includes the use of the initial system, going through all stages of development until the functional tests. Afterwards, a survey with potential users of this system to collect information about the use and improvements needed. With this study it was possibly to conclude that Augmented Reality has great potential for use in maintenance tasks for electrical panels and integration with the Internet of Things, as it provides greater security and ease of training in the industrial environment. http://dx.doi.org/10.18226/23185279.v8iss2p78