Reality Assembly Research Articles

Evaluation of Augmented Reality instructions based on initial and dynamic assembly tolerance allocation schemes in precise manual assembly

Augmented Reality (AR) assembly instructions can be used to guide precise manual assembly, providing detailed real-time information to assist in executing operations. Most AR instructions for precise manual assembly are based on initial assembly tolerance allocation schemes (ATASs), which provide a precision constrained zone and require assembly according to the initial zone. Dynamic ATASs can adjust the subsequent tolerances according to the completed assembly errors, thereby reducing the assembly difficulty. However, ATASs do not affect the AR cues for assembly guidance, and their effects on the usability of AR instructions are subtle and unclear. This study focuses on evaluating the effects of initial and dynamic ATASs on the usability of AR instructions. An AR instruction based on an initial ATAS and an AR instruction based on a dynamic ATAS were developed for a precise manual assembly task. A user study was conducted with 16 participants to compare the two AR instructions. The results showed significant differences in assembly speed, assembly quality, perceived ease-of-use, and perceived workload. We found that the AR instruction based on the dynamic ATAS can expedite assembly with better perceived ease-of-use and lower perceived workload, but may reduce assembly quality. Based on the results, several recommendations were provided to help researchers and developers design better AR instructions for precise manual assembly.

An externally guided spatial augmented reality assembly assistance system in the aviation manufacturing industry

An externally guided spatial augmented reality assembly assistance system in the aviation manufacturing industry

Co2iAR: Co-located audio-visual enabled mobile collaborative industrial AR wiring harness assembly

Existing augmented reality (AR) assembly mainly provides visual instructions for operators from a first-person perspective, and it is hard to share individual working intents for co-located workers on the shop floor, especially for large-scale product assembly task that requires multiple operators working together. To bridge this gap for practical deployments, this paper proposes Co2iAR, a co-located audio-visual enabled mobile collaborative AR assembly. Firstly, according to the stereo visual-inertial fusion strategy, robust and accurate self-contained motion tracking is achieved for the resource-constrained mobile AR platform, followed by a co-located alignment from multiple mobile AR clients on the shop floor. Then, a lightweight text-aware network for online wiring harness character recognition is proposed, as well as the audio-based confirming strategy, enabling natural audio-visual interaction among co-located workers within a shared immersive workplace, which can also monitor the current wiring assembly status and activate the step-by-step tutorials automatically. The novelty of this work is focused on the deployment of audio-visual aware interaction using the same device that is being used to deploy the co-located collaborative AR work instructions, establishing shared operating intents among multiple co-located workers. Finally, comprehensive experiments are carried out on the collaborative performance among multiple AR clients, and results illustrate that the proposed Co2iAR can alleviate the cognitive load and achieve superior performance for the co-located AR assembly tasks, providing a more human-centric collaborative assembly performance.

A Novel Method of Multitarget Augmented Reality Assembly Result Inspection for Large Complex Scenes

A Novel Method of Multitarget Augmented Reality Assembly Result Inspection for Large Complex Scenes

Assessing user performance in augmented reality assembly guidance for industry 4.0 operators

In the realm of smart manufacturing, Augmented Reality (AR) technology has gained increasing attention among researchers and manufacturers due to its practicality and adaptability. For this reason, it has been widely embraced in various industrial fields, especially for helping operators assemble products. Despite its widespread adoption, there is a debate in the research community about how effective AR is for improving user performance in assembly tasks, particularly when using handheld devices. These disparities can be attributed to differences in experimental approaches, such as the frequent use of qualitative methods, the inclusion of non-representative users, and the limited number of comprehensive case studies.In response to this, the paper delved into the benefits of AR applications, with a specific focus on measuring user performance and the cognitive workload perceived by users during assembly activities. To this end, an AR assembly guidance tool has been developed to assist users during assembly tasks, running on a mobile device, specifically a tablet, for freedom of movement and high portability. Experimentation involved the assembly of a comprehensive case study and a diverse user group, allowing the comparison representative users and experienced industrial operators. The results were promising, indicating that AR technology effectively enhances user performance during assembly-guided activities compared to conventional methods, particularly when users are unfamiliar with the task at hand. This study brings valuable insights by addressing previous research limitations and providing strong evidence of AR's positive impact on user performance in real-world assembly scenarios.

Perception and Decision-Making for Multi‑Modal Interaction Based on Fuzzy Theory in the Dynamic Environment

The projection-augmented reality assembly assistance system, which supports natural multi-modal human–computer interaction, has the potential to improve interaction efficiency. However, the effectiveness of the interaction method is compromised by the interference caused by the dynamically changing industrial environment. To address this issue, this study proposed a multi-model interaction model based on fuzzy theory and developed an environment perception decision-making interaction system. This system continuously monitors real-time environment data, evaluates interaction performance, and visually recommends the optimal method to assist users in completing their tasks. A comparative experiment was conducted between the proposed system and a non-environment perception interaction system. The collected data, including interaction success rate, task time, system usability, and user experience, were analyzed. The results demonstrated that the proposed system effectively enhances the adaptability and comfort of computer systems in complex environments and improves communication efficiency between humans and machines.

Towards tailored cognitive support in augmented reality assembly work instructions

AbstractBackgroundAugmented reality (AR) is receiving increasing interest as a tool to create an interactive and motivating learning environment. Yet, it is unclear how instructional support affects performance in AR.ObjectivesThis study sought to explore how varying the instructional support in AR can affect performance‐related behaviours of students with low cognitive abilities during assembly work.MethodsA total of 90 Belgian secondary school students repeatedly executed four different realistic assembly tasks. Three levels of instructional support (low, medium, and high) in AR as well as a control condition with paper instructions with a high level of detail were systematically varied across tasks and participants.Results and ConclusionsMultilevel regression analyses showed that AR instructions yielded lower assembly times and a lower perceived physical effort than paper instructions. Additionally, participants perceived tasks as less complex when given AR instructions with a high or a medium level of detail than when given a low level of detail. No effects of instructional support were established for other performance‐related behaviours, namely necessary assistance, error‐making, cognitive load, competence frustration, and stress. Effect sizes were small, at least among the instructional support conditions studied, yielding a limited base for adaptivity. Presumably, tailoring the instructional support in AR is only beneficial for highly complex tasks. The results might be useful for the design and implementation of AR in educational settings.

Augmented Reality Based Competency Based Learning on Computer Network Learning in Vocational Education Vocational School

This research is motivated by the complexity of the subject matter of Computer Networks, the large number of teachers who teach these subjects with different backgrounds, and not all of them can be practised directly due to limited laboratory facilities, accompanied by a learning model that is not yet competency-based which also makes learning outcomes not optimal. This study aims to develop three-dimensional computer network learning media using Augmented Reality Assembly Edu software in vocational education. This type of research is created using the ADDIE model, which includes analysis, design, development, implementation, and evaluation. Collection methods using observation and interviews. The data collection instrument used a questionnaire. Data analysis techniques using qualitative and quantitative descriptive analysis. The results showed that Competency-based learning-based Augmented Reality media, validated by a team of media experts and material experts, was very valid. In contrast, students stated well because this media made it easier to understand the material and motivated them to try and learn. It was concluded that Competency-based learning-based Augmented Reality media could be used in learning.

Reflective texture-less object registration using multiple edge features for augmented reality assembly

Augmented reality assembly technology has developed rapidly in recent years. As one of the key technologies in augmented reality assembly, object registration is to match the target’s location and posture between virtual objects and entities. However, the existing methods cannot effectively register reflective texture-less objects such as metal parts. In this paper, an object registration method based on multiple edge features is proposed, which includes a sparse uniform template sampling method and a multiple edge feature matching method. At the same time, we also used two oil pumps to carry out augmented reality assembly experiments and verify the accuracy of object registration. The experimental results show that this method can achieve high accuracy which is in line with the requirements of the augmented reality assembly in the industry.

Projected augmented reality assembly assistance system supporting multi-modal interaction

Projected augmented reality assembly assistance system supporting multi-modal interaction

Can you hear it? Stereo sound-assisted guidance in augmented reality assembly

Can you hear it? Stereo sound-assisted guidance in augmented reality assembly

Gamification of virtual reality assembly training: Effects of a combined point and level system on motivation and training results

Virtual Reality (VR) offers novel possibilities for remote training regardless of the availability of the actual equipment, the presence of specialists, and the training locations. Research shows that training environments that adapt to users' preferences and performance can promote more effective learning. However, the observed results can hardly be traced back to specific adaptive measures but the whole new training approach. This study analyzes the effects of a combined point and leveling VR-based gamification system on assembly training targeting specific training outcomes and users' motivations. The Gamified-VR-Group with 26 subjects received the gamified training, and the Non-Gamified-VR-Group with 27 subjects received the alternative without gamified elements. Both groups conducted their VR training at least three times before assembling the actual structure. The study found that a level system that gradually increases the difficulty and error probability in VR can significantly lower real-world error rates, self-corrections, and support usages. According to our study, a high error occurrence at the highest training level reduced the Gamified-VR-Group's feeling of competence compared to the Non-Gamified-VR-Group, but at the same time also led to lower error probabilities in real-life. It is concluded that a level system with a variable task difficulty should be combined with carefully balanced positive and negative feedback messages. This way, better learning results, and an improved self-evaluation can be achieved while not causing significant impacts on the participants' feeling of competence.

A novel approach to generate augmented reality assembly assistance automatically from CAD models

Many studies have already shown where the advantages of Augmented Reality (AR) lie and how great the potential of it is. Nevertheless, the technology still lacks its industrial suitability, and until now, the generation of AR assembly assistances is highly demanding and it requires a lot of manual effort. This paper presents an approach to derive AR assembly assistances from 3D assembly models in a fully automated way. In our approach, the assembly information is derived from a CAD model, processed in an exchange format on a server and then transferred to a Microsoft HoloLens 2 with two different user interfaces.

User-oriented AR assembly guideline: a new classification method of assembly instruction for user cognition

In augmented reality (AR) assembly, due to the unintuitive expression of the information content of the assembly instructions and the irregular design of the information form, the user’s assembly efficiency is low and the operation error rate is high. This paper elaborates and defines a user-oriented classification method of assembly instructions. This method reclassifies traditional assembly instructions based on the user’s cognitive needs for assembly tasks to improve users’ cognitive efficiency of physical tasks. In this article, the definition of traditional assembly instructions is first reiterated, and the geometrical relationship between it and AR assembly instructions is explained. Then, the definition of AR assembly instructions at the information level is given, the traditional assembly instructions are classified according to this definition, and the explanation of each classification is made. We compared the role of the old and new instructions in typical assembly use cases. The data shows that there are significant differences in performance between the new and old instructions. The new instructions significantly improve the user’s performance in terms of assembly time and operating experience (including enjoyment, concentration, confidence, natural intuition, feasibility, effectiveness, usability, and understandability), through the use of AR instructions on the information level and auxiliary reclassification, so as to achieve efficient and concise operation purposes. In addition, we also discussed the significance of this research and future research directions.

Augmented bricklaying

Augmented bricklaying explores the manual construction of intricate brickwork through visual augmentation, and applies and validates the concept in a real-scale building project—a fair-faced brickwork facade for a winery in Greece. As shown in previous research, robotic systems have proven to be very suitable to achieve various differentiated brickwork designs with high efficiency but show certain limitations, for example, in regard to spatial freedom or the usage of mortar on site. Hence, this research aims to show that through the use of a craft-specific augmented reality system, the same geometric complexity and precision seen in robotic fabrication can be achieved with an augmented manual process. Towards this aim, a custom-built augmented reality system for in situ construction was established. This process allows bricklayers to not depend on physical templates, and it enables enhanced spatial freedom, preserving and capitalizing on the bricklayer’s craft of mortar handling. In extension to conventional holographic representations seen in current augmented reality fabrication processes that have limited context-awareness and insufficient geometric feedback capabilities, this system is based on an object-based visual–inertial tracking method to achieve dynamic optical guidance for bricklayers with real-time tracking and highly precise 3D registration features in on-site conditions. By integrating findings from the field of human–computer interfaces and human–machine communication, this research establishes, explores, and validates a human–computer interactive fabrication system, in which explicit machine operations and implicit craftsmanship knowledge are combined. In addition to the overall concept, the method of implementation, and the description of the project application, this paper also quantifies process parameters of the applied augmented reality assembly method concerning building accuracy and assembly speed. In the outlook, this paper aims to outline future directions and potential application areas of object-aware augmented reality systems and their implications for architecture and digital fabrication.

Research on the visual elements of augmented reality assembly processes

BackgroundTo solve the problem of visualization in augmented reality (AR), for assembly process information, we report here on our study into the composition of AR assembly process information. MethodsOur work led us to classify the visual elements of assembly processes into six categories, and after looking further into visual element expression characteristics used in assembly process information in the AR environment, standard assembly process elements have been identified and visual element layout principles studied. ConclusionTypical visualization elements have been presented, using an AR-based assembly instruction system.

A fast object registration method for augmented reality assembly with simultaneous determination of multiple 2D-3D correspondences

Abstract A fast and robust object registration is one of the fundamental steps of an augmented reality (AR) assembly guidance system. To manage the registration of texture-less objects from monocular images, this paper presents a three-dimensional (3D) object registration solution in the view-based framework, which incorporates a new image feature named as chain-of-lines feature (COLF). The COLF, constructed by several directed line segments with geometric constraints, enhances the robustness of the registration by establishing multiple correspondences between the two-dimensional (2D) image and 3D model simultaneously. In order to save on computational cost and improve the efficiency of the COLF-based registration, a compression algorithm based on the local breadth-first search (LBFS) strategy is also proposed; this adaptively reduces the number of keyframes according to the target model. In the experimental evaluation, the combination of COLF and LBFS-based keyframe compression shows higher registration success rate and faster speed in comparison with conventional registration methods. Moreover, an AR assembly guidance prototype system for reductors is introduced to illustrate the application of the proposed registration method.

A Virtual Reality Assembly Assessment Benchmark for Measuring VR Performance & Limitations

With an increasing product complexity in manufacturing industry, virtual reality (VR) offers the possibility to immersively assess assembly processes already in early product development stages. Within production validation phases, engineers visually assess product part assembly and interactively validate corresponding production processes. Nevertheless, by now research does not give answers on how VR assembly system’s performance can be measured with respect to its technical limitations. The proposed Virtual Reality Assembly Assessment (VR2A) benchmark is an open, standardized experiment design for evaluating the overall VR assembly assessment performance in terms of sizes and clearances instead of measuring single technical impact factors within the interaction cycle, such as tracking, rendering and visualization limitations. VR2A benchmark focusses on the overall production engineer’s assessment objective generating quantifiable metrics. Using VR2A, users gain practical insights on their overall VR assessment system’s performance and limitations. An in-depth evaluation with production engineers (N=32) revealed, that negative clearances can be detected more easily than positive ones, part sizes directly correlate with the assessment performance. Additionally, the evaluation showed that VR2A is easy to use, universally usable and generates objective insights on the applied VR system.

Synchronous AR assembly assistance and monitoring system based on ego-centric vision

PurposeAiming at presenting an interaction-free assembly assistance tool, the purpose of this paper is to propose a synchronous augmented reality (AR) assembly assistance and monitoring system. The system monitors operator’s hands activity and process completeness to recognize the assembly state, then display the AR contents contextually.Design/methodology/approachAn assembly behavior recognition method is proposed based on gesture recognition. An assembly completeness inspection method is proposed based on SURF feature matching. Assembly state and AR display state are solved by a novel sequential hybrid AR display control strategy. A synchronous multi-channel AR view output strategy is proposed based on QR matrix decomposition.FindingsA prototype system has been developed, and case study is performed on an industrial product. Experiments are performed to verify the feasibility, efficiency and recognition accuracy of the proposed methods.Research limitations/implicationsThe proposed system assists users to perform assembly tasks with automatic visual guidance and vision monitoring, avoiding distractions caused by redundant human–computer interactions.Practical implicationsAll methods are integrated to work on only one head-worn device, making the proposed system portable and cheaper. The vision processing pipelines and the view output channels are reconfigurable for customization.Originality/valueThis paper proposes an interaction-free AR assembly assistance and monitoring system. Assembly behavior recognition and assembly completeness inspection methods are integrated to monitor the assembly state. A sequential hybrid AR display control strategy is proposed to contextually update the AR contents. A synchronous multi-channel AR view output strategy is proposed to fulfill different visualization needs.

Evaluating Commodity Hardware and Software for Virtual Reality Assembly Training

Evaluating Commodity Hardware and Software for Virtual Reality Assembly Training