Virtual Assembly Environment Research Articles

A virtual environment for complex products collaborative assembly operation simulation

As complex products such as automobiles and ships are usually developed at geographically dispersed locations, it brings the difficulty for component model verification and assembly process evaluation. To cope with this challenge, a relatively comprehensive system scheme which supports complex product real-time collaborative interactive assembly is proposed. To get high performance, the technologies such as parallel rendering for complex scene based on pc-cluster, high-efficient collision detection method, the mechanism of event synchronization based on HLA/RTI (High level Architecture/Run-time Infrastructure) are applied. A prototype system named Distributed Parallel Virtual Assembly Environment (DPVAE) is developed. With DPVAE, users at different location can do assembly operation collaboratively by hand or by tool, to conduct the component model verification and assembly process evaluation at the same time. Through a case study of RA750 car assembly, the validity of system functions has been demonstrated successfully.

A real-time simulation grid for collaborative virtual assembly of complex products

Simulation of collaborative virtual assembly (CVA) processes is a helpful tool for product development. However, existing collaborative virtual assembly environments (CVAE) have many disadvantages with regard to computing capability, data security, stability, and scalability, and moreover it is difficult to create enterprise applications in these environments. To support large-scale CVAEs offering high fidelity and satisfactory interactive performance among various distributed clients, highly effective system architectures are needed. In this paper, a collaborative virtual assembly scheme based on grid technology is proposed. This scheme consists of two parts: one is a grid-based virtual assembly server (GVAS) which can support parallel computing, the other a set of light clients which can support real-time interaction. The complex and demanding computations required for simulation of virtual assembly (VA) operations, such as model rendering, image processing (fusion), and collision detection, are handled by the GVAS using network resources. Users at the light clients input operation commands that are transferred to the GVAS and receive the results of these operations (images or video streams) from the GVAS. Product data are managed independently by the GVAS using the concept of RBAC (role-based access control), which is secure enough for this application. The key related technologies are discussed, and a prototype system is developed based on the web services and VA components identified in the paper. A case study involving a car-assembly workstation simulation has been used to verify the scheme.

Haptic-based interactive path planning for a virtual robot arm

In this paper, based on the structural similarity between the PHANToM haptic device and a six-DOF (Degrees of Freedom) articulated robot arm, a six-DOF virtual robot arm driven by the PHANToM device is modeled. In order to enable the virtual robot arm to haptically interact with virtual prototypes in a virtual assembly (VA) environment, a workspace mapping method based on robot kinematics analysis is proposed. The haptic-based virtual robot arm is used in interactive modeling in free path planning and constraint-based assembly path planning operations in the VA system. In both planning processes, the user can interactively edit an assembly path with the guiding forces as feedback. Lastly, A few experiments have been conducted to verify the effectiveness of the proposed methods. The haptic-based virtual robot arm presented in this paper provides a new human-computer interaction method for a VA system.

Kinematics Analysis of Mechanisms Based on Dynamics Multibody Systems in Virtual Assembly Environment

In recent years, in view of the absence of kinematics analysis function of mechanisms in the virtual assembly system, the method of kinematics analysis of mechanisms based on dynamics of multibody systems in virtual assembly environment is presented. A mechanism is considered as a multibody system and the kinematics equations are established based on Descartes coordinate system. All the kinematics parameters, just like position, speed and acceleration, can be obtained through solving the kinematics equations. In addition, the realization method of mechanism motion simulation in virtual assembly system is also proposed which contains the hiberarchy, data structure, module classes and simulation process of the system. The method is implemented and validated based on the prototype system VAPP(Virtual Assembly Process Planning).

Study on Mixed Assembly Modeling of Rigid and Flexible Parts for Aerospace Product Virtual Assembly

According to the characteristics of aerospace product, the model representation and storage of the rigid part and cable harness is proposed. The Dendritic Correlative Model of Mixed Assembly of Rigid and Flexible Parts (DCMMA-RFP) is introduced based on the hierarchical model and graph model. The assembly model which realizes the lightweight storage can clearly represent the hierarchy relation and assembly process of aerospace products. The assembly modeling process in the virtual assembly environment is detailed described. Finally the effectiveness of DCMMA-RFP is verified by an example.

Pipeline Design and Layout in Virtual Environment

Due to the difficulty of pipeline design and layout for complex mechanical products, a virtual reality-based method is proposed for pipeline design and routing in virtual environment. A parameter database is established for geometric design of pipelines according to their basic parameters in 3D models. Controlled-points and controlled-vectors are introduced to determine the position and direction of pipelines. Dynamic adjusting algorithm for straight pipelines and elbow pipelines in virtual environment are implemented respectively to joint and layout pipelines according to surrounding environment and features. A pipeline oriented virtual assembly environment system is developed, and some examples are given to demonstrate its function and validation.

Distributed parallel virtual assembly environment for automobile development

PurposeAutomobile development needs more and more collaborative work involving geographical dispersed designers, that brings difficulty for model verification, conception review and assembly process evaluation, so a collaborative virtual environment for automobile based on network is required. In this kind of environment, designers can do interactive assembly operations collaboratively, such as grasp, move, release, collision detection (CD), assembly evaluation report generation, etc. Furthermore, automobile structure becomes more complicated, how to process this large real‐time data effectively in real‐time interactive virtual environment is a great challenge. The purpose of this paper is focus on this.Design/methodology/approachA distributed parallel virtual assembly environment (DPVAE) is developed. In this environment, the mechanism of event synchronization based on high‐level architecture/run‐time infrastructure) is applied to realize multi‐user collaboratively interactive operation. To meet the large data set real‐time processing demand, a creative parallel processing approach supported by a single supercomputer or a parallel processing environment composed of common personal computer in a high‐speed local area network is developed. The technologies such as real‐time CD, multiple interactive operation modals are applied in DPVAE and several auxiliary tools are provide to help achieving whole scheme review, component model verification and assembly evaluation.FindingsThis paper finds that DPVAE system is an available and efficient tool to support automobile collaborative assembly design.Practical implicationsDesigners can discuss and verify the assembly scheme to realize the previous design scenario in DPVAE, so it is useful for reducing costs, improving quality and shortening the time to market, especially for new type automobile development.Originality/valueA combination of distributed technology and parallel computing technology is applied in product virtual assembly, solving the problems including collaborative work of multi‐user and large data real‐time processing successfully, that provides a useful tool for automobile development.

Integrated Virtual Assembly Environment and its application in ship piping layout

Complex pipe layout plays an important role in ship building. Pipes prefabricating and assembling often depends on testing and rebuilding in outfitting. This leads to more workload, high cost and longer manufacturing cycles. In this paper, a new virtual assembly system called Integrated Virtual Assembly Environment (IVAE) is introduced, in which the ship outfitting can be simulated and process planning can be completed. With the help of assembly constraints management, collision detection and response, assembly sequence and paths planning, users can assemble the parts interactively in a virtual outfitting environment, just as in an actual cabin. As well as for virtual ship outfitting, IVAE can also be used in general product development. As a typical case, the pipe layout in the engine room of a 5000 M3 hydraulic dredge from Hudong-Zhonghua Shipbuilding (Group) Co., Ltd is given to test virtual ship outfitting in IVAE.

Creation and evaluation of a multi-sensory virtual assembly environment

The multi-modal information presentation, integrated into the virtual environment (VE), has potential for stimulating different senses, improving the user’s impression of immersion, and increasing the amount of information that is accepted and processed by the user’s perception system. The increase of the useful feedback information may reduce the user’s cognitive load, thus enhancing the user’s efficiency and performance while interacting with VEs. This paper presents our creation of a multi-sensory virtual assembly environment (VAE) and the evaluation of the effects of multi-sensory feedback on the usability. The VAE brings together complex technologies such as constraint-based assembly simulation, optical motion tracking technology, and real-time 3D sound generation technology around a virtual reality workbench and a common software platform. The usability evaluation is in terms of its three attributes: efficiency of use, user satisfaction, and reliability. These are addressed by using task completion times (TCTs), questionnaires, and human performance error rates (HPERs), respectively. Two assembly tasks have been used to perform the experiments, using sixteen participants. The outcomes showed that the multi-sensory feedback could improve the usability. They also indicated that the integrated feedback offered better usability than either feedback used in isolation. Most participants preferred the integrated feedback to either feedback (visual or auditory) or no feedback. The participants’ comments demonstrated that nonrealistic or inappropriate feedback had negative effects on the usability, and easily made them feel frustrated. The possible reasons behind the outcomes are also analysed by using a unifying human computer interaction framework. The implications, concluded from the outcomes of this work, can serve as useful guidelines for improving VE system design and implementation.

The affect of contact force sensations on user performance in virtual assembly tasks

The development of a realistic virtual assembly environment is challenging because of the complexity of the physical processes and the limitation of available VR technology. Many research activities in this domain primarily focused on particular aspects of the assembly task such as the feasibility of assembly operations in terms of interference between the manipulated parts. The virtual assembly environment reported in this research is focused on mechanical part assembly. The approach presented addresses the problem of part-to-part contacts during the mating phase of assembly tasks. The system described calculates contact force sensations by making their intensity dependent on the depth of penetration. However the penetration is not visible to the user who sees a separate model, which does not intersect the mating part model. The two 3D models of the part, the off-screen rendered model and the on-screen rendered model are connected by a spring-dumper arrangement. The force calculated is felt by the operator through the haptic interface when parts come in contact during the mating phase of the assembly task. An evaluation study investigating the effect of contact force sensation on user performance during part-to-part interface was conducted. The results showed statistically significant effect of contact force sensation on user performance in terms of task completion time. The subjective evaluation based on feedback from users confirmed that contact force sensation is a useful cue for the operator to find the relative positions of components in the final assembly state.

Research on constraint-based virtual assembly technologies

To realize a constraint-based virtual assembly operation, the unified representations of the assembly constraint, the equivalent relation between the constraint and the degree of freedom (DOF), and the movement DOF reduction in a virtual environment are proposed. Several algorithms about the constraint treatment are submitted. First, the automatic recognition algorithm based on the assembly relation is used to determine the position and orientation relation between two geometry elements of constraint whether they meet the given errors. Second, to satisfy the new constraint, according to the positing solving algorithm, the position and orientation of an active part are modified with minimal adjustment after the part has satisfied the affirmed constraints. Finally, the algorithm of movement navigation based on the generalized coordinate system is put forward, and the part movement is guided. These algorithms have been applied to the integrated virtual assembly environment (IVAE) system. Experiments have indicated that these algorithms have well supported constraint treatments in the IVAE and realized the closer combination of the virtual and the real assembly processes.

Evaluation of Multi-sensory Feedback on the Usability of a Virtual Assembly Environment

Virtual assembly environment (VAE) technology has the great potential for benefiting the manufacturing applications in industry. Usability is an important aspect of the VAE. This paper presents the usability evaluation of a developed multi-sensory VAE. The evaluation is conducted by using its three attributes: (a) efficiency of use; (b) user satisfaction; and (c) reliability. These are addressed by using task completion times (TCTs), questionnaires, and human performance error rates (HPERs), respectively. A peg-in-a-hole and a Sener electronic box assembly task have been used to perform the experiments, using sixteen participants. The outcomes showed that the introduction of 3D auditory and/or visual feedback could improve the usability. They also indicated that the integrated feedback (visual plus auditory) offered better usability than either feedback used in isolation. Most participants preferred the integrated feedback to either feedback (visual or auditory) or no feedback. The participants’ comments demonstrated that nonrealistic or inappropriate feedback had negative effects on the usability, and easily made them feel frustrated. The possible reasons behind the outcomes are also analysed.

Evaluation of Auditory and Visual Feedback on Task Performance in a Virtual Assembly Environment

This paper presents the creation of an assembly simulation environment with multisensory feedback (auditory and visual), and the evaluation of the effects of auditory and visual feedback on the task performance in the context of assembly simulation in a virtual environment (VE). This VE experimental system platform brings together complex technologies such as constraint-based assembly simulation, optical motion tracking technology, and real time 3D sound generation technology around a virtual reality workbench and a common software platform. A peg-in-a-hole and a Sener electronic box assembly task have been used as the task cases to conduct the human factor experiment, using sixteen participants. Both objective performance data (i.e., task completion time, TCT; and human performance error rate, HPER) and subjective opinions (i.e., questionnaires) on the utilization of auditory and visual feedback in a virtual assembly environment (VAE) have been gathered from the experiment. Results showed that the introduction of auditory and/or visual feedback into VAE did improve the assembly task performance. They also indicated that integrated feedback (auditory plus visual) offered better assembly task performance than either feedback used in isolation. Most participants preferred integrated feedback to either individual feedback (auditory or visual) or no feedback. The participants' comments demonstrated that nonrealistic or inappropriate feedback had a negative effect on the task performance, and easily made them frustrated.

Virtual assembly technologies based on constraint and DOF analysis

Assembly constraint is an important factor for product assembly. Constraint is not only necessary for accurate locating of parts but also an important way to realize assembly operation process in virtual assembly (VA) system. In this study, to realize VA reasonably, uniform representations of assembly constraint, equivalent relation between constraint and degree of freedom (DOF), and movable DOF reduction are defined. Several algorithms including based-constraint assembly relation recognition, assembly parts location solving under free and constrained spaces, and movement navigation are developed. These algorithms have been applied to integration virtual assembly environment (IVAE) software system. An example is illustrated, and the results prove that the algorithms provide a good support for the constraint treatments in IVAE and the high consistence between virtual and real assembly process is realized.

Interactive visualization of complex dynamic virtual environments for industrial assemblies

Interactive visualization of assemblies in a virtual environment is important to the industrial applications like design verification and maintenance processes training. In general, the difficulty and the complication of the interactive visualization of industrial assemblies arise from the facts that an assembly model generally has a large number of primitives and high depth complexity; furthermore, dynamic environments are required for simulating assembly manipulations. In this paper, a dynamic data model for visualization of industrial assemblies is proposed. The data model integrates a spatial data set of hierarchical model representation and a dynamic transformation mechanism for runtime adaptation to the changes and movements of the assembly structures. Based on the dynamic spatial database, a visibility optimization approach that can efficiently reduce the overall geometry is presented. The complexity reduction is accomplished through view frustum culling, non-conservative occlusion culling, and geometry simplification. By which, significant speedup in user interaction rates has been achieved. Since all the techniques developed here are supported by mainstream graphics hardware, thus the solution is able to provide users with practical real-time visualization of complex dynamic virtual assembly environments with affordable graphics hardware.

A CAD-linked virtual assembly environment

This paper presents a novel approach to integrate VR and CAD for virtual assembly, namely, CAD linked virtual assembly environment. Compared with most existing VR and CAD integration approaches, the proposed approach is a kind of external integration with a CAD system through automation interfaces. A novel hybrid virtual assembly model is developed in this paper to support this external integration. It works as a combinator that links the precisely represented design dataset in CAD models and the corresponding hierarchically structured virtual assembly dataset in the VR application. With this model, a framework has been developed to embed and apply CAD assembly constraints on the virtual assembly structure. Constraint-based manipulation is then implemented in the virtual assembly environment. Moreover, with this framework, advanced computations, such as runtime DOF determination, disassembly directionality computation, and assembly/disassembly sequence generation, which rely on assembly constraint information, can be feasibly integrated into the virtual assembly application with much less effort. The approach is also more practical in terms of implementation due to two advantages: (1) It does not require the implementation of a complicated support for solid modelling in the virtual assembly application. (2) A virtual assembly environment can be automatically created and updated from the linked CAD models.

Development of a virtual and physical work cell to assemble micro-devices

The term ‘micro-assembly’ is used to describe the assembly of micron-sized parts that are extremely small (in the order of 10 −6 m). As manual assembly of micro-devices is extremely difficult and tedious, there is a need to design computer-controlled approaches to facilitate rapid assembly. In this context, the design of automated or semi-automated environments for micro-assembly applications becomes important. In this paper, the design of a micro-assembly cell is discussed along with the development of a virtual assembly environment to study micro-assembly problems and issues. A genetic algorithm-based assembly sequence generator is discussed which works in coordination with a 3D path planning approach. A physical micro-assembly cell has been created to aid in the assembly of such micro-devices. This physical cell is interfaced with a virtual reality (VR)-based assembly environment to assist in the micro-assembly activities and comprises of micro-positioners, a micro-gripper (for pick-and-place operations to complete specific micro-assembly tasks), and a camera (to provide guidance and feedback during and after assembly).

Construction of feature-matching perception in virtual assembly

An important characteristic of virtual assembly is interaction. Traditional direct manipulation in virtual assembly relies on dynamic collision detection, which is very time-consuming and even impossible in desktop virtual assembly environment. Feature-matching is a critical process in harmonious virtual assembly, and is the premise of assembly constraint sensing. This paper puts forward an active object-based feature-matching perception mechanism and a feature-matching interactive computing process, both of which make the direct manipulation in virtual assembly break away from collision detection. They also help to enhance virtual environment understandability of user intention and promote interaction performance. Experimental results show that this perception mechanism can ensure that users achieve real-time direct manipulation in desktop virtual environment.

Physically Based Modeling in Virtual Assembly

Virtual assembly is a promising application of virtual reality in design and manufacturing that has drawn much attention from industry and research institutes. Physically based modeling has been an important research topic in computer graphics and virtual reality. In this paper, physically based modeling issues in virtual assembly are investigated. The specific requirements and characteristics of physically based modeling in virtual assembly versus those in traditional computer graphics are analyzed and studied. The mass properties of the assembly models are extracted from the Computer Aided Design (CAD) system while the design models are transferred from the CAD system to the virtual assembly environment. The assembly models are categorized using human strength survey data. The interaction between the parts, the environment objects, and the human are analyzed. In the fully immersed virtual environment, it is discovered that the gravity acceleration needs to be scaled down to achieve maximum realistic feeling. Finally, the benefits and limitations of physically based modeling in virtual environments are discussed.

仮想組立環境における反力呈示のための階層処理に関する研究

This paper deals with a method for presenting reaction forces in virtual assembly environments, which is used for instructing or training techniques of assembly works. There is an important demand for the reaction force on stress of parts in the virtual assembly environment. In this paper, the C-Space (Configuration Space) representation and the hierarchical processing are proposed for satisfying the demand. The C-Space can represent all stress relations of parts clearly, and can indicate the current location immediately by making the space divided into some zones of geometrically important states for works. The hierarchical processing is realized with two-levels layer and reduces the calculation delay for controlling the reaction force. The qualitative level processing recognizes a task by kinematic reasoning with the C-Space, and the quantitative level processing calculates the reaction force by task based reaction models and physical data. A case study with two experiments shows that the method is effective for presenting reaction forces on a screw tightening work.