Manipulation Of Virtual Objects Research Articles

Changes in body perception following virtual object manipulation are accompanied by changes of the internal reference scale

Changes in body perception often arise when observers are confronted with related yet discrepant multisensory signals. Some of these effects are interpreted as outcomes of sensory integration of various signals, whereas related biases are ascribed to learning-dependent recalibration of coding individual signals. The present study explored whether the same sensorimotor experience entails changes in body perception that are indicative of multisensory integration and those that indicate recalibration. Participants enclosed visual objects by a pair of visual cursors controlled by finger movements. Then either they judged their perceived finger posture (indicating multisensory integration) or they produced a certain finger posture (indicating recalibration). An experimental variation of the size of the visual object resulted in systematic and opposite biases of the perceived and produced finger distances. This pattern of results is consistent with the assumption that multisensory integration and recalibration had a common origin in the task we used.

Bimanual Motor Strategies and Handedness Role in Human-Robot Haptic Interaction.

Bimanual object manipulation involves using both hands to interact with objects in the environment, and the process requires the central nervous system to process sensory feedback and translate it into motor commands. Although there have been significant advancements in haptics and robotics, the kinematic strategies involved in bimanual coupled tasks are still not fully understood. This study aimed to investigate the dynamic interaction between hands during the manipulation of a shared object using two impedance-controlled exoskeletons programmed to simulate bimanual coupled manipulation of virtual objects. Twenty-six participants (right-handed and left-handed) were asked to use both hands to grab and place simulated objects in specific locations. The virtual objects were rendered with four different dynamic properties, affecting the manipulation strategies used to complete the tasks. The results showed that force asymmetries were related to movement direction and handedness preference, with right-handers exhibiting asymmetries related to movement direction and left-handers showing better control of the force applied between their hands. This is possibly due to their constant exposure to objects designed for right-handed use. Additionally, the haptic properties of the virtual objects influenced task performance in terms of timing and failure for all participants. This study demonstrates the potential of advanced technologies to provide realistic simulations of multi-joint movements involving the entire upper extremities. The findings have implications for the development of training programs for bimanual object manipulation tasks and the design of virtual environments that can enhance the learning process.

Design and Evaluation of Bare-Hand Interaction for Precise Manipulation of Distant Objects in AR

Interaction with virtual objects is one of the essential features of Augmented Reality (AR) systems. One of its main issues is how to provide precise manipulation of distant virtual objects in AR. In this work, we explore bare-hand manipulation of distant objects in AR with DOF (degree-of-freedom) separation, motion scaling, and near-field metaphors. We developed two manipulation techniques: the distant widget-based metaphor (DWBM), and the near-field widget-based metaphor (NFWBM). We conducted a user study with 20 participants to compare the two techniques in terms of performance and user experience. We found that NFWBM has faster speed, lower mental effort, better ease-of-use, and a friendlier user experience. However, there is no significant difference in terms of precision; both techniques could manipulate distant objects precisely, with an average position error of less than 0.7 cm and an average orientational error of less than 1°. We also discussed the limitations of this research and directions for future work.

Immersive Virtual Reality Avatars for Embodiment Illusions in People With Mild to Borderline Intellectual Disability: User-Centered Development and Feasibility Study.

Immersive virtual reality (IVR) has been investigated as a tool for treating psychiatric conditions. In particular, the practical nature of IVR, by offering a doing instead of talking approach, could support people who do not benefit from existing treatments. Hence, people with mild to borderline intellectual disability (MBID; IQ=50-85) might profit particularly from IVR therapies, for instance, to circumvent issues in understanding relevant concepts and interrelations. In this context, immersing the user into a virtual body (ie, avatar) appears promising for enhancing learning (eg, by changing perspectives) and usability (eg, natural interactions). However, design requirements, immersion procedures, and proof of concept of such embodiment illusion (ie, substituting the real body with a virtual one) have not been explored in this group. Our study aimed to establish design guidelines for IVR embodiment illusions in people with MBID. We explored 3 factors to induce the illusion, by testing the avatar's appearance, locomotion using IVR controllers, and virtual object manipulation. Furthermore, we report on the feasibility to induce the embodiment illusion and provide procedural guidance. We conducted a user-centered study with 29 end users in care facilities, to investigate the avatar's appearance, controller-based locomotion (ie, teleport, joystick, or hybrid), and object manipulation. Overall, 3 iterations were conducted using semistructured interviews to explore design factors to induce embodiment illusions in our group. To further understand the influence of interactions on the illusion, we measured the sense of embodiment (SoE) during 5 interaction tasks. IVR embodiment illusions can be induced in adults with MBID. To induce the illusion, having a high degree of control over the body outweighed avatar customization, despite the participants' desire to replicate their own body image. Similarly, the highest SoE was measured during object manipulation tasks, which required a combination of (virtual) locomotion and object manipulation behavior. Notably, interactions that are implausible (eg, teleport and occlusions when grabbing) showed a negative influence on SoE. In contrast, implementing artificial interaction aids into the IVR avatar's hands (ie, for user interfaces) did not diminish the illusion, presuming that the control was unimpaired. Nonetheless, embodiment illusions showed a tedious and complex need for (control) habituation (eg, motion sickness), possibly hindering uptake in practice. Balancing the embodiment immersion by focusing on interaction habituation (eg, controller-based locomotion) and lowering customization effort seems crucial to achieve both high SoE and usability for people with MBID. Hence, future studies should investigate the requirements for natural IVR avatar interactions by using multisensory integrations for the virtual body (eg, animations, physics-based collision, and touch) and other interaction techniques (eg, hand tracking and redirected walking). In addition, procedures and use for learning should be explored for tailored mental health therapies in people with MBID.

A Novel Gesture-Based Control System for Fluorescence Volumetric Data in Virtual Reality.

With the development of light microscopy, it is becoming increasingly easy to obtain detailed multicolor fluorescence volumetric data. The need for their appropriate visualization has become an integral part of fluorescence imaging. Virtual reality (VR) technology provides a new way of visualizing multidimensional image data or models so that the entire 3D structure can be intuitively observed, together with different object features or details on or within the object. With the need for imaging advanced volumetric data, demands for the control of virtual object properties are increasing; this happens especially for multicolor objects obtained by fluorescent microscopy. Existing solutions with universal VR controllers or software-based controllers with the need to define sufficient space for the user to manipulate data in VR are not usable in many practical applications. Therefore, we developed a custom gesture-based VR control system with a custom controller connected to the FluoRender visualization environment. A multitouch sensor disk was used for this purpose. Our control system may be a good choice for easier and more comfortable manipulation of virtual objects and their properties, especially using confocal microscopy, which is the most widely used technique for acquiring volumetric fluorescence data so far.

The SmARtphone Controller

Abstract Current interaction modalities for mobile Augmented Reality (AR) are tedious and lack expressiveness. To overcome these prevalent limitations, we developed and evaluated a multimodal interaction concept by pairing a smartphone as an input and output modality for mobile AR. In a user study (n = 24), we investigated the effects on interaction speed, accuracy, and task load for (1) virtual object manipulation as well as (2) interaction with established graphical user interfaces (GUIs). Our findings show that a smartphone-based AR controller results in significantly faster and more accurate object manipulation with reduced task load than state-of-art mid-air gestures. Our results also indicate a significant enhancement for using the physical touchscreen as a modality compared to mid-air gestures for GUI interaction. We conclude that interaction in mobile AR environments can be improved by utilizing a smartphone as an omnipresent controller. Additionally, we discuss how future AR systems can benefit from tangible touchscreens as an additional and complementary interaction modality.

Using the Visuo-Haptic Illusion to Perceive and Manipulate Different Virtual Objects in Augmented Reality

The prop-based 3D virtual object manipulation method is widely used for interaction in Augmented Reality (AR) due to its convenience and flexibility. However, when the represented virtual object is different from the physical prop, the look and feel of the object are not well aligned. To address this problem, we present a dynamic finger remapping approach to creating a visuo-haptic illusion that dynamically adjusts the presented virtual hand posture to fit different sizes and shapes of virtual objects in AR. The finger movement toward a physical prop is synchronously remapped to the movement of the virtual fingers towards the corresponding virtual object. We developed a system that enables users to perceive consistent visual and tactile feedback while grasping and releasing various virtual objects represented by a physical prop. We conducted a user study to explore the effect of this visuo-haptic illusion on the perceived size of virtual objects, setting the sizes of the rendered virtual object and the physical prop as independent variables. We found that the perceived size of a virtual object varied with its rendered size in an almost linear fashion, while the physical prop size did not significantly affect the perception. We also conducted a second study to compare our system with a current prop-based method on virtual object manipulation. The results indicated that the remapped hands could effectively improve the realism and naturalness of the experience.

USE OF AUGMENTED REALITY TECHNOLOGIES IN SOFTWARE FOR INTERACTIVE VISUALIZATION OF THREE-DIMENSIONAL OBJECTS ON INDUSTRIAL SUBJECTS

The digital economy is an important vector of the National Economic Strategy of Ukraine. The digital economy opens new opportunities and incentives for various areas of domestic business, including the development of industrial enterprises. Among the modern available digital technologies, the article considers one of the most promising technologies – augmented reality. Interest in augmented reality is shown by industrial enterprises that invest in its development, which contributes to its spread and improvement. This paper investigates the methods of application of AR in industry. The software application with the technology of interactive visualization of three-dimensional objects is implemented in the work. objects by means of augmented reality in real time and on a real scale with the function of storing and transmitting scenes with objects in space to mass mobile devices for implementation in industrial enterprises. Thanks to the existing scientific achievements in this field, analysis and synthesis of publicly available scientific works, the authors formed the main provisions of augmented reality and considered the main elements of this process in the chosen field. The experimental part of the work was performed using a set of development tools for ArCore tracking. A feature of the proposed system is the ability to create and manage a group of objects, presenting objects in real size and in real time for mass mobile devices. It was possible to change the parameters of augmented reality objects when visualized by entering a block of generalized parameters, which speeds up the interaction and includes the ability to change in real time and simplify the user interface. The mechanism and algorithm of manipulation of virtual three-dimensional objects in space with the use of input methods with the possibility of simultaneous movement and rotation, as well as the speed and stability of work and the technology of hiding objects are implemented. A promising area of the study is the application of developed technologies in various industries.

Interaction with Virtual Content using Augmented Reality

Assembly procedures are a common task in several domains of application. Augmented Reality (AR) has been considered as having great potential in assisting users while performing such tasks. However, poor interaction design and lack of studies, often results in complex and hard to use AR systems. This paper considers three different interaction methods for assembly procedures (Touch gestures in a mobile device; Mobile Device movements; 3D Controllers and See-through HMD). It also describes a controlled experiment aimed at comparing acceptance and usability between these methods in an assembly task using Lego blocks. The main conclusions are that participants were faster using the 3D controllers and Video see-through HMD. Participants also preferred the HMD condition, even though some reported light symptoms of nausea, sickness and/or disorientation, probably due to limited resolution of the HMD cameras used in the video see-through setting and some latency issues. In addition, although some research claims that manipulation of virtual objects with movements of the mobile device can be considered as natural, this condition was the least preferred by the participants.

Workspace Enlargement of Cable-Based Haptic Device Using Linear Actuator

Haptic displays have been attracting attention for several decades and a broad range of devices have been developed to allow interaction with virtual objects. Among these, cablebased architectures were researched in order to overcome the lack of flexibility and the intrusiveness inherent to rigid haptic displays. Cable robots allow the utilization of several end effectors, which can be proved to be particularly useful for providing haptic rendering on several fingers for the bimanual manipulation of virtual objects. Nonetheless, interference can appear when cables touch each other during manipulation. Also, some configurations may require a high number of cables to make full use of the frame of the device. Here, we propose a novel architecture of a haptic display based on SPIDAR. Our purpose is to enlarge the usable workspace of an existing device configuration while lowering the number of cables and without hindering the quality of the display. A hybrid planar haptic device architecture with n+1 cables that includes a linear module is introduced. Design considerations and the kinematics used are detailed. Then, simple tasks are performed to verify the usability of the device as a haptic display.

Mobile augmented reality: A pedagogical strategy in the university setting

<p>The mobile augmented reality (M-AR) besides being a booming computer technology is an innovative tool that can support the pedagogical process in university classrooms, that is why the present research aims to show a methodological proposal for its implementation, in order to facilitate the learning of spatial reasoning of students, through the visualization and manipulation of three-dimensional virtual objects, promoting the motivation of learning knowledge and topics of the course of industrial design and technical drawing for the career of industrial engineering. A collection of geometric figures has been elaborated with the help of technological tools such as 2D and 3D modeling software, computer-aided design software and augmented reality application software. An updated methodology is proposed, available to any teacher, oriented to the stimulation of mental processes related to spatial reasoning of students, which integrates technological tools in the didactics of the dihedral system and the different graphic projections.</p>

Mobile augmented reality: A pedagogical strategy in the university setting

<p>The mobile augmented reality (M-AR) besides being a booming computer technology is an innovative tool that can support the pedagogical process in university classrooms, that is why the present research aims to show a methodological proposal for its implementation, in order to facilitate the learning of spatial reasoning of students, through the visualization and manipulation of three-dimensional virtual objects, promoting the motivation of learning knowledge and topics of the course of industrial design and technical drawing for the career of industrial engineering. A collection of geometric figures has been elaborated with the help of technological tools such as 2D and 3D modeling software, computer-aided design software and augmented reality application software. An updated methodology is proposed, available to any teacher, oriented to the stimulation of mental processes related to spatial reasoning of students, which integrates technological tools in the didactics of the dihedral system and the different graphic projections.</p>

Intuitive Virtual Objects Manipulation in Augmented Reality: Interaction between User’s Hand and Virtual Objects

Augmented Reality (AR) is a technological advancement which brings to reality the intuitive interaction that a user is attracted to virtual objects. In this paper, we focus on intuitive manipulation of virtual objects with bare hands. Specifically, we propose a method to deform, move, and join virtual objects by intuitive gestures, and aim at intuitive interactions with virtual objects. In order to manipulate virtual objects with bare hands, the position coordinates of the user’s fingers are acquired using the Leap Motion Controller. Additionally, we use Bullet Physics as an engine to represent virtual objects. On the other hand, in AR technology, since the three-dimensional (3D) model is superimposed on the image of the real space afterward, it is always displayed on the front end rather than the user’s hand (occlusion problem). Thus, the user cannot perform an intuitive manipulation. We solve the occlusion problem by performing appropriate hidden surface removal. From the results of a study using data gathered from a questionnaire survey, we found support for the intuitive interaction between the user and virtual objects.

Effect of Cutaneous Feedback on the Perception of Virtual Object Weight during Manipulation

Haptic interface technologies for virtual reality applications have been developed to increase the reality and manipulability of a virtual object by creating a diverse tactile sensation. Most evaluation of the haptic technologies, however, have been limited to the haptic perception of the tactile stimuli via static virtual objects. Noting this, we investigated the effect of lateral cutaneous feedback, along with kinesthetic feedback on the perception of virtual object weight during manipulation. We modeled the physical interaction between a participant’s finger avatars and virtual objects. The haptic stimuli were rendered with custom-built haptic feedback systems that can provide kinesthetic and lateral cutaneous feedback to the participant. We conducted two virtual object manipulation experiments, 1. a virtual object manipulation with one finger, and 2. the pull-out and lift-up of a virtual object grasped with a precision grip. The results of Experiment 1 indicate that the participants felt the virtual object rendered with lateral cutaneous feedback significantly heavier than with only kinesthetic feedback (p < 0.05 for mref = 100 and 200 g). Similarly, the participants of Experiment 2 felt the virtual objects significantly heavier when lateral cutaneous feedback was available (p < 0.05 for mref = 100, 200, and 300 g). Therefore, the additional lateral cutaneous feedback to the force feedback led the participants to feel the virtual object heavier than without the cutaneous feedback. The results also indicate that the contact force applied to a virtual object during manipulation can be a function of the perceived object weight (p = 0.005 for Experiment 1 and p = 0.02 for Experiment 2).

3D Mirrored Object Selection for Occluded Objects in Virtual Environments

There is an increasing demand for the manipulation of virtual objects in 3D virtual reality (VR) space, which begins with the user selecting a desired object. Existing selection methods aim to provide an intuitive and natural experience that minimizes user fatigue. Although typical ray-casting methods are effective when objects in the virtual space are sparsely or evenly placed, the selection becomes challenging when these objects are smaller or farther away, particularly when they overlap. In this study, we propose a method to address these challenges via an effective selection process. Based on 3D mirroring, the proposed method provides a new interaction metaphor that allows for a more efficient selection of occluded virtual objects. We conduct systematic experiments with 21 subjects and use various levels of target visibility determined by object size, spatial density, and degrees of occlusion. The results indicate that, in occluded 3D environments, the proposed 3D mirrored selection process outperforms traditional gaze-supported selection processes and offers better user satisfaction.

A bimodal soft electronic skin for tactile and touchless interaction in real time

The emergence of smart electronics, human friendly robotics and supplemented or virtual reality demands electronic skins with both tactile and touchless perceptions for the manipulation of real and virtual objects. Here, we realize bifunctional electronic skins equipped with a compliant magnetic microelectromechanical system able to transduce both tactile—via mechanical pressure—and touchless—via magnetic fields—stimulations simultaneously. The magnetic microelectromechanical system separates electric signals from tactile and touchless interactions into two different regions, allowing the electronic skins to unambiguously distinguish the two modes in real time. Besides, its inherent magnetic specificity overcomes the interference from non-relevant objects and enables signal-programmable interactions. Ultimately, the magnetic microelectromechanical system enables complex interplay with physical objects enhanced with virtual content data in augmented reality, robotics, and medical applications.

Virtual Object Manipulation by Combining Touch and Head Interactions for Mobile Augmented Reality

This paper proposes an interaction method to conveniently manipulate a virtual object by combining touch interaction and head movements for a head-mounted display (HMD), which provides mobile augmented reality (AR). A user can conveniently manipulate a virtual object with touch interaction recognized from the inertial measurement unit (IMU) attached to the index finger’s nail and head movements tracked by the IMU embedded in the HMD. We design two interactions that combine touch and head movements, to manipulate a virtual object on a mobile HMD. Each designed interaction method manipulates virtual objects by controlling ray casting and adjusting widgets. To evaluate the usability of the designed interaction methods, a user evaluation is performed in comparison with the hand interaction using Hololens. As a result, the designed interaction method receives positive feedback that virtual objects can be manipulated easily in a mobile AR environment.

AR Object Manipulation on Depth-Sensing Handheld Devices

Recently released, depth-sensing-capable, and moderately priced handheld devices support the implementation of augmented reality (AR) applications without the requirement of tracking visually distinct markers. This relaxed constraint allows for applications with significantly increased augmentation space dimension, virtual object size, and user movement freedom. Being relatively new, there is currently a lack of study on issues concerning direct virtual object manipulation for AR applications on these devices. This paper presents the results from a survey of the existing object manipulation methods designed for traditional handheld devices and identifies potentially viable ones for newer, depth-sensing-capable devices. The paper then describes the following: a test suite that implements the identified methods, test cases designed specifically for the characteristics offered by the new devices, the user testing process, and the corresponding results. Based on the study, this paper concludes that AR applications on newer, depth-sensing-capable handheld devices should manipulate small-scale virtual objects by mapping directly to device movements and large-scale virtual objects by supporting separate translation and rotation modes. Our work and results are the first step in better understanding the requirements to support direct virtual object manipulation for AR applications running on a new generation of depth-sensing-capable handheld devices.

On perceptual biases in virtual object manipulation: Signal reliability and action relevance matter.

This study examined the role of visual reliability and action relevance in mutual visual-proprioceptive attraction in a virtual grasping task. Participants initially enclosed either the width or the height of a visual rectangular object with two cursors controlled by the movements of the index finger and thumb. Then, either the height or the width of this object or the distance between the fingers was judged. The judgments of object's size were attracted by the felt finger distance, and, vice versa, the judged finger distance was attracted by the size of the grasped object. The impact of the proprioceptive information on object judgments increased, whereas the impact of visual object information on finger judgments decreased when the reliability of the visual stimulus was reduced. Moreover, the proprioceptive bias decreased for the action-relevant stimulus dimension as compared with the action-irrelevant stimulus dimension. These results indicate sensory integration of spatially separated sensory signals in the absence of any direct spatial or kinematic relation between them. We therefore suggest that the basic principles of sensory integration apply to the broad research field on perceptual-motor interactions as well as to many virtual interactions with external objects.

Design and Implementation of Decoupled IoT Application Store: A Novel Prototype for Virtual Objects Sharing and Discovery

The internet of things (IoT) has been growing towards the conception of a cyber-physical space in which physical things could be discovered, examined, activated, interlinked, and updated, for the realization of possible interactions in both virtual and physical space. The fundamental idea of IoT is the concept of a virtual object, which is regarded as the digital equivalent of a physical entity. Currently, in every IoT platform, the use of virtual objects has become a vital component. These virtual objects form building blocks of IoT applications, support the discovery of services, nurture the production of complex applications, foster the objects’ energy and power management efficiency, as well as address heterogeneity and scalability issues. Idea sharing and reusing are widesplread, and different solutions are built for this purpose to share applications or their components with similar domains to avoid duplication of efforts. In this paper, we design and implement a cloud-centric IoT application store that serves a purpose for hosting virtual objects of different IoT domains so that technology tinkerers can consume them and integrate them to build IoT applications. The proposed system is different than existing IoT marketplaces in the sense that they provide full-fledged IoT applications which include software and hardware that users can plug and play. The application store is aimed to be decoupled and can expose virtual objects of different IoT domains so that similar areas can use them with little or no modification. Moreover, it is also aimed to be modular, scalable, secure and support heterogeneity, which are considered vital attributes of IoT applications. An IoT testbed client application is developed to reuse some of the virtual objects from the proposed application store and to share specialized virtual objects to the proposed system for the use of other clients applications with similar goals. The performance and load of the platform are tested and found to be within an acceptable response time for up to 40 simultaneous users.