Motion Mapping Research Articles

Rotation sensing and gesture control of a robot joint via triboelectric quantization sensor

In human-machine interaction, robotic hands are expected to work like human's hands and to be even more powerful or delicate in certain situations. To operate robotic hands via human gesture instead of handle or button will make this human-robot interface more natural and precise. Here, we designed a joint motion triboelectric quantization sensor (jmTQS) for constructing a robotic hand synchronous control system. Based on the ultrahigh sensitivity of a triboelectric nanogenerator (TENG) to mechanical displacement, the jmTQS designed as grating-sliding mode realized directly quantifying a joint's flexion-extension degree/speed. Through counting the pulses induced by jmTQS and signing the positive/negative of the pulses to represent flexion/extension, the joint's angular position can be determined with absolute value on the basis of the initial human-robotic synchronizing position value. In the whole operating course, the intuitionistic human-robotic hand two-dimensional motion mapping can be preserved. The minimum resolution angle of the fabricated jmTQSs is 3.8° and can be further improved by decreasing the grating width. This direct quantization and intuitionistic mapping at the sensing stage greatly simplified the signal processing and classification algorithms, which contributes to achieving the natural, high-precision and real-time interface.

Feature-based visual odometry prior for real-time semi-dense stereo SLAM

Robust and fast motion estimation and mapping is a key prerequisite for autonomous operation of mobile robots. The goal of performing this task solely on a stereo pair of video cameras is highly demanding and bears conflicting objectives: on one hand, the motion has to be tracked fast and reliably, on the other hand, high-level functions like navigation and obstacle avoidance depend crucially on a complete and accurate environment representation. In this work, we propose a two-layer approach for visual odometry and SLAM with stereo cameras that runs in real-time and combines feature-based matching with semi-dense direct image alignment. Our method initializes semi-dense depth estimation, which is computationally expensive, from motion that is tracked by a fast but robust keypoint-based method. Experiments on public benchmark and proprietary datasets show that our approach is faster than state-of-the-art methods without losing accuracy and yields comparable map building capabilities. Moreover, our approach is shown to handle large inter-frame motion and illumination changes much more robustly than its direct counterparts.

Haptic based teleoperation with master-slave motion mapping and haptic rendering for space exploration

This paper presents a new solution to haptic based teleoperation to control a large-sized slave robot for space exploration, which includes two specially designed haptic joysticks, a hybrid master-slave motion mapping method, and a haptic feedback model rendering the operating resistance and the interactive feedback on the slave side. Two devices using the 3R and DELTA mechanisms respectively are developed to be manipulated to control the position and orientation of a large-sized slave robot by using both of a user’s two hands respectively. The hybrid motion mapping method combines rate control and variable scaled position mapping to realize accurate and efficient master-slave control. Haptic feedback for these two mapping modes is designed with emphasis on ergonomics to improve the immersion of haptic based teleoperation. A stiffness estimation method is used to calculate the contact stiffness on the slave side and play the contact force rendered by using a traditional spring-damping model to a user on the master side stably. Experiments by using virtual environments to simulate the slave side are conducted to validate the effectiveness and efficiency of the proposed solution.

Video representation by dense trajectories motion map applied to human activity recognition

ABSTRACTThis paper introduces an efficient video representation method based on the dense trajectory motion map (DTM). We utilize the salient features of dense trajectories and motion descriptor to integrate the discriminative information of a video into a map. Firstly, we extract the dense trajectories features by using dense optical flow then multiple descriptors are computed along trajectories to capture appearance and motion information. This result is then integrated into frames difference to integrate entire discriminative information and motion energy to get our first motion map. For the final DTM each generated motion map will be integrated with the absolute frame difference of next two frames till the end of entire video. Finally, we process the resultant DTM by exploring the efficient long-term recurrent convolutional network module for encoding and action label generation. The developed approach is shown better and had comparable recognition results over the existing methods when applied to the publically available human action datasets.

Badania dokładności i powtarzalności pozycjonowania robota przemysłowego w środowiskach off i online

The paper discusses issues concerning the accuracy and repeatability tests of the positioning of the Kuka KR 16-2 industrial robot. The results of laboratory tests of an industrial robot, as well as a comparison of robot motion paths in the Robcad environment with the real robot motion paths are presented. In order to register movement paths in the laboratory conditions, the laser tracker Faro Vantage was used. Frequent necessity to correct programs of industrial robots created in the offline environment, is a results, among others, from the insufficient experience of people who carry out programming, the environment in which robots work and the parameters of the robots themselves, and therefore their accuracy and repeatability. It is connected with the extension of the start-up time and high costs. The work describes the measurement method and attempts to determine the influence of the type of route and motion parameters on the accuracy and repeatability of robot. The accuracy of mapping of simulated robot motion in a virtual environment was also verified.

Human Action Recognition via Depth Maps Body Parts of Action

Human actions can be recognized from depth sequences. In the proposed algorithm, we initially construct depth, motion maps (DMM) by projecting each depth frame onto three orthogonal Cartesian planes and add the motion energy for each view. The body part of the action (BPoA) is calculated by using bounding box with an optimal window size based on maximum spatial and temporal changes for each DMM. Furthermore, feature vector is constructed by using BPoA for each human action view. In this paper, we employed an ensemble based learning approach called Rotation Forest to recognize different actions Experimental results show that proposed method has significantly outperforms the state-of-the-art methods on Microsoft Research (MSR) Action 3D and MSR DailyActivity3D dataset.

A novel method for natural motion mapping as a strategy of game immediacy

A novel method for natural motion mapping as a strategy of game immediacy

Online action recognition from RGB-D cameras based on reduced basis decomposition

Human action recognition from RGB-D cameras has recently become one of the major fields of research. While accuracy improvement was given more importance in previous action/gesture recognition methods, there are opportunities to work on improving the computational efficiency too. This paper introduces an efficient dimensionality reduction technique and classification mechanism to recognize actions from depth motion map features. For our proposed work, a recently introduced technique called reduced basis decomposition (RBD) is employed, which manages faster dimensional reduction with its unique mechanism of generating compressed basis vectors. The RBD has an offline error-determination and an online approximation mechanism, and it is faster than PCA/SVD. For classification, this paper employs a Probabilistic Collaborative Representation Classifier (Pro-CRC). The recommended classifier works based on probability in connection with $${l_2}$$-regularization. The combined effect of the methods above helps in achieving the state-of-the-art efficiency. In the standard protocol tests carried out in the MSR-Action3D dataset, our proposed method achieved a considerable accuracy of 91.7% which is better than the currently efficient method. Further, our proposed method also proved its effectiveness in the challenging, subject-generic test with a reported accuracy of 89.64% and an average accuracy of 85.70% in the cross fixed tests which included 252 combinations of all the subjects without repetition.

Combining CNN streams of RGB-D and skeletal data for human activity recognition

Inspired by the success of deep learning methods, for human activity recognition based on individual vision cues, this paper presents a ConvNets based approach for activity recognition by combining multiple vision cues. Moreover, a new method of creating skeleton images, from skeleton joint sequences, representing motion information is presented in this paper. Motion representation images, namely, Motion History Image (MHI), Depth Motion Maps (DMMs) and skeleton images are constructed from RGB, depth and skeletal data of RGB-D sensor. These images are then separately trained on ConvNets and respective softmax scores are fused at the decision level. The combination of these distinct vision cues, leads to complete utilization of data, available from RGB-D sensor. To evaluate the effectiveness of the proposed 5-CNNs approach, we conduct our experiments on three well known and challenging RGB-D datasets, CAD-60, SBU Kinect interaction and UTD-MHAD. Results show that the proposed approach of combining multiple cues by means of decision level fusion is competitive with other state of the art methods.

Accurate Monocular Visual-Inertial SLAM Using a Map-Assisted EKF Approach

This paper presents a novel tightly-coupled monocular visual-inertial Simultaneous Localization and Mapping algorithm, which provides accurate and robust localization within the globally consistent map in real time on a standard CPU. This is achieved by firstly performing the visual-inertial extended kalman filter(EKF) to provide motion estimate at a high rate. However the filter becomes inconsistent due to the well known linearization issues. So we perform a keyframe-based visual-inertial bundle adjustment to improve the consistency and accuracy of the system. In addition, a loop closure detection and correction module is also added to eliminate the accumulated drift when revisiting an area. Finally, the optimized motion estimates and map are fed back to the EKF-based visual-inertial odometry module, thus the inconsistency and estimation error of the EKF estimator are reduced. In this way, the system can continuously provide reliable motion estimates for the long-term operation. The performance of the algorithm is validated on public datasets and real-world experiments, which proves the superiority of the proposed algorithm.

Detailed spatiotemporal brain mapping of chromatic vision combining high-resolution VEP with fMRI and retinotopy.

Neuroimaging studies have identified so far, several color-sensitive visual areas in the human brain, and the temporal dynamics of these activities have been separately investigated using the visual-evoked potentials (VEPs). In the present study, we combined electrophysiological and neuroimaging methods to determine a detailed spatiotemporal profile of chromatic VEP and to localize its neural generators. The accuracy of the present co-registration study was obtained by combining standard fMRI data with retinotopic and motion mapping data at the individual level. We found a sequence of occipito activities more complex than that typically reported for chromatic VEPs, including feed-forward and reentrant feedback. Results showed that chromatic human perception arises by the combined activity of at the least five parieto-occipital areas including V1, LOC, V8/VO, and the motion-sensitive dorsal region MT+. However, the contribution of V1 and V8/VO seems dominant because the re-entrant activity in these areas was present more than once (twice in V8/VO and thrice in V1). This feedforward and feedback chromatic processing appears delayed compared with the luminance processing. Associating VEPs and neuroimaging measures, we showed for the first time a complex spatiotemporal pattern of activity, confirming that chromatic stimuli produce intricate interactions of many different brain dorsal and ventral areas.

Mobile Robot Control and Navigation: A Global Overview

The aim of this paper is to provide a global overview of mobile robot control and navigation methodologies developed over the last decades. Mobile robots have been a substantial contributor to the welfare of modern society over the years, including the industrial, service, medical, and socialization sectors. The paper starts with a list of books on autonomous mobile robots and an overview of survey papers that cover a wide range of decision, control and navigation areas. The organization of the material follows the structure of the author’s recent book on mobile robot control. Thus, the following aspects of wheeled mobile robots are considered: kinematic modeling, dynamic modeling, conventional control, affine model-based control, invariant manifold-based control, model reference adaptive control, sliding-mode control, fuzzy and neural control, vision-based control, path and motion planning, localization and mapping, and control and software architectures.

Action recognition with motion map 3D network

Abstract Recently, deep neural networks have demonstrated remarkable progresses for human action recognition in videos. However, most existing deep frameworks can not handle variable-length videos properly, which leads to the degradation in classification performance. In this paper, we propose a Motion Map 3D ConvNet(MM3D), which can represent the content of a video with arbitrary video length by a motion map. In our MM3D model, a novel generation network is proposed to learn a motion map to represent a video clip by iteratively integrating a current video frame into a previous motion map. A discrimination network is also introduced for classifying actions based on the learned motion map. Experiments on the UCF101 and the HMDB51 datasets prove the effectiveness of our method for human action recognition.

Indoor Pedestrian Motion Detection Via Spatial Clustering and Mapping

The proliferation of location-based services has driven the rapid development of indoor localization technology. The conventional location fingerprinting based localization approach suffers from the time-consuming and labor-intensive process of fingerprint database construction, and meanwhile, the widely known simultaneous localization and mapping seriously depends on various motion sensors. In response to these compelling problems, we propose a new indoor pedestrian motion detection approach without the assistance of location fingerprinting or motion sensing. Specifically, first of all, the motion paths are constructed by the motion model of the pedestrian in the target environment. Second, after collecting the wireless local area network (WLAN) received signal strength (RSS) sequence on each motion path, the concept of density-based spatial clustering is adopted to generate the RSS clusters, and in each, the RSS data are with the similar motion patterns. Third, based on the hotspot mapping relations between the physical subareas and RSS clusters, the target is located into the physical subarea mapped by the RSS cluster, which is most similar to the newly collected RSS data. Finally, the experimental results demonstrate that the proposed approach is capable of performing target localization as well as exploring the motion behavior of the pedestrian in an indoor environment.

Visual SLAM and Structure from Motion in Dynamic Environments

In the last few decades, Structure from Motion (SfM) and visual Simultaneous Localization and Mapping (visual SLAM) techniques have gained significant interest from both the computer vision and robotic communities. Many variants of these techniques have started to make an impact in a wide range of applications, including robot navigation and augmented reality. However, despite some remarkable results in these areas, most SfM and visual SLAM techniques operate based on the assumption that the observed environment is static. However, when faced with moving objects, overall system accuracy can be jeopardized. In this article, we present for the first time a survey of visual SLAM and SfM techniques that are targeted toward operation in dynamic environments. We identify three main problems: how to perform reconstruction (robust visual SLAM), how to segment and track dynamic objects, and how to achieve joint motion segmentation and reconstruction. Based on this categorization, we provide a comprehensive taxonomy of existing approaches. Finally, the advantages and disadvantages of each solution class are critically discussed from the perspective of practicality and robustness.

A convex polynomial model for planar sliding mechanics: theory, application, and experimental validation

We propose a polynomial model for planar sliding mechanics. For the force–motion mapping, we treat the set of generalized friction loads as the 1-sublevel set of a polynomial whose gradient directions correspond to generalized velocities. The polynomial is confined to be convex even-degree homogeneous in order to obey the maximum work inequality, symmetry, shape invariance in scale, and fast invertibility. We present a simple and statistically efficient model identification procedure using a sum-of-squares convex relaxation. We then derive the kinematic contact model that resolves the contact modes and instantaneous object motion given a position controlled manipulator action. The inherently stochastic object-to-surface friction distributions are modeled by sampling polynomial parameters from distributions that preserve sum-of-squares convexity. Thanks to the model smoothness, the mechanics of patch contact is captured while being computationally efficient without mode selection at support points. Simulation and robotic experiments on pushing and grasping validate the accuracy and efficiency of our approach.

Smartphone-Based Cooperative Indoor Localization with RFID Technology.

In GPS-denied indoor environments, localization and tracking of people can be achieved with a mobile device such as a smartphone by processing the received signal strength (RSS) of RF signals emitted from known location beacons (anchor nodes), combined with Pedestrian Dead Reckoning (PDR) estimates of the user motion. An enhacement of this localization technique is feasible if the users themselves carry additional RF emitters (mobile nodes), and the cooperative position estimates of a group of persons incorporate the RSS measurements exchanged between users. We propose a centralized cooperative particle filter (PF) formulation over the joint state of all users that permits to process RSS measurements from both anchor and mobile emitters, as well as PDR motion estimates and map information (if available) to increase the overall positioning accuracy, particularly in regions with low density of anchor nodes. Smartphones are used as a convenient mobile platform for sensor measurements acquisition, low-level processing, and data transmission to a central unit, where cooperative localization processing takes place. The cooperative method is experimentally demonstrated with four users moving in an area of 1600 , with 7 anchor nodes comprised of active RFID (radio frequency identification) tags, and additional mobile tags carried by each user. Due to the limited coverage provided by the anchor beacons, RSS-based individual localization is inaccurate (6.1 m median error), but this improves to 4.9 m median error with the cooperative PF. Further gains are produced if the PDR information is added to the filter: median error of 3.1 m (individual) and 2.6 m (cooperative); and if map information is also considered, the results are 1.8 m (individual) and 1.6 m (cooperative). Thus, for each version of the particle filter, cooperative localization outperforms individual localization in terms of positioning accuracy.

Variable motion mapping to enhance stiffness discrimination and identification in robot hand teleoperation

In robot hand teleoperation system, force feedback is critical for operator to perceive the physical property of grasped objects. However, it is challenging to implement accurate force feedback if the force accuracy of a haptic device is limited. This paper proposes a variable motion mapping method to enhance stiffness discrimination of the remote object. In this method, the motion mapping coefficient is regulated according to the object stiffness so that the operator can perceive the stiffness difference of the remote objects. To validate the proposed approach, we conducted two experiments on a three-finger robot hand (BarrettHand BH8-280) teleoperation system. In the first experiment, we measured the minimum relative change in feedback stiffness that could be detected by the operator. The result shows that the relative change should be 70% for a correct detection rate of about 91%. Based on the result, the motion mapping coefficient was selected for the system. In the second experiment, the variable motion mapping method was compared to a constant motion mapping method. The experimental task in the comparison is identifying four objects through stiffness perception alone. The experimental results show that the variable motion mapping method enhances the discrimination of the objects. The operator can identify individual object within a group of four without visual feedback with the variable motion mapping method.

Utilising the Virtual Environment for Radiotherapy Training System to Support Undergraduate Teaching of IMRT, VMAT, DCAT Treatment Planning, and QA Concepts

The use of three-dimensional virtual reality technologies in education has been widely reported in the literature. The goal of this article is to review how the virtual environment for radiotherapy training (VERT) can be utilised to support the teaching of intensity-modulated radiotherapy (IMRT), volumetric-modulated arc therapy, and dynamic conformal arc therapy techniques. Utilising Pinnacle v14 (Philips Radiation Oncology Systems, Madison, WI) and Monaco v 5.10 (Elekta CMS, Maryland Heights, MO), we exported IMRT, volumetric-modulated arc therapy, and dynamic conformal arc therapy treatment plans to VERT. Quality assurance (QA) plans were also exported from the Monaco planning software to demonstrate theoretical concepts and virtual plan delivery using the ArcCHECK phantom (Sun Nuclear Corp, Melbourne, FL) and solid water IMRT phantom. Several VERT features were utilised to critically evaluate dose coverage and organ at risk sparing on inverse treatment plans. The display of gantry and multileaf collimator motion and fluence maps could be a useful feature in the education of treatment planning concepts. QA could be delivered on VERT to demonstrate patient-specific QA concepts in a virtual environment. Anecdotal evidence shows that the use of VERT for collaborative plan evaluation sessions is likely to engage the students and improve their ability to evaluate treatment plans. VERT can be utilised to reinforce understanding of treatment plan evaluation skills, QA, and treatment delivery of inverse plans in educational environments.

Motion mapping of the heterogeneous master–slave system for intuitive telemanipulation

Heterogeneous master–slave robots are widely used as both the assistive robot system for the elder/disabled people and the teleoperation robot system in dangerous environments. For the sake of intuitive teleoperation, motion mapping should be applied to keep the slave arm possessing the similar configuration of the master arm in the whole course of telemanipulation. We propose a motion mapping on the joint analog level based on unit dual quaternions (UDQs) for the telecontrol of a heterogeneous slave robot arm via the motion of human master arm. First, we analogize the links of the slave robot to the links of the human master arm, and accordingly group the slave arm joints into “shoulder,” “elbow,” and “wrist” joint analog sets. Then, we capture the motion of the human master arm via a wearable motion capture system and take it as the motion order of the slave arm. Finally, we solve the motion of slave robot arm’s joint via UDQ-based 1-, 2-, and 3-Degree of freedom (DOF) orientation error UDQ decomposition algorithms that are suit for solving a typical class of robotic slave arms. As a result, the operator can manipulate the heterogeneous robot system intuitively and conveniently, and the fatigue degree and error rate of the user can be significantly reduced, and thus the safety of teleoperation in an unstructured and constrained environment can be significantly improved.