Motion Mapping Research Articles

Understanding Socially Aware Robot Navigation

Navigation is an essential skill for autonomous robots and it becomes a cumbersome task in human populated environments. Robots need to perform the tasks without disturbing the humans around them and ensure comfort and safety of humans as well. It is further influenced by various factors like social norms, geometry of environment and surrounding people. It is essential to comprehend three components i.e. Social Conventions (SC), Human Motion (HM) and Context Aware Mapping (CAM) for establishing effective socially aware robot navigation (SARN). This article discusses these different aspects of these components which should be taken into consideration while designing an efficient and optimal SARN. Further, it reports recent experiments conducted by different institutes pertaining to these components.

Measuring Sea Level Rise Along the Coast

Scientists created a global map of vertical land motion to show how the solid ground is moving relative to the planet’s rising seas.

Bilateral teleoperation with object-adaptive mapping

Task space mapping approaches for bilateral teleoperation, namely object-centered ones, have yielded the most promising results. In this paper, we propose an invertible mapping approach to realize teleoperation through online motion mapping by taking into account the locations of objects or tools in manipulation skills. It is applied to bilateral teleoperation, with the goal of handling different object/tool/landmark locations in the user and robot workspaces while the remote objects are moving online. The proposed approach can generate trajectories in an online manner to adapt to moving objects, where impedance controllers allow the user to exploit the haptic feedback to teleoperate the robot. Teleoperation experiments of pick-and-place tasks and valve turning tasks are carried out with two 7-axis torque-controlled Panda robots. Our approach shows higher efficiency and adaptability compared with traditional mappings.

Robotic Template Library

Robotic Template Library (RTL) is a set of tools for dealing with geometry and point cloud processing, especially in robotic applications. The software package covers basic objects such as vectors, line segments, quaternions, rigid transformations, etc., however, its main contribution lies in the more advanced modules: The segmentation module for batch or stream clustering of point clouds, the fast vectorization module for approximation of continuous point clouds by geometric objects of higher grade and the LaTeX export module enabling automated generation of high-quality visual outputs. It is a header-only library written in C++17, uses the Eigen library as a linear algebra back-end, and is designed with high computational performance in mind. RTL can be used in all robotic tasks such as motion planning, map building, object recognition and many others, but the point cloud processing utilities are general enough to be employed in any field touching object reconstruction and computer vision applications as well.

A comparison of digitized rotations of neighborhood motion maps of closest neighbors on 2D regular grids

A comparison of digitized rotations of neighborhood motion maps of closest neighbors on 2D regular grids

Extending Reach Inside the MRI Bore: A 7-DOF, Low-Friction, Hydrostatic Teleoperator

We present a hydrostatic teleoperator that provides physicians remote access inside the MRI bore and enables real-time MRI guided interventions such as liver biopsies. The device consists of a custom six-axis arm and a needle insertion end effector. The manipulator is passive and backdrivable with a near one-to-one mapping of motions and forces between the input and output. The six-axis arm translates and orients the needle during insertion and passively reflects respiratory motion while maintaining contact with the skin surface. Arm joints employ novel rotary rolling-diaphragms that provide stiff and low-friction rotational motion without the need for cables, belts, or gear mechanisms found in other solutions. We perform experiments to characterize the device's force and position tracking and demonstrate its functionality with path following tasks. Our results find a system roll-off frequency at 20Hz and that teleoperation tasks are performed comparably to holding the output directly. We motivate the need for force transparency in MRI guided biopsies with a user study in which five radiologists perform phantom membrane puncture biopsies using the needle insertion mechanism. The results indicate a 50% reduction in scans and a 14.5% decrease in membrane overshoot when force feedback is present.

Exploiting In-Hand Knowledge in Hybrid Joint-Cartesian Mapping for Anthropomorphic Robotic Hands

Replication of human hand motions on anthropomorphic robotic hands is typically treated in literature as the combination of two sub-problems: the measurement of human hand motions, and the mapping of such motions on the robotic hand. In this letter we focus on the second one. Different approaches have been proposed to deal with this problem, but none of them preserves both master finger shapes and fingertip positions on the robotic hand, i.e. ensuring predictability and natural motion for the teleoperator. In this article, we propose a novel hybrid approach that combines both joint and Cartesian mappings in a single solution. In particular, we exploit the a priori, in-hand information related to the areas of the workspace in which thumb and finger fingertips can get in contact. This allows to define, for each finger, a zone of transition from joint to Cartesian mapping. As a consequence, both hand shape during volar grasps and correctness of the fingertip positions for precision grasps are preserved, despite the master-slave kinematic dissimilarities. The proposed hybrid mapping is presented and experimentally evaluated both in simulation and with a real slave anthropomorphic robotic hand.

Kinematic Modelling and Motion Mapping of Robotic Arms

Robotic systems are known for their precision and accuracy. To achieve a desired set of operating parameters the system must have a good control system. Robotic arms can vary in size shape and purpose of use. With increasing DOF (Degree of Freedom) the complexity of the system increases and hence a DAQ system is required to check various parameters of the process. This paper discusses the implementation of kinematic and inverse kinematic modelling of a 6 Degree of Freedom (DOF) hydraulic arm in MATLAB environment. The forward kinematic model is based on Denavit-Hartenberg parameters. To obtain a given set of position and orientation of end effector, inverse kinematics gives the required set of joint angles. Potentiometers mounted on arm joint are used as the prime feedback elements for extracting the joint angles. All six joints of the arm are revolute joints. The values of potentiometer obtained are compared to the desired input values. Proportional direction control valve is the main controlling element of the hydraulic circuit. The forward and inverse kinematics is validated and implemented in MATLAB robotics toolbox (developed by Peter Croke). A GUI is also used to ease the process of retrieving and feeding the information in the system. The results of forward and inverse kinematics from MATLAB are verified by the physical model discussed in this paper.

Comparison of maximum likelihood and conventional PET scatter scaling methods for 18 F-FDG and 68 Ga-DOTATATE PET/CT.

We aim to quantify differences between a new maximum likelihood (ML) background scaling (MLBS) algorithm and two conventional scatter scaling methods for clinical PET/CT. A common source of reduced image quantification with conventional scatter corrections is attributed to erroneous scaling of the initial scatter estimate to match acquired scattered events in the sinogram. MLBS may have performance advantages over conventional methods by using all available data intersecting the subject. A retrospective analysis was performed on subjects injected with 18 F-FDG (N=71) and 68 Ga-DOTATATE (N=11) and imaged using time-of-flight (TOF) PET/CT. The scatter distribution was estimated with single scatter simulation approaches. Conventional scaling algorithms included (a) tail fitted background scaling (TFBS), which scales the scatter to "tails" outside the emission support, and (b) absolute scatter correction (ABS), which utilizes the simulated scatter distribution with no scaling applied. MLBS consisted of an alternating iterative reconstruction with a TOF-based ML activity image update allowing negative values (NEG-ML) and nested loop ML scatter scaling estimation. Scatter corrections were compared using reconstructed images as follows: (a) normalized relative difference images were generated and used for voxel-wise analysis, (b) liver and suspected lesion ROIs were drawn to compute mean SUVs, and (c) a qualitative analysis of overall diagnostic image quality, impact of artifacts, and lesion conspicuity was performed. Absolute quantification and normalized relative differences were also assessed with an 18 F-FDG phantom study. For human subjects 18 F-FDG data, Bland-Altman plots demonstrated that the largest normalized voxel-wise differences were observed close to the lower limit (SUV=1.0). MLBS reconstructions trended towards higher scatter fractions compared to TFBS and ABS images, with median voxel differences across all subjects for TFBS-MLBS measured at 1.7% and 7.6% for 18 F-FDG and 68 Ga-DOTATATE, respectively. For mean SUV analysis, there was a high degree of correlation between the scatter corrections. For 18 F-FDG, ABS scatter correction reconstructions trended towards higher liver mean SUVs relative to MLBS. The qualitative image analysis revealed no significant differences between TFBS and MLBS image reconstructions. For a uniformly filled relatively large 37cm diameter phantom, MLBS produced the lowest bias in absolute quantification, while normalized voxel-wise differences showed a trend in scatter correction performance consistent with the human subjects study. For 18 F-FDG, MLBS is at least a valid substitute to TFBS, providing reconstructed image performance comparable to TFBS in most subjects but exhibiting quantitative differences in cases where TFBS is typically prone to inaccuracies (e.g., due to patient motion and CT-based attenuation map truncation). Particularly for low contrast regions, quantification differs for ABS compared to MLBS and TFBS, and caution should be taken when utilizing ABS for decision-making based on quantitative metrics.

Automatic Screening of Lung Diseases by 3D Active Contour Method for Inhomogeneous Motion Estimation in CT Image Pairs

Lung diseases are now the third leading cause of death worldwide because of the many risk factors we are exposed to daily, such as air pollution, tobacco use, viruses (such as COVID-19), and bacteria. This work introduces a new approach of the 3D Active Contour Model (3D ACM) to estimate an inhomogeneous motion of lungs, which can be used to analyze lung disease patterns using a hierarchical predictive model. The biophysical model of lungs consists of End Expiratory (EE) and End Inspiratory (EI) models, generated by high-resolution computed tomography images (HRCT). A proposed technique uses the 3D ACM to estimate the velocity vector by using the corresponding points on the parametric surface model of the EE model to the EI model. The external energy from the EI models is the external force that pushes the 3D parametric surface to reach the boundary. The external forces, such as the balloon force and Gradient Vector Flow (GVF), were adjusted adaptively based on the which was calculated from the ratio of the maximum value of EI to EE on the Z axis. Next, the feature representation is studied and evaluated based on the lung structure, separated into five lobes. The stepwise regression, Support Vector Machine (SVM), and Artificial Neural Network (ANN) techniques are applied to classify the lung diseases into normal, obstructive lung, and restrictive lung diseases. In conclusion, the inhomogeneous motion pattern of lungs integrated with medical-based knowledge can be used to analyze lung diseases by differentiating normal and abnormal motion patterns and separating restrictive and obstructive lung diseases. HIGHLIGHTS Inhomogeneous motion analysis from the expanding and shrinking lungs of HRCT pair Adaptive 3D Active Contour Model (ACM) for detecting the shape of the lung by balancing the balloon force with the stopping condition Lung lopes separation using oblique fissure and anatomical location Structure the velocity vector map of lung motion using bag of words of the magnitude Neural Network model for predicting obstructive and restrictive lung diseases GRAPHICAL ABSTRACT

Robot-assisted sleeve gastrectomy in patients with obesity with a novel Chinese domestic MicroHand SII surgical system

BackgroundA new device has been added to the Chinese MicroHand surgical robot family, developed based on the successful application of control algorithms. As a benefit of using these specialized control algorithms, the motion mapping relation can be accommodated into the system without the help of a built-in image system, resulting in a novel Chinese domestic surgical robot with two arms called MicroHand SII, which is different from the former MicroHand S and da Vinci systems. In this study, we investigate the performance of a novel MicroHand SII robotic platform in patients with obesity.MethodsFrom March 2018 to April 2019, a total of 7 patients whose BMI ranged from 29.9 to 49.8 kg/m2 were operated on with the robot-assisted technique using the MicroHand SII surgical system. Data regarding demography, surgical procedure and the 3-month outcome postoperation were collected.ResultsThere were 2 female and 5 male patients with a median age (range) of 35 (20–51) years. The median operative time was 160 (149–195) minutes. None were converted to open surgery. All anthropometry indices improved significantly (p < 0.05) at 3 months postoperatively. There were no cases of surgical site infection, gastrointestinal/abdominal bleeding, or conversion to an open operation.ConclusionsThe initial experience showed that the Chinese domestic robot surgical system MicroHand SII could be feasibly and safely applied in sleeve gastrectomy in patients with obesity. Because of the unique design of this system such as a two-hand robot, an array of master–slave motion strategies, and a roll joint at the end of the instruments that allows 7 degrees of freedom, this robotic platform has presented its own obvious advantages.

Exploring 3D Human Action Recognition Using STACOG on Multi-View Depth Motion Maps Sequences.

This paper proposes an action recognition framework for depth map sequences using the 3D Space-Time Auto-Correlation of Gradients (STACOG) algorithm. First, each depth map sequence is split into two sets of sub-sequences of two different frame lengths individually. Second, a number of Depth Motion Maps (DMMs) sequences from every set are generated and are fed into STACOG to find an auto-correlation feature vector. For two distinct sets of sub-sequences, two auto-correlation feature vectors are obtained and applied gradually to -regularized Collaborative Representation Classifier (-CRC) for computing a pair of sets of residual values. Next, the Logarithmic Opinion Pool (LOGP) rule is used to combine the two different outcomes of -CRC and to allocate an action label of the depth map sequence. Finally, our proposed framework is evaluated on three benchmark datasets named MSR-action 3D dataset, DHA dataset, and UTD-MHAD dataset. We compare the experimental results of our proposed framework with state-of-the-art approaches to prove the effectiveness of the proposed framework. The computational efficiency of the framework is also analyzed for all the datasets to check whether it is suitable for real-time operation or not.

Regional Attention with Architecture-Rebuilt 3D Network for RGB-D Gesture Recognition

Human gesture recognition has drawn much attention in the area of computer vision. However, the performance of gesture recognition is always influenced by some gesture-irrelevant factors like the background and the clothes of performers. Therefore, focusing on the regions of hand/arm is important to the gesture recognition. Meanwhile, a more adaptive architecture-searched network structure can also perform better than the block-fixed ones like ResNet since it increases the diversity of features in different stages of the network better. In this paper, we propose a regional attention with architecture-rebuilt 3D network (RAAR3DNet) for gesture recognition. We replace the fixed Inception modules with the automatically rebuilt structure through the network via Neural Architecture Search (NAS), owing to the different shape and representation ability of features in the early, middle, and late stage of the network. It enables the network to capture different levels of feature representations at different layers more adaptively. Meanwhile, we also design a stackable regional attention module called Dynamic-Static Attention (DSA), which derives a Gaussian guidance heatmap and dynamic motion map to highlight the hand/arm regions and the motion information in the spatial and temporal domains, respectively. Extensive experiments on two recent large-scale RGB-D gesture datasets validate the effectiveness of the proposed method and show it outperforms state-of-the-art methods. The codes of our method are available at: https://github.com/zhoubenjia/RAAR3DNet.

WGLSM: An end-to-end line matching network based on graph convolution

Line matching plays an essential role in Structure from Motion (SFM) and Simultaneous Localization and Mapping (SLAM), especially in low-texture scenes, where feature points are hard to be detected. In this paper, we present a new method by combining Convolutional Neural Networks and Graph Convolutional Networks to match line segments in pairs of images. We design a graph-based method to predict the assignment matrix of two feature sets with solving a relaxed optimal transport problem. In contrast to handcrafted line matching algorithms, our approach learns the line segment features and performs matching simultaneously through end-to-end weakly supervised training. The experiment results show that our method outperforms the state-of-the-art techniques and is robust to various image transformations. Besides, the generalization experiment illustrates that our method has good generalization ability without fine-tuning. The code of our work is available athttps://github.com/mameng1/GraphLineMatching.

Mechanical characterization of abdominal aortas using multi-perspective ultrasound imaging

Mechanical characterization of abdominal aortic aneurysms using personalized biomechanical models is being widely investigated as an alternative criterion to assess risk of rupture. These methods rely on accurate wall motion detection and appropriate model boundary conditions. In this study, multi-perspective ultrasound is combined with finite element models to perform mechanical characterization of abdominal aortas in volunteers.Multi-perspective biplane radio frequency ultrasound recordings were made under seven angles (-45° to 45°) in one phantom set-up and eight volunteers, which were merged using automatic image registration. 2-D displacement fields were estimated in the seven longitudinal ultrasound views, creating a sparse, high resolution 3-D map of the wall motion at relatively high frame rates (20–27 Hz). The displacements were used to personalize the subject-specific finite element model of which the geometry of the aorta, spine, and surrounding tissue were determined from a single 3-D ultrasound acquisition.Automatic registration of the multi-perspective images was successful in six out of eight cases with an average error of 5.4° compared to the ground truth. Displacements of the aortic wall were measured and cyclic strain of the aortic diameter was found ranging from 4.2% to 8.6%. The subject-specific mesh and inverse FE analysis was performed yielding shear moduli estimates for the wall between 104 and 215 kPa. Comparative results from a single-perspective workflow revealed very low aortic wall motion signal, which resulted in relatively high modulus estimates, between 230 and 754 kPa.Multi-perspective biplane ultrasound imaging was used to personalize finite element models of the abdominal aorta and its surroundings, and performing mechanical characterization of the aortic shear modulus. The method was found to be a more robust method compared to a single-perspective 3-D ultrasound approach. Future research will focus on investigating the use of multiple 3-D ultrasound acquisitions, the feasibility of free-hand scanning, the creation of a full 3-D automatic registration process, and with that, enable a clinical continuation of this study.

Research on Human Motion Recognition Based on Data Redundancy Technology

Aiming at the problems of low recognition rate and slow recognition speed of traditional body action recognition methods, a human action recognition method based on data deduplication technology is proposed. Firstly, the data redundancy technology and perceptual hashing technology are combined to form an index, and the image is filtered from the structure, color, and texture features of human action image to achieve image redundancy processing. Then, the depth feature of processed image is extracted by depth motion map; finally, feature recognition is carried out by convolution neural network so as to achieve the purpose of human action recognition. The simulation results show that the proposed method can obtain the optimal recognition results and has strong robustness. At the same time, it also fully proves the importance of human motion recognition.

A 3D cine-MRI acquisition technique and image analysis framework to quantify bowel motion demonstrated in gynecological cancer patients.

PurposeMagnetic resonance imaging (MRI) is increasingly used in radiation oncology for target delineation and radiotherapy treatment planning, for example, in patients with gynecological cancers. As a consequence of pelvic radiotherapy, a part of the bowel is irradiated, yielding risk of bowel toxicity. Existing dose‐effect models predicting bowel toxicity are inconclusive and bowel motion might be an important confounding factor. The exact motion of the bowel and dosimetric effects of its motion are yet uncharted territories in radiotherapy. In diagnostic radiology methods on the acquisition of dynamic MRI sequences were developed for bowel motility visualization and quantification. Our study aim was to develop an imaging technique based on three‐dimensional (3D) cine‐MRI to visualize and quantify bowel motion and demonstrate it in a cohort of gynecological cancer patients.MethodsWe developed an MRI acquisition suitable for 3D bowel motion quantification, namely a balanced turbo field echo sequence (TE = 1.39 ms, TR = 2.8 ms), acquiring images in 3.7 s (dynamic) with a 1.25 × 1.25 × 2.5 mm3 resolution, yielding a field of view of 200 × 200 × 125 mm3. These MRI bowel motion sequences were acquired in 22 gynecological patients. During a 10‐min scan, 160 dynamics were acquired. Subsequent dynamics were deformably registered using a B‐spline transformation model, resulting in 159 3D deformation vector fields (DVFs) per MRI set. From the 159 DVFs, the average vector length was calculated per voxel to generate bowel motion maps. Quality assurance was performed on all 159 DVFs per MRI, using the Jacobian Determinant and the Harmonic Energy as deformable image registration error metrics. In order to quantify bowel motion, we introduced the concept of cumulative motion–volume histogram (MVH) of the bowel bag volume. Finally, interpatient variation of bowel motion was analyzed using the MVH parameters M10%, M50%, and M90%. The M10%/M50%/M90% represents the minimum bowel motion per frame of 10%/50%/90% of the bowel bag volume.ResultsThe motion maps resulted in a visualization of areas with small and large movements within the bowel bag. After applying quality assurance, the M10%, M50%, and M90% were 4.4 (range 2.2–7.6) mm, 2.2 (range 0.9–4.1) mm, and 0.5 (range 0.2–1.4) mm per frame, on average over all patients, respectively.ConclusionWe have developed a method to visualize and quantify 3D bowel motion with the use of bowel motion specific MRI sequences in 22 gynecological cancer patients. This 3D cine‐MRI‐based quantification tool and the concept of MVHs can be used in further studies to determine the effect of radiotherapy on bowel motion and to find the relation with dose effects to the small bowel. In addition, the developed technique can be a very interesting application for bowel motility assessment in diagnostic radiology.

Modular and scalable automation for field robots

Abstract This article describes a modular and scalable charging and navigation concept for electrified field robots and other agricultural machines. The concept consists of an underbody charging system on a trailer and a modular navigation box. The underlying conductive charging process is compared to other charging techniques. Charging time in relation to charging current and mean power consumption in field use is displayed. In the navigation box, data of various sensors are combined by means of multi-sensor fusion regarding the precise time of arrival. Time synchronization is achieved by a novel method for compensating the data latency jitter by employing Kalman-based timestamp filtering. Furthermore, navigation functionalities, such as motion planning and mapping, are presented.

Estimation of People Movement in Video Based on Optical Flow Block Method and Motion Maps

An algorithm for detecting and tracking moving people on video sequences using the block optical flow method and motion maps is proposed. To reduce time expenditures, a pyramidal representation of the frame and template search are used at the stage of building a preliminary map of motion vectors. The integral optical flow allows one to reduce the resulting amplitudes of the background displacement vectors and increase the resulting amplitudes of the displacement vectors of foreground objects. To improve the accuracy for localization of objects, the additive minimax similarity function is used in the analysis of motion vectors. Objects are tracked based on a modified tracing algorithm using the Kalman filter. The developed algorithm allows one not only to detect a moving object but also to show the trajectory of its movement. The results of experiments are presented that allow evaluating the effectiveness of the algorithm.

RigidFusion: Robot Localisation and Mapping in Environments With Large Dynamic Rigid Objects

This work presents a novel RGB-D SLAM approach to simultaneously segment, track and reconstruct the static background and large dynamic rigid objects that can occlude major portions of the camera view. Previous approaches treat dynamic parts of a scene as outliers and are thus limited to a small amount of changes in the scene, or rely on prior information for all objects in the scene to enable robust camera tracking. Here, we propose to treat all dynamic parts as one rigid body and simultaneously segment and track both static and dynamic components. We, therefore, enable simultaneous localisation and reconstruction of both the static background and rigid dynamic components in environments where dynamic objects cause large occlusion. We evaluate our approach on multiple challenging scenes with large dynamic occlusion. The evaluation demonstrates that our approach achieves better motion segmentation, localisation and mapping without requiring prior knowledge of the dynamic object's shape and appearance.