Motion Disparity Research Articles

Validity of Inertial Measurement Unit (IMU Sensor) for Measurement of Cervical Spine Motion, Compared with Eight Optoelectronic 3D Cameras Under Spinal Immobilization Devices.

The assessment of cervical spine motion is critical for out-of-hospital patients who suffer traumatic spinal cord injuries, given the profound implications such injuries have on individual well-being and broader public health concerns. 3D Optoelectronic systems (BTS SmartDX) are standard devices for motion measurement, but their price, complexity, and size prevent them from being used outside of designated laboratories. This study was designed to evaluate the accuracy and reliability of an inertial measurement unit (IMU) in gauging cervical spine motion among healthy volunteers, using a 3D optoelectronic motion capture system as a reference. Twelve healthy volunteers participated in the study. They underwent lifting, transferring, and tilting simulations using a long spinal board, a Sked stretcher, and a vacuum mattress. During these simulations, cervical spine angular movements-including flexion-extension, axial rotation, and lateral flexion-were concurrently measured using the IMU and an optoelectronic device. We employed the Wilcoxon signed-rank test and the Bland-Altman plot to assess reliability and validity. A single statistically significant difference was observed between the two devices in the flexion-extension plane. The mean differences across all angular planes ranged from -1.129° to 1.053°, with the most pronounced difference noted in the lateral flexion plane. Ninety-five percent of the angular motion disparities ascertained by the SmartDX and IMU were less than 7.873° for the lateral flexion plane, 11.143° for the flexion-extension plane, and 25.382° for the axial rotation plane. The IMU device exhibited robust validity when assessing the angular motion of the cervical spine in the axial rotation plane and demonstrated commendable validity in both the lateral flexion and flexion-extension planes.

Composite Motion Learning with Task Control

We present a deep learning method for composite and task-driven motion control for physically simulated characters. In contrast to existing data-driven approaches using reinforcement learning that imitate full-body motions, we learn decoupled motions for specific body parts from multiple reference motions simultaneously and directly by leveraging the use of multiple discriminators in a GAN-like setup. In this process, there is no need of any manual work to produce composite reference motions for learning. Instead, the control policy explores by itself how the composite motions can be combined automatically. We further account for multiple task-specific rewards and train a single, multi-objective control policy. To this end, we propose a novel framework for multi-objective learning that adaptively balances the learning of disparate motions from multiple sources and multiple goal-directed control objectives. In addition, as composite motions are typically augmentations of simpler behaviors, we introduce a sample-efficient method for training composite control policies in an incremental manner, where we reuse a pre-trained policy as the meta policy and train a cooperative policy that adapts the meta one for new composite tasks. We show the applicability of our approach on a variety of challenging multi-objective tasks involving both composite motion imitation and multiple goal-directed control. Code is available at https://motion-lab.github.io/CompositeMotion .

MONITORING WORK PROGRESS OF DAM CONSTRUCTION BASED ON PHOTOGRAMMETRIC POINT CLOUDS AND BIM: PRACTICAL APPROACH TO TEACHING INDUSTRY

Abstract. Industry 4.0 encompasses a range of ideas, technologies, and procedures that impact engineering education and other fields. Building information modeling (BIM) has become a widely adopted practice in Indonesia's construction industry, prompting polytechnic universities to incorporate BIM skills and concepts into their construction engineering and management degree programs. Nowdays, more than 50 dam construction works occurred in Indonesia from 2014-2024. Monitoring progress regularly is crucial to ensure a construction project's successful completion. This can be done by visually tracking the project's advancement, which enables the identification of any changes or deviations that occur. Accurate measurement data is essential for reliable progress management, as it allows for the prediction of future project success or failure. Image-based object detection is highly beneficial for retrieving site information and monitoring construction progress. To monitor the progress of a construction project, comparisons between the building's as-built and as-planned states are necessary. BIM is employed to establish the intended condition of a building by providing information regarding its geometric characteristics and construction schedule. In this paper, we present a novel method for producing a point cloud that accurately represents the dam building site using photogrammetry. To overcome the limitation of capturing photographs from every possible angle at a construction site, we employ a combination of structure from motion and control points. This approach allows us to generate a scaled point cloud in a standardized coordinate system. The building's as-built and as-planned states are then compared using this point cloud. Structure from Motion (SfM) disparity maps are fused to create dense point clouds, which are then compared to the goal state for updating 4D BIM. This makes it possible to identify missing building components. To validate the presence of building elements, a secondary examination is conducted using the positions in front of and behind the model planes constructed as per the original blueprint. The research paper explores potential approaches and presents empirical results obtained from an actual case study.

Rehabilitation Tracking of Athletes Post Anterior Cruciate Ligament Reconstruction (ACL-R) Surgery Through Causal Analysis of Gait Data & Computational Modeling.

Early identification of motion disparities in Anterior Cruciate Ligament reconstructed (ACL-R) athletes may better post-operative decision making when returning athletes to sport. Existing return to play assessments consist of assessments of muscle strength, functional tasks, patient-reported outcomes, and 3D coordinate tracking. However, these methods primarily depend on the medical provider's intuition to release them to participate in an unrestricted activity after ACL-R that may cause reinjury or long-term impacts. This study proposes a wearable sensor-based system that helps track athlete rehabilitation progress and return to sport decision making. For this, we capture gait data from 89 ACL-R athletes during their walking and jogging trials. The raw gyroscope data collected from this system is used to extract causal features based on Nolte's phase slope index. Features extracted from this study are used to develop computational models that classify ACL-R athletes based on their reconstructed knee during two visits (3-6 months & 9 months) post ACL-R surgery. The classifier's performance degradation in detecting ACL-R athletes injured knee during multiple visits supports athletic trainers and physicians' decision-making process to confirm an athlete's safe return to sport.Clinical Relevance- This study develops computational models based on causal analysis of gait data to support athletic trainers and medical practitioners' decision to return athletes to sport post ACL-R surgery.

Human primary visual cortex shows larger population receptive fields for binocular disparity-defined stimuli

The visual perception of 3D depth is underpinned by the brain’s ability to combine signals from the left and right eyes to produce a neural representation of binocular disparity for perception and behaviour. Electrophysiological studies of binocular disparity over the past 2 decades have investigated the computational role of neurons in area V1 for binocular combination, while more recent neuroimaging investigations have focused on identifying specific roles for different extrastriate visual areas in depth perception. Here we investigate the population receptive field properties of neural responses to binocular information in striate and extrastriate cortical visual areas using ultra-high field fMRI. We measured BOLD fMRI responses while participants viewed retinotopic mapping stimuli defined by different visual properties: contrast, luminance, motion, correlated and anti-correlated stereoscopic disparity. By fitting each condition with a population receptive field model, we compared quantitatively the size of the population receptive field for disparity-specific stimulation. We found larger population receptive fields for disparity compared with contrast and luminance in area V1, the first stage of binocular combination, which likely reflects the binocular integration zone, an interpretation supported by modelling of the binocular energy model. A similar pattern was found in region LOC, where it may reflect the role of disparity as a cue for 3D shape. These findings provide insight into the binocular receptive field properties underlying processing for human stereoscopic vision.

A novel dynamic random-dot stereopsis assessment to measure stereopsis in intermittent exotropia.

BackgroundTo investigate dynamic stereopsis in intermittent exotropia [X(T)] patients using a novel dynamic random-dot stereopsis assessment method.MethodsA novel dynamic random-dot stereopsis test was performed using novel self-programmed software, which consisted of red and green dots and four blocks on the screen. The test included motion + disparity (MD), motion (M), and disparity (D), where the D cues ranged from 200 to 1,200 arc-seconds. The characteristics of preoperative dynamic stereopsis in 83 X(T) patients and associations with clinical features were analysed, and the prognosis was followed up on the 1st day and at the 2nd, 6th and 12th months postoperatively.ResultsPreoperatively, the mean reciprocal values of near and far stereopsis were 0.013±0.0016 and 0.0011±0.0005 arc-seconds in static stereopsis patients, respectively, and the MD, M, and D values were 0.002±0.0002, 0.0018±0.0001, and 0.0012±0.0002 arc-seconds in dynamic stereopsis, respectively. The MD value was negatively correlated with the Newcastle control score, disease course, and prism deviations (P<0.05), M was correlated with disease course and the Newcastle control score (P<0.05), and D was not correlated with any clinical features. Postoperative dynamic stereopsis improved from the 1st day and gradually peaked at the 6th month, while static stereopsis showed a decreased tendency in near but not far stereopsis.ConclusionsThe dynamic stereopsis quantitative evaluation method based on random dots is a feasible test and shows that destruction of X(T) patients’ dynamic stereopsis is affected by age at surgery, disease course, strabismus controllability and the strabismus degree.

Ventral motion parallax enhances fruit fly steering to visual sideslip.

Flies and other insects use incoherent motion (parallax) to the front and sides to measure distances and identify obstacles during translation. Although additional depth information could be drawn from below, there is no experimental proof that they use it. The finding that blowflies encode motion disparities in their ventral visual fields suggests this may be an important region for depth information. We used a virtual flight arena to measure fruit fly responses to optic flow. The stimuli appeared below (n = 51) or above the fly (n = 44), at different speeds, with or without parallax cues. Dorsal parallax does not affect responses, and similar motion disparities in rotation have no effect anywhere in the visual field. But responses to strong ventral sideslip (206° s−1) change drastically depending on the presence or absence of parallax. Ventral parallax could help resolve ambiguities in cluttered motion fields, and enhance corrective responses to nearby objects.

Border ownership-dependent tilt aftereffect for shape defined by binocular disparity and motion parallax.

Discerning objects from their surrounds (i.e., figure-ground segmentation) in a way that guides adaptive behaviors is a fundamental task of the brain. Neurophysiological work has revealed a class of cells in the macaque visual cortex that may be ideally suited to support this neural computation: border ownership cells (Zhou H, Friedman HS, von der Heydt R. J Neurosci 20: 6594–6611, 2000). These orientation-tuned cells appear to respond conditionally to the borders of objects. A behavioral correlate supporting the existence of these cells in humans was demonstrated with two-dimensional luminance-defined objects (von der Heydt R, Macuda T, Qiu FT. J Opt Soc Am A Opt Image Sci Vis 22: 2222–2229, 2005). However, objects in our natural visual environments are often signaled by complex cues, such as motion and binocular disparity. Thus for border ownership systems to effectively support figure-ground segmentation and object depth ordering, they must have access to information from multiple depth cues with strict depth order selectivity. Here we measured in humans (of both sexes) border ownership-dependent tilt aftereffects after adaptation to figures defined by either motion parallax or binocular disparity. We find that both depth cues produce a tilt aftereffect that is selective for figure-ground depth order. Furthermore, we find that the effects of adaptation are transferable between cues, suggesting that these systems may combine depth cues to reduce uncertainty (Bülthoff HH, Mallot HA. J Opt Soc Am A 5: 1749–1758, 1988). These results suggest that border ownership mechanisms have strict depth order selectivity and access to multiple depth cues that are jointly encoded, providing compelling psychophysical support for their role in figure-ground segmentation in natural visual environments.NEW & NOTEWORTHY Figure-ground segmentation is a critical function that may be supported by “border ownership” neural systems that conditionally respond to object borders. We measured border ownership-dependent tilt aftereffects to figures defined by motion parallax or binocular disparity and found aftereffects for both cues. These effects were transferable between cues but selective for figure-ground depth order, suggesting that the neural systems supporting figure-ground segmentation have strict depth order selectivity and access to multiple depth cues that are jointly encoded.

Cyclic fatigue comparison of different manufactured endodontic files

Aim: The aim of this study was to compare the cyclic fatigue resistance of endodontic files of different manufacturers operated in disparate motions. Materials and Methods: Six different brands of nickel–titanium instruments, such as ProFile Vortex™, Vortex Blue™, Twisted File™, HyFlex™, WaveOne™, and S1™ all with a tip size ISO 25 with 0.06 taper, except for WaveOne™ tip size ISO 25 with 0.08 taper, were included in the study. Six groups of 20 rotary files from each system were tested for cyclic fatigue resistance. All files were rotated in a simulated root canal with a certain diameter, angle of curvature, and a radius of curvature of a specific cyclic fatigue testing device until fracture occurred. Time to fracture was recorded for each instrument in each group in seconds. The mean values and standard deviation were then calculated. Data were compared using repeated measures ANOVA for individual comparisons followed by Bonferroni's correction for multiple comparisons. Significance was set at the 95% confidence level. Results: Time of fracture had statistically significant differences among all groups tested (P 0.5). Thus, the S1 file offered a long life span, followed by WaveOne™, ProFile Vortex™, HyFlex™, Twisted Files™, and then Vortex Blue™. Conclusion: The reciprocating instruments (S1 and WaveOne™) had a higher cyclic fatigue resistance than all rotary files used in this study. However, the Profile Vortex (M-wire) and HyFlex (CM) showed better cyclic fatigue resistance than other rotating files in this study. Hence, the blue and R-phase heat treatments did not enhance the cyclic fatigue resistance for rotating instruments.

Motion and binocular disparity processing: Two sides of two different coins.

From a mathematical point of view, extracting motion and disparity signals from a binocular visual stream requires very similar operations, applied over time for motion and across eyes for disparity. This similarity is reflected in the theories that have been proposed to describe the neural mechanisms used by the brain to extract these signals. At the behavioral level there are, however, several differences in how humans react to these stimuli, which presumably reflect differences in how these signals are processed by the brain. Here we highlight three such differences: the degree to which different axes of motion/disparity are treated isotropically, the importance of reference signals, and the rules that underlie the combination of 1D signals to extract 2D signals.

Full-Reference Stability Assessment of Digital Video Stabilization Based on Riemannian Metric.

Assessing the quality of the motion stability is important to evaluating the performance of video stabilization algorithms. This paper presents a novel quality assessment scheme for the video motion stability in a full-reference (FR) manner. Given ideally stable videos and their corresponding shaky videos, our method measures the geodesic distance between motion paths of the stable and the stabilized videos. Due to the use of the Riemannian metric defined on the manifold of spatial transformations, our method enables the intrinsic and faithful measurement on pairwise motion disparities. To facilitate the FR assessment, a data set of stable and shaky videos is constructed by directly capturing realistic stable/shaky videos with a customized device. Then, digital video stabilization algorithms can be run on shaky videos to obtain the stabilized sequence of frames, whereupon their performances are evaluated by using our stability assessment. The experiments demonstrate that our stability assessment gains good concordance with the subjective assessment.

Estimating Depth Range Required for 3-D Displays to Show Depth-Compressed Scenes Without Inducing Sense of Unnaturalness

It is often difficult to show deep 3-D scenes with sufficient quality on 3-D displays with light-ray reconstruction, such as light-field and integral 3-D displays, because their depth-reconstruction range is restricted. Scenes with substantial depth cannot be shown properly because the reconstructed images outside the range inevitably blur. Although promising methods for providing better quality 3-D visualization by contracting the scene depth to fit within the restricted depth range have been proposed, the size of the depth range required for showing deep 3-D scenes with perceptually appealing quality is not yet known. To reveal the required depth, we conducted evaluation experiments under practical viewing conditions using a 3-D-display simulator that provides binocular and motion disparities. Even with deep 3-D scenes (originally with a depth of 250 m), we found that a depth range of at least 1 m was necessary to show these scenes with sufficient quality in terms of naturalness using a nonlinear depth compression method. These results provide a design goal for future 3-D-television development, suggesting that 3-D displays can naturally show a variety of scenes that originally had substantial depths if they can reproduce a depth of 1 m.

A Hybrid Framework for Understanding and Predicting Human Reaching Motions.

Robots collaborating naturally with a human partner in a confined workspace need to understand and predict human motions. For understanding, a model-based approach is required as the human motor control system relies on the biomechanical properties to control and execute actions. The model-based control models explain human motions descriptively, which in turn enables predicting and analyzing human movement behaviors. In motor control, reaching motions are framed as an optimization problem. However, different optimality criteria predict disparate motion behavior. Therefore, the inverse problem—finding the optimality criterion from a given arm motion trajectory—is not unique. This paper implements an inverse optimal control (IOC) approach to determine the combination of cost functions that governs a motion execution. The results indicate that reaching motions depend on a trade-off between kinematics and dynamics related cost functions. However, the computational efficiency is not sufficient for online prediction to be utilized for HRI. In order to predict human reaching motions with high efficiency and accuracy, we combine the IOC approach with a probabilistic movement primitives formulation. This hybrid model allows an online-capable prediction while taking into account motor variability and the interpersonal differences. The proposed framework affords a descriptive and a generative model of human reaching motions which can be effectively utilized online for human-in-the-loop robot control and task execution.

Multi-level complexity reduction for HEVC multiview coding

Standardized in 2014, multiview extension of high efficiency video coding (MV-HEVC) offers significantly better compression performance of up to 50% for multiview and 3D videos compared to multiple independent single view HEVC coding. However, the extreme high computational complexity of MV-HEVC demands significant optimization of the encoder. In this work, we propose a series of optimization techniques at various levels of abstraction: non-aggregation massively parallel motion estimation (ME) and disparity estimation (DE) for prediction units, fractional and bidirectional ME/DE, quantization parameter-based early termination of coding tree unit (CTU), and optimized resource-scheduled wave front parallel processing for CTU. When evaluated over three views for all available official multiview video coding test sequences, proposed optimization outperforms the anchor encoder by average factor of 5.4 at the cost of 4.4% bitrate (DBR) increase at no loss in PSNR, or alternatively a PSNR degradation of 0.12 dB at no change to the DBR.

Parallax360: Stereoscopic 360° Scene Representation for Head-Motion Parallax.

We propose a novel 360° scene representation for converting real scenes into stereoscopic 3D virtual reality content with head-motion parallax. Our image-based scene representation enables efficient synthesis of novel views with six degrees-of-freedom (6-DoF) by fusing motion fields at two scales: (1) disparity motion fields carry implicit depth information and are robustly estimated from multiple laterally displaced auxiliary viewpoints, and (2) pairwise motion fields enable real-time flow-based blending, which improves the visual fidelity of results by minimizing ghosting and view transition artifacts. Based on our scene representation, we present an end-to-end system that captures real scenes with a robotic camera arm, processes the recorded data, and finally renders the scene in a head-mounted display in real time (more than 40 Hz). Our approach is the first to support head-motion parallax when viewing real 360° scenes. We demonstrate compelling results that illustrate the enhanced visual experience - and hence sense of immersion-achieved with our approach compared to widely-used stereoscopic panoramas.

Insect Vision: Judging Distance with Binocular Motion Disparities

Praying mantises, like humans, calculate distance by triangulating angular disparities of corresponding image regions between their eyes. New work shows they can do this using just dynamic luminance changes, and thus judge distance in scenes that we see as flat.

Low-complexity encoding scheme for multiview video content

On-site multiview video viewing system can provide people with more immersive viewing experience on various live video shows, such as sports and concerts. To address the high computational complexity of multi-view video coding, several researches have worked on fast motion estimation, fast disparity estimation, and/or fast mode decision. In this paper, we propose a low-complexity encoding scheme that can further exploit inter-view dependencies and depth information. A fast mode selection with threshold-based early termination is presented to select a minimal set of mode candidates according to the reference macroblocks of neighbouring views and expedite the mode decision process. Further, this study also demonstrates that disparity estimation computation and search ranges of both motion estimation and disparity estimation can be significantly refined by using inter-view dependencies and depth information. The experimental results show that the proposed encoding method reduces the coding time by 80% with a negligible PSNR loss.

Insect Neurobiology: An Eye to Forward Motion

For many animals, visual motion provides essential information for navigating through the environment. A new study in flies reveals novel neurons capable of multiplexing information of a visual scene and encoding relative depth perception from motion disparity.

Global Motion Processing in Human Visual Cortical Areas V2 and V3.

Global motion perception entails the ability to extract the central direction tendency from an extended area of visual space containing widely disparate local directions. A substantial body of evidence suggests that local motion signals generated in primary visual cortex (V1) are spatially integrated to provide perception of global motion, beginning in the middle temporal area (MT) in macaques and its counterpart in humans, hMT. However, V2 and V3 also contain motion-sensitive neurons that have larger receptive fields than those found in V1, giving the potential for spatial integration of motion signals. Despite this, V2 and V3 have been overlooked as sites of global motion processing. To test, free of local-global confounds, whether human V2 and V3 are important for encoding global motion, we developed a visual stimulus that yields a global direction yet includes all possible local directions and is perfectly balanced at the local motion level. We then attempted to decode global motion direction in such stimuli with multivariate pattern classification of fMRI data. We found strong sensitivity to global motion in hMT, as expected, and also in several higher visual areas known to encode optic flow. Crucially, we found that global motion direction could be decoded in human V2 and, particularly, in V3. The results suggest the surprising conclusion that global motion processing is a key function of cortical visual areas V2 and V3. A possible purpose is to provide global motion signals to V6. Humans can readily detect the overall direction of movement in a flock of birds despite large differences in the directions of individual birds at a given moment. This ability to combine disparate motion signals across space underlies many aspects of visual motion perception and has therefore received considerable research attention. The received wisdom is that spatial integration of motion signals occurs in the cortical motion complex MT+ in both human and nonhuman primates. We show here that areas V2 and V3 in humans are also able to perform this function. We suggest that different cortical areas integrate motion signals in different ways for different purposes.

Are face representations depth cue invariant?

The visual system can process three-dimensional depth cues defining surfaces of objects, but it is unclear whether such information contributes to complex object recognition, including face recognition. The processing of different depth cues involves both dorsal and ventral visual pathways. We investigated whether facial surfaces defined by individual depth cues resulted in meaningful face representations-representations that maintain the relationship between the population of faces as defined in a multidimensional face space. We measured face identity aftereffects for facial surfaces defined by individual depth cues (Experiments 1 and 2) and tested whether the aftereffect transfers across depth cues (Experiments 3 and 4). Facial surfaces and their morphs to the average face were defined purely by one of shading, texture, motion, or binocular disparity. We obtained identification thresholds for matched (matched identity between adapting and test stimuli), non-matched (non-matched identity between adapting and test stimuli), and no-adaptation (showing only the test stimuli) conditions for each cue and across different depth cues. We found robust face identity aftereffect in both experiments. Our results suggest that depth cues do contribute to forming meaningful face representations that are depth cue invariant. Depth cue invariance would require integration of information across different areas and different pathways for object recognition, and this in turn has important implications for cortical models of visual object recognition.