3D Motion Detection Research Articles

Fault Signal Perception of Nanofiber Sensor for 3D Human Motion Detection Using Multi-Task Deep Learning

Once a fault occurs in the nanofiber sensor, the scientific and reliable three-dimensional (3D) human motion detection results will be compromised. It is necessary to accurately and rapidly perceive the fault signals of the nanofiber sensor and determine the type of fault, to enable it to continue operating in a sustained and stable manner. Therefore, we propose a fault signal perception method for 3D human motion detection nanofiber sensor based on multi-task deep learning. First, through obtaining the fault characteristic parameters of the nanofiber sensor, the fault of the nanofiber sensor is reconstructed to complete the fault location of the nanofiber sensor. Second, the fault signal of the nanofiber sensor is mapped by the penalty function, and the feature extraction model of the fault signal of the nanofiber sensor is constructed by combining the multi-task deep learning. Finally, the multi-task deep learning algorithm is used to calculate the sampling frequency of the fault signal, and the key variable information of the fault of the nanofiber sensor is extracted according to the amplitude of the state change of the nanofiber sensor, to realize the perception of the fault signal of the nanofiber sensor. The results show that the proposed method can accurately perceive the fault signal of a nanofiber sensor in 3D human motion detection, the maximum sensor fault location accuracy is 97%, and the maximum noise content of the fault signal is only 5 dB, which shows that the method can be widely used in fault signal perception.

Performance optimization of capacitive motion sensing (CMS) system for intra-fraction motion detection during stereotactic radiosurgery

The purpose of this investigation is to improve intra-fractional motion detection during cranial stereotactic radiosurgery with a novel capacitive motion sensing (CMS) system. Previous work showed that a capacitive detection system, based on a MPR121 capacitance-to-digital converter, provided a number of advantages over existing patient imaging systems used in the clinic, by uniquely offering ionizing-radiation-free and continuous monitoring without modification to the immobilization mask or treatment room. However, in order to provide submillimeter detection accuracy, the MPR121-based CMS system required relatively large sensors in close proximity to the patient. Therefore, the aim of this investigation was to improve sensitivity of the system, allowing reduction in sensor size and preserving its stable operation in the linear accelerator environment. For this, we developed, characterized and compared motion detection capabilities of four CMS systems based on different capacitance-to-digital converters: MPR121, CPT212B, FDC1004 and FDC2214. Among all candidates, the FDC2214-based system was found to uniquely combine accurate 3D motion detection in real time, with stable performance under ionizing radiation. It exhibited an order of magnitude improvement in sensitivity in comparison with the proof-of-study system, allowing a spatial precision as low as 0.3 mm, and its overall performance was found to satisfy the AAPM practice guidelines of positioning tolerance within 1 mm. Furthermore, the high sensitivity of the system allows both reduction of the sensor area and location more distant from the patient surface, which are key improvements with regard to development of a clinical device.

Simultaneous 3D Motion Detection, Long-Term Tracking and Model Reconstruction for Multi-Objects

Motion detection and object tracking play important roles in unsupervised human–machine interaction systems. Nevertheless, the human–machine interaction would become invalid when the system fails to detect the scene objects correctly due to occlusion and limited field of view. Thus, robust long-term tracking of scene objects is vital. In this paper, we present a 3D motion detection and long-term tracking system with simultaneous 3D reconstruction of dynamic objects. In order to achieve the high precision motion detection, an optimization framework with a novel motion pose estimation energy function is provided in the proposed method by which the 3D motion pose of each object can be estimated independently. We also develop an accurate object-tracking method which combines 2D visual information and depth. We incorporate a novel boundary-optimization segmentation based on 2D visual information and depth to improve the robustness of tracking significantly. Besides, we also introduce a new fusion and updating strategy in the 3D reconstruction process. This strategy brings higher robustness to 3D motion detection. Experiments results show that, for synthetic sequences, the root-mean-square error (RMSE) of our system is much smaller than Co-Fusion (CF); our system performs extremely well in 3D motion detection accuracy. In the case of occlusion or out-of-view on real scene data, CF will suffer the loss of tracking or object-label changing, by contrast, our system can always keep the robust tracking and maintain the correct labels for each dynamic object. Therefore, our system is robust to occlusion and out-of-view application scenarios.

(Invited) How to Integrate MEMS on Foundry-Fabricated CMOS Backplanes

A large variety of new applications for Internet of Things (IoT), Industrie 4.0 and other consumer products are pushing volume and manufacturing of microelectromechanical systems (MEMS) into a new phase. Due to massive use of these sensors and actuators in mobile and industrial applications smaller systems combined with low power consumption and higher versatility are required. MEMS are transducers that sense or control physical, chemical or optical quantities creating devices for applications in the area of 3D-motion tracking, pressure sensing, light shaping or detection of irradiation. The combination of MEMS with an application specific integrated circuit (ASIC) based on CMOS technology enables the entire system to interact with outside world. These ASICs are providing required features such as analog-to-digital conversion, amplification, filtering, information processing and storage as well as communication to the outside world. There are different solutions available to combine MEMS with customized CMOS circuits. One choice known as system-in-package (SiP) is the manufacturing of MEMS and CMOS on separate wafers and its subsequent integration in a multi-chip module using interposer/ rewiring technologies. Key advatanges of this technique are high flexibility, high modularity and a complete decoupled manufacturing of CMOS and MEMS. This enables a rapid development and leads to low development costs. And, there is no impact of different sizes between CMOS and MEMS chip. In order to enhance the integration density, to drive system feature size down, to suppress potential parasitic capacitances and to reduce power consumption an integrated manufacturing on the same substrate is a must and known as system-on-chip (SoC) solution. On one hand this approach can be still realized by a multi-wafer processing and a final integration using bonding technologies. This technology is often named as heterogeneous integration. Key advantage of multi-wafer technology is the possibility to use high-performance MEMS materials such as monocrystalline silicon as actuation/sensing material and its integration on CMOS wafers. Disadvantages are the required alignment accuracies when using a metallic bond or limitation for integration densities when using via-last approach. A more consistent way is a complete monolithic integration of MEMS on CMOS substrates. Especially when large transducer arrays are required (e.g. bolometers, micro-mirror arrays, CMUT arrays) a monolithic integration of MEMS on CMOS is the best solution. All advantages are now on hand – high integration densities, low parasitic capacitances and an effective usage of CMOS area. Fraunhofer IPMS applies surface and bulk micromachining technologies for the integration of MEMS on customized CMOS backplanes. To reduce development costs and time-to-market IPMS combines standard CMOS processes offered from CMOS foundries with a subsequent integration of MEMS part in our MEMS fab. Due to the application specific design of the CMOS backplane this concept allows a perfect match between ASIC and MEMS functionalities. In a typical project flow CMOS and MEMS will be manufactured and tested in parallel to shorten development time. After ASIC design is finished it will be tested using Multi- Project- Wafer run (MPW) in a CMOS foundry. Using these MPW runs a complete characterization of CMOS functionality can be realized in an early project state at low costs. Parallel to CMOS testing an entire process development of MEMS part can be done on passive devices in our MEMS fab. If CMOS and MEMS characterization is completed the MEMS part will be monolithically integrated on these customized CMOS backplanes. But there are still some challenges for the integration of MEMS on foundry fabricated CMOS backplanes, e.g.: Modifications of last CMOS/ interconnect/ ILD layers to realize an appropriate interface to MEMS Clarification of PCM (process control monitoring) test environment, depth of test structures and possible arising contamination issues In-chip surface topology after interconnect processing to define necessary actions to achieve required planarity for further processing Tuning mix-and-match lithography to achieve high overlay accuracies (Picture 1: Reached overlay accuracy between last 0,18µm CMOS and first MEMS layer of about 30- 40nm after correction step using i-line stepper) Required chip design features when using sacrificial layer technology IPMS offers a wide range of surface and bulk micromachining technologies which are particularly suitable for the fabrication of sensors and actuators on pre-fabricated CMOS wafers using monolithic integration. Especially surface micromachining using inorganic sacrificial layer technology allows the realization of rather complex MEMS structures on CMOS backplanes. Based on examples such as spatial light modulators (SLM) and capacitive micromachined ultrasonic transducers (CMUT) we will present solutions for the integration of surface micromachined MEMS on customized CMOS substrates. Figure 1

Bionic compound eye for 3D motion detection using an optical freeform surface

To achieve a system combining large field of view (FOV), high image quality, high 3D detection accuracy, and high-speed tracking in a close shot, a single-layered compound eye with seven ommatidia is developed. The functional relationship among the FOV, optical axis angle, and overlapping area is established mathematically. Based on the relationship, one center ommatidium and six hexagonally arranged marginal ommatidia with large apertures are set on a spherical substrate to guarantee a large FOV and sufficient overlapping information for 3D imaging. To control the off-axis aberration of the marginal ommatidia, an optical freeform surface is applied. The algorithms of regions of interest and Kalman filter are applied to achieve high-speed capturing. Experiments show that the system can obtain a high-quality image with sufficient overlapping information for 3D tracking in shifting distance.

A study to quantify the effect of patient motion and develop methods to detect and correct for motion during myocardial perfusion imaging on a CZT solid-state dedicated cardiac camera

BackgroundDue to differences in the design and acquisition parameters on the solid-state CZT cardiac camera the effect of patient motion may vary compared to Anger cameras. This study evaluates the effect of motion, two new methods of three-dimensional (3D) motion detection and a method of motion correction. MethodPhantom acquisitions were offset in the X, Y, and Z directions and combined to simulate different types of motion. Motion artifacts were identified using the total perfusion defect and blinded visual interpretation. Motion was detected by registering planar and reconstructed 30 second images, and corrected by summing the aligned reconstructed images. Validation was performed on phantom data. These techniques were then applied to 40 patient studies. ResultsMotion ≥10 mm and ≥60 seconds in duration introduced significant artifacts. There was no significant difference (P = .258) between the two methods of motion detection. Motion correction removed artifacts from 9/10 phantom simulations. Superior-inferior motion ≥8 mm was measured on 10% of patient studies, and 5% were affected by motion. Motion in the lateral and anterior-posterior directions was <8 mm. ConclusionSuperior-inferior patient motion artifacts have been identified on myocardial perfusion images acquired on a CZT camera. Routine QC to identify studies with significant motion is recommended.

Holographic three-dimensional motion detection of an optically trapped sub-100 nm gold nanoparticle

We demonstrated, for the first time, three-dimensional (3D) motion detection of a gold nanoparticle held in optical tweezers in water using holographic microscopy. Motion detection was performed with an in-line, low-coherence digital holographic microscope. The nanoparticle diameter was 60nm, which is the smallest reported particle whose 3D motion can be measured. The motion of the optically-trapped nanoparticle had an axial variation of 7.6nm (standard deviation) when a 1070nm laser beam with an intensity of more than 27MW/cm2 was focused with a 1.25 NA objective lens. From a comparison of a gold nanoparticle fixed on a glass substrate and a gold nanoparticle trapped by optical tweezers with a sufficient light intensity, we found that most of the variation was caused by noise in the experimental setup, and not the motion itself. The lateral variations for the fixed and trapped nanoparticles were the same, but the axial variation for the trapped nanoparticle was slightly larger. This suggested that the optically trapped nanoparticle had a motion with a single-nanometer order in the standard deviation only along the axial direction when the laser intensity was sufficiently large for trapping, which in this experiment was more than 27MW/cm2.

Expectation Modulates the Effect of Deep Brain Stimulation on Motor and Cognitive Function in Tremor-Dominant Parkinson's Disease

Expectation contributes to placebo and nocebo responses in Parkinson's disease (PD). While there is evidence for expectation-induced modulations of bradykinesia, little is known about the impact of expectation on resting tremor. Subthalamic nucleus (STN) deep brain stimulation (DBS) improves cardinal PD motor symptoms including tremor whereas impairment of verbal fluency (VF) has been observed as a potential side-effect. Here we investigated how expectation modulates the effect of STN-DBS on resting tremor and its interaction with VF. In a within-subject-design, expectation of 24 tremor-dominant PD patients regarding the impact of STN-DBS on motor symptoms was manipulated by verbal suggestions (positive [placebo], negative [nocebo], neutral [control]). Patients participated with (MedON) and without (MedOFF) antiparkinsonian medication. Resting tremor was recorded by accelerometry and bradykinesia of finger tapping and diadochokinesia were assessed by a 3D ultrasound motion detection system. VF was quantified by lexical and semantic tests. In a subgroup of patients, the effect of STN-DBS on tremor was modulated by expectation, i.e. tremor decreased (placebo response) or increased (nocebo response) by at least 10% as compared to the control condition while no significant effect was observed for the overall group. Interestingly, nocebo responders in MedON were additionally characterized by significant impairment in semantic verbal fluency. In contrast, bradykinesia was not affected by expectation. These results indicate that the therapeutic effect of STN-DBS on tremor can be modulated by expectation in a subgroup of patients and suggests that tremor is also among the parkinsonian symptoms responsive to placebo and nocebo interventions. While positive expectations enhanced the effect of STN-DBS by further decreasing the magnitude of tremor, negative expectations counteracted the therapeutic effect and at the same time exacerbated a side-effect often associated with STN-DBS. The present findings underscore the potency of patients' expectation and its relevance for therapeutic outcomes.

Inverse synthetic aperture radar image focusing using fast adaptive joint time–frequency and three-dimensional motion detection on experimental radar data

This study presents two algorithms for enhancing the capabilities of the adaptive joint time-frequency (AJTF) approach and the three-dimensional (3D) motion detection method, respectively, on inverse synthetic aperture radar (ISAR) imaging. The first method is a modified AJTF motion compensation algorithm that is able to employ more than two prominent point scatterers for translation and rotation motion compensation, respectively. The second algorithm enhances a current 3D motion detection method. This algorithm improves the computation time for identifying 3D motion in ISAR images and selects the imaging time intervals with the least 3D motion. With these enhanced detection algorithms, the authors have the ability to distinguish the time intervals when the target undergoes smooth two-dimensional (2D) motion from those containing more chaotic 3D motion. As a result, the authors can reliably detect those time intervals where 2D target motions are predominant to form well-focused ISAR images that are used in automatic target recognition applications.

A respiratory self‐gating technique with 3D‐translation compensation for free‐breathing whole‐heart coronary MRA

Respiratory motion remains a major challenge for robust coronary MR angiography (MRA). Diaphragmatic navigator (NAV) suffers from indirect measurement of heart position. Respiratory self-gating (RSG) approaches improve motion detection only in the head-feet direction, leaving motion in the other two dimensions unaccounted for. The purpose of this study was to extend conventional RSG (1D RSG) to RSG capable of 3D motion detection (3D RSG) by acquiring additional RSG projections with transverse-motion-encoding gradients. Simulation and volunteer studies were conducted to validate the effectiveness of this new method. Preliminary comparison was performed between coronary artery images reconstructed from the same datasets using different motion correction methods. Our simulation illustrates that a proper motion-encoding gradient and derivation method enable accurate 3D motion detection. Results from whole-heart coronary MRA show that 3D RSG can further reduce motion artifacts as compared to NAV and 1D RSG and enables use of larger gating windows for faster coronary imaging.

Detecting binocular 3D motion in static 3D noise: no effect of viewing distance.

Relative binocular disparity cannot tell us the absolute 3D shape of an object, nor the 3D trajectory of its motion, unless the visual system has independent access to how far away the object is at any moment. Indeed, as the viewing distance is changed, the same disparate retinal motions will correspond to very different real 3D trajectories. In this paper we were interested in whether binocular 3D motion detection is affected by viewing distance. A visual search task was used, in which the observer is asked to detect a target dot, moving in 3D, amidst 3D stationary distractor dots. We found that distance does not affect detection performance. Motion-in-depth is consistently harder to detect than the equivalent lateral motion, for all viewing distances. For a constant retinal motion with both lateral and motion-in-depth components, detection performance is constant despite variations in viewing distance that produce large changes in the direction of the 3D trajectory. We conclude that binocular 3D motion detection relies on retinal, not absolute, visual signals.

Robust Regression for the Detection of Independent 3D Motion by a Binocular Observer

A method is proposed for the visual detection of objects that move independently of the observer in a 3D dynamic environment. Many of the existing techniques for solving this problem are based on 2D motion models, which is equivalent to assuming that all the objects in a scene are at a constant depth from the observer. Although such methods perform well if this assumption holds, they may give erroneous results when applied to scenes with large depth variations. Additionally, many of the existing techniques rely on the computation of optical flow, which amounts to solving the ill-posed correspondence problem. In this paper, independent 3D motion detection is formulated using 3D models and is approached as a problem of robust regression applied to visual input acquired by a binocular, rigidly moving observer. Similar analysis is applied both to the stereoscopic data taken by a non-calibrated stereoscopic system and to the motion data obtained from successive frames in time. Least Median of Squares (LMedS) estimation is applied to stereoscopic data to produce maps of image regions characterized by a dominant depth. LMedS is also applied to the motion data that are related to the points at the dominant depth, to segment the latter with respect to 3D motion. In contrast to the methods that rely on 2D models, the proposed method performs accurately, even in the case of scenes with large depth variations. Both stereo and motion processing is based on the normal flow field, which (in contrast to the optical flow field) can be robustly computed from the spatiotemporal derivatives of the image intensity function. Although parts of the proposed scheme have non-trivial computational requirements, computations can be expedited by various ways which are discussed in detail. This is also demonstrated by an on-board implementation of the method on a mobile robotic platform. The method has been evaluated using synthetic as well as real data. Sample results show the effectiveness and robustness of the proposed scheme.