Motion Jitter Research Articles

YOLOv8‐ESW: An Improved Oncomelania hupensis Detection Model

ABSTRACTTraditional Oncomelania hupensis detection relies on human eye observation, which results in reduced efficiency due to easy fatigue of the human eye and limited individual cognition, an improved YOLOv8 O. hupensis detection algorithm, YOLOv8‐ESW(expectation–maximization attention [EMA], Small Target Detection Layer, and Wise‐IoU), is proposed. The original dataset is augmented using the OpenCV library. To imitate image blur caused by motion jitter, salt and pepper, and Gaussian noise were added to the dataset; to imitate images from different angles captured by the camera in an instant, affine, translation, flip, and other transformations were performed on the original data, resulting in a total of 6000 images after data enhancement. Considering the insufficient feature fusion problem caused by lightweight convolution, We present the expectation–EMA module (E), which innovatively incorporates a coordinate attention mechanism and convolutional layers to introduce a specialized layer for small target detection (S). This design significantly improves the network's ability to synergize information from both superficial and deeper layers, better focusing on small target O. hupensis and occluded O. hupensis. To tackle the challenge of quality imbalance among O. hupensis samples, we employ the Wise‐IoU (WIoU) loss function (W). This approach uses a gradient gain distribution strategy and improves the model convergence speed and regression accuracy. The YOLOv8‐ESW model, with 16.8 million parameters and requiring 98.4 GFLOPS for computations, achieved a mAP of 92.74% when tested on the O. hupensis dataset, marking a 4.09% improvement over the baseline model. Comprehensive testing confirms the enhanced network's efficacy, significantly elevating O. hupensis detection precision, minimizing both missed and false detections, and fulfilling real‐time processing criteria. Compared with the current mainstream models, it has certain advantages in detection accuracy and has reference value for subsequent research in actual detection.

Enhancing Multispectral Breast Imaging Quality Through Frame Accumulation and Hybrid GA-CPSO Registration.

Multispectral transmission imaging has emerged as a promising technique for imaging breast tissue with high resolution. However, the method encounters challenges such as low grayscale, noisy transmission images with weak signals, primarily due to the strong absorption and scattering of light in breast tissue. A common approach to improve the signal-to-noise ratio (SNR) and overall image quality is frame accumulation. However, factors such as camera jitter and respiratory motion during image acquisition can cause frame misalignment, degrading the quality of the accumulated image. To address these issues, this study proposes a novel image registration method. A hybrid approach combining a genetic algorithm (GA) and a constriction factor-based particle swarm optimization (CPSO), referred to as GA-CPSO, is applied for image registration before frame accumulation. The efficiency of this hybrid method is enhanced by incorporating a squared constriction factor (SCF), which speeds up the registration process and improves convergence towards optimal solutions. The GA identifies potential solutions, which are then refined by CPSO to expedite convergence. This methodology was validated on the sequence of breast frames taken at 600 nm, 620 nm, 670 nm, and 760 nm wavelength of light and proved the enhancement of accuracy by various mathematical assessments. It demonstrated high accuracy (99.93%) and reduced registration time. As a result, the GA-CPSO approach significantly improves the effectiveness of frame accumulation and enhances overall image quality. This study explored the groundwork for precise multispectral transmission image segmentation and classification.

VividWav2Lip: High-Fidelity Facial Animation Generation Based on Speech-Driven Lip Synchronization

Speech-driven lip synchronization is a crucial technology for generating realistic facial animations, with broad application prospects in virtual reality, education, training, and other fields. However, existing methods still face challenges in generating high-fidelity facial animations, particularly in addressing lip jitter and facial motion instability issues in continuous frame sequences. This study presents VividWav2Lip, an improved speech-driven lip synchronization model. Our model incorporates three key innovations: a cross-attention mechanism for enhanced audio-visual feature fusion, an optimized network structure with Squeeze-and-Excitation (SE) residual blocks, and the integration of the CodeFormer facial restoration network for post-processing. Extensive experiments were conducted on a diverse dataset comprising multiple languages and facial types. Quantitative evaluations demonstrate that VividWav2Lip outperforms the baseline Wav2Lip model by 5% in lip sync accuracy and image generation quality, with even more significant improvements over other mainstream methods. In subjective assessments, 85% of participants perceived VividWav2Lip-generated animations as more realistic compared to those produced by existing techniques. Additional experiments reveal our model’s robust cross-lingual performance, maintaining consistent quality even for languages not included in the training set. This study not only advances the theoretical foundations of audio-driven lip synchronization but also offers a practical solution for high-fidelity, multilingual dynamic face generation, with potential applications spanning virtual assistants, video dubbing, and personalized content creation.

Blurred spectral images restoration technology for AOTF imaging spectrometer based on dual-path architecture.

The acousto-optic tunable filter (AOTF) imaging spectrometer is a staring-type instrument that acquires spectral images of different bands asynchronously and sequentially, and is susceptible to platform jitter and target motion. When the integration time of the detector increases, serious spectral images degradation occurs along with a large band-to-band misregistration. The dual-path optical configuration, capable of simultaneously acquiring diffracted and undiffracted beams of AOTF, has been shown to be effective in solving the problem of band-to band misregistration. Hence by solving the remaining spectral image degradation problems, the application range of AOTF will transcend the previous limitation. Therefore, this paper presents a new, to the best of our knowledge, method of capturing a series of short-exposure undiffracted beam images to calculate the blur kernel of the diffracted spectral images, since the high-intensity characteristics of the undiffracted beam can also be exploited in the dual-path system. The paper starts with analyzing the impact of the platform on spectral images, and then demonstrates the development and calibration of a dual-path based spectral image deblurring method. Finally, laboratory experiment verifies that the resulting blur kernel can be effectively used for spectral image restoration, thus demonstrating the robustness of this technique.

Research on Fast Self-alignment Method of SINS under Vehicle Jitter and Leveling Disturbance

Using the combination of simulation and actual measurement, analyze the impact of disturbances such as the jitter of the launch vehicle drive motor and the leveling motion of the launch vehicle on the output signals of the gyroscope and accelerometer, clarify the alignment error source of the SINS, and combine the error model to design based on the Kalman classical filter. For the algorithm’s variable mode filtering scheme, a variable mode filtering alignment method using the output pulse threshold of the horizontal gyro is proposed, and the filtering method in the fine alignment stage is improved. Vehicle experiments show that the proposed variable mode filtering method has a simple algorithm and a small amount of calculation, and can effectively suppress dynamic disturbances such as engine jitter and vehicle leveling motion, improve the convergence speed of the azimuth error of the strapdown inertial navigation system, and reduce the azimuth misalignment angle estimate bias.

Fully Coherent Integration and Measurement of Optimized Frequency Agile Waveform for Weak Target High-Resolution ISAR Imaging

Frequency agile waveform effectively decreases interception probability and increases the anti-jamming ability for radar probing in the electromagnetic countermeasure environment. However, the transmitting agile frequency introduces jitter phases causing de-coherence of echoes, which decreases signal-to-noise ratio (SNR) accumulation gain and leads to difficult motion estimation. Therefore, stable target detection, motion parameters estimation, and inverse synthetic aperture radar (ISAR) imaging for frequency-agile radar are still intractable problems in the low SNR environment. Aiming at these problems, we proposed a fully coherent processing method for weak target detection and ISAR imaging for frequency agile waveform in this paper. The agile waveform with low range and Doppler side lobes is designed to improve motion parameters estimation and ISAR imaging performance. Then, the joint jitter phases and motion parameters estimation based on the generalized likelihood ratio test (GLRT) is proposed to realize robust target detection and motion parameters estimation in low SNR conditions. The translational motion compensation and sparse ISAR imaging methods are also presented based on the detection and motion parameters estimation results. Finally, both simulated and real-measured data are used to verify the remarkable detection, motion parameters estimation, and ISAR imaging performance compared with the traditional non-coherent accumulation and mixture of coherent and non-coherent accumulation methods.

PAMSGAN: Pyramid Attention Mechanism-Oriented Symmetry Generative Adversarial Network for Motion Image Deblurring

Motion blur is a common problem in optical imaging, which is caused by the relative displacement between the subject and the camera in the exposure process of the camera. This can result in motion blur of the acquired image, reduce the image resolution and affect the imaging quality. Motion blur image restoration technology uses the existing motion blur image to restore the clear image through the modeling of imaging physical process and mathematical solution without re-photographing the target scene. It has an important application value in the civil and military fields. Solving the problem of motion blur caused by camera jitter and object motion during camera imaging is a very challenging problem. When the popular generative adversarial network model is directly applied to the image blur blind removal task, serious pattern collapse phenomenon will occur. In this paper, we propose a novel motion image deblurring model based on pyramid attention mechanism-oriented symmetry generative adversarial network. This new method does not need to predict the fuzzy kernel of the blurred images, and can directly realize the blind removal of image motion blur. Based on the original CycleGan, the network structure and loss function of the symmetry generative adversarial network are improved. The accuracy of blind removal of motion images is improved, and the stability of the network is greatly enhanced in the case of limited samples. The generative network adopts the encoding and decoding structure, and introduces the feature pyramid attention mechanism. The combination of multi-scale pyramid features and attention mechanism can capture more rich advanced features to improve the model performance. In the experiment, the RMSProp algorithm is used to optimize the network training. Finally, a clear image is obtained through network adversarial training between generative and discriminant network. Experimental results on the related image blur benchmark datasets show that the restoration quality of the proposed method is higher in terms of subjective and objective evaluation. Meanwhile, the restoration results can achieve better results in subsequent object detection tasks.

Context Module Based Multi-patch Hierarchical Network for Motion Deblurring

Single image blind motion deblurring refers to transferring a blurred motion image into a corresponding clear image, which is a challenging and classic problem in the field of computer vision. The spatially variant blur is usually caused by many factors, such as camera jitter and object motion. Since image deblurring can be regarded as the task of image transformation, deep learning methods based on coarse-to-fine scheme, especially those using multi-scale architectures become popular. However, they have the disadvantages of unsatisfactory image quality and time-consuming running caused by large kernel size. In this paper, we propose a novel end-to-end network structure based on Deep Hierarchical Multi-patch network architecture integrated with Context Module and additional ResBlocks in order to tackle deblurring problem. Compared with recently proposed networks, it generates images with better visual effect as well as higher image quality index. Besides, our model significantly reduces the test time. We evaluate the proposed network structure on public GoPro dataset, a large-scale image dataset with complex synthetic blur. The experiments on the benchmark dataset prove that our effective method outperforms other state-of-the-art blind deblurring algorithms both qualitatively and quantitatively, which demonstrates the effectiveness of Context Module in the task of single image blur removal.

Zitterbewegung-mediated RKKY coupling in topological insulator thin films

The dynamics of itinerant electrons in topological insulator (TI) thin films is investigated using a multi-band decomposition approach. We show that the electron trajectory in the 2D film is anisotropic and confined within a characteristic region. Remarkably, the confinement and anisotropy of the electron trajectory are associated with the topological phase transition of the TI system, which can be controlled by tuning the film thickness and/or applying an in-plane magnetic field. Moreover, persistent electron wavepacket oscillation can be achieved in the TI thin film system at the phase transition point, which may assist in the experimental detection of the jitter motion (Zitterbewegung). The implications of the microscopic picture of electron motion in explaining other transport-related effects, e.g., electron-mediated RKKY coupling in the TI thin film system, are also discussed.

A visual SLAM method based on point-line fusion in weak-matching scene

Visual simultaneous localization and mapping (SLAM) is well-known to be one of the research areas in robotics. There are many challenges in traditional point feature-based approaches, such as insufficient point features, motion jitter, and low localization accuracy in low-texture scenes, which reduce the performance of the algorithms. In this article, we propose an RGB-D SLAM system to handle these situations, which is named Point-Line Fusion (PLF)-SLAM. We utilize both points and line segments throughout the process of our work. Specifically, we present a new line segment extraction method to solve the overlap or branch problem of the line segments, and then a more rigorous screening mechanism is proposed in the line matching section. Instead of minimizing the reprojection error of points, we introduce the reprojection error based on points and lines to get a more accurate tracking pose. In addition, we come up with a solution to handle the jitter frame, which greatly improves tracking success rate and availability of the system. We thoroughly evaluate our system on the Technische Universität München (TUM) RGB-D benchmark and compare it with ORB-SLAM2, presumably the current state-of-the-art solution. The experiments show that our system has better accuracy and robustness compared to the ORB-SLAM2.

Software-Based Integral Product Architecture for Modular Motion Control System of a RFC Linear Motor Motion Stage: Model-Based DOB for Residual Vibration Suppression

Together with strong ICT technology and concept of smart factory, motion control system is driving innovation and increased productivity across the industrial landscape. Thus, motion control systems built with modular approach has ever-longer lists of demands for performance improvement. This paper presents software-based integral product architecture for the modular motion control system of a RFC linear motor motion stage: a model-based DOB for residual vibration suppression. First, the modular motion control system of a linear motor motion stage with air bearing is introduced and its digital twin or multi-rate simulation model is verified with experiments. Then, software-based integral product approach or a model-based DOB is applied to improve the performances of the linear motor stage and its feasibility is confirmed based on the simulation model. Auxiliary software to implement the model-based DOB into a commercial motion control are developed including excitation, DOB user code and data import. Finally, the model-based DOB is shown to improve both motion jitter and settling time of the motion stage through experiments. Precisely, 93% of the following error and 84% the settling time of ± 0.5 μm window are improved.

Global Jitter Motion of the Retinal Image Dynamically Alters the Receptive Field Properties of Retinal Ganglion Cells.

Fixational eye movements induce aperiodic motion of the retinal image. However, it is not yet fully understood how fixational eye movements affect retinal information processing. Here we show that global jitter motion, simulating the image motion during fixation, alters the spatiotemporal receptive field properties of retinal ganglion cells. Using multi-electrode and whole-cell recording techniques, we investigated light-evoked responses from ganglion cells in the isolated goldfish retina. Ganglion cells were classified into six groups based on the filtering property of light stimulus, the membrane properties, and the cell morphology. The spatiotemporal receptive field profiles of retinal ganglion cells were estimated by the reverse correlation method, where the dense noise stimulus was applied on the dark or random-dot background. We found that the jitter motion of the random-dot background elongated the receptive filed along the rostral-caudal axis and temporally sensitized in a specific group of ganglion cells: Fast-transient ganglion cells. At the newly emerged regions of the receptive field local light stimulation evoked excitatory postsynaptic currents with large amplitude and fast kinetics without changing the properties of inhibitory postsynaptic currents. Pharmacological experiments suggested two presynaptic mechanisms underlying the receptive field alteration: (i) electrical coupling between bipolar cells, which expands the receptive field in all directions; (ii) GABAergic presynaptic inhibition from amacrine cells, which reduces the dorsal and ventral regions of the expanded receptive field, resulting in elongation along the rostral-caudal axis. Our study demonstrates that the receptive field of Fast-transient ganglion cells is not static but dynamically altered depending on the visual inputs. The receptive field elongation during fixational eye movements may contribute to prompt firing to a target in the succeeding saccade.

Vision-Based Obstacle Avoidance Strategies for MAVs Using Optical Flows in 3-D Textured Environments.

Due to payload restrictions for micro aerial vehicles (MAVs), vision-based approaches have been widely studied with their light weight characteristics and cost effectiveness. In particular, optical flow-based obstacle avoidance has proven to be one of the most efficient methods in terms of obstacle avoidance capabilities and computational load; however, existing approaches do not consider 3-D complex environments. In addition, most approaches are unable to deal with situations where there are wall-like frontal obstacles. Although some algorithms consider wall-like frontal obstacles, they cause a jitter or unnecessary motion. To address these limitations, this paper proposes a vision-based obstacle avoidance algorithm for MAVs using the optical flow in 3-D textured environments. The image obtained from a monocular camera is first split into two horizontal and vertical half planes. The desired heading direction and climb rate are then determined by comparing the sum of optical flows between half planes horizontally and vertically, respectively, for obstacle avoidance in 3-D environments. Besides, the proposed approach is capable of avoiding wall-like frontal obstacles by considering the divergence of the optical flow at the focus of expansion and navigating to the goal position using a sigmoid weighting function. The performance of the proposed algorithm was validated through numerical simulations and indoor flight experiments in various situations.

A Hardware–Software Co-Design Framework for Real-Time Video Stabilization

Live digital video is a valuable source of information in security, broadcast and industrial quality control applications. Motion jitter due to camera and platform instability is a common artefact found in captured video which renders it less effective for subsequent computer vision tasks such as detection and tracking of objects, background modeling, mosaicking, etc. The process of algorithmically compensating for the motion jitter is hence a mandatory pre-processing step in many applications. This process, called video stabilization, requires estimation of global motion from consecutive video frames and is constrainted by additional challenges such as preservation of intentional motion and native frame resolution. The problem is exacerbated in the presence of local motion of foreground objects and requires robust compensation of the same. As such achieving real-time performance for this computationally intensive operation is a difficult task for embedded processors with limited computational and memory resources. In this work, development of an optimized hardware–software co-design framework for video stabilization has been investigated. Efficient video stabilization depends on the identification of key points in the frame which in turn requires dense feature calculation at the pixel level. This task has been identified to be most suitable for offloading the pipelined hardware implemented in the FPGA fabric due to the involvement of complex memory and computation operations. Subsequent tasks to be performed for the overall stabilization algorithm utilize these sparse key points and have been found to be efficiently handled in the software. The proposed Hardware–Software (HW–SW) co-design framework has been implemented on Zedboard FPGA platform which houses Xilinx Zynq SOC equipped with ARM A9 processor. The proposed implementation scheme can process real-time video stream input at 28 frames per second and is at least twice faster than the corresponding software-only approach. Two different hardware accelerator designs have been implemented using different high-level synthesis tools using rapid prototyping principle and consume less than 50% of logic resources available on the host FPGA while being at least 30% faster than contemporary designs.

Training peripheral vision to read: Reducing crowding through an adaptive training method

Reading is slow and difficult for people with central vision loss who must rely on their peripheral vision. It has been shown that practicing on a letter-recognition task can increase peripheral reading speed, and that the training-related improvement is attributable mainly to reduced crowding. Since there is a high degree of variability in the vision conditions across people with central vision loss, a one-size-fits-all training protocol may not be adequate or appropriate for these patients. In this study, we target two aspects of training—training task and individual customization, and propose a training paradigm that focuses on reducing crowding and tailors training for each individual using an adaptive method. Seven normally-sighted adults were trained with four daily sessions of identifying crowded letters presented at various positions 10° below fixation in a pre/post design. During the training, a dynamic cue (jitter motion) was applied to target letters to modulate crowding. Amplitude of motion was varied on a block by block basis according to individual performance to maintain task difficulty near a pre-defined level (80% accuracy in letter recognition). We found that motion amplitude gradually reduced as training progressed, indicating a reduction in crowding. Following training, reading speed (measured using RSVP method) showed a significant improvement in both the trained (49%) and untrained (50%) visual fields. Despite showing similar improvement as observed in the previous training studies, our adaptive training method demands less effort and, most importantly, offers customization for each individual trainee.

Element-Weighted Neutrosophic Correlation Coefficient and Its Application in Improving CAMShift Tracker in RGBD Video

Neutrosophic set (NS) is a new branch of philosophy to deal with the origin, nature, and scope of neutralities. Many kinds of correlation coefficients and similarity measures have been proposed in neutrosophic domain. In this work, by considering that there may exist different contributions for the neutrosophic elements of T (Truth), I (Indeterminacy), and F (Falsity), a method of element-weighted neutrosophic correlation coefficient is proposed, and it is applied for improving the CAMShift tracker in RGBD (RGB-Depth) video. The concept of object seeds is proposed, and it is employed for extracting object region and calculating the depth back-projection. Each candidate seed is represented in the single-valued neutrosophic set (SVNS) domain via three membership functions, T, I, and F. Then the element-weighted neutrosophic correlation coefficient is applied for selecting robust object seeds by fusing three kinds of criteria. Moreover, the proposed correlation coefficient is applied for estimating a robust back-projection by fusing the information in both color and depth domains. Finally, for the scale adaption problem, two alternatives in the neutrosophic domain are proposed, and the corresponding correlation coefficient between the proposed alternative and the ideal one is employed for the identification of the scale. When considering challenging factors like fast motion, blur, illumination variation, deformation, and camera jitter, the experimental results revealed that the improved CAMShift tracker performs well.

Digital Image Stabilization Method Based on Variational Mode Decomposition and Relative Entropy

Cameras mounted on vehicles frequently suffer from image shake due to the vehicles’ motions. To remove jitter motions and preserve intentional motions, a hybrid digital image stabilization method is proposed that uses variational mode decomposition (VMD) and relative entropy (RE). In this paper, the global motion vector (GMV) is initially decomposed into several narrow-banded modes by VMD. REs, which exhibit the difference of probability distribution between two modes, are then calculated to identify the intentional and jitter motion modes. Finally, the summation of the jitter motion modes constitutes jitter motions, whereas the subtraction of the resulting sum from the GMV represents the intentional motions. The proposed stabilization method is compared with several known methods, namely, medium filter (MF), Kalman filter (KF), wavelet decomposition (MD) method, empirical mode decomposition (EMD)-based method, and enhanced EMD-based method, to evaluate stabilization performance. Experimental results show that the proposed method outperforms the other stabilization methods.

A detector interferometric calibration experiment for high precision astrometry

Context: Exoplanet science has made staggering progress in the last two decades, due to the relentless exploration of new detection methods and refinement of existing ones. Yet astrometry offers a unique and untapped potential of discovery of habitable-zone low-mass planets around all the solar-like stars of the solar neighborhood. To fulfill this goal, astrometry must be paired with high precision calibration of the detector. Aims: We present a way to calibrate a detector for high accuracy astrometry. An experimental testbed combining an astrometric simulator and an interferometric calibration system is used to validate both the hardware needed for the calibration and the signal processing methods. The objective is an accuracy of 5e-6 pixel on the location of a Nyquist sampled polychromatic point spread function. Methods: The interferometric calibration system produced modulated Young fringes on the detector. The Young fringes were parametrized as products of time and space dependent functions, based on various pixel parameters. The minimization of func- tion parameters was done iteratively, until convergence was obtained, revealing the pixel information needed for the calibration of astrometric measurements. Results: The calibration system yielded the pixel positions to an accuracy estimated at 4e-4 pixel. After including the pixel position information, an astrometric accuracy of 6e-5 pixel was obtained, for a PSF motion over more than five pixels. In the static mode (small jitter motion of less than 1e-3 pixel), a photon noise limited precision of 3e-5 pixel was reached.

Time dilation in a jittering motion perceived in a stationary stimulus

Time dilation in a jittering motion perceived in a stationary stimulus

Signature of Topological Phases in Zitterbewegung

We have studied the Zitterbewegung effect on an infinite two-dimensional sheet with honeycomb lattice. By tuning the perpendicular electric field and the magnetization of the sheet, it can enter different topological phases. We have shown that the phase and magnitude of Zitterbewegung effect, i.e., the jittering motion of electron wavepackets, correlates with the various topological phases. The topological phase diagram can be reconstructed by analyzing these features. Our findings are applicable to materials like silicene, germanene, stanene, etc.