Motion Detection Model Research Articles

Unveiling the power of Haar frequency domain: Advancing small target motion detection in dim light

Visual small target motion detection finds successful applications in varied scenarios. However, dim-light conditions, such as the tunnel scenes and nighttime environments, present significant challenges to existing detection methods which mainly operate within the spatiotemporal domain. This is because the transmission of small target motion information suffers from the inevitable interference of image noise caused by dim light in the spatiotemporal domain, resulting in the detriment of extracting essential spatiotemporal features of the target motion. Given the significant obstacles posed by dim-light imaging to small target motion detection within the spatiotemporal domain, the exploration of an alternative observation domain for small target motion, alongside the development of a corresponding detection method, emerges as a viable solution. To address this, in this paper, we discovered the remarkable potential of the Haar frequency domain in characterizing the small target motion in dim light. To investigate the advantages of integrating Haar frequency processing in small target motion detection, we introduce a Haar-windowed summation mechanism into an existing bio-inspired small target motion detection model. The proposed mechanism integrates visual information in spatiotemporal windows regulated by frequency parameters of Haar wavelets and effectively discriminates the small target motion from the disturbance of random noise caused by dim light. Theoretical analysis and numerical experiments confirm the superior performance of integrating the Haar frequency processing. This study provides a new vision of small target motion detection through the lens of the frequency domain and extends the limits of existing bio-inspired models for practical applications in dim light.

Velocimetry of coarse particles in pipeline flow based on GMM model and flow direction constraints

Coarse particle vertical pipeline hydraulic transport has been widely used in deep sea mining, petrochemical industry and shaft slag removal. The precise measurement of the velocity of coarse particles within a solid–liquid two-phase flow in a vertical pipe constitutes a pivotal and challenging aspect when investigating particle motion characteristics. We introduce a method for measuring coarse particle velocity that combines a GMM motion detection model with flow direction constraint matching. In this approach, a high-speed camera system captures the flow image of solid–liquid two-phase flow within a pipeline, and the GMM model analyzes the image to detect the motion particles. Subsequently, a rapid particle matching algorithm, incorporating flow direction constraints, is formulated to establish the particle matching relationships. Simulation and real experiments substantiate the efficacy of this method. Moreover, the method achieves high-precision matching with an accuracy of up to 99%. For videos captured at 30 frames per second (fps), the method enables real-time speed measurement, with an overall measurement error of less than 5%. In summary, this is an effective method for measuring coarse particle velocity field which is important to the study of the motion characteristics of solid–liquid two-phase flow.

A penalized motion detection model for extracting ionospheric echoes from low signal-to-noise ratio Ionogram video images

A penalized motion detection model for extracting ionospheric echoes from low signal-to-noise ratio Ionogram video images

A spatiotemporal energy model based on spiking neurons for human motion perception.

Inspired by the motion processing pathway, this paper proposes a bio-inspired feedforward spiking network model based on Hodgkin-Huxley neurons for human motion perception. The proposed network mimics the mechanisms of direction selectivity found in simple and complex cells of the primary visual cortex. Simple cells' receptive fields are modeled using Gabor energy filters, while complex cells' receptive fields are constructed by integrating the responses of simple cells in an energy model. To generate the motion map, the spiking output of the network integrates motion information encoded by the responses of complex cells with various preferred directions. Simulation results demonstrate that the spiking neuron-based network effectively replicates the directional selectivity operation of the visual cortex when presented with a sequence of time-varying images. We evaluate the proposed model against state-of-the-art spiking neuron-based motion detection models using publicly available datasets. The results highlight the model's capability to extract motion energy from diverse video sequences, akin to human visual motion perception models. Additionally, we showcase the application of the proposed model in motion segmentation tasks and compare its performance with state-of-the-art motion-based segmentation models using challenging video segmentation benchmarks. The results indicate competitive performance. The motion maps generated by the proposed model can be utilized for action recognition in input videos.

Divisive normalization processors in the early visual system of the Drosophila brain

Divisive normalization is a model of canonical computation of brain circuits. We demonstrate that two cascaded divisive normalization processors (DNPs), carrying out intensity/contrast gain control and elementary motion detection, respectively, can model the robust motion detection realized by the early visual system of the fruit fly. We first introduce a model of elementary motion detection and rewrite its underlying phase-based motion detection algorithm as a feedforward divisive normalization processor. We then cascade the DNP modeling the photoreceptor/amacrine cell layer with the motion detection DNP. We extensively evaluate the DNP for motion detection in dynamic environments where light intensity varies by orders of magnitude. The results are compared to other bio-inspired motion detectors as well as state-of-the-art optic flow algorithms under natural conditions. Our results demonstrate the potential of DNPs as canonical building blocks modeling the analog processing of early visual systems. The model highlights analog processing for accurately detecting visual motion, in both vertebrates and invertebrates. The results presented here shed new light on employing DNP-based algorithms in computer vision.

A fractional-order visual neural model for small target motion detection

Detecting small moving targets consisting of one or few pixels is technically demanding due to their limited visual features. Motivated by nature, some bio-inspired models have been developed that simulate the behavior of small target motion detectors (STMDs), a class of specialized neurons found in insects’ visual neural systems dominating in detecting small moving targets during activities such as predation or courtship. However, the existing models’ high dependence on the sampling frequency of input videos becomes a bottleneck that seriously hinders their real-time applications in the physical world. This is because massive computational power is required to capture and process high-sampling-frequency videos. While model detection performance in low-sampling-frequency videos is plagued by significant spatial errors and weak responses. To address these issues, we propose an STMD-based visual neural model with a fractional-order difference operator for small target motion detection. The fractional-order operator captures instantaneous luminance change and integrates it with memory information, where the instantaneous information dominates the integrated signal. The STMD network further separates the rising and falling luminance components, which are aligned in the time domain and then multiplied to predict the location of moving small targets. Due to the rapid response of instantaneous information and the supplement of memory information, the proposed model locates the small moving targets accurately and robustly in low-sampling-frequencies. Numerical experiments show that the proposed model significantly improves the detection performance for low-sampling-frequency videos.

Vascular wall motion detection models based on long short-term memory in plane-wave-based ultrasound imaging

Objective. Vascular wall motion can be used to diagnose cardiovascular diseases. In this study, long short-term memory (LSTM) neural networks were used to track vascular wall motion in plane-wave-based ultrasound imaging. Approach. The proposed LSTM and convolutional LSTM (ConvLSTM) models were trained using ultrasound data from simulations and tested experimentally using a tissue-mimicking vascular phantom and an in vivo study using a carotid artery. The performance of the models in the simulation was evaluated using the mean square error from axial and lateral motions and compared with the cross-correlation (XCorr) method. Statistical analysis was performed using the Bland–Altman plot, Pearson correlation coefficient, and linear regression in comparison with the manually annotated ground truth. Main results. For the in vivo data, the median error and 95% limit of agreement from the Bland–Altman analysis were (0.01, 0.13), (0.02, 0.19), and (0.03, 0.18), the Pearson correlation coefficients were 0.97, 0.94, and 0.94, respectively, and the linear equations were 0.89x + 0.02, 0.84x + 0.03, and 0.88x + 0.03 from linear regression for the ConvLSTM model, LSTM model, and XCorr method, respectively. In the longitudinal and transverse views of the carotid artery, the LSTM-based models outperformed the XCorr method. Overall, the ConvLSTM model was superior to the LSTM model and XCorr method. Significance. This study demonstrated that vascular wall motion can be tracked accurately and precisely using plane-wave-based ultrasound imaging and the proposed LSTM-based models.

Modeling and Quantitative Analysis of Motion Detection in CSI-Based Sensing

Channel state information (CSI) based sensing approaches have unique advantages for fine-grained motion sensing. However, due to the mismatch between the sensing position and detection probability, mapping motion-sensing precision to optimal detection zones is an involved problem. In this letter, we propose a general CSI-based motion detection model, in which the Fresnel Zones and radar detection theory are used to analyze the key factors affecting detection probability. Besides, analytic expressions for the detection probability at arbitrary positions with available and unavailable phases are derived. Finally, numerical results corroborate the theoretical analysis and show that appropriate transceiver deployment and sensing frequency selection can effectively improve sensing performance.

Mathematical study of neural feedback roles in small target motion detection.

Building an efficient and reliable small target motion detection visual system is challenging for artificial intelligence robotics because a small target only occupies few pixels and hardly displays visual features in images. Biological visual systems that have evolved over millions of years could be ideal templates for designing artificial visual systems. Insects benefit from a class of specialized neurons, called small target motion detectors (STMDs), which endow them with an excellent ability to detect small moving targets against a cluttered dynamic environment. Some bio-inspired models featured in feed-forward information processing architectures have been proposed to imitate the functions of the STMD neurons. However, feedback, a crucial mechanism for visual system regulation, has not been investigated deeply in the STMD-based neural circuits and its roles in small target motion detection remain unclear. In this paper, we propose a time-delay feedback STMD model for small target motion detection in complex backgrounds. The main contributions of this study are as follows. First, a feedback pathway is designed by transmitting information from output-layer neurons to lower-layer interneurons in the STMD pathway and the role of the feedback is analyzed from the view of mathematical analysis. Second, to estimate the feedback constant, the existence and uniqueness of solutions for nonlinear dynamical systems formed by feedback loop are analyzed via Schauder's fixed point theorem and contraction mapping theorem. Finally, an iterative algorithm is designed to solve the nonlinear problem and the performance of the proposed model is tested by experiments. Experimental results demonstrate that the feedback is able to weaken background false positives while maintaining a minor effect on small targets. It outperforms existing STMD-based models regarding the accuracy of fast-moving small target detection in visual clutter. The proposed feedback approach could inspire the relevant modeling of robust motion perception robotics visual systems.

A Novel Method for Detecting Misclassifications of the Locomotion Mode in Lower-Limb Exoskeleton Robot Control

Lower-limb exoskeleton robots can support hemiplegic patients’ affected limbs and assist in their rehabilitation. In order to set effective control strategies, it is necessary to obtain the user’s motion intention accurately and timeously. These requirements pose many challenges. The surface electromyography (sEMG) signal has long been used for detecting a person’s locomotion mode. However, most traditional myoelectric motion recognition methods need to collect multi-channel sEMG signals (more than eight) and use feature engineering to obtain the desired accuracy. Additionally, traditional methods make decisions when the recognition probability is still uncertain, resulting in misclassification during continuous recognition. In order to overcome these limitations, a dual-purpose autoencoder-guided temporal convolution network (DA-TCN) is proposed, which is used as a motion detection model for a lower-limb exoskeleton robot based on sEMG. In the proposed method, we first obtain four-channel sEMG signals without any feature engineering. Next, the DA-TCN is used to obtain the discriminative deep features of the sEMG signals. Compared with separable features from common TCNs, discriminative features assist in obtaining a more discriminative twins-loss. Finally, the classification result is redetermined by the decoder, based on the reconstruction of deep features, and misclassifications are rejected based on the value of the twins-loss. The performance of the proposed method was evaluated by obtaining data from seven volunteer subjects in seven locomotion modes. The proposed method was able to effectively detect misclassifications and significantly improved the stability of continuous recognition.

Pedestrian Localization in a Video Sequence Using Motion Detection and Active Shape Models

There is increasing interest in video object detection for many situations, such as industrial processes, surveillance systems, and nature exploration. In this work, we were concerned with the detection of pedestrians in video sequences. The aim was to deal with issues associated with the background, scale, contrast, or resolution of the video frames, which cause inaccurate detection of pedestrians. The proposed method was based on the combination of two techniques: motion detection by background subtraction (MDBS) and active shape models (ASM). The MDBS technique aids in the identification of a moving region of interest in the video sequence, which potentially includes a pedestrian; then, the ASM algorithm actively finds and adjusts the silhouette of the pedestrian. We tested the proposed MDBS + ASM method with video sequences from open repositories, and the results were favorable in scenes where pedestrians were in a well-illuminated environment. The mean fit error was up to 4.5 pixels. In contrast, in scenes where reflections, occlusions, or pronounced movement are present, the identification was slightly affected; the mean fit error was 8.3 pixels in the worst case. The main contribution of this work was exploring the potential of the combination of MDBS and ASM for performance improvements in the contour-based detection of a moving pedestrian walking in a controlled environment. We present a straightforward method based on classical algorithms which have been proven effective for pedestrian detection. In addition, since we were looking for a practical process that could work in real-time applications (for example, closed-circuit television video or surveillance systems), we established our approach with simple techniques.

The Hodgkin–Huxley neuron model for motion detection in image sequences

In this paper, we consider a biologically inspired spiking neural network model for motion detection. The proposed model simulates the neurons’ behavior in the cortical area MT to detect different kinds of motion in image sequences. We choose the conductance-based neuron model of the Hodgkin–Huxley to define MT cell responses. Based on the center-surround antagonism of MT receptive fields, we model the area MT by its great proportion of cells with directional selective responses. The network’s spiking output corresponds to an MT neuron population’s firing rates and enables to extract motion boundaries. We conduct several experiments on real image sequences. The experimental results show the proposed network’s ability to segregate multiple moving objects from an image sequence and reproduce the MT cells’ responses. We perform a quantitative evaluation on the YouTube Motion Boundaries (YMB) dataset, and we compare the result to state-of-the-art methods for boundary detection in videos: boundary flow estimation (BF) and temporal boundary difference (BD). The proposed network model provides the best results on YMB compared to BF and BD methods.

A Dual Generation Adversarial Network for Human Motion Detection Using Micro-Doppler Signatures

Radar sensors and micro-Doppler signatures have been widely used to recognize human motions. Apart from the motion classification task, human motion detection has attracted much attention as an emerging topic. A majority of existing motion detectors are designed for a specific motion, such as falling. In some scenarios, however, a broader range of human actions is of interest, hence a general motion detector is desired. In this paper, we propose a radar-based motion detection model named dual generative adversarial network (DGN). The proposed model tackles the detection task as a one-class classification problem and is applicable to detecting various motions. Unlike prior fall detection algorithms, which depend on manually collected alien data, the DGN employs a dual generation scheme to automatically produce valid alien samples in both the pixel level and the semantic level. The model is verified on two measured radar datasets containing individual motions and interactive motions, respectively. The experimental results show that our method outperforms other existing models on the human motion detection task.

Predicting individual neuron responses with anatomically constrained task optimization

Artificial neural networks trained to solve sensory tasks can develop statistical representations that match those in biological circuits. However, it remains unclear whether they can reproduce properties of individual neurons. Here, we investigated how artificial networks predict individual neuron properties in the visual motion circuits of the fruit fly Drosophila. We trained anatomically constrained networks to predict movement in natural scenes, solving the same inference problem as fly motion detectors. Units in the artificial networks adopted many properties of analogous individual neurons, even though they were not explicitly trained to match these properties. Among these properties was the split into ON and OFF motion detectors, which is not predicted by classical motion detection models. The match between model and neurons was closest when models were trained to be robust to noise. These results demonstrate how anatomical, task, and noise constraints can explain properties of individual neurons in a small neural network.

Mechanical characterization of abdominal aortas using multi-perspective ultrasound imaging

Mechanical characterization of abdominal aortic aneurysms using personalized biomechanical models is being widely investigated as an alternative criterion to assess risk of rupture. These methods rely on accurate wall motion detection and appropriate model boundary conditions. In this study, multi-perspective ultrasound is combined with finite element models to perform mechanical characterization of abdominal aortas in volunteers.Multi-perspective biplane radio frequency ultrasound recordings were made under seven angles (-45° to 45°) in one phantom set-up and eight volunteers, which were merged using automatic image registration. 2-D displacement fields were estimated in the seven longitudinal ultrasound views, creating a sparse, high resolution 3-D map of the wall motion at relatively high frame rates (20–27 Hz). The displacements were used to personalize the subject-specific finite element model of which the geometry of the aorta, spine, and surrounding tissue were determined from a single 3-D ultrasound acquisition.Automatic registration of the multi-perspective images was successful in six out of eight cases with an average error of 5.4° compared to the ground truth. Displacements of the aortic wall were measured and cyclic strain of the aortic diameter was found ranging from 4.2% to 8.6%. The subject-specific mesh and inverse FE analysis was performed yielding shear moduli estimates for the wall between 104 and 215 kPa. Comparative results from a single-perspective workflow revealed very low aortic wall motion signal, which resulted in relatively high modulus estimates, between 230 and 754 kPa.Multi-perspective biplane ultrasound imaging was used to personalize finite element models of the abdominal aorta and its surroundings, and performing mechanical characterization of the aortic shear modulus. The method was found to be a more robust method compared to a single-perspective 3-D ultrasound approach. Future research will focus on investigating the use of multiple 3-D ultrasound acquisitions, the feasibility of free-hand scanning, the creation of a full 3-D automatic registration process, and with that, enable a clinical continuation of this study.

Deteksi Api pada Video dengan Gaussian Mixture Model Untuk Deteksi Gerakan Dan Segmentasi Warna Api Dalam Ruang Warna YCbCr

Fire is a disaster that can endanger lives and cause property loss. The solution to detect fire that is commonly used today is to use a sensor. Fire sensors can be used together with surveillance cameras (CCTV) which are now being installed in many office buildings. This study tries to build a model for detecting fire in video with a digital image processing approach using the Gaussian Mixture Model for motion detection and fire color segmentation in the YCbCr color space. The model is then tested with metrics for accuracy, precision, recall, and processing speed. The dataset used is in the form of videos with small, medium, large fire sizes, and videos that only have fire-like objects. The test results show that the algorithm is able to detect fire when the size of the fire is not too small or the position of the fire is close enough to the camera. For videos with a resolution of 800x600 and a framerate of 30 fps, it can achieve 66.89% accuracy, 73.77% precision, and 66.66% recall. The performance during the day is relatively better than at night. Algorithm processing speed is too slow to be implemented in real-time

Bio-inspired visual neural network on spatio-temporal depth rotation perception

In primates’ cerebral cortex, depth rotation sensitive (DRS) neurons have the property of preferential selectivity for depth rotation motion, whereas such a property is rarely adopted to create computational models for depth rotation motion detection. To fill this gap, a novel feedforward visual neural network is developed to execute depth rotation object detection, based on the recent neurophysiologic achievements on the mammalian vision system. The proposed neural network consists of two parts, i.e., presynaptic and postsynaptic neural networks. The former comprises multiple lateral inhibition neural sub-networks for the capture of visual motion information, and the latter extracts the cues of translational and depth motion and later, synthesizes such clues to perceive the process of depth rotation of an object. Experimentally, the neural network is sufficiently examined by different types of depth rotation under multiple conditions and settings. Numerical experiments show that not only it can effectively detect the spatio-temporal energy change of depth rotation of a moving object, but also its output excitation curve is a quasi-sinusoidal one, which is compatible with the hypothesis suggested by Johansson and Jansson in projective geometry. This research is a critical step toward the construction of artificial vision system for depth rotation object recognition.

A Flame Detection Method Based on Fusion Feature and SVM for Substation Inspection

For the safety inspection of substation, this paper proposes a flame image detection method based on the combination of fusion feature and support vector machine (SVM). Firstly, the suspected flame area is detected by motion detection and flame color model. Then, four features of B-channel color variation coefficient, roughness, flicker frequency and flame area change rate are extracted. Finally, four features are sent to SVM to learn the flame classifier and used in video flame detection. The experimental results show that the proposed method can effectively identify the flame.

A minimal synaptic model for direction selective neurons in Drosophila.

Visual motion estimation is a canonical neural computation. In Drosophila, recent advances have identified anatomic and functional circuitry underlying direction-selective computations. Models with varying levels of abstraction have been proposed to explain specific experimental results but have rarely been compared across experiments. Here we use the wealth of available anatomical and physiological data to construct a minimal, biophysically inspired synaptic model for Drosophila’s first-order direction-selective T4 cells. We show how this model relates mathematically to classical models of motion detection, including the Hassenstein-Reichardt correlator model. We used numerical simulation to test how well this synaptic model could reproduce measurements of T4 cells across many datasets and stimulus modalities. These comparisons include responses to sinusoid gratings, to apparent motion stimuli, to stochastic stimuli, and to natural scenes. Without fine-tuning this model, it sufficed to reproduce many, but not all, response properties of T4 cells. Since this model is flexible and based on straightforward biophysical properties, it provides an extensible framework for developing a mechanistic understanding of T4 neural response properties. Moreover, it can be used to assess the sufficiency of simple biophysical mechanisms to describe features of the direction-selective computation and identify where our understanding must be improved.

Human Action Classification Using the Motion Detection Model of the Insect Visual System and the Echo State Network

Human Action Classification Using the Motion Detection Model of the Insect Visual System and the Echo State Network