Background Motion Research Articles

Following Forrest Gump: Smooth pursuit related brain activation during free movie viewing

Most fMRI studies investigating smooth pursuit (SP) related brain activity have used simple synthetic stimuli such as a sinusoidally moving dot. However, real-life situations are much more complex and SP does not occur in isolation but within sequences of saccades and fixations. This raises the question whether the same brain networks for SP that have been identified under laboratory conditions are activated when following moving objects in a movie.Here, we used the publicly available studyforrest data set that provides eye movement recordings along with 3 ​T fMRI recordings from 15 subjects while watching the Hollywood movie “Forrest Gump”. All three major eye movement events, namely fixations, saccades, and smooth pursuit, were detected with a state-of-the-art algorithm. In our analysis, smooth pursuit (SP) was the eye movement of interest, while saccades were acting as the steady state of viewing behaviour due to their lower variability. For the fMRI analysis we used an event-related design modelling saccades and SP as regressors initially. Because of the interdependency of SP and content motion, we then added a new low-level content motion regressor to separate brain activations from these two sources.We identified higher BOLD-responses during SP than saccades bilaterally in MT+/V5, in middle cingulate extending to precuneus, and in the right temporoparietal junction. When the motion regressor was added, SP showed higher BOLD-response relative to saccades bilaterally in the cortex lining the superior temporal sulcus, precuneus, and supplementary eye field, presumably due to a confounding effect of background motion. Only parts of V2 showed higher activation during saccades in comparison to SP.Taken together, our approach should be regarded as proof of principle for deciphering brain activity related to SP, which is one of the most prominent eye movements besides saccades, in complex dynamic naturalistic situations.

Fly eyes are not still: a motion illusion in Drosophila flight supports parallel visual processing.

Most animals shift gaze by a 'fixate and saccade' strategy, where the fixation phase stabilizes background motion. A logical prerequisite for robust detection and tracking of moving foreground objects, therefore, is to suppress the perception of background motion. In a virtual reality magnetic tether system enabling free yaw movement, Drosophila implemented a fixate and saccade strategy in the presence of a static panorama. When the spatial wavelength of a vertical grating was below the Nyquist wavelength of the compound eyes, flies drifted continuously and gaze could not be maintained at a single location. Because the drift occurs from a motionless stimulus - thus any perceived motion stimuli are generated by the fly itself - it is illusory, driven by perceptual aliasing. Notably, the drift speed was significantly faster than under a uniform panorama, suggesting perceptual enhancement as a result of aliasing. Under the same visual conditions in a rigid-tether paradigm, wing steering responses to the unresolvable static panorama were not distinguishable from those to a resolvable static pattern, suggesting visual aliasing is induced by ego motion. We hypothesized that obstructing the control of gaze fixation also disrupts detection and tracking of objects. Using the illusory motion stimulus, we show that magnetically tethered Drosophila track objects robustly in flight even when gaze is not fixated as flies continuously drift. Taken together, our study provides further support for parallel visual motion processing and reveals the critical influence of body motion on visuomotor processing. Motion illusions can reveal important shared principles of information processing across taxa.

Moving Target Detection Based on an Adaptive Low-Rank Sparse Decomposition

For the exact detection of moving targets in video processing, an adaptive low-rank sparse decomposition algorithm is proposed in this paper. In the paper's algorithm, the background model and the solved frame vector are first used to construct an augmented matrix, then robust principal component analysis (RPCA) is used to perform a low-rank sparse decomposition on the enhanced augmented matrix. The separated low-rank part and sparse noise correspond to the background and motion foreground of the video frame, respectively, the incremental singular value decomposition method and the current background vector are used to update the background model. The experimental results show that the algorithm can deal with complex scenes such as light changes and background motion better, and the algorithm's delay and memory consumption can be reduced effectively.

A method of security management of early warning based on mean shift algorithm and data mining

Facing COVID-19 epidemic, many countries have recently strengthened epidemic prevention and control measures. The reliability of safety management is of great significance to personnel management and control during the COVID-19 epidemic period. The focus of security management of early warning is to monitor and identify the moving target. The current optical flow method is vulnerable to the influence of light changes and background movement, and it is not very accurate for moving target detection in dynamic complex background. In this paper, aiming at the traditional Lucas Kanade optical flow method, the inter frame difference method, mean shift clustering algorithm and morphological processing are combined to optimize and improve on the original basis, so that the moving target detection effect in both simple and complex environments is significantly improved. At the same time, the improved algorithm also reduces the execution time to a certain extent, and has a certain resistance to noise interference such as light changes. This has a certain ability test value for personnel control during the epidemic.

Vision Tracking: A Survey of the State-of-the-Art

Vision tracking is a well-studied framework in vision computing. Developing a robust visual tracking system is challenging because of the sudden change in object motion, cluttered background, partial occlusion and camera motion. In this study, the state-of-the art visual tracking methods are reviewed and different categories are discussed. The overall visual tracking process is divided into four stages—object initialization, appearance modeling, motion estimation, and object localization. Each of these stages is briefly elaborated and related researches are discussed. A rapid growth of visual tracking algorithms is observed in last few decades. A comprehensive review is reported on different performance metrics to evaluate the efficiency of visual tracking algorithms which might help researchers to identify new avenues in this area. Various application areas of the visual tracking are also discussed at the end of the study.

Exposure-Structure Blending Network for High Dynamic Range Imaging of Dynamic Scenes

This paper presents a deep end-to-end network for high dynamic range (HDR) imaging of dynamic scenes with background and foreground motions. Generating an HDR image from a sequence of multi-exposure images is a challenging process when the images have misalignments by being taken in a dynamic situation. Hence, recent methods first align the multi-exposure images to the reference by using patch matching, optical flow, homography transformation, or attention module before the merging. In this paper, we propose a deep network that synthesizes the aligned images as a result of blending the information from multi-exposure images, because explicitly aligning photos with different exposures is inherently a difficult problem. Specifically, the proposed network generates under/over-exposure images that are structurally aligned to the reference, by blending all the information from the dynamic multi-exposure images. Our primary idea is that blending two images in the deep-feature-domain is effective for synthesizing multi-exposure images that are structurally aligned to the reference, resulting in better-aligned images than the pixel-domain blending or geometric transformation methods. Specifically, our alignment network consists of a two-way encoder for extracting features from two images separately, several convolution layers for blending deep features, and a decoder for constructing the aligned images. The proposed network is shown to generate the aligned images with a wide range of exposure differences very well and thus can be effectively used for the HDR imaging of dynamic scenes. Moreover, by adding a simple merging network after the alignment network and training the overall system end-to-end, we obtain a performance gain compared to the recent state-of-the-art methods.

Pedestrian detection using a moving camera: A novel framework for foreground detection

While background subtraction techniques have been widely applied to detect moving objects in a video stream captured by a static camera, detecting moving objects using a moving camera still represents a challenging task. In this context, pedestrian detection using a camera placed on the top of a vehicle’s windshield has been rarely investigated. This is mainly due to the background ego-motion. Since the scene captured by the camera seems in motion, it is very difficult to distinguish the moving pedestrians from the others that belong to the static part of the scene. For this reason, a compensation step is needed to suppress the ego-motion. This paper presents a study on the main challenges facing pedestrian detection systems as well as methods proposed to handle these challenges. A novel trajectory classification framework for detecting pedestrians even in challenging real-world environments is proposed. The proposed method models the background motion between two consecutive frames in order to compensate the camera motion. Then, it defines a classification process that differentiates between the background and the foreground in the frame. Using the defined foreground, we consequently identify the presence of pedestrians in the scene. The proposed method was validated on a public benchmark dataset: CVC-14 containing both visible and far infrared video sequences in day and night times. Experimental results confirm the effectiveness of the proposed approach in capturing the dynamic aspect between frames and therefore detecting the presence of pedestrians in the scene.

VR motion sickness recognition by using EEG rhythm energy ratio based on wavelet packet transform

Background and objectivesVirtual reality motion sickness (VRMS) is one of the main factors hindering the development of VR technology. At present, the VRMS recognition methods using electroencephalogram (EEG) signals have poor applicability to multiple subjects. MethodsAiming at this dilemma, the wavelet packet transform (WPT), was used to propose a feature extraction method for EEG rhythm energy ratios of delta (δ), theta (θ), alpha (α), and beta (β) in this research. Moreover, VRMS was recognized by combining k-Nearest Neighbor classifier (k-NN), support vector machine (SVM) with polynomial kernel (polynomial-SVM) and radial basis function kernel (RBF-SVM), respectively. The method is that the raw EEG signals were de-noised by an elliptical band-pass filter and segmented by a fixed window, 7-level db4 WPT was performed on each EEG segment, and the wavelet packet energy ratios of delta, theta, alpha and beta rhythms from FP1, FP2, C3, C4, P3, P4, O1 and O2 channels were calculated and combined to form feature vectors for recognizing VRMS. ResultsUnder the condition of 4-s window size, the average VRMS recognition accuracy of polynomial-SVM for the single subject was 92.85%, and the VRMS recognition accuracy of 18 subjects was about 79.25%. ConclusionsCompared with other VRMS recognition methods, this method does not only have a higher recognition accuracy to a single subject, but also have better applicability to multiple subjects. Meanwhile, when using the EEG four rhythm energy ratios of FP1, FP2, C3, C4, P3, P4, O1 and O2 channels as feature vectors, the polynomial-SVM achieved better VRMS recognition performance than the k-NN and RBF-SVM.

Mechanically-assisted and non-invasive ventilation for radiation therapy: A safe technique to regularize and modulate internal tumour motion

Background and purposeCurrent motion mitigation strategies, like margins, gating, and tracking, deal with geometrical uncertainties in the tumour position, induced by breathing during radiotherapy (RT). However, they often overlook motion variability in amplitude, respiratory rate, or baseline position, when breathing spontaneously. Consequently, this may negatively affect the delivered dose conformality in comparison to the plan. We previously demonstrated on volunteers that 3 different modes of mechanically-assisted and non-invasive ventilation (MANIV) may reduce variability in breathing motion. The volume-controlled mode (VC) constraints the amplitude and respiratory rate (RR) in physiologic condition. The shallow-controlled mode (SH), derived from VC, increases the RR and decreases amplitude. The slow-controlled mode (SL) induces repeated breath holds with constrained ventilation pressure. In this study, we compared these mechanical ventilation modes to spontaneous breathing or breath hold and assessed their tolerance and effects on internal tumour motion in patients receiving RT. Material and methodsThe VC and SH modes were evaluated in ten patients with lung or liver cancers (cohort A). The SL mode was evaluated in 12 left breast cancer patients (cohort B). After a training and simulation session, the patients underwent 2 MRI sessions to analyze the internal motion of breast and tumour. ResultsMANIV was well tolerated, without any adverse events or oxymetric changes, even in patients with respiratory comorbidities. In cohort A, when compared to spontaneous breathing (SP), VC reduced significantly inter-session variations of the tumour motion amplitude (p = 0.01), as well as intra- and inter-session variations of the RR (p < 0.05). As to SH, the RR increased, while its variations within and across sessions decreased when compared to SP (p < 0.001). SH reduced the median amplitude of the tumour motion by 6.1 mm or 38.2% (p ≤ 0.01) compared to VC. In cohort B, breast position stability over the end-inspiratory plateaus obtained spontaneously or with SL remained similar. Median duration of the plateaus in SL was 16.6 s. ConclusionMANIV is a safe and well tolerated ventilation technique for patients receiving radiotherapy. MANIV could thus make current motion mitigation strategies less critical and more robust. Clinical implementation might be considered, provided the ventilation mode is carefully selected with respect to the treatment indication and patient individualities.

Unsupervised Moving Object Segmentation from Stationary or Moving Camera based on Multi-frame Homography Constraints.

Moving object segmentation is the most fundamental task for many vision-based applications. In the past decade, it has been performed on the stationary camera, or moving camera, respectively. In this paper, we show that the moving object segmentation can be addressed in a unified framework for both type of cameras. The proposed method consists of two stages: (1) In the first stage, a novel multi-frame homography model is generated to describe the background motion. Then, the inliers and outliers of that model are classified as background trajectories and moving object trajectories by the designed cumulative acknowledgment strategy. (2) In the second stage, a super-pixel-based Markov Random Fields model is used to refine the spatial accuracy of initial segmentation and obtain final pixel level labeling, which has integrated trajectory classification information, a dynamic appearance model, and spatial temporal cues. The proposed method overcomes the limitations of existing object segmentation algorithms and resolves the difference between stationary and moving cameras. The algorithm is tested on several challenging open datasets. Experiments show that the proposed method presents significant performance improvement over state-of-the-art techniques quantitatively and qualitatively.

Residual attention unit for action recognition

3D CNNs are powerful tools for action recognition that can intuitively extract spatio-temporal features from raw videos. However, most of the existing 3D CNNs have not fully considered the disadvantageous effects of the background motion that frequently appears in videos. The background motion is usually misclassified as a part of human action, which may undermine modeling the dynamic pattern of the action. In this paper, we propose the residual attention unit (RAU) to address this problem. RAU aims to suppress the background motion by upweighting the values associated with the foreground region in the feature maps. Specifically, RAU contains two separate submodules in parallel, i.e., spatial attention as well as channel-wise attention. Given an intermediate feature map, the spatial attention works in a bottom-up top-down manner to generate the attention mask, while the channel-wise attention recalibrates the feature responses of all channels automatically. As applying the attention mechanism directly to the input features may lead to the loss of discriminative information, we design a bypass to preserve the integrity of the original features by a shortcut connection between the input and output of the attention module. Notably, our RAU can be embedded into 3D CNNs easily and enables end-to-end training along with the networks. The experimental results on UCF101 and HMDB51 demonstrate the validity of our RAU.

Accurate Tracking of Manoeuvring Target using Scale Estimation and Detection

Camera zoom operation and fast approaching/receding target causes scaling of acquired target in video frames. Fast moving target manifests in large inter-frame motion. In general, non-uniform background degrades performance of tracking algorithms. Fast Fourier transform (FFT)-based Correlation algorithms improve tracking in this scenario, but their applications is limited to small inter-frame motion. Increasing search region has implication on execution speed of the algorithms. Rapid target scaling, non-uniform background and large inter-frame motion of target hinder accurate and long term visual tracking. These challenges have been addressed for extended target tracking by augmenting fast discriminative scale space tracking (fDSST) algorithm with probable target location prediction and target detection. Localisation of fast motion has been achieved by applying fused outputs of Kalman filter and quadratic regression based prediction before applying fDSST. It has helped in accurate localisation of fast motion without increasing search region. In each frame, target location and size have been estimated using fDSST and further refined by target detection near this location. Smoothing and limiting of trajectory and size of detected target has enhanced tracking performance. Experimental results show considerable improvement of precision, success rate and centre location error tracking performance against state-of-the-art trackers in stringent conditions.

Global Jitter Motion of the Retinal Image Dynamically Alters the Receptive Field Properties of Retinal Ganglion Cells.

Fixational eye movements induce aperiodic motion of the retinal image. However, it is not yet fully understood how fixational eye movements affect retinal information processing. Here we show that global jitter motion, simulating the image motion during fixation, alters the spatiotemporal receptive field properties of retinal ganglion cells. Using multi-electrode and whole-cell recording techniques, we investigated light-evoked responses from ganglion cells in the isolated goldfish retina. Ganglion cells were classified into six groups based on the filtering property of light stimulus, the membrane properties, and the cell morphology. The spatiotemporal receptive field profiles of retinal ganglion cells were estimated by the reverse correlation method, where the dense noise stimulus was applied on the dark or random-dot background. We found that the jitter motion of the random-dot background elongated the receptive filed along the rostral-caudal axis and temporally sensitized in a specific group of ganglion cells: Fast-transient ganglion cells. At the newly emerged regions of the receptive field local light stimulation evoked excitatory postsynaptic currents with large amplitude and fast kinetics without changing the properties of inhibitory postsynaptic currents. Pharmacological experiments suggested two presynaptic mechanisms underlying the receptive field alteration: (i) electrical coupling between bipolar cells, which expands the receptive field in all directions; (ii) GABAergic presynaptic inhibition from amacrine cells, which reduces the dorsal and ventral regions of the expanded receptive field, resulting in elongation along the rostral-caudal axis. Our study demonstrates that the receptive field of Fast-transient ganglion cells is not static but dynamically altered depending on the visual inputs. The receptive field elongation during fixational eye movements may contribute to prompt firing to a target in the succeeding saccade.

Localized Trajectories for 2D and 3D Action Recognition

The Dense Trajectories concept is one of the most successful approaches in action recognition, suitable for scenarios involving a significant amount of motion. However, due to noise and background motion, many generated trajectories are irrelevant to the actual human activity and can potentially lead to performance degradation. In this paper, we propose Localized Trajectories as an improved version of Dense Trajectories where motion trajectories are clustered around human body joints provided by RGB-D cameras and then encoded by local Bag-of-Words. As a result, the Localized Trajectories concept provides an advanced discriminative representation of actions. Moreover, we generalize Localized Trajectories to 3D by using the depth modality. One of the main advantages of 3D Localized Trajectories is that they describe radial displacements that are perpendicular to the image plane. Extensive experiments and analysis were carried out on five different datasets.

Traffic extreme situations detection in video sequences based on integral optical flow

Road traffic analysis is an important task in many applications and it can be used in video surveillance systems to prevent many undesirable events. In this paper, we propose a new method based on integral optical flow to analyze cars movement in video and detect flow extreme situations in real-world videos. Firstly, integral optical flow is calculated for video sequences based on optical flow, thus random background motion is eliminated; secondly, pixel-level motion maps which describe cars movement from different perspectives are created based on integral optical flow; thirdly, region-level indicators are defined and calculated; finally, threshold segmentation is used to identify different cars movements. We also define and calculate several parameters of moving car flow including direction, speed, density, and intensity without detecting and counting cars. Experimental results show that our method can identify cars directional movement, cars divergence and cars accumulation effectively.

A New Hybrid Proposed Algorithm for Multiple Vehicle Detection and Tracking in a Day-Time Environment

Multiple Vehicle detection and tracking is one of the hot research topics in the field of intelligent transportation systems, image processing, computer vision, robotics whereas applications are real time traffic monitoring, lane estimation, accident avoidance, alarm signal to indicate road accidents to save the public safety and so on. There exists a numerous higher level applications are motivated by a young researchers and scientists to identify the newly advanced techniques in which to solve the real time traffic problems using machine learning and deep learning methods to track multiple vehicles accurately. To addresses the various existing challenges in machine learning and deep learning based multiple vehicle detection and tracking algorithms namely camera oscillation, shadowing, changing in background motion, cluttering, camouflage etc. for the detection rate decreases dramatically when the distribution of the training samples and the scene target samples do not match. To address this issue, a new hybrid model of two-tier classifier of Haar+HOG, SVM+AdaBoost classifier algorithm based on a feature extraction algorithm is proposed in this paper. Inspired by the Adaptive Discrete Classifiers mechanism multiple relatively independent source samples are first used to build multiple classifiers and then particle grouping is used to generate the target training samples with confident scores. The global manual feature extraction ability of deep convolutional neural network is then used to perform source-target scene feature similarity calculation with a deep auto encoder in order to design a composite deep structure based adaptive discrete classifier and its global training method. The main contributions of this paper are threefold: 1) To improve the overall accuracy rate of multiple vehicle detection and tracking of front-view vehicles alone rather than full-sided vehicles. 2) The novelty of our proposed work is for particle grouping of multi-vehicles such as car, bus and lorry. 3) To propose the tracking of front- view multi- vehicles in linear and non-linear motion using particle and extended kalman filter along with hybrid new multi-vehicle tracking algorithm and attains 93.6% of accuracy is shown in the experimental results. We evaluates our proposed method with standard data sets PETS 2016 and 5 self-data sets iROAD were manually collected on traffic road and compared with the existing state of the art approaches and along with the Experiments on the Kitti dataset and a 3 different self -data set captured by our group demonstrate that the proposed method performs better than the existing machine-learning based vehicle detection methods. In addition, compared with the existing automatic feature extraction and region based object detection methods, our new hybrid method improves the overall detection rate by an average of approximately 5% of existing methods.

Is the manual following response an attempt to compensate for inferred self-motion?

If the surrounding of a visual target unexpectedly starts to move during a fast goal-directed hand movement, the hand reflexively moves along with it. This is known as the ‘manual following response’. One explanation for this response is that it is a compensation for inferred self-motion in space. Previous studies have shown that background motion gives rise to both postural responses and deviations in goal-directed hand movements. To evaluate whether compensation for inferred self-motion is responsible for the manual responses we examined whether galvanic stimulation of the vestibular system would give rise to similar deviations in hand movements. Standing participants tried to quickly tap on targets that were presented on a horizontal screen. Participants could infer self-motion on some of the trials, either from galvanic vestibular stimulation or from background motion. Both perturbations took place during the hand movement. It took both the head and hand about 45 ms longer to respond to background motion than to respond to galvanic stimulation. The head responded in a similar manner to both types of perturbations. The hand responded about as expected to galvanic stimulation, but much more vigorously to background motion. Thus, the manual response to background motion is probably not a direct consequence of trying to compensate for inferred self-motion. Perhaps the manual following response is a consequence of an error in binding motion information to objects.

Joint Dynamic Pose Image and Space Time Reversal for Human Action Recognition from Videos

Human action recognition aims to classify a given video according to which type of action it contains. Disturbance brought by clutter background and unrelated motions makes the task challenging for video frame-based methods. To solve this problem, this paper takes advantage of pose estimation to enhance the performances of video frame features. First, we present a pose feature called dynamic pose image (DPI), which describes human action as the aggregation of a sequence of joint estimation maps. Different from traditional pose features using sole joints, DPI suffers less from disturbance and provides richer information about human body shape and movements. Second, we present attention-based dynamic texture images (att-DTIs) as pose-guided video frame feature. Specifically, a video is treated as a space-time volume, and DTIs are obtained by observing the volume from different views. To alleviate the effect of disturbance on DTIs, we accumulate joint estimation maps as attention map, and extend DTIs to attention-based DTIs (att-DTIs). Finally, we fuse DPI and att-DTIs with multi-stream deep neural networks and late fusion scheme for action recognition. Experiments on NTU RGB+D, UTD-MHAD, and Penn-Action datasets show the effectiveness of DPI and att-DTIs, as well as the complementary property between them.

Dim small moving target detection and tracking method based on spatial-temporal joint processing model

In order to solve the problem of weak target detection in complex moving background, a joint algorithm model of space-time domain is proposed in this paper. The algorithm first selects appropriate length of the time domain window and loop step size. In each time domain window: (1) In time domain processing stage, the algorithm fully considers the factors of background movement, and uses neighborhood similarity measure based on Pearson correlation coefficient to suppress most part of background area. (2) In space domain processing stage, a target detection method based on regional gray level (RGL) is proposed to suppress residual background and re-enhance the target. In each iteration, this joint detection model can obtain part of the trajectory of the target in real time. Therefore, after all iterations are completed (all images in the sequence have been processed), the target can be detected and the target trajectory can be efficiently extracted. The experimental results in this paper show that the proposed algorithm can effectively detect targets with maximum speed of 5 pixel/frame in the image sequence with average SCR ≤ 1 and background moving speed ≤0.8 pixel/frame.

Spatial suppression promotes rapid figure-ground segmentation of moving objects

Segregation of objects from their backgrounds is a fundamental visual function and one that is particularly effective when objects are in motion. Theoretically, suppressive center-surround mechanisms are well suited for accomplishing motion segregation. This longstanding hypothesis, however, has received limited empirical support. We report converging correlational and causal evidence that spatial suppression of background motion signals is critical for rapid segmentation of moving objects. Motion segregation ability is strongly predicted by both individual and stimulus-driven variations in spatial suppression strength. Moreover, aging-related superiority in perceiving background motion is associated with profound impairments in motion segregation. This segregation deficit is alleviated via perceptual learning, but only when motion segregation training also causes decreased sensitivity to background motion. We argue that perceptual insensitivity to large moving stimuli effectively implements background subtraction, which, in turn, enhances the visibility of moving objects and accounts for the observed link between spatial suppression and motion segregation.