Objects In Image Sequences Research Articles

Three dimensional tracking of rigid objects in motion using 2D optical flows

Detecting and tracking objects in image sequences is paramount for any video analytic. While object detectors have become increasingly robust, motion estimates based on the straightforward approach of running a detector and linking detections are prone to bounding box noise. An algorithm for the monocular estimation of the 3D motion of rigid objects is presented, combining an object detector, minimal camera calibration, and 2D optical flows observed on the image plane. The algorithm utilizes the 2D Delaunay triangulation over geometrically consistent optical flow tracks to identify regions common to the same object. The algorithm is evaluated on the special case of image sequences of vehicles on roads. Experiments on BrnoCompSpeed dataset show that speed estimates from both detector-based tracking (mean error of 1.62 km/h) and optical flow based tracking (mean error of 2.19 km/h) perform competitively on straight roads. An extensive empirical evaluation is also conducted on a new dataset containing synthetic and real world scenes that also include vehicle trajectories involving rotation. A naive baseline bounding box track based method obtained a mean error of 5.29 km/h for vehicle speed, but optical flow tracks performed significantly better with mean error of 1.65 km/h on the new dataset including rotation. With the new method, video analytics such as vehicle speed estimation and lane change detection can obtain precise information about the 3D trajectory of rigid objects in motion.

Effectiveness Of Moving Objects Detecting And Tracking In Airspace By Images In NearInfrared

Objects in nearinfrared (NIR) images can often have different linear scales and shapes than the same objects in optical images for visible spectrum (RedGreenBlue, RGB). Therefore they can require different computer vision methods for detection, tracking, and classification. This paper devoted to the methods by which the problems of moving objects detecting and tracking in NIR images are solved. The main characteristics of moving objects in image sequences are highlighted. Advantages and disadvantages of different methods for detecting and tracking objects in NIR images of airspace are considered and two of the most promising methods classes are selected. Studies have been carried out on the effectiveness of LucasKanade method, which is one of the methods of local optical flow, and the ORB method of scaleinvariant transformation of features when detecting and tracking moving objects in NIR images. In numerical experiments, more than 5000 NIR images containing moving objects of three types were used as well as three combinations of considered methods. It is shown which combination is the most accurate in the tasks of moving objects detection and tracking and can be used for airspace automatic operational control and management based on computer vision systems. Probably, other combinations of methods from the two considered classes also can help to increase accuracy of moving objects detection and tracking in airspace by NIR images.

Dynamic mode decomposition via convolutional autoencoders for dynamics modeling in videos

Extracting the underlying dynamics of objects in image sequences is one of the challenging problems in computer vision. Besides, dynamic mode decomposition (DMD) has recently attracted attention as a method for obtaining modal representations of nonlinear dynamics from general multivariate time-series data without explicit prior information about the dynamics. In this paper, we propose a convolutional autoencoder (CAE)-based DMD (CAE-DMD) to perform accurate modeling of underlying dynamics in videos. We develop a modified CAE model that encodes images to latent vectors and incorporated DMD on the latent vectors to extract DMD modes. These modes are split into background and foreground modes for foreground modeling in videos, or used for video classification tasks. And the latent vectors are mapped so as to recover the input image sequences through a decoder. We perform the network training in an end-to-end manner, i.e., by minimizing the mean square error between the original and reconstructed images. As a result, we obtain accurate extraction of underlying dynamic information in the videos. We empirically investigate the performance of CAE-DMD in two applications background foreground extraction and video classification on synthetic and publicly available datasets.

Graph relation network for person counting in construction site using UAV

Due to its portability and maneuverability, Unmanned Aerial Vehicles (UAVs) are increasingly used in industrial fields. Our work aims to develop a framework of person counting executed by an UAV, which can count the total number of persons accurately. To solve the problem of multiple counts for the same person, this work proposes a novel Graph Similarity-based Person Counting Network (GSPCN) which consists of three modules: person detection network, person re-identification module, and person counting module. To begin with, we detect the bounding boxes and the corresponding images for each object in image sequence. Secondly, we calculate the visual similarity between persons. And then, each person is taken as the root node to construct a graph, then we calculate the similarity between different graphs as the graph similarity. After that, we comprehensively consider the visual similarity and graph similarity to determine whether the person is repeated. Finally, by removing all duplicate persons, we can get the total number of persons. The proposed framework is tested in a real-world scenario and it empirically outperforms the existing state-of-the-art methods.

Real-Time Adaptive Object Detection and Tracking for Autonomous Vehicles

The recent development of deep neural networks attempts to go deeper through the layered architecture to solve complex problems. As expected, the deepening impacts the processing times for inferences in tasks like object classification and localization. This demands solutions for balancing computational resource requirements and extend the versatility of a given model to suit a variety of applications. Often the solutions rely on architectural modifications to ease training times while achieving more accurate results. In this work, we propose a multistage algorithm with an initial stage adopting the YOLOv3 network for object detection and HART approach for object tracking. This is followed by an adaptive post-localization stage shift system taking into consideration the processing times of stage inferences, which are the number of located objects in image sequences. The goal is to change the localization strategy to achieve optimal processing time performance. For evaluation, we present results in terms of precision and processing times in varying traffic conditions. The results demonstrate the effectiveness of the proposed adaptive multistage model in comparison to other real-time state-of-the-art detection strategies, as it achieves a frame rate gain of 49% over YOLOv3 while maintaining competitive task accuracy.

One Shot Segmentation: Unifying Rigid Detection and Non-Rigid Segmentation Using Elastic Regularization.

This paper proposes a novel approach for the non-rigid segmentation of deformable objects in image sequences, which is based on one-shot segmentation that unifies rigid detection and non-rigid segmentation using elastic regularization. The domain of application is the segmentation of a visual object that temporally undergoes a rigid transformation (e.g., affine transformation) and a non-rigid transformation (i.e., contour deformation). The majority of segmentation approaches to solve this problem are generally based on two steps that run in sequence: a rigid detection, followed by a non-rigid segmentation. In this paper, we propose a new approach, where both the rigid and non-rigid segmentation are performed in a single shot using a sparse low-dimensional manifold that represents the visual object deformations. Given the multi-modality of these deformations, the manifold partitions the training data into several patches, where each patch provides a segmentation proposal during the inference process. These multiple segmentation proposals are merged using the classification results produced by deep belief networks (DBN) that compute the confidence on each segmentation proposal. Thus, an ensemble of DBN classifiers is used for estimating the final segmentation. Compared to current methods proposed in the field, our proposed approach is advantageous in four aspects: (i) it is a unified framework to produce rigid and non-rigid segmentations; (ii) it uses an ensemble classification process, which can help the segmentation robustness; (iii) it provides a significant reduction in terms of the number of dimensions of the rigid and non-rigid segmentations search spaces, compared to current approaches that divide these two problems; and (iv) this lower dimensionality of the search space can also reduce the need for large annotated training sets to be used for estimating the DBN models. Experiments on the problem of left ventricle endocardial segmentation from ultrasound images, and lip segmentation from frontal facial images using the extended Cohn-Kanade (CK+) database, demonstrate the potential of the methodology through qualitative and quantitative evaluations, and the ability to reduce the search and training complexities without a significant impact on the segmentation accuracy.

A novel deformable body partition model for MMW suspicious object detection and dynamic tracking

In recent years, millimeter wave (MMW) imaging techniques have developed rapidly and been widely used in public security field. Traditional security techniques on MMW images mostly based on manual analysis and simple image processing, which cannot achieve real-time and accurate performance. This paper proposes a deep learning-based deformable body partition (DBP) model for detecting suspicious objects hidden in human body on MMW image dataset. Considering the characteristics of MMW security images, we combine the location information of concealed object with the corresponding human body part for object detection. According to the human body structure, 17 deformable regions are partitioned with 12 body keypoints to address body region misalignment problems of different people. By training convolutional neural network (CNN) with different body regions, the suspicious objects can be detected, together with their location information. To make full use of spatio-temporal context information of MMW image sequences, we design an object dynamic tracking method to lock the concealed targets moving from different angles’ MMW images of the same still body. Compared with conventional global-based object detection methods, the proposed method can not only accurately detect suspicious objects but also output the location information of them. Importantly, DBP model is more adaptive than the fixed coordinate-based body partition method and can automatically change the size of each body region according to each person’s body shape. Moreover, our object dynamic tracking method can utilize the positional relationship of suspicious objects in image sequences to reduce the object searching area of each image. Experimental results prove the effectiveness of the proposed method. The detection accuracy and speed on MMW images are favorable in practical application.

Deep learning cardiac motion analysis for human survival prediction.

Motion analysis is used in computer vision to understand the behaviour of moving objects in sequences of images. Optimising the interpretation of dynamic biological systems requires accurate and precise motion tracking as well as efficient representations of high-dimensional motion trajectories so that these can be used for prediction tasks. Here we use image sequences of the heart, acquired using cardiac magnetic resonance imaging, to create time-resolved three-dimensional segmentations using a fully convolutional network trained on anatomical shape priors. This dense motion model formed the input to a supervised denoising autoencoder (4Dsurvival), which is a hybrid network consisting of an autoencoder that learns a task-specific latent code representation trained on observed outcome data, yielding a latent representation optimised for survival prediction. To handle right-censored survival outcomes, our network used a Cox partial likelihood loss function. In a study of 302 patients the predictive accuracy (quantified by Harrell’s C-index) was significantly higher (p = .0012) for our model C=0.75 (95% CI: 0.70 - 0.79) than the human benchmark of C=0.59 (95% CI: 0.53 - 0.65). This work demonstrates how a complex computer vision task using high-dimensional medical image data can efficiently predict human survival.

Segmentation of moving objects in image sequence based on perceptual similarity of local texture and photometric features

The segmentation of moving objects in image sequence can be formulated as a background subtraction problem—the separation of objects from the background in each image frame. The background scene is learned and modeled. A pixelwise process is employed to classify each pixel as an object or background based on its similarity with the background model. The segmentation is challenging due to the occurrence of dynamic elements such as illumination change and background motions. We propose a framework for object segmentation with a novel feature for background representation and new mechanisms for updating the background model. A ternary pattern is employed to characterize the local texture. The pattern and photometric features are used for background modeling. The classification of pixel is performed based on the perceptual similarity between the current pixel and the background model. The segmented object is refined by taking into account the spatial consistency of the image feature. For the background model update, we propose two mechanisms that are able to adapt to abrupt background change and also merge new background elements into the model. We compare our framework with various background subtraction algorithms on video datasets.

Improved background subtraction method for detecting moving objects based on GMM

The background subtraction (BS) method is one of the most commonly used components in the detection of moving objects in computer vision systems, in which moving objects in image sequences are detected by comparing the background model with the current frame. Traditional moving object detection algorithms have some problems, such as changes in dynamic background, background variation due to changes of illumination, and being more sensitive to noise and shadows. In this study, an improved BS method based on the Gaussian mixture model (GMM) is proposed for the challenge in object detection. Not only is the influence of changes of illumination, and noise and shadows reduced, but the dynamic changes of natural scenes can also be tackled by using this method. The contribution lies in the following aspects: (i) a Gaussian background modeling method with less running time is proposed in the background modeling stage. The background is reconstructed based on GMM of the mean images of image blocks, aiming to simplify the calculations so as to improve the speed of the corresponding operations. (ii) In the foreground detection stage, a wavelet‐based, denoising method with the semisoft threshold function is applied to denoise the object images of the foreground. (iii) In the background maintenance stage, an adaptive background maintenance algorithm is proposed to dynamically update the background. Experimental results show that the computational complexity is reduced, while the adaptability and performance are improved by using this method. It outperforms the traditional methods in terms of both efficiency and robustness. © 2018 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.

Infrared moving object detection based on local saliency and sparse representation

The key issue of infrared object detection is to locate moving object in image sequence. In order to improve detection precision, an infrared object detection method based on local saliency and sparse representation is proposed in this paper. Motion information, such as velocity, acceleration components are added into the eigenvectors to build local saliency model. And the approximate position of the infrared target is located based on the local saliency. To accurately extract the infrared object, sparse representation is used to capture complete edge of the object. Experiments show that the proposed method can accurately detect infrared moving objects, and has good robustness to external disturbances and dynamic background.

A Cluster Tracking Method Based on Cam-shift

The classic Mean-Shift algorithm lacks the necessary template update, because window size remains the same in tracking process, tracking will fail when the template scale change, track will be ineffective when the template is faster, the feature of histogram seems simple in the object color characteristic described aspects and lacks space information. This paper presents a Cam-Shift clustering algorithm, regarding the centroid position of the moving object which is detected as the first iteration of the input frame, narrowing the visual search range, shortening the matching time. Through the template update of color model and cluster, the algorithm resists the interference of light, strain, shelter and achieving more precise moving object motion parameter estimation and tracking results than the existing algorithms to a certain extent. Experiment is used to verify each algorithm in this article. Selecting aerial image sequence multiple moving vehicle object to study, results are capable of getting sustained and effective track to multiple moving objects in image sequences.

Parallel multi-modal background modeling

Background subtraction is a widely used technique for detecting moving objects in image sequences. Very often background subtraction approaches assume the availability of one or more clear (i.e., without foreground objects) frames at the beginning of the sequence in input. However, this assumption is not always true, especially when dealing with dynamic background or crowded scenes. In this paper, we present the results of a multi-modal background modeling method that is able to generate a reliable initial background model even if no clear frames are available. The proposed algorithm runs in real-time on HD images. Quantitative experiments have been conducted taking into account six different quality metrics on a set of 14 publicly available image sequences. The obtained results demonstrate a high-accuracy in generating the background model in comparison with several other methods.

Real-time object motion estimation based on frequency domain matching from a single camera

In this paper, we propose a new method for estimating motions of a moving object in image sequences from a single camera. Log-polar transform and phase correlation matching method is applied to estimate rotation and scale change between the object in the template image and the object in the next frames. Kalman filtering is utilized to smooth high frequency variation of the estimated rotation and scale motions. Translation of the object is estimated through template matching by dynamically updating template for each frame. Based on the proposed method, we implemented experiments to check accuracy of results from phase correlation matching and demonstrated an application for tracking object motion in real-time image sequences.

Motion Driven Tonal Stabilization.

This paper addresses the problem of tonal fluctuation in videos. Due to the automatic settings of consumer cameras, the colors of objects in image sequences might change over time. We propose here a fast and computationally light method to stabilize this tonal appearance, while remaining robust to motion and occlusions. To do so, a minimally viable color correction model is used, in conjunction with an effective estimation of dominant motion. The final solution is a temporally weighted correction, explicitly driven by the motion magnitude, both visually efficient and very fast, with potential to real time processing. Experimental results obtained on a variety of sequences outperform the current state of the art in terms of tonal stability, at a much reduced computational complexity.

Survey on Algorithms for Object Tracking in Video

Object tracking is a very essential task in many applications of computer vision such as surveillance, vehicle navigation, autonomous robot navigation, etc. It contains detection of interesting moving objects and tracking of such objects from frame to frame. Its main task is to find and follow a moving object or multiple objects in image sequences. Normally there are three stages of video analysis: object detection, object tracking and object reorganization. This paper presents a brief survey of various video object tracking techniques like point tracking, kernel tracking and Silhouette tracking algorithms. General Terms Human-computer Interactions, 2D image, noise image, Proximity.

A Comparative Study of Object TrackingTechniques

Object tracking is very essential task in many application of computer vision such as surveillance, vehicle navigation, autonomous robot navigation, etc. It contains detection of interesting moving objects and tracking of such objects from frame to frame. Its main task is to find and follow a moving object or multiple objects in image sequences. Normally there are three stages of video analysis; object detection, object tracking, and object reorganization. This paper present a brief survey of various video object tracking techniques like point tracking, kernel tracking and Silhouette tracking algorithms. Also it presents Comparative study of all the techniques.

An Implementation of Combined Local-Global Optical Flow

Optical Flow (OF) approaches for motion estimation calculate vector fields for the apparent velocities of objects in image sequences. In 1981 Horn and Schunck (HS) introduced two basic assumptions: 'brightness value constancy' and 'smooth variation' to estimate a smooth OF field over the entire image -global approach-. In parallel, Lucas and Kanade (LK) assumed constant motion patterns for image patches, estimating piecewise-homogeneous OF fields -local approach-. Several variations of these approaches exist today. Here we present the combined local-global (CLG) approach by Bruhn et al. which encompasses properties of HS-OF and LK-OF, aiming to improve the OF accuracy for small-scale variations, while delivering the HS-OF dense and smooth fields. A multiscale implementation is provided for 2D images, together with two numerical solvers: Successive Over-Relaxation and the faster Pointwise-Coupled Gauss-Seidel by Bruhn et al.. The algorithm works on gray-scale (single channel) images, with color images being converted prior to the OF computation.

Visual Object Tracking in Scale-space by Artificial Neural Network with Reference Frames

A novel approach of tracking objects in image sequences is proposed, which can both adapt to variation of objects’ features, and jointly determine objects’ position and scale provided that their typical appearances are available prior to tracking. Most classical algorithms fail in the presence of an object’s significant simultaneous variation of appearance, scale and surrounding illumination, because the features they employ are obtained from object’s region in the frame where the tracking is initiated and ignore the deviation that happens during tracking. In this paper, we propose to extract object’s features from its typical appearances in reference frames and represent them as the coefficients of an BP neural network. Due to the BP network’s strong ability of nonlinear classification, this new approach can discriminate the object from background despite its transitions between appearances during tracking. We also propose to determine object’s scale by performing scale space blob detection in the particle filtering framework, which overcomes the drawbacks of falling into local minima and enhances the tracker’s robustness. Experimental results validate the effectiveness of the proposed approach, and demonstrate its improved performance in tracking precision and robustness.

Moving Cast Shadow Detection

Motion perception is an amazing innate ability of the creatures on the planet. This adroitness entails a functional advantage that enables species to compete better in the wild. The motion perception ability is usually employed at different levels, allowing from the simplest interaction with the ’physis’ up to the most transcendental survival tasks. Among the five classical perception system , vision is the most widely used in the motion perception field. Millions years of evolution have led to a highly specialized visual system in humans, which is characterized by a tremendous accuracy as well as an extraordinary robustness. Although humans and an immense diversity of species can distinguish moving object with a seeming simplicity, it has proven to be a difficult and non trivial problem from a computational perspective.In the field of Computer Vision, the detection of moving objects is a challenging and fundamental research area. This can be referred to as the ’origin’ of vast and numerous vision-based research sub-areas. Nevertheless, from the bottom to the top of this hierarchical analysis, the foundations still relies on when and where motion has occurred in an image.Pixels corresponding to moving objects in image sequences can be identified by measuring changes in their values. However, a pixel’s value (representing a combination of color and brightness) could also vary due to other factors such as: variation in scene illumination, camera noise and nonlinear sensor responses among others. The challenge lies in detecting if the changes in pixels’ value are caused by a genuine object movement or not. An additional challenging aspect in motion detection is represented by moving cast shadows. The paradox arises because a moving object and its cast shadow share similar motion patterns. However, a moving cast shadow is not a moving object. In fact, a shadow represents a photometric illumination effect caused by the relative position of the object with respect to the light sources.Shadow detection methods are mainly divided in two domains depending on the application field. One normally consists of static images where shadows are casted by static objects, whereas the second one is referred to image sequences where shadows are casted by moving objects. For the first case, shadows can provide additional geometric and semantic cues about shape and position of its casting object as well as the localization of the light source. Although the previous information can be extracted from static images as well as video sequences, the main focus in the second area is usually change detection, scene matching or surveillance. In this context, a shadow can severely affect with the analysis and interpretation of the scene.The work done in the thesis is focused on the second case, thus it addresses the problem of detection and removal of moving cast shadows in video sequences in order to enhance the detection of moving object.