Optical Flow Gradient Research Articles

Методи iнтелектуалiзацiї процесiв просторового монiторингу сцени

Introduction. The development of intelligent technologies requires the active use of advanced technologies and innovative approaches for the intellectualization of spatial scene monitoring processes. The relevance of the topic lies in the great need to improve the quality of video content production. In particular, there is interest in the automation and further intellectualization of shooting processes. The use of new methods of intellectualization leads to a reduction of permissible errors when creating a creative video project. Intellectualization of data processing processes from markers, namely the use of artificial intelligence (AI) methods, allows to obtain a controlled level of quality with minimal human involvement. Intellectualization of stage production contributes to the creation of exciting and innovative performances that captivate the audience. It allows creating new ways of interacting with the audience and providing them with unique impressions from cultural events. The purpose of the paper is to study the methods of intellectualization of data processing from markers during the use of automatic video cameras in tasks of observing stage action for video-photography. The results. The issue of the interaction of markers with cameras in three-dimensional space, which is completely identical to the built 3D model, is considered. Conclusions. The information technology of spatial monitoring of the scene can increase the efficiency and simplify the use of automatic video cameras in the tasks of monitoring the stage action for video-photo shooting. There is no one universal "best" method, as each algorithm has its own advantages and disadvantages. However, the optical flow gradient calculation method may be considered more suitable for use in stage production. The introduction of information technology for spatial scene monitoring based on the optical flow gradient calculation method will improve efficiency and simplify the use of automatic video cameras. The use of surveillance information technology will reduce the burden on the personnel who maintain and manage the filming and are involved in the work. Keywords: intellectualization of data processing processes, intelligent monitoring, automatic video camera, animation, optical flow gradient, computer vision

Spatiotemporal Detection and Localization of Object Removal Video Forgery with Multiple Feature Extraction and Optimized Residual Network

Video forgery detection and localization is one of the most important issue due to the advanced editing software that provides strengthen to tools for manipulating the videos. Object based video tampering destroys the originality of the video. The main aim of the video forensic is to eradicate the forgeries from the original video that are useful in various applications. However, the research on detecting and localizing the object based video forgery with advanced techniques still remains the open and challenging issue. Many of the existing techniques have focused only on detecting the forged video under static background that cannot be applicable for detecting the forgery in tampered video. In addition to this, conventional techniques fail to extract the essential features in order to investigate the depth of the video forgery. Hence, this paper brings a novel technique for detecting and localizing the forged video with multiple features. The steps involved in this research are keyframe extraction, pre-processing, feature extraction and finally detection and localization of forged video. Initially, keyframe extraction uses the Gaussian mixture model (GMM) to extract frames from the forged videos. Then, the pre-processing stage is manipulated to convert the RGB frame into a grayscale image. Multi-features need to be extracted from the pre-processed frames to study the nature of the forged videos. In our proposed study, speeded up robust features (SURF), principal compound analysis histogram oriented gradients (PCA-HOG), model based fast digit feature (MBFDF), correlation of adjacent frames (CAF), the prediction residual gradient (PRG) and optical flow gradient (OFG) features are extracted. The dataset used for the proposed approach is collected from REWIND of about 40 forged and 40 authenticated videos. With the help of the DL approach, video forgery can be detected and localized. Thus, this research mainly focuses on detecting and localization of forged video based on the ResNet152V2 model hybrid with the bidirectional gated recurrent unit (Bi-GRU) to attain maximum accuracy and efficiency. The performance of this approach is finally compared with existing approaches in terms of accuracy, precision, F-measure, sensitivity, specificity, false-negative rate (FNR), false discovery rate (FDR), false-positive rate (FPR), Mathew’s correlation coefficient (MCC) and negative predictive value (NPV). The proposed approach assures the performance of 96.17% accuracy, 96% precision, 96.14% F-measure, 96.58% sensitivity, 96.5% specificity, 0.034 FNR, 0.04 FDR, 0.034 FPR, 0.92 MCC and 96% NPV, respectively. Along with is, the mean square error (MSE) and peak-to-signal-noise ratio (PSNR) for the GMM model attained about 104 and 27.95, respectively.

Video object segmentation via random walks on two-frame graphs comprising superpixels

We propose a novel video object segmentation method employing random walkers to travel on graphs constructed on two consecutive frames. First, we estimate the initial foreground and background distributions by minimising an energy function that incorporates the stationary distributions of the random walks. The random walkers frequently travel between similar nodes of the graph constructed on two adjacent frames, which enables the incorporation of the inter-frame information into the energy function effectively and elegantly. Then, we refine the initial results by simulating the movements of multiple random walkers. We process the sequence in a recursive manner, which naturally propagates the previous segmentation labels to the subsequent frames. Additionally, we develop a strategy for adjusting the superpixel number using region similarity and the average Frobenius norm of optical flow gradient. This strategy can improve performance significantly. Furthermore, we discuss the feature selection problem in the method to select a more effective feature representation. Extensive and comparable experiments on Segtrack and Segtrack v2 demonstrate that the proposed algorithm yields higher performance than several recent state-of-the-art approaches.

Inter-frame forgery detection in H.264 videos using motion and brightness gradients

In the midst of low cost and easy-to-use multimedia editing software, which make it exceedingly simple to tamper with digital content, the domain of digital multimedia forensics has attained considerable significance. This research domain deals with production of tools and techniques that enable authentication of digital evidence prior to its use in various critical and consequential matters, such as politics, criminal investigations, defense planning. This paper presents a forensic scheme for detection of frame-based tampering in digital videos, especially those captured by surveillance cameras. Frame-based tampering, which involves insertion, removal or duplication of frames into or from video sequences, is usually very difficult to detect via simple visual inspection. Such forgeries, however, disturb the temporal correlation among successive frames of the tampered video. These disturbances, when analyzed in an appropriate manner, help reveal the evidence of forgery. The forensic technique presented in this paper relies on objective analysis of prediction residual and optical flow gradients for the detection of frame-based tampering in MPEG-2 and H.264 encoded videos. The proposed technique is also capable of determining the exact location of the forgery in the given video sequence. Results of extensive experimentation in diverse and realistic forensic set-ups show that the proposed technique can detect and locate tampering with an average accuracy of 83% and 80% respectively, regardless of the number of frames inserted, removed or duplicated.

Video Inter-frame Forgery Detection Approach for Surveillance and Mobile Recorded Videos

We are living in an age where use of multimedia technologies like digital recorders and mobile phones is increasing rapidly. On the other hand, digital content manipulating softwares are also increasing making it easy for an individual to doctor the recorded content with trivial consumption of time and wealth. Digital multimedia forensics is gaining utmost importance to restrict unethical use of such easily available tampering techniques. These days, it is common for people to record videos using their smart phones. We have also witnessed a sudden growth in the use of surveillance cameras, which we see inhabiting almost every public location. Videos recorded using these devices usually contains crucial evidence of some event occurence and thereby most susceptible to inter-frame forgery which can be easily performed by insertion/removal/replication of frame(s). The proposed forensic technique enabled detection of inter-frame forgery in H.264 and MPEG-2 encoded videos especially mobile recorded and surveillance videos. This novel method introduced objectivity for automatic detection and localization of tampering by utilizing prediction residual gradient and optical flow gradient. Experimental results showed that this technique can detect tampering with 90% true positive rate, regardless of the video codec and recording device utilized and number of frames tampered.

Moving object detection based on optical flow field analysis in dynamic scenes

To overcome the limitation of existing algorithms for detecting moving objects from the dynamic scenes, a foreground detection algorithm based on optical flow field analysis is proposed. Firstly, the object boundary information is determined by detecting the differences in optical flow gradient magnitude and optical flow vector direction between foreground and background. Then, the pixels inside the objects are obtained based on the point-in-polygon problem from computational geometry. Finally, the superpixels per frame are acquired by over-segmenting method. And taking the superpixels as nodes, the Markov Random field model is built, in which the appearance information fitted by Gaussian Mixture Model is combined with spatiotemporal constraints of each superpixel. The final foreground detection result is obtained by finding the minimum value of the energy function. The proposed algorithm does not need any priori assumptions, and can effectively realize the moving object detection in dynamic and stationary background. The experimental results show that the proposed algorithm is superior to the existing state-of-the-art algorithms in the detection accuracy, robustness and time consuming.

Action recognition new framework with robust 3D-TCCHOGAC and 3D-HOOFGAC

Action recognition has very high academic research value, potential commercial value and wide market application prospect in computer vision. In order to improve the action recognition accuracy, two kinds of dynamic descriptors based on dense trajectories are proposed in this paper. Firstly, to capture the local position information that action occurs, dense sampling in motion regions is done by constraining and clustering of optical flow. Secondly, the motion corners of object are selected as feature points which are then tracked to obtain motion trajectories. Finally, the gradient information and optical flow gradient information are extracted respectively in the video cube centered at the trajectories, then the auto-correlation and normalization processing are carried out on the two above information to obtain two dynamic descriptors named 3D histograms of oriented gradients in trajectory centered cube auto-correlation and 3D histograms of oriented optical flow gradients auto-correlation, which can resist a certain degree of interferences caused by camera motion and complex background. However, the diversity of realistic videos makes dynamic or static descriptors alone unable to achieve accurate action classification. A new framework is proposed, which makes the dynamic descriptors and static descriptors fuse and supplement mutually to further improve the action recognition accuracy. This paper adopts the leave-one-out cross validation on datasets of Weizmann and UCF-Sports with action recognition accuracy of 100 % and 96.00 %, and adopts the four-fold cross validation on datasets of KTH and YouTube with action recognition accuracy of 97.17 % and 88.23 %, which has the better performance over the references.

Monocular Concurrent Recovery of Structure and Motion Scene Flow

This paper describes a variational method of joint three-dimensional structure and motion scene flow recovery from a single image sequence. A basic scheme is developed by minimizing a functional with a term of conformity of scene flow and depth to the image sequence spatiotemporal variations, and quadratic smoothness regularization terms. The data term follows by re-writing optical velocity in the optical flow gradient constraint in terms of scene flow and depth. As a result, this problem statement is analogous to the classical Horn and Schunck optical flow formulation except that it involves scene flow and depth rather than image motion. When discretized, the Euler-Lagrange equations give a large scale sparse system of linear equations in the unknowns of the scene flow three coordinates and depth. The equations can be ordered in such a way that its matrix is symmetric positive definite such that they can be solved efficiently by Gauss-Seidel iterations. Experiments are shown to verify the scheme’s validity and efficiency.

Evaluation of advanced Lukas–Kanade optical flow on thoracic 4D-CT

Extensive use of high frequency imaging in medical applications permit the estimation of velocity fields which corresponds to motion of landmarks in the imaging field. The focus of this work is on the development of a robust local optical flow algorithm for velocity field estimation in medical applications. Local polynomial fits to the medical image intensity-maps are used to generate convolution operators to estimate the spatial gradients. A novel polynomial window function with a compact support is used to differentially weight the optical flow gradient constraints in the region of interest. Tikhonov regularization is exploited to synthesize a well posed optimization problem and to penalize large displacements. The proposed algorithm is tested and validated on benchmark datasets for deformable image registration. The ten datasets include large and small deformations, and illustrate that the proposed algorithm outperforms or is competitive with other algorithms tested on this dataset, when using mean and variance of the displacement error as performance metrics.

GPU‐based acceleration of bio‐inspired motion estimation model

SUMMARYIn this paper, we describe the specific and efficient implementation of a gradient‐based optical flow model. This scheme was particularized using a validated neuromorphic motion estimation system for the robust extraction of image velocity. This model contains many characteristics that enhanced the capability when compared with other optical flow gradient family algorithms. Our implementation was performed using specific graphic processing units designed in an ad hoc framework for this model, which could be reused in several low‐level machine‐vision approaches. Observed performance results indicate that these accelerators be highly recommended. Furthermore, the throughput obtained in comparison with a general CPU was analyzed for the accurateness of a system built with regard to other optical flow systems. Additionally, several visual examples, commonly used for testing motion estimation sequences, were shown to reveal implementation behavior features. Copyright © 2012 John Wiley & Sons, Ltd.

Mapping the Spatiotemporal Dynamics of Calcium Signaling in Cellular Neural Networks Using Optical Flow

An optical flow gradient algorithm was applied to spontaneously forming networks of neurons and glia in culture imaged by fluorescence optical microscopy in order to map functional calcium signaling with single pixel resolution. Optical flow estimates the direction and speed of motion of objects in an image between subsequent frames in a recorded digital sequence of images (i.e., a movie). Computed vector field outputs by the algorithm were able to track the spatiotemporal dynamics of calcium signaling patterns. We begin by briefly reviewing the mathematics of the optical flow algorithm, and then describe how to solve for the displacement vectors and how to measure their reliability. We then compare computed flow vectors with manually estimated vectors for the progression of a calcium signal recorded from representative astrocyte cultures. Finally, we applied the algorithm to preparations of primary astrocytes and hippocampal neurons and to the rMC-1 Muller glial cell line in order to illustrate the capability of the algorithm for capturing different types of spatiotemporal calcium activity. We discuss the imaging requirements, parameter selection and threshold selection for reliable measurements, and offer perspectives on uses of the vector data.

3D motion estimation of bubbles of gas in fluid glass, using an optical flow gradient technique extended to a third dimension

To solve the problem of estimating velocities of gas bubbles in melted glass, a method based on optical flow constraint (OFC) has been extended to the 3D case. A single camera, whose distance to the fluid is variable in time, is used to capture a sequence of frames at different depths. Since objects are not static, we cannot obtain two frames of different height values at the same time, and to our knowledge, this prevents the use of common 3D motion estimation techniques. Since the information will be rather sparse, our estimation takes several measures around a given pixel and discards the erroneous ones, using a robust estimator. Along with the exposition of the practical application, the estimation proposed here is first contrasted in the 2D case to common benchmarks and then evaluated for a synthetic problem where velocities are known.