Real Video Images Research Articles

SAR Analysis for Real Video Super- Resolution Improvement

This paper presents a simultaneous autoregressive (SAR) analysis method to describe the unknown signal-to-noise ratio (SNR) and the texture feature of low-quality real video frames when the ground-truth images are not available. The real video images degraded by the factors, such as electronic noise, oversaturated pixels, motion blur, and compression artifacts, often result in poor motion registration estimation, which makes the performance of the existing video super-resolution (VSR) algorithms lower than expected. It is hard to estimate the SNR of the low-quality real frames without any prior knowledge. To solve this problem, we made a connection between SAR hyperparameters and the SNR of real images. The relationship expression of them was given in this paper. Using the proposed method, well-registered low-quality real video frames can be selected to decrease the root mean squared error (RMSE) of motion estimation of video frames for VSR reconstruction improvement. The anomalous low-quality frame images whose SAR hyperparameters values are inconsistent with others will be considered for removal. Synthetic experiments were designed to illustrate how the SAR hyperparameters values vary with the variation of synthetic parameters. In order to better illustrate the effectiveness of the proposed method, real low-quality videos captured under different conditions were tested under VSR reconstruction experiments. The VSR reconstruction results show that the results obtained using SAR prior analysis have sharper edges and fewer ringing artifacts than the original results. It indicates that the proposed method is helpful to obtain better results of motion registration estimation for low-quality real video images.

Human motions and emotions recognition inspired by LMA qualities

The purpose of this paper is to describe human motions and emotions that appear on real video images with compact and informative representations. We aimed to recognize expressive motions and analyze the relationship between human body features and emotions. We propose a new descriptor vector for expressive human motions inspired from the Laban movement analysis method (LMA), a descriptive language with an underlying semantics that allows to qualify human motion in its different aspects. The proposed descriptor is fed into a machine learning framework including, random decision forest, multi-layer perceptron and two multiclass support vector machines methods. We evaluated our descriptor first for motion recognition and second for emotion recognition from the analysis of expressive body movements. Preliminary experiments with three public datasets, MSRC-12, MSR Action 3D and UTkinect, showed that our model performs better than many existing motion recognition methods. We also built a dataset composed of 10 control motions (move, turn left, turn right, stop, sit down, wave, dance, introduce yourself, increase velocity, decrease velocity). We tested our descriptor vector and achieved high recognition performance. In the second experimental part, we evaluated our descriptor with a dataset composed of expressive gestures performed with four basic emotions selected from Russell’s Circumplex model of affect (happy, angry, sad and calm). The same machine learning methods were used for human emotions recognition based on expressive motions. A 3D virtual avatar was introduced to reproduce human body motions, and three aspects were analyzed (1) how expressed emotions are classified by humans, (2) how motion descriptor is evaluated by humans, (3) what is the relationship between human emotions and motion features.

Multibody motion segmentation for an arbitrary number of independent motions

Abstract We propose a new method for segmenting feature point trajectories tracked through a video sequence without assuming a number of independent motions. Our method realizes motion segmentation of feature point trajectories by hierarchically separating the trajectories into two affine spaces in a situation that we do not know the number of independently moving objects. We judge that input trajectories should be separated by comparing the likelihoods computed from those trajectories before/after separation. We also consider integration of the resulting separated trajectories for avoiding too much segmentations. By using real video images, we confirmed the efficiency of our proposed method.

Detection of Barely Discernible Subpixel Objects Using a Modified Orthogonal Subspace Projection Method

This paper deals with the problem of detection of the video/infrared small barely discernible targets in the presence of the heavy background clutter and channel noise. The majority of well-known detection algorithms are based on the assumption that the signal energy reflected from the targets' surface and acquired by the sensor is more or equal to the background clutter energy. This assumption limits the implementation of these algorithms in the case of low signal-to-clutter ratio. The presence of the subpixel target decreases the background power that hence may be different under the null and alternative hypotheses. To detect the barely discernible targets we derive the Generalized Likelihood Ratio Test (GLRT) under hypothesis dependent structured background power and contrast it with well-known Orthogonal Subspace Projection (OSP) algorithm. Experiments on simulated data and real video images demonstrate the ability of this proposed detector to detect the subpixel barely discernible targets in the presence of the heavy background clutter.

TH-AB-204-03: Cherenkov Video for Patient Positioning Validation and Movement Tracking During External Beam Radiation Therapy

Purpose: To investigate the positional accuracy possible with real time Cherenkov video imaging based patient positioning and movement tracking during EBRT. Methods: In a phase 1 clinical trial, including 12 patients undergoing post-lumpectomy whole breast irradiation, Cherenkov emission was imaged with a time-gated ICCD camera synchronized to the LINAC pulse output, during different fractions of the treatment. Patients were positioned with the aid of the AlignRT system. Inter-fraction setup variation was studied by rigid image registrations between images acquired at individual treatments to the average image from all imaged treatment fractions. The amplitude of respiratory motion was calculated from the registration of each frame of Cherenkov images to the reference. A Canny edge detection algorithm was utilized to highlight the beam field edges and biological features provided by major blood vessels apparent in the images. Results: Real-time Cherenkoscopy could monitor the treatment delivery, patient motion and alignment of the beam edge to the treatment region simultaneously. For all the imaged fractions, the patient positioning errors were within our clinical tolerances (3 mm in shifts and 3 degree in pitch angle rotation), with 4.60% exceeding no more than 1 mm in shifts. The averaged error of repetitive patient positioning was 1.2 mm in linear shift and 0.34 degrees in rotational pitch, consistent with the accuracy reported by the AlignRT system. The edge detection algorithm enhanced features such as field edges and blood vessels. Patient positioning errors and respiratory motion retrieved from rigid image registration were consistent with the edge enhanced images. Besides positioning errors caused by global inaccurate setups, edge enhanced blood vessels indicate the existence of deformations Conclusion: Real-time Cherenkoscopy imaging during EBRT is a novel imaging tool that can be used for treatment monitoring, patient positioning and motion tracking with approximately 1.2mm displacement and 0.34 degree rotational accuracy.

A Video-Analysis-Based Railway–Road Safety System for Detecting Hazard Situations at Level Crossings

Safety and security are the most discussed topics in the road and railway transportation field. Latest security initiatives in the field of railway transportation propose to implement video surveillance at level crossing (LC) environments. In this paper we explore the possibility of implementing a smart video surveillance security system that is tuned toward detecting and evaluating abnormal situations induced by users (pedestrians, vehicle drivers, and unattended objects) in LCs. This intelligent security system starts by detecting, separating, and tracking moving objects shot in the LC. Then, a hidden Markov model is developed to estimate ideal trajectories, allowing the detected targets to discard dangerous situations. After that, the level of risk of each target is instantly estimated by using the Dempster-Shafer data fusion technique. The proposed analysis allows for also recognizing hazard scenarios. The video surveillance system is connected to a communication system (the Wireless Access for Vehicular Environment), which takes the information on the dynamic status of the LC (safe or presence of a dangerous situation) and sends it to users approaching the LC. Four hazard scenarios are tested and evaluated with different real video image sequences: presence of the obstacle in the LC, presence of the stopped vehicles line, vehicle zigzagging between two closed half barriers, and pedestrian crossing the LC area.

Autonomous Flight in the Canyons of Streets on the Basis of Information Derived from the Vision System

This article presents algorithm of autonomous microplane flight control in street canyons using image from a camera as a carrier of information about microplane surroundings. Algorithm structure and video information processing are formed in a way that enables their implementation with the use of available equipment such as microcameras, advanced autopilot and signal processor DSP. On the basis of the developed algorithm model simulation research was prepared and conducted. Obtained results confirm effectiveness of this algorithm in autonomous flight control of microplane in street canyons. However, conducted simulations did not use real video image, but only simplified model of street map perspective image.

Tracking of Ocean-Floor Organisms Using a Particle Filter

This paper describes the development of a visual tracking method of an underwater vision system installed into a UVMS (underwater vehicle-manipulator system) to manipulate ocean-floor organisms such as acanthaster. First, an execution procedure of UVMS for approaching the target is shown. Next, a novel and robust target tracking method by using particle filter framework in order to achieve the procedure is proposed. We applied the proposed method to real video images, and examined its effectiveness by comparing it with other methods. The results of some experiments that evaluate the performance of the developed method are given.

Automated Aid Evaluation for Transitioning UAS Camera Views

Control applications involving multiple Unmanned Aerial Systems (UASs) will often require the operator to switch attention between UASs and their respective camera views. An automated aid that transitions between camera views is under evaluation. Instead of discretely switching from the camera view for one UAS to the camera view for another, a transition format is presented. With this format, the camera imagery seamlessly fades into a synthetic imagery correlate of the real video image and then uses a “fly-out, fly-in” metaphor over several seconds, finishing with a transition back from synthetic to real video imagery at the new camera viewpoint. The results from the present simulation evaluation provide support that an automated camera view transition aid can improve task performance and situation awareness. Moreover, the results show potential benefits of allowing an operator to decide where and when the fly-in segment starts.

An algorithm for calculi segmentation on ureteroscopic images

The purpose of the study is to develop an algorithm for the segmentation of renal calculi on ureteroscopic images. In fact, renal calculi are common source of urological obstruction, and laser lithotripsy during ureteroscopy is a possible therapy. A laser-based system to sweep the calculus surface and vaporize it was developed to automate a very tedious manual task. The distal tip of the ureteroscope is directed using image guidance, and this operation is not possible without an efficient segmentation of renal calculi on the ureteroscopic images. We proposed and developed a region growing algorithm to segment renal calculi on ureteroscopic images. Using real video images to compute ground truth and compare our segmentation with a reference segmentation, we computed statistics on different image metrics, such as Precision, Recall, and Yasnoff Measure, for comparison with ground truth. The algorithm and its parameters were established for the most likely clinical scenarii. The segmentation results are encouraging: the developed algorithm was able to correctly detect more than 90% of the surface of the calculi, according to an expert observer. Implementation of an algorithm for the segmentation of calculi on ureteroscopic images is feasible. The next step is the integration of our algorithm in the command scheme of a motorized system to build a complete operating prototype.

Incorrect Feature Tracking Detection by Affine Space Fitting

We present a new method for detecting incorrect feature point tracking. In this paper, we detect incorrect feature point tracking by imposing the constraint that under the affine camera model feature trajectories should be in an affine space in the parameter space. Introducing a statistical model of image noise, we test detected partial trajectories are sufficiently reliable. Then we detect incorrect partial trajectories. Using real video images, we demonstrate that our proposed method can detect incorrect feature point tracking fairly well.

Fast Projective Reconstruction: Toward Ultimate Efficiency

We accelerate the time-consuming iterations for projective reconstruction, a key component of self-calibration for computing 3-D shapes from feature point tracking over a video sequence. We first summarize the algorithms of the primal and dual methods for projective reconstruction. Then, we replace the eigenvalue computation in each step by the power method. We also accelerate the power method itself. Furthermore, we introduce the SOR method for accelerating the subspace fitting involved in the iterations. Using simulated and real video images, we demonstrate that the computation sometimes becomes several thousand times faster.

Dynamic face recognition: From human to machine vision

As confirmed by recent neurophysiological studies, the use of dynamic information is extremely important for humans in visual perception of biological forms and motion. Apart from the mere computation of the visual motion of the viewed objects, the motion itself conveys far more information, which helps understanding the scene. This paper provides an overview and some new insights on the use of dynamic visual information for face recognition. In this context, not only physical features emerge in the face representation, but also behavioral features should be accounted. While physical features are obtained from the subject’s face appearance, behavioral features are obtained from the individual motion and articulation of the face. In order to capture both the face appearance and the face dynamics, a dynamical face model based on a combination of Hidden Markov Models is presented. The number of states (or facial expressions) are automatically determined from the data by unsupervised clustering of expressions of faces in the video. The underlying architecture closely recalls the neural patterns activated in the perception of moving faces. Experimental results obtained from real video image data show the feasibility of the proposed approach.

Modeling and simulation with augmented reality

In applications such as airport operations, military simulations, and medical simulations, conducting simulations in accurate and realistic settings that are represented by real video imaging sequences becomes essential. This paper surveys recent work that enables visually realistic model constructions and the simulation of synthetic objects which are inserted in video sequences, and illustrates how synthetic objects can conduct intelligent behavior within a visual augmented reality.

Simulating autonomous agents in augmented reality

In many critical applications such as airport operations (for capacity planning), military simulations (for tactical training and planning), and medical simulations (for the planning of medical treatment and surgical operations), it is very useful to conduct simulations within physically accurate and visually realistic settings that are represented by real video imaging sequences. Furthermore, it is important that the simulated entities conduct autonomous actions which are realistic and which follow plans of action or intelligent behavior in reaction to current situations. We describe the research we have conducted to incorporate synthetic objects in a visually realistic manner in video sequences representing a real scene. We also discuss how the synthetic objects can be designed to conduct intelligent behavior within an augmented reality setting. The paper discusses both the computer vision aspects that we have addressed and solved, and the issues related to the insertion of intelligent autonomous objects within an augmented reality simulation.

Nonuniform video coding by means of multifoveal geometries

AbstractThis paper presents a control mechanism for video transmission that relies on transmitting nonuniform resolution images depending on the delay of the communication channel. These images are built in an active way to keep the areas of interest of the image at the highest resolution available. In order to shift the areas of high resolution over the image and to achieve a data structure that is easy to process by using conventional algorithms, a shifted foveal multiresolution geometry of adaptive size is used. If delays are too high, the resolution areas of the image can be transmitted at different rates. A functional system has been developed for corridor surveillance with static cameras. Tests with real video images have proven that the method allows an almost constant rate of images per second as long as the channel is not collapsed. A new method for determining the areas of interest is also proposed, based on hierarchical object tracking by means of adaptive stabilization of pyramidal structures. © 2002 John Wiley & Sons, Inc. Int J Imaging Syst Technol 12, 27–34, 2002

Straight lines have to be straight

Most algorithms in 3D computer vision rely on the pinhole camera model because of its simplicity, whereas video optics, especially low-cost wide-angle or fish-eye lenses, generate a lot of non-linear distortion which can be critical. To find the distortion parameters of a camera, we use the following fundamental property: a camera follows the pinhole model if and only if the projection of every line in space onto the camera is a line. Consequently, if we find the transformation on the video image so that every line in space is viewed in the transformed image as a line, then we know how to remove the distortion from the image. The algorithm consists of first doing edge extraction on a possibly distorted video sequence, then doing polygonal approximation with a large tolerance on these edges to extract possible lines from the sequence, and then finding the parameters of our distortion model that best transform these edges to segments. Results are presented on real video images, compared with distortion calibration obtained by a full camera calibration method which uses a calibration grid.

Localized photoelectrochemical etching with micrometric lateral resolution on transition metal diselenide photoelectrodes

Abstract The effect of localized photoetching on the photoresponse of n-MoSe2 and n-WSe2 single crystal electrodes has been examined with the help of the scanning microscope for semiconductor characterization (SMSC) technique. The SMSC technique has been used to obtain real video images of the photocurrent response distribution over the semiconductor van der Waals surface after photoetching. A direct comparison of the photoresponse corresponding to photocorroded and non-photocorroded areas can readily be established from these images. On the basis of various examples, it is shown that under moderate depletion conditions of the space-charge region, the highest photoresponse always corresponds to photoetched zones. This is unequivocally interpreted as an improvement of the kinetics of charge transfer at photoattached compared with non-photoattacked areas. This trend is always observed whatever the degree of roughness of the photocorroded surface. However, the beneficial effect of photoetching is more persistent on rough (stepped) than on smooth regions.

逆解析手法による3次元人体運動の再構成

Publisher Summary A system that reconstructs 3D human movement from a stream of 2D input images is outlined in this chapter. The system finds the 3D shape and motion of the human body by fitting a simple skeleton model to the markers found in the 2D image. The fitting is done using a nonlinear optimization approach. This optimization is essentially difficult because in-depth information cannot be obtained from 2D images. In order to overcome this problem, an inverse analysis technique that takes account of the following four kinds of a priori information is applied. First, the human body can be exhibited by versatile skeleton model. Second, joint angles have their limits. Third, human motion is smooth. Fourth, human keeps one's balance in slow motion. This chapter presents a reconstructed human movement from real video images by this method in order to demonstrate its applicability.

Measurement of complex optical flow with use of an augmented generalized gradient scheme

We present a method for measuring the local velocities and first-order variations in velocities in a time-varying image. The scheme is an extension of the generalized gradient model that encompasses the local variation of velocity within a local patch of the image. Motion within a patch is analyzed in parallel by 42 different spatiotemporal filters derived from 6 linearly independent spatiotemporal kernels. No constraints are imposed on the image structure, and there is no need for smoothness constraints on the velocity field. The aperture problem does not arise so long as there is some two-dimensional structure in the patch being analyzed. Among the advantages of the scheme is that there is no requirement to calculate second or higher derivatives of the image function. This makes the scheme robust in the presence of noise. The spatiotemporal kernels are of simple form, involving Gaussian functions, and are biologically plausible receptive fields. The validity of the scheme is demonstrated by application to both synthetic and real video images sequences and by direct comparison with another recently published scheme [Biol. Cybern. 63, 185 (1990)] for the measurement of complex optical flow.