Stereoscopic Image Sequences Research Articles

Fresh Concrete Properties from Stereoscopic Image Sequences

Increasing the degree of digitization and automation in concrete production can make a decisive contribution to reducing the associated CO2\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\ ext{CO}_{2}$$\\end{document} emissions. This paper presents a method which predicts the properties of fresh concrete during the mixing process on the basis of stereoscopic image sequences of the moving concrete and mix design information or a variation of these. A Convolutional Neural Network (CNN) is used for the prediction, which receives the images supported by information about the mix design as input. In addition, the network receives temporal information in the form of the time difference between image acquisition and the point in time for which the concrete properties are to be predicted. During training, the times at which the reference values were captured are used for the latter. With this temporal information, the network implicitly learns the time-dependent behavior of the concrete properties. The network predicts the slump flow diameter, the yield stress and the plastic viscosity. The time-dependent prediction opens up the possibility of forecasting the temporal development of the fresh concrete properties during mixing. This is a significant advantage for the concrete industry, as countermeasures can then be taken in a timely manner, if the properties deviate from the desired ones. In various experiments it is shown that both the stereoscopic observations and the mix design information contain valuable information for the time-dependent prediction of the fresh concrete properties.

Image-based Deep Learning for the time-dependent prediction of fresh concrete properties

Abstract. Increasing the degree of digitisation and automation in the concrete production process can play a crucial role in reducing the CO2 emissions that are associated with the production of concrete. In this paper, a method is presented that makes it possible to predict the properties of fresh concrete during the mixing process based on stereoscopic image sequences of the concretes flow behaviour. A Convolutional Neural Network (CNN) is used for the prediction, which receives the images supported by information on the mix design as input. In addition, the network receives temporal information in the form of the time difference between the time at which the images are taken and the time at which the reference values of the concretes are carried out. With this temporal information, the network implicitly learns the time-dependent behaviour of the concretes properties. The network predicts the slump flow diameter, the yield stress and the plastic viscosity. The time-dependent prediction potentially opens up the pathway to determine the temporal development of the fresh concrete properties already during mixing. This provides a huge advantage for the concrete industry. As a result, countermeasures can be taken in a timely manner. It is shown that an approach based on depth and optical flow images, supported by information of the mix design, achieves the best results.

COOPERATIVE IMAGE ORIENTATION CONSIDERING DYNAMIC OBJECTS

Abstract. In the context of image orientation, it is commonly assumed that the environment is completely static. This is why dynamic elements are typically filtered out using robust estimation procedures. Especially in urban areas, however, many such dynamic elements are present in the environment, which leads to a noticeable amount of errors that have to be detected via robust adjustment. This problem is even more evident in the case of cooperative image orientation using dynamic objects as ground control points (GCPs), because such dynamic objects carry the relevant information. One way to deal with this challenge is to detect these dynamic objects prior to the adjustment and to process the related image points separately. To do so, a novel methodology to distinguish dynamic and static image points in stereoscopic image sequences is introduced in this paper, using a neural network for the detection of potentially dynamic objects and additional checks via forward intersection. To investigate the effects of the consideration of dynamic points in the adjustment, an image sequence of an inner-city traffic scenario is used; image orientation, as well as the 3D coordinates of tie points, are calculated via a robust bundle adjustment. It is shown that compared to a solution without considering dynamic points, errors in the tie points are significantly reduced, while the median of the precision of all 3D coordinates of the tie points is improved.

Photogrammetric determination of 3D crack opening vectors from 3D displacement fields

This publication presents a procedure for the determination of all three components of crack opening vectors from stereoscopic image sequences of a specimen under load in civil engineering material testing. The method is based on analyzing stereoscopic image sequences of a concrete specimen with a surface texture, which is suitable for applying image matching techniques. Spatio-temporal correspondences are established by applying sub-pixel accuracy area based image matching techniques to a grid of surface points. Data acquisition starts at zero load. The load is stepwise or continuously increased during the experiment. The surface points are matched between the stereo images and tracked through each camera image sequence. As an intermediate result, we obtain a set of 3D object surface points for each epoch by spatial intersection. These 3D object points are triangulated into a mesh. Then, the mesh triangles are tested for deformations by transforming the triangles into 2D space and computing the norm of the 2D relative translation vector. Connected components of deformed triangles are determined and crack normals are computed. In the next step, the 3D relative translation vector can be derived for each deformed triangle. Defining local crack opening coordinate systems for the deformed triangles, the three components of the crack opening vectors can be computed. The method has been tested and validated in practical experiments. The technique is capable of quantitatively analyzing cracks with a width of less than one pixel in image space.

Advanced Satellite Image Classification of Various Resolution Image Using a Novel Approach of Deep Neural Network Classifier

Image registration is computationally intensive and applied in a variety of applications, for example, multispectral classification, change recognition, climate prediction and multi-view analysis in GIS and medicine. There are three types of registration namely multi-view, multimodal and multi-temporal. In multi-view based registration, the images of the same scene taken at different viewpoints are analyzed and modeled for the requirement. Hence stereoscopic image sequences of the same view are acquired, and accurate comparison for the image classification is essential. This paper presents a robust method which has three steps. The first phase includes obtaining hyper spectral (satellite) images and preprocessing of them, the second period subdivides into image blocks for alignment, and the final step focuses on classification based on hyper graph structure using deep learning approach. For processing of satellite images, a new method linear iterative clustering and deep neural network classification are employed. Previous works in remote sensing applications involve training samples and hence prior knowledge of image sets which incurs more computational time. The implementation of this method shows an automatic, achieving better accuracy and dynamic reconfigurable image registration in reduced complexity. The mathematical model used is hidden markov chain model for clustering which provides region-wise feature construction for evaluating region shape and contextual information. The work yields classification accuracy of 94.12% which is far better than past outcomes in this engaged field of research. The execution of the usage is examined, a comparison is additionally influenced regarding false classification ratio, time complexity and clustering accuracy is demonstrated.

3D Scene Flow Estimation with a Piecewise Rigid Scene Model

3D scene flow estimation aims to jointly recover dense geometry and 3D motion from stereoscopic image sequences, thus generalizes classical disparity and 2D optical flow estimation. To realize its conceptual benefits and overcome limitations of many existing methods, we propose to represent the dynamic scene as a collection of rigidly moving planes, into which the input images are segmented. Geometry and 3D motion are then jointly recovered alongside an over-segmentation of the scene. This piecewise rigid scene model is significantly more parsimonious than conventional pixel-based representations, yet retains the ability to represent real-world scenes with independent object motion. It, furthermore, enables us to define suitable scene priors, perform occlusion reasoning, and leverage discrete optimization schemes toward stable and accurate results. Assuming the rigid motion to persist approximately over time additionally enables us to incorporate multiple frames into the inference. To that end, each view holds its own representation, which is encouraged to be consistent across all other viewpoints and frames in a temporal window. We show that such a view-consistent multi-frame scheme significantly improves accuracy, especially in the presence of occlusions, and increases robustness against adverse imaging conditions. Our method currently achieves leading performance on the KITTI benchmark, for both flow and stereo.

Vision-Only Localization

Autonomous and intelligent vehicles will undoubtedly depend on an accurate ego localization solution. Global navigation satellite systems suffer from multipath propagation rendering this solution insufficient. Herein, we present a real-time system for six-degrees-of-freedom ego localization that uses only a single monocular camera. The camera image is harnessed to yield an ego pose relative to a previously computed visual map. We describe a process to automatically extract the ingredients of this map from stereoscopic image sequences. These include a mapping trajectory relative to the first pose, global scene signatures and local landmark descriptors. The localization algorithm then consists of a topological localization step that completely obviates the need for any global positioning sensors such as GNSS. A metric refinement step that recovers an accurate metric pose is subsequently applied. Metric localization recovers the ego pose in a factor graph optimization process based on local landmarks. We demonstrate centimeter-level accuracy by a set of experiments in an urban environment. To this end, two localization estimates are computed for two independent cameras mounted on the same vehicle. These two independent trajectories are thereafter compared for consistency. Finally, we present qualitative experiments of an augmented reality (AR) system that depends on the aforementioned localization solution. Several screen shots of the AR system are shown confirming centimeter-level accuracy and subdegree angular precision.

Improving Graph Cuts algorithm to transform sequence of stereo image to depth map

Recently, 3D display systems are getting considerable attentions not only from theater but also from home. 3D multimedia content development plays very important role in helping to setup a visual reality entertainment program. Lenticular Autostereoscopic display is one of the 3D-TV having the following advantages such as improving 3D viewing experience, supporting wider viewing angles for multiple viewers, and no requiring any special glasses. However, most of the current 3D movie and camera do not support the Autostereoscopic function. Therefore, we proposed a system that can transform the current 3D stereoscopic image sequence to the depth map sequence. These sequences can be warped into the multiplexed image by DIBR (Depth Image Based Rendering), and show with Autostereoscopic.Some recent techniques that transform the stereoscopic correspondence problem are based on Graph Cuts. They transform the matching problem to a minimization of a global energy function. However, it has been difficult to include high level information in the formulation of the Graph Cut. In this paper, we describe a new technique for generating depth map sequence from stereoscopic image sequence. We improve the Graph Cuts from pixel-based matching to region-based by using the Mean Shift 3D regions clustering to link the features of images before segmentation. And we also use the result of 3D regions clustering to assign depth values to time domain. After the sequence of depth map has been obtained, the DIBR method was used in transformation process. The experimental result shows that our system not only establishes a mechanism of depth transformation but also improves the accuracy and effectiveness on traditional Graph Cuts.

Problems of coding stereo images in human memory

This paper discusses the memorization and recall by man of a sequence of planar or stereoscopic images, including six frames that contain a planar strip (8×8 positions of the stimulus) or a volume strip (8×4×2 positions). At the recall stage, the subject chose between the stimulus and three distractors in each frame. It is shown that the times for recognition and recall are less for volume stimuli, while the percent of correct responses is greater for planar stimuli. For volume stimuli, the distribution of errors depends on the disparity between the target and the selected distractor. A model based on a heteroassociative neural network reproduces the error distribution for planar but not for volume stimuli. The resulting data are evidence that the internal representations are substantially different for planar and three-dimensional objects: disparity has a substantial effect on the memorization and recognition of three-dimensional objects.

A novel ultra-high speed camera for digital image processing applications

Multi-channel gated-intensified cameras are commonly used for capturing images at ultra-high frame rates. The use of image intensifiers reduces the image resolution and increases the error in applications requiring high-quality images, such as digital image correlation. We report the development of a new type of non-intensified multi-channel camera system that permits recording of image sequences at ultra-high frame rates at the native resolution afforded by the imaging optics and the cameras used. This camera system is based upon the concept of using a sequence of short-duration light pulses of different wavelengths for illumination and using wavelength selective elements in the imaging system to route each particular wavelength of light to a particular camera. As such, the duration of the light pulses controls the exposure time and the timing of the light pulses controls the interframe time. A prototype camera system built according to this concept comprises four dual-frame cameras synchronized with four dual-cavity pulsed lasers producing 5 ns pulses in four different wavelengths. The prototype is capable of recording four-frame full-resolution image sequences at frame rates up to 200 MHz and eight-frame image sequences at frame rates up to 8 MHz. This system is built around a stereo microscope to capture stereoscopic image sequences usable for 3D digital image correlation. The camera system is used for imaging the chip–workpiece interface area during high speed machining, and the images are used to map the strain rate in the primary shear zone.

An improved, wavelet-based, stereoscopic image sequence codec with SNR and spatial scalability

A novel stereoscopic moving-image compression structure is presented in this paper that allows for SNR and spatial scalability. The right picture sequence, referred to as the reference sequence, is encoded in the wavelet domain using motion-compensated prediction, where intra-pictures and prediction error are encoded using an adaptively scanned wavelet-difference-reduction (ASWDR) algorithm. The left picture sequence, referred to as the target sequence, is coded using disparity-compensated prediction, where the prediction error is also coded with ASWDR. Variable-block disparity and motion compensation and a novel loop filtering scheme to suppress partition artifacts induced by imperfect disparity and motion compensation are the key features of the proposed algorithm. The hierarchical structure of the two-dimensional DWT is exploited to obtain discrete levels of spatial scalability.

Stereoscopic video generation based on efficient layered structure and motion estimation from a monoscopic image sequence

This paper presents a novel object-based method for the generation of a stereoscopic image sequence from a monoscopic video, using bidirectional two-dimensional motion estimation for the recovery of rigid motion and structure and a Bayesian framework to handle occlusions. The latter is based on extended Kalman filters and an efficient method for reliably tracking object masks. Experimental results show that the layered object scene representation, combined with the proposed algorithm for reliably tracking object masks throughout the sequence, yields very accurate results.

A new look at binocular stereopsis

We report a new phenomenon, which illustrates that the role of binocular disparity in 3D shape perception critically depends on whether the parts are interpreted as belonging to a single object. The nature of this phenomenon was studied in four experiments. In the first two experiments the subjects were shown a sequence of stereoscopic images of a cube, in which binocular disparity indicated that the individual parts move towards or away from one eye. However, when the parts of the cube were perceived as elements of a single object, they appeared to move in a rigid fashion and the direction of motion was orthogonal to that predicted by the binocular disparities. The third experiment generalized these results to more complex polyhedra. The last experiment showed that constraints related to motion, such as rigidity, are important, but not critical for this phenomenon to occur. All these results imply that the interpretation as to what corresponds to a single object affects the importance (weight) of binocular disparity and may even eliminate its contribution altogether; the percept of a 3D shape is dominated by a priori constraints, and depth cues play a secondary role.

Quadtree-based disparity estimation for intermediate view synthesis of stereoscopic image sequences

In stereoscopic or multiview 3-D display systems, the synthesis of intermediate sequences is essential to assure lookaround capability and continuous motion parallax so that it can enhance comfortable 3-D perception. The quadtree-based disparity estimation is one of the most popular methods for the synthesis of intermediate sequences, due to the simplicity of its algorithm and hardware implementation. We propose two solutions in order to reduce annoying flicker at object boundaries of synthesized intermediate sequences using quadtree-based disparity estimation. The first requires a new splitting scheme, providing more consistent quadtree splitting strategies during the disparity estimation. The second involves adaptive temporal smoothing, using the dissimilarity between the present frame and the previous one to reduce the error of disparity estimation. These two proposals are tested by using several stereoscopic sequences, and the results show that flicker is remarkably reduced by them.

Stereo video coding based on quad-tree decomposition of B–P frames by motion and disparity interpolation

A new optimised technique for coding stereoscopic image sequences is presented and compared with already known methods. The proposed technique, called enhanced interpolated motion and disparity estimation (EIMDE), is based on the joint method, which encodes the frames of the right image sequence by exploiting both the temporal redundancy of the same sequence and the disparity redundancy with the left image sequence. In the proposed method, a variable block size scheme has been employed for motion and disparity estimation. The block size is controlled by quad-tree decomposition of the processed frame based on a rate-distortion splitting criterion. For the prediction of a macroblock, optimised motion and disparity vectors are jointly estimated and the participating proportion of each similarity is suitably searched. In this way, the energy of the resulted residual frame is minimised and the whole framework is optimised. Finally, the residual frame is decomposed by a discrete wavelet transform and is further compressed by morphological encoding the resulting coefficients. The proposed coder has been experimentally evaluated on real image sequences, where it produced good performance over other known methods.

Efficient Stereoscopic Video Coding Using Joint Disparity-Motion Estimation

The goal of this paper is to determine dense disparity and motion fields jointly and efficiently, given a sequence of stereoscopic images. At each time instant, the motion field of left sequences is estimated by using the current disparity field, the next disparity field, and the motion field of right sequences. The improved concepts for the proposed joint estimation are based on an edge-preserving regularization, an ambiguous vector region detection, and a median filtering for block-based vector generation. The reported approach is verified by processing a set of stereo sequences. Results are given with real stereoscopic data.

A Bayesian Approach for Segmentation in Stereo Image Sequences

Stereoscopic image sequence processing has been the focus of considerable attention in recent literature for videoconference applications. A novel Bayesian scheme is proposed in this paper, for the segmentation of a noisy stereoscopic image sequence. More specifically, occlusions and visible foreground and background regions are detected between the left and the right frame while the uncovered-background areas are identified between two successive frames of the sequence. Combined hypotheses are used for the formulation of the Bayes decision rule which employs a single intensity-difference measurement at each pixel. Experimental results illustrating the performance of the proposed technique are presented and evaluated in videoconference applications.

Generation of Temporally Consistent Multiple Virtual Camera Views from Stereoscopic Image Sequences

The emergence of a new generation of 3D auto stereoscopic displays is driving the requirement for multi-baseline images. The dominant form of this display technology requires multiple views of the ...

A wavelet multiresolution compression technique for 3D stereoscopic image sequence based on mixed-resolution psychophysical experiments

Three-dimensional visual communications, made through the use of stereoscopic images, are able to achieve total display realism. In order to create a 3D system with two images (left and right) that should be transmitted simultaneously, the large amount of information contained in the images should be reduced, and an efficient coding appropriate for stereoscopic images is developed. Stereoscopic image sequence compression involves the exploitation of the spatial redundancy between the left and right image frames to achieve the compression ratio higher than that are by the independent compression of the two frames. In order to further achieve the higher compression ratio, in this paper, we employ the mixed-resolution psychophysical experiments to the stereo image compression. The experiments have shown that a stereo image pair with one high-resolution image and one lower-resolution image is sufficient to provide good stereoscopic depth perception. Thus, one image sequence is compressed independent of the other sequence using the motion compensation, while the other sequence is estimated at a lower resolution from this stream using the low-resolution disparity compensation. To implement the mixed-resolution coding, a wavelet multiresolution framework has been adopted to facilitate such an estimation of motion and disparity vectors at different resolutions. Experimental results indicate that the compression ratio for a typical stereoscopic image sequence is about 90, without any significant loss in the perceived 3D stereoscopic image quality.

Photogrammetric set-up for the analysis of particle motion in aerosol under microgravity conditions

An optical set-up to investigate the motion of aerosol particles under microgravity conditions has been designed and manufactured. It provides accurate measurement of 3D coordinates of up to 1000 particles, discrimination between particles of different nature and estimation of the angular velocity of small aerosol crystals. A photogrammetric technique is applied to track independently moving particles, using the acquisition and processing of stereoscopic image sequences. The optical scheme of the set-up is based on a unique objective that collects the light coming from different directions from the back- and front-illuminated aerosol, and transfers it to four different cameras. Driving parameters for the optical head design and image quality are discussed. The set-up performance was verified on optical test objects, aerosol simulators and also real aerosol under microgravity conditions during parabolic flight. The set-up has been designed for use in the JET experiment on board the Maser 8 sounding rocket. The experiment aims at the first direct observation of the chemojet motion of free-flying small growing crystals. Investigation of chemojet motion along with elaboration of the image analysis technique is focused on the development of highly sensitive, real-time, non-perturbative analysis of the reactions on the surface of aerosol particles.