Multi-view Method Research Articles

Full-parallax and high-quality multiview 3D image acquisition method using camera slider

Full-parallax and high-quality multiview 3D image acquisition method using camera slider

Development of dense models of the Earth’s surface based on the semiglobal matching method for the case of multiple overlapping images

A method is developed for constructing a three-dimensional digital surface model based on the use of aerial images with multiple overlapping. The specific feature of this study is the use of the multiview matching method instead of stereo matching. The method is based on adapting the energy aggregation algorithm, which was proposed in the semiglobal matching (SGM) method, to the object space, as well as using the one-to-many scheme of cost calculation. The reconstructed scene is represented as a voxel grid. A high-performance implementation of the digital surface model construction at all stages is proposed based on the massive parallelization of computations on a graphics processing unit.

The review and results of different methods for facial recognition

In recent years, facial recognition draws much attention due to its wide potential applications. As a unique technology in Biometric Identification, facial recognition represents a significant improvement since it could be operated without cooperation of people under detection. Hence, facial recognition will be taken into defense system, medical detection, human behavior understanding, etc. Several theories and methods have been established to make progress in facial recognition: (1) A novel two-stage facial landmark localization method is proposed which has more accurate facial localization effect under specific database; (2) A statistical face frontalization method is proposed which outperforms state-of-the-art methods for face landmark localization; (3) It proposes a general facial landmark detection algorithm to handle images with severe occlusion and images with large head poses; (4) There are three methods proposed on Face Alignment including shape augmented regression method, pose-indexed based multi-view method and a learning based method via regressing local binary features. The aim of this paper is to analyze previous work of different aspects in facial recognition, focusing on concrete method and performance under various databases. In addition, some improvement measures and suggestions in potential applications will be put forward.

A study on facial expressions recognition

In terms of communication, postures and facial expressions of such feelings like happiness, anger and sadness play important roles in conveying information. With the development of the technology, recently a number of algorithms dealing with face alignment, face landmark detection, classification, facial landmark localization and pose estimation have been put forward. However, there are a lot of challenges and problems need to be fixed. In this paper, a few technologies have been concluded and analyzed, and they all relate to handling facial expressions recognition and poses like pose-indexed based multi-view method for face alignment, robust facial landmark detection under significant head pose and occlusion, partitioning the input domain for classification, robust statistics face formalization.

Interaction Style Recognition Based on Multi-Layer Multi-View Profile Representation

Interaction Style ( IS ) refers to patterns of interaction containing highly contextual and innate information. Awareness of our IS can help us discover interpersonal conflicts and guide us how to interact with others. Recently, automatic IS recognition is becoming increasingly important in the design of a dialogue system for harmonious interaction. With the goal to select appropriate responses, four IS types proposed by Berens are selected as the basis for our study. In this study, multiple views (multi-views) of the utterances during interaction, including emotions and dialogue topics, are recognized first. Inspired by the emotion profile theory, the IS profiles are then extracted using the multi-view features to better characterize the IS of the interactional utterances. Similar to the multilayer architectures in deep neural networks, a multi-layer multi-view IS profile representation method, structured layer by layer through embedding the multi-views, is proposed to better interpret intermediate representations in the feature space of the interactional utterances based on a probabilistic fusion model. The IS is finally recognized by using the Support Vector Machine (SVM) based on the obtained IS profiles. Experimental results demonstrate that the proposed method achieved an encouraging IS recognition accuracy and outperformed the previous method.

Multiview Community Discovery Algorithm via Nonnegative Factorization Matrix in Heterogeneous Networks

With the rapid development of the Internet and communication technologies, a large number of multimode or multidimensional networks widely emerge in real-world applications. Traditional community detection methods usually focus on homogeneous networks and simply treat different modes of nodes and connections in the same way, thus ignoring the inherent complexity and diversity of heterogeneous networks. It is challenging to effectively integrate the multiple modes of network information to discover the hidden community structure underlying heterogeneous interactions. In our work, a joint nonnegative matrix factorization (Joint-NMF) algorithm is proposed to discover the complex structure in heterogeneous networks. Our method transforms the heterogeneous dataset into a series of bipartite graphs correlated. Taking inspiration from the multiview method, we extend the semisupervised learning from single graph to several bipartite graphs with multiple views. In this way, it provides mutual information between different bipartite graphs to realize the collaborative learning of different classifiers, thus comprehensively considers the internal structure of all bipartite graphs, and makes all the classifiers tend to reach a consensus on the clustering results of the target-mode nodes. The experimental results show that Joint-NMF algorithm is efficient and well-behaved in real-world heterogeneous networks and can better explore the community structure of multimode nodes in heterogeneous networks.

A depth-based Multi-view Super-Resolution Method Using Image Fusion and Blind Deblurring

A depth-based Multi-view Super-Resolution Method Using Image Fusion and Blind Deblurring

Effective energy-based multi-view piecewise planar stereo

For piecewise planar scene modeling, many challenging issues still persist, in particular, how to generate sufficient candidate planes and how to assign an optimal plane for each spatial patch. To address these issues, we present a novel multi-view piecewise planar stereo method for the complete reconstruction. In our method, reconstruction is formulated as an energy-based plane labeling problem, where photo-consistency and geometric constraints are incorporated to a unified superpixel-level MRF (Markov Random Field) framework. To enhance the efficacy of the plane inference and optimization, an effective multi-direction plane sweeping with much restricted search space is carried out to generate sufficient and reliable candidate planes. Experiments show that our method can effectively handle many challenging factors (e.g., slant surfaces, textureless regions) and achieve satisfactory results.

Multi-view collaborative modeling method for complex system

It is hard to model a complex system by simply combining its partial characteristics. This paper presents a multi-view collaborative modeling method for complex system. Multi users, multi subjects, multi granularities and multi models in complex system modeling are unified to multi views. The principles and schemes for the decomposition of complex systems are introduced. According to the principle of separation of concerns, a complex system can be decomposed to a variety of model views. Collaborative modeling means that the views can be associated and integrated to form complete system model through their semantic. According to the form of representation, the models are generalizes to three kinds, i.e., view models, semantic models and storage models, which provides a unified framework for conversion and mapping between the models. Under the guidance of the method, a tool for the modeling of warship C2 system is developed. Applications show that the tool can support collaborative modeling and design for C2 system outstandingly.

See-through multi-projection three-dimensional display using transparent anisotropic diffuser.

We propose a see-through multi-projection three-dimensional (3D) display using a transparent anisotropic diffuser. By immersing a metal-coated anisotropic diffuser into index matching oil which has the same refractive index of anisotropic diffuser, a transparent anisotropic diffuser is implemented. The reflectance of the transparent anisotropic diffuser is analyzed with the transfer matrix. Two multi-projection methods are proposed based on reflection type integral imaging and multi-view method. Especially, the reflection type multi-view-based system is realized with a curved anisotropic diffuser. High resolution see-through 3D display can be realized with the proposed methods. They can be used in various applications with the two multi-projection methods. In order to show the augmented reality features, real objects and virtual 3D images are presented at the same time in the experimental setup.

Multiview road sign detection via self-adaptive color model and shape context matching

The multiview appearance of road signs in uncontrolled environments has made the detection of road signs a challenging problem in computer vision. We propose a road sign detection method to detect multiview road signs. This method is based on several algorithms, including the classical cascaded detector, the self-adaptive weighted Gaussian color model (SW-Gaussian model), and a shape context matching method. The classical cascaded detector is used to detect the frontal road signs in video sequences and obtain the parameters for the SW-Gaussian model. The proposed SW-Gaussian model combines the two-dimensional Gaussian model and the normalized red channel together, which can largely enhance the contrast between the red signs and background. The proposed shape context matching method can match shapes with big noise, which is utilized to detect road signs in different directions. The experimental results show that compared with previous detection methods, the proposed multiview detection method can reach higher detection rate in detecting signs with different directions.

Multi-View 3D Human Tracking in Crowded Scenes

This paper presents a robust multi-view method for tracking people in 3D scene. Our method distinguishes itself from previous works in two aspects. Firstly, we define a set of binary spatial relationships for individual subjects or pairs of subjects that appear at the same time, e.g. being left or right, being closer or further to the camera, etc. These binary relationships directly reflect relative positions of subjects in 3D scene and thus should be persisted during inference. Secondly, we introduce an unified probabilistic framework to exploit binary spatial constraints for simultaneous 3D localization and cross-view human tracking. We develop a cluster Markov Chain Monte Carlo method to search the optimal solution. We evaluate our method on both public video benchmarks and newly built multi-view video dataset. Results with comparisons showed that our method could achieve state-of-the-art tracking results and meter-level 3D localization on challenging videos.

Multi-Dimension Support Vector Machine Based Crowd Detection and Localisation Framework for Varying Video Sequences

In this paper, we propose a novel method for anomalous crowd behaviour detection and localization with divergent centers in intelligent video sequence through multiple SVM (support vector machines) based appearance model. In multi-dimension SVM crowd detection, many features are available to track the object robustly with three main features which include 1) identification of an object by gray scale value, 2) histogram of oriented gradients (HOG) and 3) local binary pattern (LBP). We propose two more powerful features namely gray level co-occurrence matrix (GLCM) and Gaber feature for more accurate and authenticate tracking result. To combine and process the corresponding SVMs obtained from each features, a new collaborative strategy is developed on the basis of the confidence distribution of the video samples which are weighted by entropy method. We have adopted subspace evolution strategy for reconstructing the image of the object by constructing an update model. Also, we determine reconstruction error from the samples and again automatically build an update model for the target which is tracked in the video sequences. Considering the movement of the targeted object, occlusion problem is considered and overcome by constructing a collaborative model from that of appearance model and update model. Also if update model is of discriminative model type, binary classification problem is taken into account and overcome by collaborative model. We run the multi-view SVM tracking method in real time with subspace evolution strategy to track and detect the moving objects in the crowded scene accurately. As shown in the result part, our method also overcomes the occlusion problem that occurs frequently while objects under rotation and illumination change due to different environmental conditions.

Fast and accurate multi‐view reconstruction by multi‐stage prioritised matching

In this study, the authors propose a multi-view stereo reconstruction method which creates a three-dimensional point cloud of a scene from multiple calibrated images captured from different viewpoints. The method is based on a prioritised match expansion technique, which starts from a sparse set of seed points, and iteratively expands them into neighbouring areas by using multiple expansion stages. Each seed point represents a surface patch and has a position and a surface normal vector. The location and surface normal of the seeds are optimised using a homography-based local image alignment. The propagation of seeds is performed in a prioritised order in which the most promising seeds are expanded first and removed from the list of seeds. The first expansion stage proceeds until the list of seeds is empty. In the following expansion stages, the current reconstruction may be further expanded by finding new seeds near the boundaries of the current reconstruction. The prioritised expansion strategy allows efficient generation of accurate point clouds and their experiments show its benefits compared with non-prioritised expansion. In addition, a comparison to the widely used patch-based multi-view stereo software shows that their method is significantly faster and produces more accurate and complete reconstructions.

Multi-view Classification for Identification of Alzheimer's Disease.

In this paper, we propose a multi-view learning method using Magnetic Resonance Imaging (MRI) data for Alzheimer's Disease (AD) diagnosis. Specifically, we extract both Region-Of-Interest (ROI) features and Histograms of Oriented Gradient (HOG) features from each MRI image, and then propose mapping HOG features onto the space of ROI features to make them comparable and to impose high intra-class similarity with low inter-class similarity. Finally, both mapped HOG features and original ROI features are input to the support vector machine for AD diagnosis. The purpose of mapping HOG features onto the space of ROI features is to provide complementary information so that features from different views can not only be comparable (i.e., homogeneous) but also be interpretable. For example, ROI features are robust to noise, but lack of reflecting small or subtle changes, while HOG features are diverse but less robust to noise. The proposed multi-view learning method is designed to learn the transformation between two spaces and to separate the classes under the supervision of class labels. The experimental results on the MRI images from the Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset show that the proposed multi-view method helps enhance disease status identification performance, outperforming both baseline methods and state-of-the-art methods.

Moving Object Tracking Using Multiple Views and Data Association

In this paper, a moving object tracking method using multiple views of the same scene taken by three cameras are presented. The object of interest is the pedestrian in the street. Unlike the single view method, the proposed multi-view method can effectively exploit extra information available in the other views when the object of interest in one of the views falls into an occlusion or clutter environment. Making a relation of an object detected in one view with the ones in the other views is critical and it is obtained by data association. The simulation results show a significant performance improvement over the conventional one.

A Novel Multi-view Three-dimensional Visual Measurement Method Based On Hexahedron Targets

In order to meet the demand of coded light multi-view 3D reconstruction, we propose a highly efficient and accurate technique for point cloud registration, which can acquire the whole surface information of the measured piece without the help of other locating equipment. Method: First of all, we illustrate the principle of the hexahedron targets registration, establish the index point couples and present the registration patterns for each side of the target, as well as the recognition approach. Then, based on the quaternion preliminary registration, we propose an accurate registration method which combines the orthogonal Grey code and ICP. The result of the simulated experiment shows that the registration error of the hexahedron targets method is approximately 0.03mm, and during the actual experiment, the number is approximately 0.8mm, which is close to the global camera method and a little bit higher than the surface method. The new technique solves the problem that the measured surface cannot be placed with index points. Furthermore, this technique ensures the registration accuracy without any other apparatus.

Hybrid parallel computing architecture for multiview phase shifting

Abstract. The multiview phase-shifting method shows its powerful capability in achieving high resolution three-dimensional (3-D) shape measurement. Unfortunately, this ability results in very high computation costs and3-D computations have to be processed offline. To realize real-time 3-D shape measurement, a hybrid parallelcomputing architecture is proposed for multiview phase shifting. In this architecture, the central processingunit can co-operate with the graphic processing unit (GPU) to achieve hybrid parallel computing. The highcomputation cost procedures, including lens distortion rectification, phase computation, correspondence,and 3-D reconstruction, are implemented in GPU, and a three-layer kernel function model is designed to simul-taneously realize coarse-grained and fine-grained paralleling computing. Experimental results verify that thedeveloped system can perform 50 fps (frame per second) real-time 3-D measurement with 260 K 3-D pointsper frame. A speedup of up to 180 times is obtained for the performance of the proposed technique using aNVIDIA GT560Ti graphics card rather than a sequential C in a 3.4 GHZ Inter Core i7 3770.

Fast multiview three-dimensional reconstruction method using cost volume filtering

As the number of customers who want to record three-dimensional (3-D) information using a mobile electronic device increases, it becomes more and more important to develop a method which quickly reconstructs a 3-D model from multiview images. A fast multiview-based 3-D reconstruction method is presented, which is suitable for the mobile environment by constructing a cost volume of the 3-D height field. This method consists of two steps: the construction of a reliable base surface and the recovery of shape details. In each step, the cost volume is constructed using photoconsistency and then it is filtered according to the multiscale. The multiscale-based cost volume filtering allows the 3-D reconstruction to maintain the overall shape and to preserve the shape details. We demonstrate the strength of the proposed method in terms of computation time, accuracy, and unconstrained acquisition environment.

A Multi-View Sonar Image Fusion Method Based on Nonsubsampled Contourlet Transform and Morphological Modification

In underwater detection, a single image from the image sonar doesnt have the capacity to fully describe the surface of the interested target. In practice, a target is usually detected from various views, and similar contours and textures in a series of multi-view images can be employed. This work offers a novel technique for the fusion of a series of sonar images in multi-view detection of the same target, to improve the quality of images and repair the regions of target. The method takes advantage of Nonsubsampled Contourlet Transform (NSCT) to acquire coefficient matrixes of different resolutions and directions. Besides, a framework for sonar image fusion is set up based on morphology modification. The coefficient matrixes in NSCT domain are fused in the multi-scale framework and revised in decision-making. Experimental results show that target regions in the fused sonar images are effectively repaired and image quality also get improved evidently.