View-independent Representation Research Articles

Enhanced view-independent representation method for skeleton-based human action recognition

Human action recognition is an important branch of computer vision science. It is a challenging task based on skeletal data because joints have complex spatiotemporal information. In this work, we propose a method for action recognition, which consists of three parts: view-independent representation, combination with cumulative Euclidean distance, and combined model. First, the action sequence becomes view-independent representations independent of the view. Second, these representations are further combined with cumulative Euclidean distances, so the joints more closely associated with the action are emphasised. Then, a combined model is adopted to extract these representation features and classify actions. It consists of two parts, a regular three-layer BLSTM network, and a temporal attention module. Experimental results on two multi-view benchmark datasets, Northwestern-UCLA and NTU RGB + D, demonstrate the effectiveness of our complete method. Despite its simple architecture and the use of only one type of action feature, it can still significantly improve recognition performance and has strong robustness.

Enhanced view-independent representation method for skeleton-based human action recognition

Enhanced view-independent representation method for skeleton-based human action recognition

View-independent representation with frame interpolation method for skeleton-based human action recognition

Human action recognition is an important branch of computer vision science. It is a challenging task based on skeletal data because of joints’ complex spatiotemporal information. In this work, we propose a method for action recognition, which consists of three parts: view-independent representation, frame interpolation, and combined model. First, the action sequence becomes view-independent representations independent of the view. Second, when judgment conditions are met, differentiated frame interpolations are used to expand the temporal dimensional information. Then, a combined model is adopted to extract these representation features and classify actions. Experimental results on two multi-view benchmark datasets Northwestern-UCLA and NTU RGB+D demonstrate the effectiveness of our complete method. Although using only one type of action feature and a simple architecture combined model, our complete method still outperforms most of the referential state-of-the-art methods and has strong robustness.

Flexible point cloud matching method based on three-dimensional image feature points

Flexible and robust point cloud matching is important for three-dimensional surface measurement. This article proposes a new matching method based on three-dimensional image feature points. First, an intrinsic shape signature algorithm is used to detect the key shape feature points, using a weighted three-dimensional occupational histogram of the data points within the angular space, which is a view-independent representation of the three-dimensional shape. Then, the point feature histogram is used to represent the underlying surface model properties at a point whose computation is based on the combination of certain geometrical relations between the point’s nearest k-neighbors. The two-view point clouds are robustly matched using the proposed double neighborhood constraint of minimizing the sum of the Euclidean distances between the local neighbors of the point and feature point. The proposed optimization method is immune to noise, reduces the search range for matching points, and improves the correct feature point matching rate for a weak surface texture. The matching accuracy and stability of the proposed method are verified using experiments. This method can be used for a flat surface with weak features and in other applications. The method has a larger application range than the traditional methods.

Surface Motion Capture Animation Synthesis.

We propose to generate novel animations from a set of elementary examples of video-based surface motion capture, under user-specified constraints. 4D surface capture animation is motivated by the increasing demand from media production for highly realistic 3D content. To this aim, data driven strategies that consider video-based information can produce animation with real shapes, kinematics and appearances. Our animations rely on the combination and the interpolation of textured 3D mesh data, which requires examining two aspects: (1) Shape geometry and (2) appearance. First, we propose an animation synthesis structure for the shape geometry, the Essential graph, that outperforms standard Motion graphs in optimality with respect to quantitative criteria, and we extend optimized interpolated transition algorithms to mesh data. Second, we propose a compact view-independent representation for the shape appearance. This representation encodes subject appearance changes due to viewpoint and illumination, and due to inaccuracies in geometric modelling independently. Besides providing compact representations, such decompositions allow for additional applications such as interpolation for animation.

The BEVPS: A new test battery to assess visual perceptual and spatial processing abilities in 5–14 year-old children

ABSTRACTThis study aims to examine the different levels of visual perceptual object recognition (early, intermediate, and late) defined in Humphreys and Riddoch’s model as well as basic visual spatial processing in children using a new test battery (BEVPS). It focuses on the age sensitivity, internal coherence, theoretical validity, and convergent validity of this battery. French-speaking, typically developing children (n = 179; 5 to 14 years) were assessed using 15 new computerized subtests. After selecting the most age-sensitive tasks though ceiling effect and correlation analyses, an exploratory factorial analysis was run with the 12 remaining subtests to examine the BEVPS’ theoretical validity. Three separate factors were identified for the assessment of the stimuli’s basic features (F1, four subtests), view-dependent and -independent object representations (F2, six subtests), and basic visual spatial processing (F3, two subtests). Convergent validity analyses revealed positive correlations between F1 and F2 and the Beery-VMI visual perception subtest, while no such correlations were found for F3. Children’s performances progressed until the age of 9–10 years in F1 and in view-independent representations (F2), and until 11–12 years in view-dependent representations (F2). However, no progression with age was observed in F3. Moreover, the selected subtests, present good-to-excellent internal consistency, which indicates that they provide reliable measures for the assessment of visual perceptual processing abilities in children.

Compact Multiview Representation of Documents Based on the Total Variability Space

Mapping text documents in an LDA-based topic-space is a classical way to extract high-level representation of text documents. Unfortunately, LDA is highly sensitive to hyper-parameters related to the number of classes, or word and topic distribution, and there is no systematic way to pre-estimate optimal configurations. Moreover, various hyper-parameter configurations offer complementary views on the document. In this paper, we propose a method based on a two-step process that, first, expands the representation space by using a set of topic spaces and, second, compacts the representation space by removing poorly relevant dimensions. These two steps are based respectively on multi-view LDA-based representation spaces and factor-analysis models. This model provides a view-independent representation of documents while extracting complementary information from a massive multi-view representation. Experiments are conducted on the DECODA conversation corpus and the Reuters-21578 textual dataset. Results show the efficiency of the proposed multiview compact representation paradigm. The proposed categorization system reaches an accuracy of 86.5% with automatic transcriptions of conversations from DECODA corpus and a Macro-F1 of 80% during a classification task of the well-known Reuters-21578 corpus, with a significant gain compared to the baseline (best single topic space configuration), as well as methods and document representations previously studied.

View-independent action recognition: a hybrid approach

In this paper, we propose a new framework for view independent action recognition, which uses a combination of a view-dependent representation and a view-independent representation. The view-dependent representation reduces the number of possible action's labels prior to the view-independent representation. We used the entropy of silhouette's distance transformation as view-dependent representation and the self-similarity matrix of the trajectory of uniformly distributed feature points over the human body as view-independent representation. The experiment results show that the proposed method outperforms recent action recognition approaches despite its low computational cost.

Temporal Coherence in Image-Based Visual Hull Rendering

Image-based visual hull rendering is a method for generating depth maps of a desired viewpoint from a set of silhouette images captured by calibrated cameras. It does not compute a view-independent data representation, such as a voxel grid or a mesh, which makes it particularly efficient for dynamic scenes. When users are captured, the scene is usually dynamic, but does not change rapidly because people move smoothly within a subsecond time frame. Exploiting this temporal coherence to avoid redundant calculations is challenging because of the lack of an explicit data representation. This paper analyzes the image-based visual hull algorithm to find intermediate information that stays valid over time and is, therefore, worth to make explicit. We then derive methods that exploit this information to improve the rendering performance. Our methods reduce the execution time by up to 25 percent. When the user's motions are very slow, reductions of up to 50 percent are achieved.

On the Motion Dimension Reduction by Laplacian Eigenmap

Laplacian eigenmap is used for motion dimension reduction. Eight Kinds of artificially typical 2D motion data are made as input of Laplacian eigenmap, and meaningful conclusions are obtained, which can be easily extended to complex data analysis in real applications. Advanced motion analysis methods have also been discussed briefly, such as 3D motion analysis, view independent motion analysis, pose representation and estimation as well as structural video analysis.

A new pose-based representation for recognizing actions from multiple cameras

We address the problem of recognizing actions from arbitrary views for a multi-camera system. We argue that poses are important for understanding human actions and the strength of the pose representation affects the overall performance of the action recognition system. Based on this idea, we present a new view-independent representation for human poses. Assuming that the data is initially provided in the form of volumetric data, the volume of the human body is first divided into a sequence of horizontal layers, and then the intersections of the body segments with each layer are coded with enclosing circles. The circular features in all layers (i) the number of circles, (ii) the area of the outer circle, and (iii) the area of the inner circle are then used to generate a pose descriptor. The pose descriptors of all frames in an action sequence are further combined to generate corresponding motion descriptors. Action recognition is then performed with a simple nearest neighbor classifier. Experiments performed on the benchmark IXMAS multi-view dataset demonstrate that the performance of our method is comparable to the other methods in the literature.

Perceptual learning of view-independence in visuo-haptic object representations

We previously showed that cross-modal recognition of unfamiliar objects is view-independent, in contrast to view-dependence within-modally, in both vision and haptics. Does the view-independent, bisensory representation underlying cross-modal recognition arise from integration of unisensory, view-dependent representations or intermediate, unisensory but view-independent representations? Two psychophysical experiments sought to distinguish between these alternative models. In both experiments, participants began from baseline, within-modal, view-dependence for object recognition in both vision and haptics. The first experiment induced within-modal view-independence by perceptual learning, which was completely and symmetrically transferred cross-modally: visual view-independence acquired through visual learning also resulted in haptic view-independence and vice versa. In the second experiment, both visual and haptic view-dependence were transformed to view-independence by either haptic-visual or visual-haptic cross-modal learning. We conclude that cross-modal view-independence fits with a model in which unisensory view-dependent representations are directly integrated into a bisensory, view-independent representation, rather than via intermediate, unisensory, view-independent representations.

When Faces Are Heads: View-Dependent Recognition of Faces Altered Relationally or Componentially

When faces are viewed from different angles the appearance of facial features undergoes dramatic changes. We investigated two types of 3D-head models in frontal and three-quarter views, varying either in componential information such as different eyes, mouths and noses, or in relational information. Variations of the latter can only be investigated using 3D-head versions. Experiment 1 revealed high costs of transfer in recognition performance when views change, that were similar for both componentially and relationally altered faces. In Experiment 2, whole-to-part superiority was investigated by presenting isolated parts of critical features in addition to the whole face. Recognition of the whole face was only superior when views were identical. The results support the hypothesis of picture-based and view-dependent processing. Thus, there seems to be no efficient view-independent representation, at least for relatively unfamiliar faces.

Scalable 3D video of dynamic scenes

In this paper we present a scalable 3D video framework for capturing and rendering dynamic scenes. The acquisition system is based on multiple sparsely placed 3D video bricks, each comprising a projector, two grayscale cameras, and a color camera. Relying on structured light with complementary patterns, texture images and pattern-augmented views of the scene are acquired simultaneously by time-multiplexed projections and synchronized camera exposures. Using space–time stereo on the acquired pattern images, high-quality depth maps are extracted, whose corresponding surface samples are merged into a view-independent, point-based 3D data structure. This representation allows for effective photo-consistency enforcement and outlier removal, leading to a significant decrease of visual artifacts and a high resulting rendering quality using EWA volume splatting. Our framework and its view-independent representation allow for simple and straightforward editing of 3D video. In order to demonstrate its flexibility, we show compositing techniques and spatiotemporal effects.

新映像メディアとその応用 全周計測距離データを用いた顔基本６表情の解析と合成

In most conventional approaches to the synthesis of human facial expressions, facial images are generated by manually moving feature points on a face based on the concept of FACS (Facial Action Coding System), primarily with 3D models, such as a wireframe model This paper describes a synthesis-by-analysis approach using range images for producing human facial 3D images with primary expressions First view-independent representations of 3D locations of facial feature points are obtained using an object-centered coordinate system defined on the face Then we quantify feature point locations for the neutral expression and six primary expressions. Applying an image warping technique on both registered range and surface texture images, we finally generate 3D facial expression images from a neutral expression image and motion vectors of facial feature points.