View Descriptors Research Articles

RJAN: Region‐based joint attention network for 3D shape recognition

AbstractAs an essential field of multimedia and computer vision, 3D shape recognition has attracted much research attention in recent years. Multiview‐based approaches have demonstrated their superiority in generating effective 3D shape representations. Typical methods usually extract the multiview global features and aggregate them together to generate 3D shape descriptors. However, there exist two disadvantages: First, the mainstream methods ignore the comprehensive exploration of local information in each view. Second, many approaches roughly aggregate multiview features by adding or concatenating them together. The information loss for some discriminative characteristics limits the representation effectiveness. To address these problems, a novel architecture named region‐based joint attention network (RJAN) was proposed. Specifically, the authors first design a hierarchical local information exploration module for view descriptor extraction. The region‐to‐region and channel‐to‐channel relationships from different granularities can be comprehensively explored and utilised to provide more discriminative characteristics for view feature learning. Subsequently, a novel relation‐aware view aggregation module is designed to aggregate the multiview features for shape descriptor generation, considering the view‐to‐view relationships. Extensive experiments were conducted on three public databases: ModelNet40, ModelNet10, and ShapeNetCore55. RJAN achieves state‐of‐the‐art performance in the tasks of 3D shape classification and 3D shape retrieval, which demonstrates the effectiveness of RJAN. The code has been released on https://github.com/slurrpp/RJAN.

Evaluation of window view preference using quantitative and qualitative factors of window view content

A high-quality window view plays an essential role in people's indoor life, and assessing the content of the window view is essential for evaluating view quality. Although many studies have evaluated specific view content preferences of occupants, only a few have lumped different quantitative and qualitative factors together. This study aims to assess occupants' window view preferences by quantitative and qualitative factors of view contents. We conduct a systematic study by providing 80 static photos of window views to the participants, selecting quantitative (i.e., natural feature ratio, artificial feature ratio, number of layers) and qualitative (i.e., presence of rule of thirds, horizontal layers, and far elements) window view descriptors, developing and administering questionnaires, and analyzing the obtained data. The findings reveal that the percentage of natural elements, including the sky and greenery, are highly correlated to occupants' subjective window view preference. In addition, the number of layers and the existence of far elements are also factors with more significant impacts.

Self-Supervised Graph Convolutional Network for Multi-View Clustering

Despite the promising preliminary results, existing graph convolutional network (GCN) based multi-view learning methods directly use the graph structure as view descriptor, which may inhibit the ability of multi-view learning for multimedia data. The major reason is that, in real multimedia applications, the graph structure may contain outliers. Moreover, they fail to take advantage of the information embedded in the inaccurate clustering labels obtained from their proposed methods, resulting in inferior clustering results. These observations motivate us to study whether there is a better alternative GCN based framework for multi-view clustering. To this end, in this paper, we propose an end-to-end self-supervised graph convolutional network for multi-view clustering (SGCMC). Specifically, SGCMC constructs a new view descriptor for graph-structured data by mapping the raw node content into the complex space via Euler transformation, which not only suppresses outliers but also reveals non-linear patterns embedded in data. Meanwhile, the proposed SGCMC uses the clustering labels to guide the learning of the latent representation and coefficient matrix, and the latter in turn is used to conduct the subsequent node clustering. By this way, clustering and representation learning are seamlessly connected, with the aim to achieve better clustering results. Extensive experiments indicate that the proposed SGCMC outperforms the state-of-the-art methods.

Multi-view graph embedding clustering network: Joint self-supervision and block diagonal representation

Multi-view clustering has become an active topic in artificial intelligence. Yet, similar investigation for graph-structured data clustering has been absent so far. To fill this gap, we present a Multi-View Graph embedding Clustering network (MVGC). Specifically, unlike traditional multi-view construction methods, which are only suitable to describe Euclidean structure data, we leverage Euler transform to augment the node attribute, as a new view descriptor, for non-Euclidean structure data. Meanwhile, we impose block diagonal representation constraint, which is measured by the ℓ1,2-norm, on self-expression coefficient matrix to well explore the cluster structure. By doing so, the learned view-consensus coefficient matrix well encodes the discriminative information. Moreover, we make use of the learned clustering labels to guide the learnings of node representation and coefficient matrix, where the latter is used in turn to conduct the subsequent clustering. In this way, clustering and representation learning are seamlessly connected, with the aim to achieve better clustering performance. Extensive experimental results indicate that MVGC is superior to 11 state-of-the-art methods on four benchmark datasets. In particular, MVGC achieves an Accuracy of 96.17% (53.31%) on the ACM (IMDB) dataset, which is an up to 2.85% (1.97%) clustering performance improvement compared with the strongest baseline.

MV-LFN: Multi-view based local information fusion network for 3D shape recognition

3D shape recognition is a challenging task due to the difficulty of representing the complex structure of 3D shapes. Recently, the view-based approaches that utilize the multiple views rendered from the shape for visual information extraction and feature aggregation to generate a global shape descriptor, achieved promising performance. However, the view-based approaches commonly ignore the exploration and utilization of local information in the multiple views, which influences the effectiveness of generated features. In this paper, we design a novel Multi-view based Local Information Fusion Network (MV-LFN) for the 3D shape recognition task. The local correlation attention mechanism (LCAM) is introduced to exploit the local correlations in the feature maps for generating a more effective view descriptor. Then, we hierarchically aggregate the multi-view feature maps to generate a shape super matrix (SSM). The local information is effectively extracted and maintained during the multi-view aggregation process, and the discrimination of shape descriptors is significantly improved. We conduct comparative experiments on the ModelNet and ShapeNetCore55 databases. The experimental performances effectively validate the superiority of MV-LFN.

Application of network protocol improvement and image content search in mathematical calculus 3D modeling video analysis

This article connects mobile terminals through improved network protocols, and connects monitoring devices through improved network protocols in the local area network. At the same time, it can convert mobile terminal requests into requests that comply with improved network protocols and forward them to monitoring devices. Method of three-dimensional feature extraction based on dynamic data. First, normalize the coordinates and scales of the three-dimensional calculus model, and use two kinds of data (voxel representation and pixel representation) to deal with video object recognition. Then construct a convolutional neural network as an input to the network, extract visual information and geometric information, and use deep learning to improve the recognition ability of single-modal features. Based on the convolutional neural network, the geometric descriptor and the view descriptor of the three-dimensional calculus model are extracted separately, and the two feature descriptors are fused in a network to find the association between the patterns. After forming the fused feature descriptor, it is applied to micro Classification and retrieval of integral 3D features. The multi-feature fusion layer can not only learn more distinguishing features of the two descriptors, but also supplement relevant information between the two descriptors. Compared with using these two representations alone, this method produces a much better classifier and improves retrieval efficiency.

Fuzzy Control of a Low Cost Mobile Robot Based on Vertical Lines in An Edge Detected Image

Edge detection is a computationally efficient and economical image processing technique as this method retains only the edges of surrounding objects for guiding the navigation and localization of a mobile robot. However, present edge detection based methods are heavily dependent on fusion with odometrical data which gradually accumulates localization errors. Moreover, they lack descriptive capabilities. In this paper, a fuzzy view descriptor which fused the information of the detected adjacent primary colours of the vertical lines in the mobile robot’s view with the fuzzified digital compass readings was used to ascertain the actual view. This fusion provided a sense of direction in addition to recognizing the frontal view. The fuzzified distance of the base pixel coordinates of the vertical line of interest determined the attraction of the mobile robot towards it or the repulsion from it. A fuzzy control rule base guided the mobile robot towards the vertical line of interest by constantly reducing its deviation in the edge image. The mobile robot managed to execute its consecutive movements in an intuitive manner without a precise mathematical model. The deviations of the vertical lines were kept small and the detection of the correct views was constantly reliable.

Exploring Macroscopic and Microscopic Fluctuations of Elicited Facial Expressions for Mood Disorder Classification

In the clinical diagnosis of mood disorder, a large proportion of patients with bipolar disorder (BD) are misdiagnosed as having unipolar depression (UD). Generally, long-term tracking is required for patients with BD to conduct an appropriate diagnosis by using traditional diagnosis tools. A one-time diagnosis system for facilitating diagnosis procedures is thus highly desirable. Accordingly, in this study, the facial expressions of patients with BD, patients with UD, and healthy controls elicited by emotional video clips were used for conducting mood disorder classification; the classification was performed by exploring the temporal fluctuation characteristics among the three groups. First, macroscopic facial expressions characterized by action units (AUs) were applied for describing the temporal transformation of muscles. Modulation spectrum analysis was applied to extract short-term intensity variations in the AUs. An interval-based multilayer perceptron (MLP) neural network was then used to classify mood disorder on the basis of the detected AU intensities. Moreover, motion vectors (MVs) were employed to describe subtle changes in facial expressions in the microscopic view. Eight basic orientations of MV change were considered for representing microfluctuation. Wavelet decomposition was then applied to extract entropy and energy features in different frequency bands. A long short-term memory model was finally used to model long-term variations for conducting mood disorder classification. A decision-level fusion approach was conducted on the combined results of macroscopic and microscopic facial expressions. For evaluating the described methods, the facial expressions elicited from the 36 subjects (12 from each of the BD, UD, and control groups) were used in 12-fold cross-validation experiments. Approaches for macroscopic and microscopic expressions achieved classification accuracies of 63.9 and 66.7 percent, respectively, and the accuracy of the fusion approach reached 72.2 percent. The results indicate that macroscopic and microscopic view descriptors are complementary to each other and helpful for conducting mood disorder classification.

2A2-L09 局所地図記述子による簡潔・識別的なマップマッチング

We propose a solution to the "map-to-text (M2T)" problem, which involves the generation of text descriptions of local map content based on scene understanding to facilitate fast succinct text-based map matching. Unlike previous local feature approaches that trade discriminativitv for viewpoint invariance, we develop a holistic view descriptor that is view-dependent and highly discriminative. Because the success of our holistic view descriptor depends on the assumption that the viewpoint is unique given a local map, we also address the issue of viewpoint planning that provides similar views for similar local maps.

Representational image generation for 3D objects

Finding good representational images for 3D object exploration is a highly subjective problem in the cognitive field. The "best" or "good" definitions do not depend on any metric. We have explained the VKL distance concept and introduced a novel view descriptor called vSKL distance for finding "good" representational images. The image generation is done by projecting the surfaces of 3D objects onto the screen or any planar surface. The projection process depends on parameters such as camera position, camera vector, up vector, and clipping plane positions. In this work we present a technique to find such camera positions that the 3D object is projected in "good" or "best" way where those subjective definitions are mapped to Information Theoretical distances. We compared greedy view selection integrated vSKL with two well known techniques: VKL and VMI. vSKL performs very close to the other two, hence face coverage perturbation is minimal, but it is 3 to 4 times faster. Furthermore, the saliency information is conveyed to users with generated images.

Automatic view selection through depth-based view stability analysis

Although the real world is composed of three-dimensional objects, we communicate information using two-dimensional media. The initial 2D view we see of an object has great importance on how we perceive it. Deciding which of the all possible 2D representations of 3D objects communicates the maximum information to the user is still challenging, and it may be highly dependent on the addressed task. Psychophysical experiments have shown that three-quarter views (oblique views between frontal view and profile view) are often preferred as representative views for 3D objects; however, for most models, no knowledge of its proper orientation is provided. Our goal is the selection of informative views without any user intervention. In order to do so, we analyze some stability-based view descriptors and present a new one that computes view stability through the use of depth maps, without prior knowledge on the geometry or orientation of the object. We will show that it produces good views that, in most of the analyzed cases, are close to three-quarter views.

View Influence Analysis and Optimization for Multiview Face Recognition

We present a novel method to recognize a multiview face (i.e., to recognize a face under different views) through optimization of multiple single-view face recognitions. Many current face descriptors show quite satisfactory results to recognize identity of people with given limited view (especially for the frontal view), but the full view of the human head has not yet been recognizable with commercially acceptable accuracy. As there are various single-view recognition techniques already developed for very high success rate, for instance, MPEG-7 advanced face recognizer, we propose a new paradigm to facilitate multiview face recognition, not through a multiview face recognizer, but through multiple single-view recognizers. To retrieve faces in any view from a registered descriptor, we need to give corresponding view information to the descriptor. As the descriptor needs to provide any requested view in space, we refer to it as 3D information that it needs to contain. Our analysis in various angled views checks the extent of each view influence and it provides a way to recognize a face through optimized integration of single view descriptors covering the view plane of horizontal rotation from -90° to 90° and vertical rotation from -30° to 30° The resulting face descriptor based on multiple representative views, which is of compact size, shows reasonable face recognition performance on any view. Hence, our face descriptor contains quite enough information of a person's face to help for recognition and eventually for search, retrieval, and browsing of photographs, videos, and 3D-facial model databases.

Breast cancer CADx based on BI-RADS™ descriptors from two mammographic views

In this study we compared the performance of computer aided diagnosis (CADx) algorithms based on Breast Imaging Reporting And Data System (BI-RADS) descriptors from one or two views. To select cases for the study with different mediolateral (MLO) and craniocaudal (CC) view descriptors, we assessed the agreement in BI-RADS lesion descriptors, BI-RADS assessment, and subtlety ratings for 1626 cases from the Digital Database for Screening Mammogrpahy (DDSM) using kappa statistics. We used 115 mass caseswith different descriptors for the two views to design linear discriminant analysis (LDA) based CADx algorithms. The CADx algorithms used BI-RADS descriptors and patient age as features. Thealgorithms based on BI-RADS descriptors from both the views performed marginally betterthan algorithms based on BI-RADS descriptors from a single view. A system that averaged theresults of two classifiers trained separately on the MLO and CC views displayed the best performance (Az=0.920 +/- 0.027). Thus, some improvement in performance of BI-RADS based CADx algorithms may be achieved by combining information from two mammographic views.

On Visual Similarity Based 3D Model Retrieval

Abstract A large number of 3D models are created and available on the Web, since more and more 3D modelling anddigitizing tools are developed for ever increasing applications. The techniques for content‐based 3D model retrievalthen become necessary. In this paper, a visual similarity‐based 3D model retrieval system is proposed.This approach measures the similarity among 3D models by visual similarity, and the main idea is that if two 3Dmodels are similar, they also look similar from all viewing angles. Therefore, one hundred orthogonal projectionsof an object, excluding symmetry, are encoded both by Zernike moments and Fourier descriptors as features forlater retrieval. The visual similarity‐based approach is robust against similarity transformation, noise, model degeneracyetc., and provides 42%, 94% and 25% better performance (precision‐recall evaluation diagram) thanthree other competing approaches: (1) the spherical harmonics approach developed by Funkhouser et al., (2) theMPEG‐7 Shape 3D descriptors, and (3) the MPEG‐7 Multiple View Descriptor. The proposed system is on the Webfor practical trial use (http://3d.csie.ntu.edu.tw), and the database contains more than 10,000 publicly available3D models collected from WWW pages. Furthermore, a user friendly interface is provided to retrieve 3D modelsby drawing 2D shapes. The retrieval is fast enough on a server with Pentium IV 2.4 GHz CPU, and it takes about2 seconds and 0.1 seconds for querying directly by a 3D model and by hand drawn 2D shapes, respectively. Categories and Subject Descriptors (according to ACM CCS): H.3.1 [Information Storage and Retrieval]: Indexing Methods