Latent Alignment Research Articles

Discriminative feature alignment: Improving transferability of unsupervised domain adaptation by Gaussian-guided latent alignment

In this paper, we focus on the unsupervised domain adaptation problem where an approximate inference model is to be learned from a labeled data domain and expected to generalize well to an unlabeled domain. The success of unsupervised domain adaptation largely relies on the cross-domain feature alignment. Previous work has attempted to directly align features by classifier-induced discrepancies. Nevertheless, a common feature space cannot always be learned via this direct feature alignment especially when large domain gaps exist. To solve this problem, we introduce a Gaussian-guided latent alignment approach to align the latent feature distributions of the two domains under the guidance of a prior. In such an indirect way, the distributions over the samples from the two domains will be constructed on a common feature space, i.e., the space of the prior, which promotes better feature alignment. To effectively align the target latent distribution with this prior distribution, we also propose a novel unpaired L1-distance by taking advantage of the formulation of the encoder-decoder. The extensive evaluations on nine benchmark datasets validate the superior knowledge transferability through outperforming state-of-the-art methods and the versatility of the proposed method by improving the existing work significantly.

Cross-domain recommendation based on latent factor alignment

Recently, various cross-domain recommendation (CDR) models are proposed to overcome the sparsity problem, which leverage relatively abundant rating data from the auxiliary domain to improve recommendation performance of target domain. Though matrix factorization-based collaborative filtering algorithms gain dominance in single-domain recommendation systems, they cannot be used directly in cross-domain cases as the obtained latent factors of the target and auxiliary domains may not be aligned, which will lead to inaccurate knowledge transfer from the auxiliary domain to the target one. A CDR model named CDCFLFA is presented in this paper to solve this problem. In CDCFLFA, firstly latent factors between the two domains are aligned based on pattern matching. Then, user preferences of the auxiliary domain are transferred to update the original user latent vectors of target domain. Finally, a linear least square problem is solved to compute the item latent vectors of target domain and thus unknown ratings are obtained according to the updated user and item latent vectors. CDCFLFA does not require the same user or item sets between the two domains. Extensive experiments are conducted, and the results show that CDCFLFA achieves smaller MAE and RMSE values and larger precision and recall than the previous single- and cross-domain recommendation methods. Hence, CDCFLFA can be regarded as an effective cross-domain extension of single-domain matrix factorization algorithm.

Selective Wavelet Attention Learning for Single Image Deraining

Single image deraining refers to the process of restoring the clean background scene from a rainy image. Current approaches have resorted to deep learning techniques to remove rain from a single image by leveraging some prior information. However, due to the various appearances of rain streaks and accumulation, it is difficult to separate rain and background information in the embedding space, which results in inaccurate deraining. To address this issue, this paper proposes a selective wavelet attention learning method by learning a series of wavelet attention maps to guide the separation of rain and background information in both spatial and frequency domains. The key aspect of our method is utilizing wavelet transform to learn the content and structure of rainy features because the high-frequency features are more sensitive to rain degradations, whereas the low-frequency features preserve more of the background content. To begin with, we develop a selective wavelet attention encoder–decoder network to learn wavelet attention maps guiding the separation of rainy and background features at multiple scales. Meanwhile, we introduce wavelet pooling and unpooling to the encoder–decoder network, which shows superiority in learning increasingly abstract representations while preserving the background details. In addition, we propose latent alignment learning to supervise the background features as well as augment the training data to further improve the accuracy of deraining. Finally, we employ a hierarchical discriminator network based on selective wavelet attention to adversarially improve the visual fidelity of the generated results both globally and locally. Extensive experiments on synthetic and real datasets demonstrate that the proposed approach achieves more appealing results both quantitatively and qualitatively than the recent state-of-the-art methods.

Learning Aligned Image-Text Representations Using Graph Attentive Relational Network

Image-text matching aims to measure the similarities between images and textual descriptions, which has made great progress recently. The key to this cross-modal matching task is to build the latent semantic alignment between visual objects and words. Due to the widespread variations of sentence structures, it is very difficult to learn the latent semantic alignment using only global cross-modal features. Many previous methods attempt to learn the aligned image-text representations by the attention mechanism but generally ignore the relationships within textual descriptions which determine whether the words belong to the same visual object. In this paper, we propose a graph attentive relational network (GARN) to learn the aligned image-text representations by modeling the relationships between noun phrases in a text for the identity-aware image-text matching. In the GARN, we first decompose images and texts into regions and noun phrases, respectively. Then a skip graph neural network (skip-GNN) is proposed to learn effective textual representations which are a mixture of textual features and relational features. Finally, a graph attention network is further proposed to obtain the probabilities that the noun phrases belong to the image regions by modeling the relationships between noun phrases. We perform extensive experiments on the CUHK Person Description dataset (CUHK-PEDES), Caltech-UCSD Birds dataset (CUB), Oxford-102 Flowers dataset and Flickr30K dataset to verify the effectiveness of each component in our model. Experimental results show that our approach achieves the state-of-the-art results on these four benchmark datasets.

Multi-Granularity Sequence Alignment Mapping for Encoder-Decoder Based End-to-End ASR

Encoder-decoder based automatic speech recognition (ASR) methods are increasingly popular due to their simplified processing stages and low reliance on prior knowledge. Conventional encoder-decoder based approaches usually learn a sequence-to-sequence mapping function from the source speech to target units (e.g., subwords, characters) in an end-to-end manner. However, it is still unclear how to choose the optimal target unit, or granularity of multiple units. In general, as increasing the information available for learning sequence-to-sequence mapping functions can improve modeling effectiveness, we therefore propose a multi-granularity sequence alignment (MGSA) approach. This aims to enhance cross-sequence interactions between different granularity units for both modeling and inference stages in the encoder-decoder based ASR. Specifically, a decoder module is designed to generate multi-granularity sequence predictions. We then exploit the latent alignment mapping among units having different levels of granularity, by utilizing the decoded multi-level sequences as input for model prediction. The cross-sequence interaction can also be employed to re-calibrate output probabilities in the proposed post-inference algorithm. Experimental results on both WSJ-80 hrs and Switchboard-300 hrs datasets show the superiority of the proposed method compared to traditional multi-task methods as well as to single granularity baseline systems.

Image captioning with semantic-enhanced features and extremely hard negative examples

Image captioning is a task to generate natural descriptions of images. In existing image captioning models, the generated captions usually lack semantic discriminability. Semantic discriminability is difficult as it requires the model to capture detailed differences in images. In this paper, we propose an image captioning framework with semantic-enhanced features and extremely hard negative examples. These two components are combined in a Semantic-Enhanced Module. The semantic-enhanced module consists of an image-text matching sub-network and a Feature Fusion layer which provides semantic-enhanced features of rich semantic information. Moreover, in order to improve the semantic discriminability, we propose an extremely hard negative mining method which utilize the extremely hard negative examples to improve the latent alignment between visual and language information. Experimental results on MSCOCO and Flickr30K show that our proposed framework and training method can simultaneously improve the performance of image-text matching and image captioning, achieving competitive performance against state-of-the-art methods.

Pose-Guided Multi-Granularity Attention Network for Text-Based Person Search

Text-based person search aims to retrieve the corresponding person images in an image database by virtue of a describing sentence about the person, which poses great potential for various applications such as video surveillance. Extracting visual contents corresponding to the human description is the key to this cross-modal matching problem. Moreover, correlated images and descriptions involve different granularities of semantic relevance, which is usually ignored in previous methods. To exploit the multilevel corresponding visual contents, we propose a pose-guided multi-granularity attention network (PMA). Firstly, we propose a coarse alignment network (CA) to select the related image regions to the global description by a similarity-based attention. To further capture the phrase-related visual body part, a fine-grained alignment network (FA) is proposed, which employs pose information to learn latent semantic alignment between visual body part and textual noun phrase. To verify the effectiveness of our model, we perform extensive experiments on the CUHK Person Description Dataset (CUHK-PEDES) which is currently the only available dataset for text-based person search. Experimental results show that our approach outperforms the state-of-the-art methods by 15 % in terms of the top-1 metric.

Multi-view network embedding with node similarity ensemble

Node similarity is utilized as the most popular guidance for network embedding: nodes more similar in a network should still be more similar when mapping node information from a high-dimensional vector space to a low-dimensional vector space. Most existing methods preserve a single node similarity in the network embedding, which can merely preserve one-side network structural information. Though some works try to utilize several node similarities to preserve more network information, they fail to consider the interrelationships between the latent spaces preserving different node similarities. This causes both network information insufficiency and network information redundancy. To solve the problems of existing works, we propose a novel multi-view network embedding model with node similarity ensemble. Node similarities are first selected to maximize the represented network information while minimizing the information redundancy. For each combination of the selected node similarities, a latent space is generated as a view of the network. A Canonical Correlation Analysis based approach is then used to extract the common structure of the latent spaces alignment, and a neural network based approach is used to extract the view-specific latent structure by measuring the asymmetric KL divergence of nodes’ Gaussian distribution. The common structure and the view-specific structure of multiple views are merged to perverse the overall network information. Experiments held on the real-world networks verified the superiority of the proposed method to existing works.

Dynamic attention-based explainable recommendation with textual and visual fusion

Explainable recommendation, which provides explanations about why an item is recommended, has attracted growing attention in both research and industry communities. However, most existing explainable recommendation methods cannot provide multi-model explanations consisting of both textual and visual modalities or adaptive explanations tailored for the user’s dynamic preference, potentially leading to the degradation of customers’ satisfaction, confidence and trust for the recommender system. On the technical side, Recurrent Neural Network (RNN) has become the most prevalent technique to model dynamic user preferences. Benefit from the natural characteristics of RNN, the hidden state is a combination of long-term dependency and short-term interest to some degrees. But it works like a black-box and the monotonic temporal dependency of RNN is not sufficient to capture the user’s short-term interest.In this paper, to deal with the above issues, we propose a novel Attentive Recurrent Neural Network (Ante-RNN) with textual and visual fusion for the dynamic explainable recommendation. Specifically, our model jointly learns image representations with textual alignment and text representations with topical attention mechanism in a parallel way. Then a novel dynamic contextual attention mechanism is incorporated into Ante-RNN for modelling the complicated correlations among recent items and strengthening the user’s short-term interests. By combining the full latent visual-semantic alignments and a hybrid attention mechanism including topical and contextual attentions, Ante-RNN makes the recommendation process more transparent and explainable. Extensive experimental results on two real world datasets demonstrate the superior performance and explainability of our model.

Learning Low-Dimensional Temporal Representations with Latent Alignments.

Low-dimensional discriminative representations enhance machine learning methods in both performance and complexity. This has motivated supervised dimensionality reduction (DR), which transforms high-dimensional data into a discriminative subspace. Most DR methods require data to be i.i.d. However, in some domains, data naturally appear in sequences, where the observations are temporally correlated. We propose a DR method, namely, latent temporal linear discriminant analysis (LT-LDA), to learn low-dimensional temporal representations. We construct the separability among sequence classes by lifting the holistic temporal structures, which are established based on temporal alignments and may change in different subspaces. We jointly learn the subspace and the associated latent alignments by optimizing an objective that favors easily separable temporal structures. We show that this objective is connected to the inference of alignments and thus allows for an iterative solution. We provide both theoretical insight and empirical evaluations on several real-world sequence datasets to show the applicability of our method.

Modality-specific Cross-modal Similarity Measurement with Recurrent Attention Network.

Nowadays, cross-modal retrieval plays an important role to flexibly find useful information across different modalities of data. Effectively measuring the similarity between different modalities of data is the key of cross-modal retrieval. Different modalities such as image and text have imbalanced and complementary relationship, and they contain unequal amount of information when describing the same semantics. For example, images often contain more details that cannot be demonstrated by textual descriptions and vice versa. Existing works based on Deep Neural Network (DNN) mostly construct one common space for different modalities, to find the latent alignments between them, which lose their exclusive modality-specific characteristics. Therefore, we propose modality-specific cross-modal similarity measurement (MCSM) approach by constructing the independent semantic space for each modality, which adopts an endto- end framework to directly generate modality-specific crossmodal similarity without explicit common representation. For each semantic space, modality-specific characteristics within one modality are fully exploited by recurrent attention network, while the data of another modality is projected into this space with attention based joint embedding, which utilizes the learned attention weights for guiding the fine-grained cross-modal correlation learning, and captures the imbalanced and complementary relationship between different modalities. Finally, the complementarity between the semantic spaces for different modalities is explored by adaptive fusion of the modality-specific cross-modal similarities to perform cross-modal retrieval. Experiments on the widely-used Wikipedia, Pascal Sentence, MS-COCO datasets as well as our constructed large-scale XMediaNet dataset verify the effectiveness of our proposed approach, outperforming 9 stateof- the-art methods.

Latent alliances: the Women’s March and agrarian feminism as opportunities of and for political ecology

On 21 January 2017, the day after a bitterly contested presidential inauguration, nearly 5 million women and men marched and protested in 673 places across the globe – under the banner of the Women’s March. The March exceeded and expanded expectations on many levels, from organizational projections to media classifications to scholarly analysis. One aspect of the March – and its ongoing repercussions – that has eluded attention is its latent links to and alliances with women-led agrarian movements underway in the United States and internationally. Rendering the March merely urban does a disservice to the subtle ways agricultural politics and even agrarian resistance permeated its political messages. I argue that a feminist political ecology analysis – particularly a postcolonial, intersectional one rooted in anti-racism and anti-classism and following the lead of Black agrarian womanism – helps elucidate the latent alignments between and among the Women’s March and transnational women-led agrarian movements, such as those of La Via Campesina. As such, the burgeoning field of feminist political ecology is well-equipped to help connect the applied intersectional visions at work in the Women’s March with those of agrarian feminist movements, even as this task provides an important intellectual opportunity for political ecology itself.

Latent Alignment in Pathway-Dependent Ordering of Block Copolymer Thin Films.

Block copolymers spontaneously form well-defined nanoscale morphologies during thermal annealing. Yet, the structures one obtains can be influenced by nonequilibrium effects, including processing history or pathway-dependent assembly. Here, we explore various pathways for ordering of block copolymer thin films, using oven-annealing, as well as newly disclosed methods for rapid photothermal annealing and photothermal shearing. We report the discovery of an efficient pathway for ordering self-assembled films: ultrarapid shearing of as-cast films induces "latent alignment" in the disordered morphology. Subsequent thermal processing can then develop this directly into a uniaxially aligned morphology with low defect density. This deeper understanding of pathway-dependence may have broad implications in self-assembly.

Subcategory Clustering with Latent Feature Alignment and Filtering for Object Detection

For objects with large appearance variations, it has been proved that their detection performance can be effectively improved by clustering positive training instances into subcategories and learning multi-component models for the subcategories. However, it is not trivial to generate subcategories of high quality, due to the difficulty in measuring the similarity between positive instances. In this letter we propose a new weakly supervised clustering method to achieve better sub-categorization. Our method provides a more precise measurement of the similarity by aligning the positive instances through latent variables and filtering the aligned features. As a better alternative to the initialization step of the latent-SVM algorithm for the learning of the multi-component models, our method can lead to a superior performance gain for object detection. We demonstrate this on various real-world datasets.

Uma Abordagem da Teoria do Complexo de Segurança sobre as águas transfronteiriças do Rio Jordão / An approach of the Security Complex Theory applied to the transboundary waters of the Jordan River

The aim of this study is to address the characteristics of the security complex in the environmental sector concerning the transboundary waters in the Jordan River basin in the Middle East. To do so, the interests that guide the security relations between the states involved in managing the water resources in this river basin are identified, which are marked by border conflicts, latent ethnic interests and ideological alignments with historical links. This context, according to Buzan (2009), demonstrates the nature of insecurity with different power forces, in that the states are part of a regional security sub-system that can be seen in terms of patterns of friendship and animosity confined to a particular geographical area, to wit, the sharing of transboundary waters from the Jordan river and interest in controlling or sharing these water resources. Keywords : Security Complex; Management of Transboundary Water Resources; Jordan River Basin Received on November 2, 2012 / Accepted on November 30, 2012