Interpretable Representation Research Articles

Kinematic-Structure-Preserved Representation for Unsupervised 3D Human Pose Estimation

Estimation of 3D human pose from monocular image has gained considerable attention, as a key step to several human-centric applications. However, generalizability of human pose estimation models developed using supervision on large-scale in-studio datasets remains questionable, as these models often perform unsatisfactorily on unseen in-the-wild environments. Though weakly-supervised models have been proposed to address this shortcoming, performance of such models relies on availability of paired supervision on some related task, such as 2D pose or multi-view image pairs. In contrast, we propose a novel kinematic-structure-preserved unsupervised 3D pose estimation framework, which is not restrained by any paired or unpaired weak supervisions. Our pose estimation framework relies on a minimal set of prior knowledge that defines the underlying kinematic 3D structure, such as skeletal joint connectivity information with bone-length ratios in a fixed canonical scale. The proposed model employs three consecutive differentiable transformations namely forward-kinematics, camera-projection and spatial-map transformation. This design not only acts as a suitable bottleneck stimulating effective pose disentanglement, but also yields interpretable latent pose representations avoiding training of an explicit latent embedding to pose mapper. Furthermore, devoid of unstable adversarial setup, we re-utilize the decoder to formalize an energy-based loss, which enables us to learn from in-the-wild videos, beyond laboratory settings. Comprehensive experiments demonstrate our state-of-the-art unsupervised and weakly-supervised pose estimation performance on both Human3.6M and MPI-INF-3DHP datasets. Qualitative results on unseen environments further establish our superior generalization ability.

Modeling Dynamic Missingness of Implicit Feedback for Sequential Recommendation

Implicit feedback is widely used in collaborative filtering methods for sequential recommendation. It is well known that implicit feedback contains a large number of values that are <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">missing not at random</i> (MNAR); and the missing data is a mixture of negative and unknown feedback, making it difficult to learn users’ negative preferences. Recent studies modeled <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">exposure</i> , a latent missingness variable which indicates whether an item is exposed to a user, to give each missing entry a confidence of being negative feedback. However, these studies use static models and ignore the information in temporal dependencies among items, which seems to be an essential underlying factor to subsequent missingness. To model and exploit the dynamics of missingness, we propose a latent variable named “ <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">user intent</i> ” to govern the temporal changes of item missingness, and a hidden Markov model to represent such a process. The resulting framework captures the dynamic item missingness and incorporates it into matrix factorization (MF) for recommendation. We further extend the proposed framework to capture the dynamic preference of users, which results in a unified framework that is able to model different evolution patterns of user intent and user preference. We also explore two types of constraints to achieve a more compact and interpretable representation of <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">user intents</i> . Experiments on real-world datasets demonstrate the superiority of our method against state-of-the-art recommender systems.

La India y el Renacimiento florentino: las cartas de Filippo Sassetti

Here we present the translation of two of the letters that Filippo Sassetti, the Florentine merchant and humanist of the 16th-century, sent from India to Italy with abundant and insightful observations about the religious beliefs, customs, languages, nature and social life of the subcontinent. This document ―little known and so far unpublished in Spanish (and apparently in other languages too)― is a valuable testimony of the complex process of Europe’s reception and interpretative representation of the ancient Indian culture. In this respect, Sassetti’s hermeneutic endeavours, to a large extent dependent on Florentine Renaissance humanism’s ideals, stand out. They help us understand the Indian Other beyond the stereotypes in vogue then (and now), as well as the difficulties to achieve that.

Deep Inductive Graph Representation Learning

This paper presents a general inductive graph representation learning framework called $\text{DeepGL}$ DeepGL for learning deep node and edge features that generalize across-networks. In particular, $\text{DeepGL}$ DeepGL begins by deriving a set of base features from the graph (e.g., graphlet features) and automatically learns a multi-layered hierarchical graph representation where each successive layer leverages the output from the previous layer to learn features of a higher-order. Contrary to previous work, $\text{DeepGL}$ DeepGL learns relational functions (each representing a feature) that naturally generalize across-networks and are therefore useful for graph-based transfer learning tasks. Moreover, $\text{DeepGL}$ DeepGL naturally supports attributed graphs, learns interpretable inductive graph representations, and is space-efficient (by learning sparse feature vectors). In addition, $\text{DeepGL}$ DeepGL is expressive, flexible with many interchangeable components, efficient with a time complexity of $\mathcal {O}(|E|)$ O ( | E | ) , and scalable for large networks via an efficient parallel implementation. Compared with recent methods, $\text{DeepGL}$ DeepGL is (1) effective for across-network transfer learning tasks and large (attributed) graphs, (2) space-efficient requiring up to 6x less memory, (3) fast with up to 106x speedup in runtime performance, and (4) accurate with an average improvement in AUC of 20 percent or more on many learning tasks and across a wide variety of networks.

KREDIBILITAS MUQĀTIL BIN SULAYMĀN DALAM PERIWAYATAN ISRĀ’ĪLIYYĀT

In the circle interpretation of the Quran, Muqātil Bin Sulaymān is not a unusual name. He was one of narrators of Isrā’īliyyāt from Tabi' al-Tabi'in. One of his works, Tafsir al-Kabīr, is constantly used as a representation of interpretations which are full of nuances of Isrā’īliyyāt. This tafsir is also famous as TafsirMuqātil bin Sulaymān. During his life, many negative accusations were addressed to him. This accusation caused the passiveness of Muslim scholars to examine Tafsir al-Kabīr. Despite the many negative accusations he received, Muqātil’s expertise in the field of Quranic study could not be doubted. Ahlu Hadith agree to dismiss every hadith narrated by Muqātil. However, in the caseof narration in the story of Isrā’īliyyāt, it seems that it prevailed the dismissal. This can be proven by the many interpretations that quote the narration of Muqātil. This article will discuss how Muqātil’s credibility in the narrative of the story of Isrā’īliyyāt. Will he be dismissed or vice versa. Keywords:Muqātil Bin Sulaymān, Isrā’īliyyāt, tafsir

Cores que desenham o mundo: infâncias e as marcas de gênero, raça e classe

O ato de desenhar constitui uma das expressões singulares do protagonismo infantil, reverberando a experiência de vida pela qual as crianças materializam e compartilham suas indagações frente ao mundo, atividade criadora que traz a complexidade das relações experienciadas na convivência entre elas e entre elas e com os/as adultos/as, estilos de vida e práticas culturais que deixam as marcas de sua presença. Essas marcas de experiência, por sua vez, são rastros que carregam representações que nos contam pensamentos e interpretações do que é ser uma menina ou um menino, negro/a ou branco/a e filhos/as da classe trabalhadora. Com base nos desenhos de meninas e meninos, negros/as e brancos/as, com idade entre 8 e 9 anos, procuramos compreender como as crianças do ensino fundamental concebem as infâncias vivenciadas na periferia da cidade de São Paulo – Brasil. Ao tomarmos os desenhos infantis como fonte primaria da pesquisa, os conceituamos como artefatos culturais e documentos históricos, que podem contribuir para que se reconheçam as culturas infantis expressas nos traços, símbolos e cores das criações das crianças, permitindo-nos assim pensar este momento da sociedade, com suas especificidades históricas, e as suas marcas de gênero, raça e classe social. Trata-se de uma análise interpretativa, pautada na antropologia, sociologia e pedagogia da infância.

Interpretable Representation Learning for Speech and Audio Signals Based on Relevance Weighting

The learning of interpretable representations from raw data presents significant challenges for time series data like speech. In this work, we propose a relevance weighting scheme that allows the interpretation of the speech representations during the forward propagation of the model itself. The relevance weighting is achieved using a sub-network approach that performs the task of feature selection. A relevance sub-network, applied on the output of first layer of a convolutional neural network model operating on raw speech signals, acts as an acoustic filterbank (FB) layer with relevance weighting. A similar relevance sub-network applied on the second convolutional layer performs modulation filterbank learning with relevance weighting. The full acoustic model consisting of relevance sub-networks, convolutional layers and feed-forward layers is trained for a speech recognition task on noisy and reverberant speech in the Aurora-4, CHiME-3 and VOiCES datasets. The proposed representation learning framework is also applied for the task of sound classification in the UrbanSound8K dataset. A detailed analysis of the relevance weights learned by the model reveals that the relevance weights capture information regarding the underlying speech/audio content. In addition, speech recognition and sound classification experiments reveal that the incorporation of relevance weighting in the neural network architecture improves the performance significantly.

Masked convolutional neural network for supervised learning problems.

Convolutional neural networks (CNNs) have exhibited superior performance in various types of classification and prediction tasks, but their interpretability remains to be low despite years of research effort. It is crucial to improve the ability of existing models to interpret deep neural networks from both theoretical and practical perspectives and to develop new neural network models with interpretable representations. The aim of this paper is to propose a set of novel masked CNN (MCNN) models with better ability to interpret networks and more accurate prediction. The key ideas behind MCNNs are to introduce a latent binary network to extract informative regions of interest that contain important signals for prediction and to integrate the latent binary network with CNNs to achieve better prediction in various supervised learning problems. Extensive numerical studies demonstrate the competitive performance of the proposed MCNN models.

Язык теории как объект метапародии в прозе русского постмодернизма

This article deals with the roles that metaparody plays in modern prose. Methodologically, it follows Yuri Shatin’ semiotic approach to parody. In his works, parody is regarded as a special function of aesthetic language that comically reimagines a work’s pretexts while also expanding its referential semiotic structure by pointing out its intertextual sources. Contemporary fiction expands the semantic field of the plot by having narrators or characters reflect on philological terms and concepts. Furthermore, in postmodern fiction, representation of popular theoretical interpretations often achieves the effect of metaparody, the parody of scholar metalanguage. This paper studies metaparody in A. Bitov and A. Zholkovsky’s “philological prose”. It also deals with parodic reception of well-known theories (such as V. Propp’s fairy-tale functions; Structuralist approach to language; PostStructuralist concepts of discourse and narrative) in post-modern novels by M. Uspensky, A. Lyovkin, and V. Pelevin. Investigating references to theoretical discourse in various story episodes and structure units makes it possible to define the principles of “intrigue of interpretation” found in contemporary novels. This intrigue, featuring readers as characters (including professional readers such as scholars, librarians, critics, or publishers), becomes especially valuable in post-modern situation of “lacking reality.” Texts of this variety showcase the methods of interpreting and describing them as parts of the plot, while the “superior” theorizing instance becomes interwoven into the event-line. As such, both theoretical models and basic narrative conventions appear in parodical light, including even the crucial postmodern conflict between literary solipsism and the “open structure.”

A Study of Inductive Biases for Unsupervised Speech Representation Learning

Distributed representations, or embeddings, are commonly learned without supervision on very large unannotated corpora for natural language processing. In speech processing, deep network-based representations such as bottlenecks and x-vectors have had some success,but are limited to supervised or partly supervised settings where annotations are available and are not optimized to separate underlying factors. Here, we propose a generative model with deep encoders and decoders that can learn interpretable speech representations without supervision. Our inductive biases operate as prior distributions in a variational autoencoder model and allow us to separate several latent variables along a continuous range of time-scale properties, as opposed to binary oppositions or hierarchical factorization that have been previously proposed. On simulated data, we confirm that these biases enable the model to accurately recover phonetic and speaker underlying factors. On TIMIT and LibriSpeech, they yield representations that separate phonetic and speaker information, as evidenced by unsupervised results on downstream phoneme and speaker classification tasks using a simple k-means classifier.

On the Road With 16 Neurons: Towards Interpretable and Manipulable Latent Representations for Visual Predictions in Driving Scenarios

This paper proposes a strategy for visual perception in the context of autonomous driving. Humans, when not distracted or drunk, are still the best drivers you can currently find. For this reason, we take inspiration from two theoretical ideas about the human mind and its neural organization. The first idea concerns how the brain uses structures of neuron ensembles that expand and compress information to extract abstract concepts from visual experience and code them into compact representations. The second idea suggests that these neural perceptual representations are not neutral but functional to predicting the future state of affairs in the environment. Similarly, the prediction mechanism is not neutral but oriented to the planning of future action. We identify within the deep learning framework two artificial counterparts of the aforementioned neurocognitive theories. We find a correspondence between the first theoretical idea and the architecture of convolutional autoencoders, while we translate the second theory into a training procedure that learns compact representations which are not neutral but oriented to driving tasks, from two distinct perspectives. From a static perspective, we force separate groups of neural units in the compact representations to represent specific concepts crucial to the driving task distinctly. From a dynamic perspective, we bias the compact representations to predict how the current road scenario will change in the future. We successfully learn compact representations that use as few as 16 neural units for each of the two basic driving concepts we consider: cars and lanes . We maintain the two concepts separated in the latent space to facilitate the interpretation and manipulation of the perceptual representations. The source code for this paper is available at https://github.com/3lis/rnn_vae .

Petri net controlled virtual commissioning – A virtual design-loop approach

Abstract With the increasing system complexity, in the course of the introduction of modern technologies in today’s production systems, the system-related modeling effort in the virtual production planning phase also increases not only in terms of time but also extensively and leads to extended design requirements. Virtual Commissioning (VC) as a digital planning tool for validating highly automated systems in industry or cyber-physical systems (CPS) in research, is an already established method and has to be continually developed to meet these increasing design requirements. In addition, the modelling scope of the control-related behavior models is increasing and inevitably becomes confusing, especially beyond the process designer area, which makes it difficult for process owners to understand the control-specific behavior of the systems. Therefore, this paper deals with methods for the virtual development and validation of control structures of complex system relationships, evaluates them according to the respective area of application and puts them into the context of this work in order to demonstrate the need for a method to be able to provide a clear and interpretable representation of control-related behavior models. Subsequently, a graphically based modeling approach based on a petri-network for the control of virtual production models is presented, applied to an industry-related virtual demonstrator and evaluated for process-related robustness.

Interpretability and Validity of a Bifactor Model of ADHD in Young Adults: Assessing the General “g” and Specific IA and HI Factors

The current study investigated the clinical utility and validity of a bifactor model for representing ADHD symptoms in young adults. Two-hundred and seventy-eight participants (33.8% male) aged 18–24 and informants completed well-validated measures of ADHD symptoms and internalizing and externalizing behaviors. Results across self- and informant-report methods suggested a bifactor model with inattentive and hyperactive/impulsive specific factors as the best fitting conceptualization of ADHD symptoms. Within this bifactor model, the general (“g”) factor was found to be an interpretable representation of most ADHD symptoms that demonstrated significant associations with several internalizing and externalizing behaviors. Over and above “g,” inattentive and hyperactive/impulsive specific factors appeared to account for unique information in ADHD symptoms, particularly those corresponding with difficulties following through on instructions and verbal impulsivity, respectively. Specific factors also appeared to account for unique information pertaining to anxious/depressed and intrusive behavioral dysfunction, over and above either “g” or total ADHD symptom count scores. Overall, results suggested that integration of different components of the bifactor model, particularly HI, into clinical practice may help parse heterogeneity in ADHD-related phenotypes during young adulthood.

Representation learning of genomic sequence motifs with convolutional neural networks.

Although convolutional neural networks (CNNs) have been applied to a variety of computational genomics problems, there remains a large gap in our understanding of how they build representations of regulatory genomic sequences. Here we perform systematic experiments on synthetic sequences to reveal how CNN architecture, specifically convolutional filter size and max-pooling, influences the extent that sequence motif representations are learned by first layer filters. We find that CNNs designed to foster hierarchical representation learning of sequence motifs—assembling partial features into whole features in deeper layers—tend to learn distributed representations, i.e. partial motifs. On the other hand, CNNs that are designed to limit the ability to hierarchically build sequence motif representations in deeper layers tend to learn more interpretable localist representations, i.e. whole motifs. We then validate that this representation learning principle established from synthetic sequences generalizes to in vivo sequences.

ProLFA: Representative prototype selection for local feature aggregation

Given a set of hand-crafted local features, acquiring a global representation via aggregation is a promising technique to boost computational efficiency and improve task performance. Existing feature aggregation (FA) approaches, including Bag of Words and Fisher Vectors, usually fail to capture the desired information due to their pipeline mode. In this paper, we propose a generic formulation to provide a systematical solution (named ProLFA) to aggregate local descriptors. It is capable of producing compact yet interpretable representations by selecting representative prototypes from numerous descriptors, under relaxed exclusivity constraint. Meanwhile, to strengthen the discriminability of the aggregated representation, we rationally enforce the domain-invariant projection of bundled descriptors along a task-specific direction. Furthermore, ProLFA is also provided with a powerful generalization ability to deal flexibly with the semi-supervised and fully supervised scenarios in local feature aggregation. Experimental results on various descriptors and tasks demonstrate the proposed ProLFA is considerably superior over currently available alternatives about feature aggregation.

Profitability and risk sources in global timberland investments

As a long-term investment, timberland investments offer financial benefits including portfolio diversification, attractive risk/return profile, an inflation hedge, and the potential of cash flow. Based on interviews with experts regarding ranges of input parameters used in single-hectare financial models and Monte Carlo simulation method, we examine what are the main factors that influence internal rates of returns (IRRs) in several global timber plantation investment opportunities: loblolly pine on the U.S. Atlantic coastal plain; Douglas-fir plantations in the western U.S.; loblolly pine and eucalyptus plantations in Brazil; radiata pine and eucalyptus plantations in Chile; and pine and oak stands in Poland. The results show that excluding the price of land, biological growth and timber prices were the most influential variables that impacted the IRRs across global timberland investments. In addition, some country-specific factors, such as planting costs (Chile) and management costs (Poland and the U.S.), were identified as crucial when considering timberland investments in these countries. Investments in South America’s pine plantations are characterized by the same level of returns as eucalyptus opportunities, but with lower risk. The same was found for Douglas-fir investments in the Pacific Northwest compared to loblolly pine in the U.S. South. If Poland were an investable alternative, which is not the case so far, any investments in oak and pine stands are not recommended yet, given that for the same level of risk, better returns may be achieved in Douglas-fir plantations in the U.S. PNW. The Monte Carlo method utilized provides easily interpretable representation of the robustness of timberland investment estimates in selected regions and should become standard practice in forest-business decision making. However, more accurate probability density functions need to be determined in further research, using, for instance, historical data and kernel density estimation, rather than “lack of information” (triangular) distributions.

Bridge-GAN: Interpretable Representation Learning for Text-to-Image Synthesis

Text-to-image synthesis is to generate images with the consistent content as the given text description, which is a highly challenging task with two main issues: visual reality and content consistency. Recently, it is available to generate images with high visual reality due to the significant progress of generative adversarial networks. However, translating text description to image with high content consistency is still ambitious. For addressing the above issues, it is reasonable to establish a transitional space with interpretable representation as a bridge to associate text and image. So we propose a text-to-image synthesis approach named Bridge-like Generative Adversarial Networks (Bridge-GAN). Its main contributions are: (1) A transitional space is established as a bridge for improving content consistency, where the interpretable representation can be learned by guaranteeing the key visual information from given text descriptions. (2) A ternary mutual information objective is designed for optimizing the transitional space and enhancing both the visual reality and content consistency. It is proposed under the goal to disentangle the latent factors conditioned on text description for further interpretable representation learning. Comprehensive experiments on two widely-used datasets verify the effectiveness of our Bridge-GAN with the best performance.

Discrimination of Movement-Related Cortical Potentials Exploiting Unsupervised Learned Representations From ECoGs.

Brain–Computer Interfaces (BCI) aim to bypass the peripheral nervous system to link the brain to external devices via successful modeling of decoding mechanisms. BCI based on electrocorticogram or ECoG represent a viable compromise between clinical practicality, spatial resolution, and signal quality when it comes to extracellular electrical potentials from local neuronal assemblies. Classic analysis of ECoG traces usually falls under the umbrella of Time-Frequency decompositions with adaptations from Fourier analysis and wavelets as its most prominent variants. However, analyzing such high-dimensional, multivariate time series demands for specialized signal processing and neurophysiological principles. We propose a generative model for single-channel ECoGs that is able to fully characterize reoccurring rhythm–specific neuromodulations as weighted activations of prototypical templates over time. The set of timings, weights and indexes comprise a temporal marked point process (TMPP) that accesses a set of bases from vector spaces of different dimensions—a dictionary. The shallow nature of the model admits the equivalence between latent variables and representations. In this way, learning the model parameters is a case of unsupervised representation learning. We exploit principles of Minimum Description Length (MDL) encoding to effectively yield a data-driven framework where prototypical neuromodulations (not restricted to a particular duration) can be estimated alongside the timings and features of the TMPP. We validate the proposed methodology on discrimination of movement-related tasks utilizing 32-electrode grids implanted in the frontal cortex of six epileptic subjects. We show that the learned representations from the high-gamma band (85–145 Hz) are not only interpretable, but also discriminant in a lower dimensional space. The results also underscore the practicality of our algorithm, i.e., 2 main hyperparameters that can be readily set via neurophysiology, and emphasize the need of principled and interpretable representation learning in order to model encoding mechanisms in the brain.

A Multi-domain and Multi-modal Representation Disentangler for Cross-Domain Image Manipulation and Classification.

Learning interpretable data representation has been an active research topic in deep learning and computer vision. While representation disentanglement is an effective technique for addressing this task, existing works cannot easily handle the problems in which manipulating and recognizing data across multiple domains are desirable. In this paper, we present a unified network architecture of Multi-domain and Multi-modal Representation Disentangler (M2RD), with the goal of learning domain-invariant content representation with the associated domain-specific representation observed. By advancing adversarial learning and disentanglement techniques, the proposed model is able to perform continuous image manipulation across data domains with multiple modalities. More importantly, the resulting domain-invariant feature representation can be applied for unsupervised domain adaptation. Finally, our quantitative and qualitative results would confirm the effectiveness and robustness of the proposed model over state-of-the-art methods on the above tasks.

Mapping physical functions of the shoulder to American Shoulder and Elbow Surgeons and Patient-Reported Outcomes Measurement Information System scores

We sought to correlate physical functions of the shoulder to American Shoulder and Elbow Surgeons (ASES) and Patient-Reported Outcomes Measurement Information System (PROMIS) scores. We reviewed 3300 patient encounters with completed ASES scores, representing 2447 patients. Patients were seen for shoulder-related complaints. The most common diagnoses were rotator cuff disease (56%) and arthritis (9%); 54% and 46% of encounters were in operatively and nonoperatively treated patients, respectively. A total of 2632 PROMIS Physical Function (PF), 2574 PROMIS Pain Interference (PI), and 959 PROMIS Upper Extremity (UE) scores were simultaneously collected with the ASES form. The ASES form specifically asks about the ability to perform 8 physical functions. Receiver operating characteristic curves were calculated to determine 90% positive predictive value (PPV) and 90% negative predictive value (NPV) cutoffs for the ability to perform at high function for the ASES and PROMIS-PF, -UE, and -PI scores for the entire shoulder cohort and for rotator cuff disease and arthritis subgroups. ASES scores had consistently excellent ability, PROMIS-UE scores had reasonable to excellent ability, and PROMIS-PF and PROMIS-PI scores had overall reasonable ability to determine high- and low-function states. For reaching a high shelf in the rotator cuff disease subgroup, the 90% NPV and PPV cutoff scores were 41 and 66, respectively, for the ASES instrument. For reaching a high shelf in the arthritis subgroup, the cutoff scores were 50 and 78, respectively, for the ASES instrument. The 90% NPV and PPV cutoffs for each score, physical function, and diagnosis group were depicted by visual representations ("maps") for easier interpretation. Shoulder physical functions were mapped to outcome scores. Physical function mapping adds clinical meaning to the orthopedic literature, facilitating improved, more-informed decision making between physicians and patients.