Raw Representation Research Articles

Understanding the limitations of self-supervised learning for tabular anomaly detection

While self-supervised learning has improved anomaly detection in computer vision and natural language processing, it is unclear whether tabular data can benefit from it. This paper explores the limitations of self-supervision for tabular anomaly detection. We conduct several experiments spanning various pretext tasks on 26 benchmark datasets to understand why this is the case. Our results confirm representations derived from self-supervision do not improve tabular anomaly detection performance compared to using the raw representations of the data. We show this is due to neural networks introducing irrelevant features, which reduces the effectiveness of anomaly detectors. However, we demonstrate that using a subspace of the neural network’s representation can recover performance.

Directed Acyclic Graph Learning on Attributed Heterogeneous Network

Learning the directed acyclic graph (DAG) among causal variables is a fundamental pre-task in causal discovery. Available DAG learning solutions canonically focus on homogeneous nodes with multiple variables and assume i.i.d. samples, how to learn DAG on typical attributed heterogeneous network (AHN) composed with different types of inter-dependent nodes and diverse attributes is a practical but more difficult task. In this paper, we propose HetDAG to identify DAG among nodes from heterogeneous network. HetDAG first embeds different types of node attributes and aggregates these embeddings as the node's raw representation. Then it uses contrastive learning with prior network structure to explore latent relationships between nodes and update the representation. Next, HetDAG introduces an attention-based DAG learning module that takes node representations as input to search DAG and orient edges between nodes. To the best of our knowledge, HetDAG is the first study to learn DAG on heterogeneous networks. Extensive experiments on both semi-synthetic and real data show that HetDAG can learn DAG in an efficacy way and outperforms the state-of-the-art approaches. The results on real biological networks confirm that HetDAG can find out the causal relations between lncRNAs and miRNAs.

Exploring superadditivity of coherent information of noisy quantum channels through genetic algorithms

Machine learning techniques are increasingly being used in fundamental research to solve various challenging problems. Here we explore one such technique to address an important problem in quantum communication scenario. While transferring quantum information through a noisy quantum channel, the utility of the channel is characterized by its quantum capacity. Quantum channels, however, display an intriguing property called super-additivity of coherent information. This makes the calculation of quantum capacity a hard computational problem involving optimization over an exponentially increasing search space. In this work, we first utilize a neural network ansatz to represent quantum states and then apply an evolutionary optimization scheme to address this problem. We find regions in the three-parameter space of qubit Pauli channels where coherent information exhibits this super-additivity feature. We characterised the quantum codes that achieves high coherent information, finding several non-trivial quantum codes that outperforms the repetition codes for some Pauli channels. For some Pauli channels, these codes displays very high super-additivity of the order of 0.01, much higher than the observed values in other well studied quantum channels. We further compared the learning performance of the Neural Network ansatz with the raw ansatz to find that in the three-shot case, the neural network ansatz outperforms the raw representation in finding quantum codes of high coherent information. We also compared the learning performance of the evolutionary algorithm with a simple Particle Swarm Optimisation scheme and show empirical results indicating comparable performance, suggesting that the Neural Network ansatz coupled with the evolutionary scheme is indeed a promising approach to finding non-trivial quantum codes of high coherent information.

Understanding Dynamics of Nonlinear Representation Learning and Its Application.

Representations of the world environment play a crucial role in artificial intelligence. It is often inefficient to conduct reasoning and inference directly in the space of raw sensory representations, such as pixel values of images. Representation learning allows us to automatically discover suitable representations from raw sensory data. For example, given raw sensory data, a deep neural network learns nonlinear representations at its hidden layers, which are subsequently used for classification (or regression) at its output layer. This happens implicitly during training through minimizing a supervised or unsupervised loss. In this letter, we study the dynamics of such implicit nonlinear representation learning. We identify a pair of a new assumption and a novel condition, called the on-model structure assumption and the data architecture alignment condition. Under the on-model structure assumption, the data architecture alignment condition is shown to be sufficient for the global convergence and necessary for global optimality. Moreover, our theory explains how and when increasing network size does and does not improve the training behaviors in the practical regime. Our results provide practical guidance for designing a model structure; for example, the on-model structure assumption can be used as a justification for using a particular model structure instead of others. As an application, we then derive a new training framework, which satisfies the data architecture alignment condition without assuming it by automatically modifying any given training algorithm dependent on data and architecture. Given a standard training algorithm, the framework running its modified version is empirically shown to maintain competitive (practical) test performances while providing global convergence guarantees for deep residual neural networks with convolutions, skip connections, and batch normalization with standard benchmark data sets, including MNIST, CIFAR-10, CIFAR-100, Semeion, KMNIST, and SVHN.

Understanding Disproportionate Suspension Practices for Black Students in Elementary, Middle, and High Schools

There are racial and ethnic disparities in use of out-of-school suspensions within the United States. The present study assessed for the presence of disproportionate suspension by race, special education status, and receipt of free or reduced cost meals using two separate metrics (risk ratios and raw differential representation); evaluated separate models of disproportionate suspension for students identified as Black, White, and Hispanic; and examined potential curvilinear associations between the proportion of the racial/ethnic group within the school population and disproportionate suspension. Aggregate data from elementary (n = 27), middle (n = 9), and high (n = 4) U.S. schools with over 105,000 students were included. Results indicated disproportionate suspension was present for Black-identified students, students in special education, and those with socioeconomic difficulties. Metrics of risk ratios and raw differential representation demonstrated somewhat different patterns in disproportionate suspension. We observed a significant curvilinear effect of the proportion of the school body identified as Black versus White on suspension practices. Students identified as White in schools with a larger White student body were more protected from suspension whereas students identified as Black were overrepresented among those suspended regardless of the student body composition. Findings suggest Black-identified students experience differential treatment in school settings and future tested models are encouraged to include more specific teacher and school administrator factors. IMPACT STATEMENT Theory suggests the proportion of the student body identified as Black or White may differentially influence suspension practices for Black and White students. The current study demonstrated support for this theory, with protective effects against suspension for White students in a predominantly White student body. Conversely, Black students were suspended more as the student body became more proportionally Black.

Boosting Graph Embedding on a Single GPU

Graphs are ubiquitous, and they can model unique characteristics and complex relations of real-life systems. Although using machine learning (ML) on graphs is promising, their raw representation is not suitable for ML algorithms. Graph embedding represents each node of a graph as a d-dimensional vector which is more suitable for ML tasks. However, the embedding process is expensive, and CPU-based tools do not scale to real-world graphs. In this work, we present GOSH, a GPU-based tool for embedding large-scale graphs with minimum hardware constraints. GOSH employs a novel graph coarsening algorithm to enhance the impact of updates and minimize the work for embedding. It also incorporates a decomposition schema that enables any arbitrarily large graph to be embedded with a single GPU. As a result, GOSH sets a new state-of-the-art in link prediction both in accuracy and speed, and delivers high-quality embeddings for node classification at a fraction of the time compared to the state-of-the-art. For instance, it can embed a graph with over 65 million vertices and 1.8 billion edges in less than 30 minutes on a single GPU.

Development of an Assistance System for the Intuitive Assessment of Laboratory Findings in Oncology

Abstract Due to the broad spectrum of individual functionalities, current hospital information systems in most cases do not offer quick information access or proper assistance in information assessment. This results in extensive use of raw value representations and the need to process and evaluate those cognitively through the physician. This can result in an error-prone and ineffective process, especially in complex and chronic treatment scenarios. With an example scenario of laboratory value assessment in head and neck oncology, we have evaluated the requirements for extensive support of case interpretation through the conduction of a Delphi study including five medical experts with different levels of expertise. Based on the results, we propose a solution with integrated HL7 FHIR compliant clinical decision-support functionality through a set of integrated rules from relevant clinical practice guidelines. Furthermore, we introduce the implementation of a dedicated application that allows for the definition of custom laboratory value evaluation rules to enable automated and patientindividual value interpretation.

DeepSec meets RawPower - Deep Learning for Detection of Network Attacks Using Raw Representations

The application of machine learning models to the analysis of network traffic measurements has largely grown in recent years. In the networking domain, shallow models are usually applied, where a set of expert handcrafted features are needed to fix the data before training. There are two main problems associated with this approach: firstly, it requires expert domain knowledge to select the input features, and secondly, different sets of custom-made input features are generally needed according to the specific target (e.g., network security, anomaly detection, traffic classification). On the other hand, the power of machine learning models using deep architectures (i.e., deep learning) for networking has not been yet highly explored. In this paper we explore the power of deep learning models on the specific problem of detection of network attacks, using different representations for the input data. As a mayor advantage as compared to the state of the art, we consider raw measurements coming directly from the stream of monitored bytes as the input to the proposed models, and evaluate different raw-traffic feature representations, including packet and flow-level ones.

Boosted Convolutional Decision Trees for Translationally Invariant Pattern Recognition and Transfer Learning

Decision Tree (DT) models provide a well-known class of interpretable machine learning tools for diverse pattern recognition problems. However, applying DTs to learn floating features in images and categorical data based on their raw representation has been challenging. Convolutional Neural Networks (CNNs) are the current state-of-the-art method for classifying raw images, but have their own disadvantages, including that they are often difficult to interpret, have a large number of parameters and hyperparameters, require a fixed image size, and have only partial translational invariance directly built into its architecture. We propose a novel application of Convolutional Decision Trees (CDTs) and show that our approach is more interpretable and can learn higher quality convolutional filters compared to CNNs. CDTs have full translational invariance built into the architecture and can be trained and make predictions on variable-sized images. Using two independent test cases—protein-DNA binding prediction, and hand-written digit classification—we demonstrate that our GPU-enabled implementation of the Cross Entropy (CE) optimization method for training CDTs learns informative convolutional filters that can both facilitate accurate data classifications in a tree-like pattern and be used for transfer learning to improve CNNs themselves. These results motivate further studies on developing accurate and efficient tree-based models for pattern recognition and computer vision.

Vivre Ici: Space, Place and Experiences in Contemporary French Documentary by Alison J. Murray Levine

Reviewed by: Vivre Ici: Space, Place and Experiences in Contemporary French Documentary by Alison J. Murray Levine Martine Guyot-Bender Levine, Alison J. Murray. Vivre Ici: Space, Place and Experiences in Contemporary French Documentary. Liverpool UP, 2018. ISBN 978-1-78694-041-4. Pp. 320. Film documentary is still often perceived as cinema's parent pauvre. However, cinema itself began as a new technology whose purpose was to record life. Today, thanks to smaller cameras, film documentary is flourishing all over the word, most notably in France. Film documentary does not merely record "life caught unaware," as Vertov claimed. It uses the same components as fiction: a camera, lighting, framing, and situations that are assembled into coherent narratives. Apart from nature films, however, documentary is rarely considered as an esthetic experience. In this context, Levine's book fills in an important gap in French film studies in that it moves away from the topic of a small set of films to focus on what matters the most—that documentaries can transmit a sensual experience to the audience. Levine examines films produced over the last twenty years in metropolitan France. However, her analysis can apply to general documentaries, past or future, French or not. For Levine, documentaries are cinematographic, experiential objects. She carefully lays out her theoretical frame, which combines theorists such as Jean-Louis Comolli and Maurice Merleau-Ponty, and a brief review of the recent history of documentary, which she calls "Renaissance?" The question mark suggests that there is no renaissance, that documentary has long been an important genre, but has only recently become a subject of analysis. Levine separates the vast theme of her book into well-defined subsets. The chapter titled "Planet" describes various cinematic strategies used in nature films—that is, the nonhuman dimension of planet Earth—to guide their audience into feeling, in a tactical way. A second chapter, with examples from Depardon, Varda, and Marker, presents road documentaries as the complement of fictional road movies: people who travel and observe their world. Considering the excitement that surrounded Nicolas Philibert's Être et avoir (2002), a chapter focused on the filming of schools was logical. Using a series of recent films on the challenges of multiculturalism in urban schools, Levine shows the French fascination for education. She evokes the tightrope on which filmmakers find themselves when attempting to represent issues such as laïcité, without voyeurism. There is also France's fascination for the transformation of rural life, which, [End Page 194] as Levine shows, is often filmed by the means of portraits, some "dark," some "golden," which contain a strong political stance. The last chapter, "Edge," addresses documentaries that focus on human misery on the margins of society, such as Claude Dextrel's extraordinary Au bord du monde (2013). Levine handles the challenge of introducing a myriad of documentary sub-genres by using a systematic, narrow focus, which will help many viewers and non-specialist educators of cinema to see documentary as cinema, instead of seeing it as a raw representation of reality, which it has never been. Martine Guyot-Bender Hamilton College (NY) Copyright © 2019 American Association of Teachers of French

Tail, Tusk, and Trunk: An Examination of What Different Metrics Reveal about Racial Disproportionality in School Discipline

There are substantial racial disparities in school discipline but little agreement on how best to measure them. The choice of metric can influence fundamental conclusions about the magnitude of racial disproportionality and which interventions are likely to address it. To inform the choice of metrics, this study used discipline data from a national sample of schools (N = 4,512) to examine the strengths and weaknesses of two common (risk ratio, risk difference) and three relatively novel (standardized effect size, raw differential representation, and discipline rate) approaches to assessing racial disproportionality. The raw number of students-of-color differentially disciplined was the most stable metric and captured the widest range of information. None of the metrics captured all relevant aspects of disproportionate discipline. Researchers and policy makers should be as deliberate as possible about their specific aims in measuring disproportionality and carefully select metrics that provide the information most responsive to their goals.

How deep learning extracts and learns leaf features for plant classification

Plant identification systems developed by computer vision researchers have helped botanists to recognize and identify unknown plant species more rapidly. Hitherto, numerous studies have focused on procedures or algorithms that maximize the use of leaf databases for plant predictive modeling, but this results in leaf features which are liable to change with different leaf data and feature extraction techniques. In this paper, we learn useful leaf features directly from the raw representations of input data using Convolutional Neural Networks (CNN), and gain intuition of the chosen features based on a Deconvolutional Network (DN) approach. We report somewhat unexpected results: (1) different orders of venation are the best representative features compared to those of outline shape, and (2) we observe multi-level representation in leaf data, demonstrating the hierarchical transformation of features from lower-level to higher-level abstraction, corresponding to species classes. We show that these findings fit with the hierarchical botanical definitions of leaf characters. Through these findings, we gained insights into the design of new hybrid feature extraction models which are able to further improve the discriminative power of plant classification systems. The source code and models are available at: https://github.com/cs-chan/Deep-Plant.

Refining raw pixel values using a value error model to drive texture synthesis

The goal in photography is generally the construction of a model of scene appearance. Unfortunately, statistical variations introduced by photon shot and other noise introduce errors in the raw value reported for each pixel sample. Rather than simply accepting those values as the best raw representation, the current work treats them as initial estimates of the correct values, and computes an error model for each pixel's value. The value error models for all pixels in an image are then used to drive a type of texture synthesis which refines the pixel value estimates, potentially increasing both accuracy and precision of each value. Each refined raw pixel value is synthesized from the value estimates of a plurality of pixels with overlapping error bounds and similar context within the same image. The error modeling and texture synthesis algorithms are implemented in and evaluated using KREMY (KentuckY Raw Error Modeler, pronounced "creamy"), a free software tool created for this purpose.

Comparing the Machine Ability to Recognize Hand-Written Hindu and Arabic Digits

AbstractThe main aim of this work is to compare Hindu and Arabic digits with respect to a machine’s ability to recognize them. This comparison is done on the raw representation (images) of the digits and on their features extracted using two feature selection methods. Three learning algorithms with different inductive biases were used in the comparison performed using the raw representation; two of them were also used to compare the digits using their extracted features. All classifiers gave better results for Hindu digits in both cases; when raw representation was used and when the selected features where used. The experiments also show that Hindu digits can be classified with better accuracy, higher confidence and using fewer features than Arabic digits. These results indicate that hand-written Hindu digits are actually easier to recognize than hand-written Arabic digits. The machine learning methods used in this work are instance based learning (the kNN algorithm), Naive Bayesian and neural networks. T...

Deep Learning and Developmental Learning: Emergence of Fine-to-Coarse Conceptual Categories at Layers of Deep Belief Network.

In this paper, I investigate conceptual categories derived from developmental processing in a deep neural network. The similarity matrices of deep representation at each layer of neural network are computed and compared with their raw representation. While the clusters generated by raw representation stand at the basic level of abstraction, conceptual categories obtained from deep representation shows a bottom-up transition procedure. Results demonstrate a developmental course of learning from specific to general level of abstraction through learned layers of representations in a deep belief network.

Mesh Convolutional Restricted Boltzmann Machines for Unsupervised Learning of Features With Structure Preservation on 3-D Meshes.

Discriminative features of 3-D meshes are significant to many 3-D shape analysis tasks. However, handcrafted descriptors and traditional unsupervised 3-D feature learning methods suffer from several significant weaknesses: 1) the extensive human intervention is involved; 2) the local and global structure information of 3-D meshes cannot be preserved, which is in fact an important source of discriminability; 3) the irregular vertex topology and arbitrary resolution of 3-D meshes do not allow the direct application of the popular deep learning models; 4) the orientation is ambiguous on the mesh surface; and 5) the effect of rigid and nonrigid transformations on 3-D meshes cannot be eliminated. As a remedy, we propose a deep learning model with a novel irregular model structure, called mesh convolutional restricted Boltzmann machines (MCRBMs). MCRBM aims to simultaneously learn structure-preserving local and global features from a novel raw representation, local function energy distribution. In addition, multiple MCRBMs can be stacked into a deeper model, called mesh convolutional deep belief networks (MCDBNs). MCDBN employs a novel local structure preserving convolution (LSPC) strategy to convolve the geometry and the local structure learned by the lower MCRBM to the upper MCRBM. LSPC facilitates resolving the challenging issue of the orientation ambiguity on the mesh surface in MCDBN. Experiments using the proposed MCRBM and MCDBN were conducted on three common aspects: global shape retrieval, partial shape retrieval, and shape correspondence. Results show that the features learned by the proposed methods outperform the other state-of-the-art 3-D shape features.

Shape-based features for reliable action recognition using spectral regression discriminant analysis

This paper deals with the classification task of human activities in videos with learning using different image sequences as input methods. Image sequences of purely binary silhouette (or shape), that is Raw Silhouette Representation (RSR), distance transform (DT) images of RSR, edge images of RSR, Silhouette History Image (SHI) and Silhouette Energy Image (SEI), image sequences of Wavelet Transform (WT) are used for training and testing using spectral regression discriminant analysis. Hausdorff distance was used for similarity measures to match the embedded action trajectories. Then action classification is achieved in a K-nearest neighbour framework. Using these different input methods we achieved 100% for all the cases except WT cases. From the results, it is evident that SHI and SEI are effective input method in terms of time and space consumption.

Application of time series discretization using evolutionary programming for classification of precancerous cervical lesions

In this work, we present a novel application of time series discretization using evolutionary programming for the classification of precancerous cervical lesions. The approach optimizes the number of intervals in which the length and amplitude of the time series should be compressed, preserving the important information for classification purposes. Using evolutionary programming, the search for a good discretization scheme is guided by a cost function which considers three criteria: the entropy regarding the classification, the complexity measured as the number of different strings needed to represent the complete data set, and the compression rate assessed as the length of the discrete representation. This discretization approach is evaluated using a time series data based on temporal patterns observed during a classical test used in cervical cancer detection; the classification accuracy reached by our method is compared with the well-known times series discretization algorithm SAX and the dimensionality reduction method PCA. Statistical analysis of the classification accuracy shows that the discrete representation is as efficient as the complete raw representation for the present application, reducing the dimensionality of the time series length by 97%. This representation is also very competitive in terms of classification accuracy when compared with similar approaches.

On the clustering of foF2 time series corresponding to disturbed ionospheric periods

This paper presents an analysis of a set of time series that represent foF2 disturbances during storm conditions, using clustering tools. The time series under study have been drawn from ionospheric observations obtained from eight European middle latitude ionosondes during a significant number of storm-time intervals and they are divided into eight groups according to the latitude (middle to low and middle to high) and the local time of the observation point at storm onset (afternoon, evening, morning, prenoon). The time series in each group have been analyzed using clustering-based methods. Specifically, each time series has been represented using two different ways of representation: the first is the raw representation while the second is through the parameters of the autoregressive (AR) model that best represents it. For each representation a hierarchy of clusterings is produced via the complete link algorithm. The two produced hierarchies are combined to a single one and the final clustering results are extracted from the produced hierarchy. The obtained results are in close agreement with the theoretical formulations concerning ionospheric storm effects at middle latitudes. In addition, they may be proved useful in the development of more accurate ionospheric forecasting methods.