High-level Representation Research Articles

Evaluation Model of New Socialist Countryside Construction Based on a Multilayer Neural Network in a Complex Environment.

The creation of a deep neural network model with numerous hidden layers enables the layer-by-layer extraction of features from the input high-dimensional data, enabling the identification of the data's low-dimensional nested structure and the development of a more efficient and abstract high-level representation. The research on deep learning is thoroughly examined, along with the direction it needs to take going forward. Supporting the construction of new socialist rural areas as a calculated move to address the problems facing farming, rural communities, and farmers is another essential step in furthering modernization. It helps to reshape the entire rural landscape, coordinate the growth of urban and rural areas, achieve the goal of a wealthy society in every way, boost demand, and support the holistic development of people. The emergence of new socialist rural communities against the background of deep learning is the subject of significant research and analysis in this work. In the research, it is examined and studied using deep learning methods and convolutional neural networks. It is evident from the research described in this paper that deep learning backgrounds have a considerable impact on the development of new rural areas—up to 54.53%. In this essay, the foundation for the future construction of a new socialist countryside is laid forth.

Dynamic Balanced Domain-Adversarial Networks for Cross-Domain Fault Diagnosis of Train Bearings

Confronting with the problems of inconsistent data distributions, noise interference, and class imbalance in fault diagnosis of train bearings under variable working conditions, this paper proposes a cross-domain fault diagnosis model called dynamic balanced domain-adversarial networks (DBDAN). The major contributions of the proposed DBDAN are that, a novel frequency band attention module (FBAM) with a clear physical interpretation is embedded in the feature extraction of the DBDAN to alleviate the noise interference, and an adaptive class-level weighting strategy is designed for the imbalanced dataset to tackle the class imbalance problem. Specifically, the FBAM highlights the time-frequency contents in the frequency bands containing state information by adaptively learning the time-frequency characteristics of bearing vibration signals. Then, high-level representations are extracted by the following feature extractor, which are used for fault recognition by the label classifier, and alignments of marginal and conditional distributions across domains by the domain discriminator. The adaptive class-level weighting strategy is used to dynamically balance the distribution alignments between the imbalanced classes. The DBDAN model employs Wasserstein distance to measure the distribution discrepancy, and is trained in an adversarial way. The experiment results of cross-domain fault diagnosis of a common rolling bearing and a train axle box bearing under variable working conditions verify the superiority of the proposed method over other domain-adversarial methods in terms of diagnostic accuracy, convergence speed and stability.

Remaining Useful Life Prediction of Machinery: A New Multiscale Temporal Convolutional Network Framework

With the rapid development of deep learning (DL) techniques, data-driven models have been increasingly used in remaining useful life (RUL) prediction, in which convolution neural network (CNN)-based RUL prognostics models have received special attention. However, there are still two main issues that need to be addressed: (1) Traditional CNN is not suitable to extract the time-sequence characteristics from the long-term historical signals. (2) The receptive field range of convolution operation is fixed, thus only learning the feature information at a specific scale, which is insufficient for complex feature extraction. To address these two issues, a multi-scale temporal convolutional network (MsTCN) which has powerful time-sequence characteristics is proposed for RUL prediction in this paper. The MsTCN adopts the temporal convolutional network (TCN) framework, which is good at extracting time-sequence information. Based on this, a new multi-scale dilated causal convolution residual block (MsDCCRB) is developed to constitute the RUL prognostics model, where multiple dilated convolutions (DCs) are based on different dilation factors are put on each layer in parallel. Furthermore, the squeeze-and-excitation (SE) unit is embedded into the MsDCCRB to adaptively recalibrate the sequence feature responses and enhance the representation learning ability. Through stacking multiple MsDCCRBs, the historical condition monitoring data can be fed directly into the proposed model to realize the high-level representations of RUL estimation. Finally, the proposed approach is validated with the accelerated whole-life degradation dataset of rolling element bearings (REBs). The experimental results exhibit the proposed MsTCN achieves a higher RUL prediction accuracy, which is superior to some state-of-the-art data-driven prognostics methods.

Perceptual Loss-Constrained Adversarial Autoencoder Networks for Hyperspectral Unmixing

Recently, the use of a deep autoencoder-based method in blind spectral unmixing has attracted great attention as the method can achieve superior performance. However, most autoencoder-based unmixing methods use non-structured reconstruction loss to train networks, leading to the ignorance of band-to-band-dependent characteristics and fine-grained information. To cope with this issue, we propose a general perceptual loss-constrained adversarial autoencoder network for hyperspectral unmixing. Specifically, the adversarial training process is used to update our framework. The discriminate network is found to be efficient in discovering the discrepancy between the reconstructed pixels and their corresponding ground truth. Moreover, the general perceptual loss is combined with the adversarial loss to further improve the consistency of high-level representations. Ablation studies verify the effectiveness of the proposed components of our framework, and experiments with both synthetic and real data illustrate the superiority of our framework when compared with other competing methods.

Forecasting Fine-Grained Urban Flows Via Spatio-Temporal Contrastive Self-Supervision

As a critical task of the urban traffic services, fine-grained urban flow inference (FUFI) benefits in many fields including intelligent transportation management, urban planning, public safety. FUFI is a technique that focuses on inferring fine-grained urban flows depending solely on observed coarse-grained data. However, existing methods always require massive learnable parameters and the complex network structures. To reduce these defects, we formulate a contrastive self-supervision method to predict fine-grained urban flows taking into account all correlated spatial and temporal contrastive patterns. Through several well-designed self-supervised tasks, uncomplicated networks have a strong ability to capture high-level representations from flow data. Then, a fine-tuning network combining with three pre-training encoder networks is proposed. We conduct experiments to evaluate our model and compare with other state-of-the-art methods by using two real-world datasets. All the empirical results not only show the superiority of our model against other comparative models, but also demonstrate its effectiveness in the resource-limited environment.

Semantic enrichment approach for low-level CAD models managed in PLM context: Literature review and research prospect

The proposed approach in the paper is dedicated to enrichment of CAD models storage in PLM (Product Lifecycle Management) systems or archive databases. The paper considers low level CAD models (i.e. frozen geometry and without CAD model tree) based on standards such as STL, IGES and STEP AP203 first edition where the CAD model tree is absent. The paper also highlights the differences in semantic richness between these standards and their degrees of industrial implementation. STEP AP242 standard is considered as a high level representation for CAD files. The paper aims to review the literature that addresses the methods for semantic enrichment of CAD models by using ontologies. The future challenges and one possible research direction are then discussed. A first application called VAQUERO for CAD enrichment using an ontology based on STEP AP242 standard is proposed.

Change Detection in Synthetic Aperture Radar Images Based on a Generalized Gamma Deep Belief Networks.

Change detection from synthetic aperture radar (SAR) images is of great significance for natural environmental protection and human societal activity, which can be regarded as the process of assigning a class label (changed or unchanged) to each of the image pixels. This paper presents a novel classification technique to address the SAR change-detection task that employs a generalized Gamma deep belief network (gΓ-DBN) to learn features from difference images. We aim to develop a robust change detection method that can adapt to different types of scenarios for bitemporal co-registered Yellow River SAR image data set. This data set characterized by different looks, which means that the two images are affected by different levels of speckle. Widely used probability distributions offer limited accuracy for describing the opposite class pixels of difference images, making change detection entail greater difficulties. To address the issue, first, a gΓ-DBN can be constructed to extract the hierarchical features from raw data and fit the distribution of the difference images by means of a generalized Gamma distribution. Next, we propose learning the stacked spatial and temporal information extracted from various difference images by the gΓ-DBN. Consequently, a joint high-level representation can be effectively learned for the final change map. The visual and quantitative analysis results obtained on the Yellow River SAR image data set demonstrate the effectiveness and robustness of the proposed method.

Natural Language Processing and Computational Linguistics

Natural Language Processing and Computational Linguistics

Automatic Sequence-Based Network for Lung Diseases Detection in Chest CT.

ObjectiveTo develop an accurate and rapid computed tomography (CT)-based interpretable AI system for the diagnosis of lung diseases.BackgroundMost existing AI systems only focus on viral pneumonia (e.g., COVID-19), specifically, ignoring other similar lung diseases: e.g., bacterial pneumonia (BP), which should also be detected during CT screening. In this paper, we propose a unified sequence-based pneumonia classification network, called SLP-Net, which utilizes consecutiveness information for the differential diagnosis of viral pneumonia (VP), BP, and normal control cases from chest CT volumes.MethodsConsidering consecutive images of a CT volume as a time sequence input, compared with previous 2D slice-based or 3D volume-based methods, our SLP-Net can effectively use the spatial information and does not need a large amount of training data to avoid overfitting. Specifically, sequential convolutional neural networks (CNNs) with multi-scale receptive fields are first utilized to extract a set of higher-level representations, which are then fed into a convolutional long short-term memory (ConvLSTM) module to construct axial dimensional feature maps. A novel adaptive-weighted cross-entropy loss (ACE) is introduced to optimize the output of the SLP-Net with a view to ensuring that as many valid features from the previous images as possible are encoded into the later CT image. In addition, we employ sequence attention maps for auxiliary classification to enhance the confidence level of the results and produce a case-level prediction.ResultsFor evaluation, we constructed a dataset of 258 chest CT volumes with 153 VP, 42 BP, and 63 normal control cases, for a total of 43,421 slices. We implemented a comprehensive comparison between our SLP-Net and several state-of-the-art methods across the dataset. Our proposed method obtained significant performance without a large amount of data, outperformed other slice-based and volume-based approaches. The superior evaluation performance achieved in the classification experiments demonstrated the ability of our model in the differential diagnosis of VP, BP and normal cases.

Sensing the Sounds of Silence: A Pilot Study on the Detection of Model Mice of Autism Spectrum Disorder from Ultrasonic Vocalisations.

Studying the animal models of human neuropsychiatric disorders can facilitate the understanding of mechanisms of symptoms both physiologically and genetically. Previous studies have shown that ultrasonic vocalisations (USVs) of mice might be efficient markers to distinguish the wild type group and the model of autism spectrum disorder (mASD). Nevertheless, in-depth analysis of these 'silence' sounds by leveraging the power of advanced computer audition technologies (e. g., deep learning) is limited. To this end, we propose a pilot study on using a large-scale pre-trained audio neural network to extract high-level representations from the USVs of mice for the task on detection of mASD. Experiments have shown a best result reaching an unweighted average recall of 79.2 % for the binary classification task in a rigorous subject-independent scenario. To the best of our knowledge, this is the first time to analyse the sounds that cannot be heard by human beings for the detection of mASD mice. The novel findings can be significant to motivate future works with according means on studying animal models of human patients.

A parallel attention-augmented bilinear network for early magnetic resonance imaging-based diagnosis of Alzheimer's disease.

Structural magnetic resonance imaging (sMRI) can capture the spatial patterns of brain atrophy in Alzheimer's disease (AD) and incipient dementia. Recently, many sMRI‐based deep learning methods have been developed for AD diagnosis. Some of these methods utilize neural networks to extract high‐level representations on the basis of handcrafted features, while others attempt to learn useful features from brain regions proposed by a separate module. However, these methods require considerable manual engineering. Their stepwise training procedures would introduce cascading errors. Here, we propose the parallel attention‐augmented bilinear network, a novel deep learning framework for AD diagnosis. Based on a 3D convolutional neural network, the framework directly learns both global and local features from sMRI scans without any prior knowledge. The framework is lightweight and suitable for end‐to‐end training. We evaluate the framework on two public datasets (ADNI‐1 and ADNI‐2) containing 1,340 subjects. On both the AD classification and mild cognitive impairment conversion prediction tasks, our framework achieves competitive results. Furthermore, we generate heat maps that highlight discriminative areas for visual interpretation. Experiments demonstrate the effectiveness of the proposed framework when medical priors are unavailable or the computing resources are limited. The proposed framework is general for 3D medical image analysis with both efficiency and interpretability.

Autonomous underwater robot for underwater image enhancement via multi-scale deformable convolution network with attention mechanism

Underwater images suffer from poor visibility quality, which results from selective attenuation and scattering. These interdependence phenomena together cause image degradation, failing autonomous underwater robots to recognize image contents. To address those problems, we propose a multi-scale deformable convolution network with an attention mechanism (MsDCANet) to enhance the quality of underwater images. The proposed model is generally implemented by an encoder-decoder architecture. Concretely, we first propose a multi-scale deformable convolutional network, which acquires deformable local receptive fields at different scales and enriches the diversity of feature representation. Coupled with spatial and channel attention mechanisms, the task-oriented high-level representations acquired from input feature maps, and the most significant features are highlighted. Considering the contributions of the encoder and decoder functions on the final enhancement task, an attention-based skip fusion scheme is designed to improve the original skip connection, which aims to reconstruct higher quality underwater images. Finally, the proposed model is optimized by a multi-task loss function, including pixel loss and perceptual loss. Experiments on underwater images captured under diverse scenes demonstrate that the proposed model produces visually pleasing results and has strong generalization ability, even significantly outperforming several state-of-the-art methods. Besides, our approach can also improve the performance of vision tasks and be applied to aquaculture development.

Behavioral and neural representations en route to intuitive action understanding

When we observe another person's actions, we process many kinds of information – from how their body moves to the intention behind their movements. What kinds of information underlie our intuitive understanding about how similar actions are to each other? To address this question, we measured the intuitive similarities among a large set of everyday action videos using multi-arrangement experiments, then used a modeling approach to predict this intuitive similarity space along three hypothesized properties. We found that similarity in the actors' inferred goals predicted the intuitive similarity judgments the best, followed by similarity in the actors' movements, with little contribution from the videos' visual appearance. In opportunistic fMRI analyses assessing brain-behavior correlations, we found suggestive evidence for an action processing hierarchy, in which these three kinds of action similarities are reflected in the structure of brain responses along a posterior-to-anterior gradient on the lateral surface of the visual cortex. Altogether, this work joins existing literature suggesting that humans are naturally tuned to process others' intentions, and that the visuo-motor cortex computes the perceptual precursors of the higher-level representations over which intuitive action perception operates.

Dynamic Electrode-to-Image (DETI) mapping reveals the human brain’s spatiotemporal code of visual information

A number of neuroimaging techniques have been employed to understand how visual information is transformed along the visual pathway. Although each technique has spatial and temporal limitations, they can each provide important insights into the visual code. While the BOLD signal of fMRI can be quite informative, the visual code is not static and this can be obscured by fMRI’s poor temporal resolution. In this study, we leveraged the high temporal resolution of EEG to develop an encoding technique based on the distribution of responses generated by a population of real-world scenes. This approach maps neural signals to each pixel within a given image and reveals location-specific transformations of the visual code, providing a spatiotemporal signature for the image at each electrode. Our analyses of the mapping results revealed that scenes undergo a series of nonuniform transformations that prioritize different spatial frequencies at different regions of scenes over time. This mapping technique offers a potential avenue for future studies to explore how dynamic feedforward and recurrent processes inform and refine high-level representations of our visual world.

CN-waterfall: a deep convolutional neural network for multimodal physiological affect detection

Affective computing solutions, in the literature, mainly rely on machine learning methods designed to accurately detect human affective states. Nevertheless, many of the proposed methods are based on handcrafted features, requiring sufficient expert knowledge in the realm of signal processing. With the advent of deep learning methods, attention has turned toward reduced feature engineering and more end-to-end machine learning. However, most of the proposed models rely on late fusion in a multimodal context. Meanwhile, addressing interrelations between modalities for intermediate-level data representation has been largely neglected. In this paper, we propose a novel deep convolutional neural network, called CN-Waterfall, consisting of two modules: Base and General. While the Base module focuses on the low-level representation of data from each single modality, the General module provides further information, indicating relations between modalities in the intermediate- and high-level data representations. The latter module has been designed based on theoretically grounded concepts in the Explainable AI (XAI) domain, consisting of four different fusions. These fusions are mainly tailored to correlation- and non-correlation-based modalities. To validate our model, we conduct an exhaustive experiment on WESAD and MAHNOB-HCI, two publicly and academically available datasets in the context of multimodal affective computing. We demonstrate that our proposed model significantly improves the performance of physiological-based multimodal affect detection.

Representing Integer Sequences Using Piecewise-Affine Loops

A formal, high-level representation of programs is typically needed for static and dynamic analyses performed by compilers. However, the source code of target applications is not always available in an analyzable form, e.g., to protect intellectual property. To reason on such applications, it becomes necessary to build models from observations of its execution. This paper details an algebraic approach which, taking as input the trace of memory addresses accessed by a single memory reference, synthesizes an affine loop with a single perfectly nested reference that generates the original trace. This approach is extended to support the synthesis of unions of affine loops, useful for minimally modeling traces generated by automatic transformations of polyhedral programs, such as tiling. The resulting system is capable of processing hundreds of gigabytes of trace data in minutes, minimally reconstructing 100% of the static control parts in PolyBench/C applications and 99.99% in the Pluto-tiled versions of these benchmarks. As an application example of the trace modeling method, trace compression is explored. The affine representations built for the memory traces of PolyBench/C codes achieve compression factors of the order of 106 and 103 with respect to gzip for the original and tiled versions of the traces, respectively.

Exploiting symmetries of high-level Petri games in distributed synthesis

Abstract Distributed Synthesis is the problem of automatically generating correct controllers for individual agents in a distributed system. Petri games model this problem by a game between two teams of players on a Petri net structure. Under some restrictions, Petri games can be solved by a reduction to a two player game. The concept of symmetries in Petri nets is closely related to high-level representations of Petri games. Applying symmetries to the states in the two-player game results in a significant state space reduction. We give an overview about (high-level) Petri games and the application of symmetries in this setting. We present ongoing work aiming to concisely describe solutions of Petri games by a high-level representation.

A model-based reinforcement learning method based on conditional generative adversarial networks

Deep reinforcement learning (DRL) integrates the advantages of the perception of deep learning and enables reinforcement learning scale to problems with high dimensional state and action spaces that were previously intractable. The success of DRL primarily relies on the high level representation ability of deep learning. To obtain a good performed representation model, excessive training samples and training time are necessary. However, collecting a large number of samples in real world is extremely expensive and time consuming. To mitigate the sample inefficiency problem, we propose a novel model-based reinforcement learning method by combining conditional generative adversarial networks (CGAN-MbRL) with the state-of-the-art policy learning method. The proposed CGAN-MbRL can directly deal with the high dimensional state, and mitigate the problem of sample inefficiency to some extent. Finally, the effectiveness of the proposed method is demonstrated through the illustrative data and the RL benchmark.

Representation learning with collaborative autoencoder for personalized recommendation

In the past decades, recommendation systems have provided lots of valuable personalized suggestions for the users to address the problem of information over-loaded. Collaborative Filtering (CF) is one of the most commonly applied and successful recommendation approaches, which refers to using the preferences of groups with similar interests to recommend information to other users. Recently, in addition to the traditional matrix factorization techniques, deep learning methods have been proposed to learn more abstract and higher-level representations for recommendation. However, most previous deep recommendation methods learn the higher-level feature representations of users and items through an identical model structure, which ignores the different characteristics of the user-based and item-based data. In addition, the rating matrix is usually sparse which may result in a significant degradation of recommendation performance. To address these problems, we propose a representation learning method with Collaborative Autoencoder for Personalized Recommendation (CAPR for short). In this method, user-based and item-based feature representations are learned by two different autoencoders for capturing different features of the data. Meanwhile, items’ attributions are combined into the feature representations with semi-autoencoder for alleviating the sparsity problem. Extensive experimental results confirm the effectiveness of our proposed method compared to other state-of-the-art matrix factorization methods and deep recommendation methods.

An Expert-Driven Probabilistic Assessment of the Safety and Security of Offshore Wind Farms

Offshore wind farms (OWFs) are important infrastructure which provide an alternative and clean means of energy production worldwide. The offshore wind industry has been continuously growing. Over the years, however, it has become evident that OWFs are facing a variety of safety and security challenges. If not addressed, these issues may hinder their progress. Based on these safety and security goals and on a Bayesian network model, this work presents a methodological approach for structuring and organizing expert knowledge and turning it into a probabilistic model to assess the safety and security of OWFs. This graphical probabilistic model allowed us to create a high-level representation of the safety and security state of a generic OWF. By studying the interrelations between the different functions of the model, and by proposing different scenarios, we determined the impacts that a failing function may have on other functions in this complex system. Finally, this model helped us define the performance requirements of such infrastructure, which should be beneficial for optimizing operation and maintenance.