Hierarchical Feature Extractor Research Articles

Comprehensive study on building chiller fault feature and diagnosis using hybrid CNN

Chillers are one of the biggest energy consumption devices in HVAC systems. Abnormal operation may undermine the performance, energy efficiency, and environment. This study comprehensively explores the hybrid applications of deep convolutional neural network (CNN) in chiller fault diagnosis and fault feature extraction. Unlike the computer vision where the feature locations are fixed, for chiller fault, it can be changed. The effect of feature sequence on diagnostic performance is carefully investigated, and found that it depends on the size and number of the convolution kernels. Small and large number of kernels may extract fine and enough features for the model to counter the location change and maintain the basic characteristics of the faults. 1-D CNN is further studied as a hierarchical feature extractor combined with the traditional machine learning, like k-nearest neighbor (KNN), decision tree (DT), and random forest (RF), to build a hybrid strategy. It is found that the highest accuracy of 99.85% is achieved by RF plus 1-D CNN with an accuracy of 100% for refrigerant leakage. Fine and clear features from deeper structure are most favorable for the weak learner like DT, but may harm the information diversity of RF and lower its accuracy.

Image fusion via hierarchical extractor and maximum feature distribution learning

Most existing learning-based image fusion solutions are prone to discard important high-frequency information such as edges and detail textures of the image due to they tend to build deeper network structures. In this work, we report a performance-efficient model for fusing infrared and visible images that combines the idea of GAN architecture with the prior knowledge of edge and intensity. We innovatively introduce the maximum edge and maximum intensity information of the paired dual-source images directly into the hierarchical feature extractor, which can diversify the inputs and mitigate the loss of intermediate information during feature extraction process. Considering that features extracted from different encoding branches contain lots of redundant information, fused images are reconstructed via the decoder after preliminarily merging of locally encoded features using the attention mechanism. The two discriminators are employed to determine the authenticity of the fusion results compared with maximum feature map rather than images themselves, thus the generated images will both contain more intensity information and edge details. A prior information-based hybrid loss function is designed to guide the proposed model to learn the maximum feature distribution of dual-source image, rather than being biased towards learning the distribution of one source image or the mean distribution of two source images, thereby improving the reconstruction performance. Extensive evaluations demonstrate that our method performs better than the leading algorithms on the public datasets in both qualitative and quantitative terms. Our fusion results not only present abundant textures, brighter thermal targets, and high-contrast appearance, but also eliminate unpleasant halo artifacts along edges.

Survival Prediction via Hierarchical Multimodal Co-Attention Transformer: A Computational Histology-Radiology Solution.

The rapid advances in deep learning-based computational pathology and radiology have demonstrated the promise of using whole slide images (WSIs) and radiology images for survival prediction in cancer patients. However, most image-based survival prediction methods are limited to using either histology or radiology alone, leaving integrated approaches across histology and radiology relatively underdeveloped. There are two main challenges in integrating WSIs and radiology images: (1) the gigapixel nature of WSIs and (2) the vast difference in spatial scales between WSIs and radiology images. To address these challenges, in this work, we propose an interpretable, weakly-supervised, multimodal learning framework, called Hierarchical Multimodal Co-Attention Transformer (HMCAT), to integrate WSIs and radiology images for survival prediction. Our approach first uses hierarchical feature extractors to capture various information including cellular features, cellular organization, and tissue phenotypes in WSIs. Then the hierarchical radiology-guided co- attention (HRCA) in HMCAT characterizes the multimodal interactions between hierarchical histology-based visual concepts and radiology features and learns hierarchical co- attention mappings for two modalities. Finally, HMCAT combines their complementary information into a multimodal risk score and discovers prognostic features from two modalities by multimodal interpretability. We apply our approach to two cancer datasets (365 WSIs with matched magnetic resonance [MR] images and 213 WSIs with matched computed tomography [CT] images). Our results demonstrate that the proposed HMCAT consistently achieves superior performance over the unimodal approaches trained on either histology or radiology data alone, as well as other state-of-the-art methods.

A deep learning method for autism spectrum disorder identification based on interactions of hierarchical brain networks

BackgroundIt has been recently shown that deep learning models exhibited remarkable performance of representing functional Magnetic Resonance Imaging (fMRI) data for the understanding of brain functional activities. With hierarchical structure, deep learning models can infer hierarchical functional brain networks (FBN) from fMRI. However, the applications of the hierarchical FBNs have been rarely studied. MethodsIn this work, we proposed a hierarchical recurrent variational auto-encoder (HRVAE) to unsupervisedly model the fMRI data. The trained HRVAE encoder can predict hierarchical temporal features from its three hidden layers, and thus can be regarded as a hierarchical feature extractor. Then LASSO (least absolute shrinkage and selection operator) regression was applied to estimate the corresponding hierarchical FBNs. Based on the hierarchical FBNs from each subject, we constructed a novel classification framework for brain disorder identification and test it on the Autism Brain Imaging Data Exchange (ABIDE) dataset, a world-wide multi-site database of autism spectrum disorder (ASD). We analyzed the hierarchy organization of FBNs, and finally used the overlaps of hierarchical FBNs as features to differentiate ASD from typically developing controls (TDC). ResultsThe experimental results on 871 subjects from ABIDE dataset showed that the HRVAE model can effectively derive hierarchical FBNs including many well-known resting state networks (RSN). Moreover, the classification result improved the state-of-the-art by achieving a very high accuracy of 82.1 %. ConclusionsThis work presents a novel data-driven deep learning method using fMRI data for ASD identification, which could provide valuable reference for clinical diagnosis. The classification results suggest that the interactions of hierarchical FBNs have association with brain disorder, which promotes the understanding of FBN hierarchy and could be applied to other brain disorder analysis.

Mixed-precision quantized neural networks with progressively decreasing bitwidth

Efficient model inference is an important and practical issue in the deployment of deep neural networks on resource constraint platforms. Network quantization addresses this problem effectively by leveraging low-bit representation and arithmetic that could be conducted on dedicated embedded systems. In the previous works, the parameter bitwidth is set homogeneously and there is a trade-off between superior performance and aggressive compression. Actually, the stacked network layers, which are generally regarded as hierarchical feature extractors, contribute diversely to the overall performance. For a well-trained neural network, the feature distributions of different categories are organized gradually as the network propagates forward. Hence the capability requirement on the subsequent feature extractors is reduced. It indicates that the neurons in posterior layers could be assigned with lower bitwidth for quantized neural networks. Based on this observation, a simple yet effective mixed-precision quantized neural network with progressively decreasing bitwidth is proposed to improve the trade-off between accuracy and compression. Extensive experiments on typical network architectures and benchmark datasets demonstrate that the proposed method could achieve better or comparable results while reducing the memory space for quantized parameters by more than 25% in comparison with the homogeneous counterparts. In addition, the results also demonstrate that the higher-precision bottom layers could boost the 1-bit network performance appreciably due to a better preservation of the original image information while the lower-precision posterior layers contribute to the regularization of k−bit networks.

Dual Sum-Product Networks Autoencoder for Multi-Label Classification

Sum-product networks (SPNs) are an expressive deep probabilistic architecture with solid theoretical foundations, which allows tractable and exact inference. SPNs always act as black-box inference machine in many artificial intelligence tasks. Due to their recursive definition, SPNs can also be naturally employed as hierarchical feature extractors. Recently, SPNs have been successfully employed as autoencoder framework in representation learning. However, SPNs autoencoder ignores the model structural duality and trains the models separately and independently. In this work, we propose a Dual-SPNs autoencoder which designs two SPNs autoencoders to compose as a dual form. This approach trains the models simultaneously, and explicitly exploits the structural duality between them to enhance the training process. Experimental results on several multilabel classification problems demonstrate that Dual-SPNs autoencoder is very competitive against with state-of-the-art autoencoder architectures.

Hierarchical symbolic dynamic filtering of streaming non-stationary time series data

Abstract This paper proposes a hierarchical feature extractor for non-stationary streaming time series based on the concept of switching observable Markov chain models. The slow time-scale non-stationary behaviors are considered to be a mixture of quasi-stationary fast time-scale segments that are exhibited by complex dynamical systems. The key idea is to model each unique stationary characteristics without apriori knowledge (e.g., number of possible unique characteristics) at a lower logical level, and capture the transitions from one low-level model to another at a higher level. In this context, the concepts in the recently developed Symbolic Dynamic Filtering is extended, to build an online algorithm suited for handling quasi-stationary data at a lower level and a non-stationary behavior at a higher level without apriori knowledge. A key observation made in this study is that the rate of change of data likelihood seems to be a better indicator of change in data characteristics compared to the traditional methods that mostly consider data likelihood for change detection. Thus, an adaptive Chinese Restaurant Process (CRP) distribution is formulated to accommodate the rate of change of data likelihood in the learning algorithm. The algorithm also aims to minimize model complexity while capturing data characteristics (likelihood). Efficacy demonstration and comparative evaluation of the proposed algorithm are performed using time series data both from a simulated system that exhibit nonlinear dynamics as well as a real residential electrical energy disaggregation case study using the publicly available reference energy disaggregation dataset (REDD). We discuss results that show that the proposed hierarchical symbolic dynamic filtering (HSDF) algorithm can identify underlying features with significantly high degree of accuracy, even under very noisy conditions. Based on our validation experiments, we demonstrate better performance of the proposed fast algorithm compared to the baseline Hierarchical Dirichlet Process-Hidden Markov Models (HDP-HMM). The proposed algorithm’s low computational complexity also makes it suitable for on-board, real time operations.

Decoding of finger trajectory from ECoG using deep learning

Objective. Conventional decoding pipeline for brain–machine interfaces (BMIs) consists of chained different stages of feature extraction, time-frequency analysis and statistical learning models. Each of these stages uses a different algorithm trained in a sequential manner, which makes it difficult to make the whole system adaptive. The goal was to create an adaptive online system with a single objective function and a single learning algorithm so that the whole system can be trained in parallel to increase the decoding performance. Here, we used deep neural networks consisting of convolutional neural networks (CNN) and a special kind of recurrent neural network (RNN) called long short term memory (LSTM) to address these needs. Approach. We used electrocorticography (ECoG) data collected by Kubanek et al. The task consisted of individual finger flexions upon a visual cue. Our model combined a hierarchical feature extractor CNN and a RNN that was able to process sequential data and recognize temporal dynamics in the neural data. CNN was used as the feature extractor and LSTM was used as the regression algorithm to capture the temporal dynamics of the signal. Main results. We predicted the finger trajectory using ECoG signals and compared results for the least angle regression (LARS), CNN-LSTM, random forest, LSTM model (LSTM_HC, for using hard-coded features) and a decoding pipeline consisting of band-pass filtering, energy extraction, feature selection and linear regression. The results showed that the deep learning models performed better than the commonly used linear model. The deep learning models not only gave smoother and more realistic trajectories but also learned the transition between movement and rest state. Significance. This study demonstrated a decoding network for BMI that involved a convolutional and recurrent neural network model. It integrated the feature extraction pipeline into the convolution and pooling layer and used LSTM layer to capture the state transitions. The discussed network eliminated the need to separately train the model at each step in the decoding pipeline. The whole system can be jointly optimized using stochastic gradient descent and is capable of online learning.

Visual tracking via shallow and deep collaborative model

In this paper, we propose a robust tracking method based on the collaboration of a generative model and a discriminative classifier, where features are learned by shallow and deep architectures, respectively. For the generative model, we introduce a block-based incremental learning scheme, in which a local binary mask is constructed to deal with occlusion. The similarity degrees between the local patches and their corresponding subspace are integrated to formulate a more accurate global appearance model. In the discriminative model, we exploit the advances of deep learning architectures to learn generic features which are robust to both background clutters and foreground appearance variations. To this end, we first construct a discriminative training set from auxiliary video sequences. A deep classification neural network is then trained offline on this training set. Through online fine-tuning, both the hierarchical feature extractor and the classifier can be adapted to the appearance change of the target for effective online tracking. The collaboration of these two models achieves a good balance in handling occlusion and target appearance change, which are two contradictory challenging factors in visual tracking. Both quantitative and qualitative evaluations against several state-of-the-art algorithms on challenging image sequences demonstrate the accuracy and the robustness of the proposed tracker.

Automated intelligent system for sound signalling device quality assurance

This paper presents a novel approach to the detection and recognition of faulty audio signalling devices as part of an automated industrial manufacturing quality assurance process. The proposed system outperforms other well-established automated systems based on mel-frequency cepstrum coefficients (MFCC) and multi-layer perceptron (MLP). It uses both unlabelled sound data and labelled historical data acquired from human experts in detecting faulty signalling devices. The unlabelled data is used to train a deep neural network generative model to create multiple levels of hierarchical feature extractors which are used to train an MLP classifier, with the intent to model the human reasoning and judging processes in respect to sound classification. This paper presents the results of real world experiments based on data pertaining to the audio signalling quality assurance process for car instrument cluster manufacturing. These results show that the proposed system is able to successfully classify speakers into two groups: “Good” and “No good” depending on the part quality. The proposed system proves to be capable enough to eliminate the need for a manual inspection within the manufacturing process and is shown to be able to diagnose a fault with a high degree of accuracy. This work can be extended to other areas of automotive inspection where there is a need for a robust solution to sound detection and where an output signal is represented by a complex and changing frequency spectrum even with significant environmental noise.

Hierarchical retinal blood vessel segmentation based on feature and ensemble learning

Segmentation of retinal blood vessels is of substantial clinical importance for diagnoses of many diseases, such as diabetic retinopathy, hypertension and cardiovascular diseases. In this paper, the supervised method is presented to tackle the problem of retinal blood vessel segmentation, which combines two superior classifiers: Convolutional Neural Network (CNN) and Random Forest (RF). In this method, CNN performs as a trainable hierarchical feature extractor and ensemble RFs work as a trainable classifier. By integrating the merits of feature learning and traditional classifier, the proposed method is able to automatically learn features from the raw images and predict the patterns. Extensive experiments have been conducted on two public retinal images databases (DRIVE and STARE), and comparisons with other major studies on the same database demonstrate the promising performance and effectiveness of the proposed method. A supervised method based on feature and ensemble learning is proposed.The whole pipeline of the proposed method is automatic and trainable.Convolutional Neural Network performs as a trainable hierarchical feature extractor.Ensemble Random Forests work as a trainable classifier.Compared with state-of-the-arts, the experimental results are promising.