Low-dimensional Feature Space Research Articles

Dlsa: Semi-supervised partial label learning via dependence-maximized label set assignment

Semi-supervised partial label learning is an emerging weakly supervised learning paradigm dealing with partially labeled data and unlabeled data simultaneously. The supervision information is weak and even missing for training instances. Most existing researches simply propagate labels to unlabeled data and focus on inducing a weighted classifier model, ignoring the importance of acquiring reliable label confidences for training instances. In this paper, by label set assignment, the valid label supervision information can be preserved and propagated to unlabeled instances to a greater extent. Then, the training instances are projected into a lower-dimensional feature space. Reliable label confidence can be obtained by maximizing the dependence between the feature description and the associated label information. After that, the predictive model can be induced by preserving the intrinsic manifold structure in both feature space and label space. Accordingly, a novel semi-supervised partial label learning approach via dependence-maximized label set assignment (Dlsa) is proposed, where an accurate and robust model can be induced based on reliable label confidences obtained by label set assignment and dependence-maximized dimensionality reduction. Extensive experiments on five real-world data sets validate the effectiveness and superiority of the proposed method.

Fifth Generation Mobile Communication Technology Network Attack Defense Based on Software Defined network Technology in Power Internet of Things

The purpose is to guarantee the security of fifth generation mobile communication technology (5G) network in power Internet of Things environment and improve the ability of wireless network communication to resist attacks. First, in terms of attack prevention, the 5G network security structure is proposed to replace the plaintext information commonly used in the original system with Ciphertext based on software defined network (SDN), thereby alleviating the security risks of the data dimension. Second, concerning attack detection, the signal is identified by using the imperfections and differences of equipment manufacturing based on the above security structure, preventing the attacker from further harming the sensitive data leaked. Researchers found that the SDN-based 5G network attack prevention scheme avoids the centralized exposure of sensitive data, improves security, reduces computational overhead, and simplifies encryption logic. Without affecting the bandwidth, the existing 5G network system is greatly prevented from attacks. The detection mechanism is not limited by the low-dimensional feature space, and it demonstrates strong robustness and stability. It can effectively detect the attacks on 5G network system in power Internet of Things (IoT). This study provides an important reference for the protection of 5G communication network attacks under IoT.

An Efficient Epileptic Seizure Detection Technique using Discrete Wavelet Transform and Machine Learning Classifiers

This paper presents an epilepsy detection method based on discrete wavelet transform (DWT) with Machine learning classifiers. Here DWT has been used for feature extraction as it provides a better decomposition of the signals in different frequency bands. At first, DWT has been applied to the EEG signal to extract the detail and approximate coefficients or different sub-bands. After the extraction of the coefficients, Principal component analysis (PCA) has been applied on different sub-bands and then a feature level fusion technique is used to extract the main features in low dimensional feature space. Three classifiers name: Support Vector Machine (SVM) classifier, K-Nearest-Neighbor (KNN) classifier, and Naive Bayes (NB) classifier have been used in the proposed work for classifying the EEG signals. The raised method is tested over Bonn databases and provides a maximum of 100% recognition accuracy for KNN, SVM, NB classifiers.

Interpretable modeling of genotype–phenotype landscapes with state-of-the-art predictive power

Large-scale measurements linking genetic background to biological function have driven a need for models that can incorporate these data for reliable predictions and insight into the underlying biophysical system. Recent modeling efforts, however, prioritize predictive accuracy at the expense of model interpretability. Here, we present LANTERN (landscape interpretable nonparametric model, https://github.com/usnistgov/lantern), a hierarchical Bayesian model that distills genotype-phenotype landscape (GPL) measurements into a low-dimensional feature space that represents the fundamental biological mechanisms of the system while also enabling straightforward, explainable predictions. Across a benchmark of large-scale datasets, LANTERN equals or outperforms all alternative approaches, including deep neural networks. LANTERN furthermore extracts useful insights of the landscape, including its inherent dimensionality, a latent space of additive mutational effects, and metrics of landscape structure. LANTERN facilitates straightforward discovery of fundamental mechanisms in GPLs, while also reliably extrapolating to unexplored regions of genotypic space.

An asymmetrical-structure auto-encoder for unsupervised representation learning of skeleton sequences

In this paper, we propose a novel framework for unsupervised representation learning using a structure-asymmetrical auto-encoder in which a 2D-CNN-based encoder learns separable spatiotemporal representations in a low-dimensional feature space under the supervision of salient skeleton motion cues. This study addresses the problem of learning action representations of skeleton sequences. The network captures not only correlations of adjacent joints but also long-term motion dependencies by using the proposed unsupervised training, which leads to the advantage that similar movements are gathered around the same cluster, whereas different movements are gathered around distinct clusters. Our method is unsupervised and does not rely on annotations to associate skeleton sequences with actions. Experimental results clearly showed the effectiveness of the proposed representation learning, and improvements compared with skeleton-based generative learning methods. When the proposed network was fine-tuned with partial labeled data, our results still outperformed some fully supervised methods.

SGFNNs: Signed Graph Filtering-based Neural Networks for Predicting Drug-Drug Interactions.

Capturing comprehensive information about drug-drug interactions (DDIs) is one of the key tasks in public health and drug development. Recently, graph neural networks (GNNs) have received increasing attention in the drug discovery domain due to their capability of integrating drugs profiles and the network structure into a low-dimensional feature space for predicting links and classification. Most of GNN models for DDI predictions are built on an unsigned graph, which tends to represent associated nodes with similar embedding results. However, semantic correlation between drugs, such as degressive effects, or even adverse side reactions should be disassortative. In this study, we put forward signed GNNs to model assortative and disassortative relationships within drug pairs. Since negative links exclude direct generalization of spectral filters on unsigned graph, we divide the signed graph into two unsigned subgraphs to dedicate two spectral filters, which captures both commonality and difference of drug pairs. For drug representations we derive two signed graph filtering-based neural networks (SGFNNs) which integrate signed graph structures and drug node attributes. Moreover, we use an end-to-end framework for learning DDIs, where an SGFNN together with a discriminator is jointly trained under a problem-specific loss function. The experimental results on two prediction problems show that our framework can obtain significant improvements compared with baselines. The case study further verifies the validation of our method.

A Deep Reinforcement Learning Framework for Fast Charging of Li-Ion Batteries

One of the most crucial challenges faced by the Li-ion battery community concerns the search for the minimum time charging without damaging the cells. This goal can be achieved by solving a large-scale constrained optimal control problem, which relies on accurate electrochemical models. However, these models are limited by their high computational cost, as well as identifiability and observability issues. As an alternative, simple output-feedback algorithms can be employed, but their performance strictly depends on trial and error tuning. Moreover, particular techniques have to be adopted to handle safety constraints. With the aim of overcoming these limitations, we propose an optimal-charging procedure based on deep reinforcement learning. In particular, we focus on a policy gradient method to cope with continuous sets of states and actions. First, we assume full state measurements from the Doyle–Fuller–Newman (DFN) model, which is projected to a lower dimensional feature space via the principal component analysis. Subsequently, this assumption is removed, and only output measurements are considered as the agent observations. Finally, we show the adaptability of the proposed policy to changes in the environment’s parameters. The results are compared with other methodologies presented in the literature, such as the reference governor and the proportional–integral–derivative approach.

Novel concept specific deep learning for disease treatment prediction on patient trajectory data model

Prediction of diseases by researchers in healthcare industries and the different machine learning algorithms involved play an important role in disease prediction. Similarly, the prediction of patient trajectory data and its analysis also plays a vital role. A novel deep learning approach named Deep Concept Specific learning has been proposed for analyzing unstructured patient trajectory data. The proposed model is structured as a generic model suitable for any patient's medical disease. To evaluate the proposed model performance, the disease related to diabetes has been employed. It implies for autoencoder model, which has been optimized with hyper parameter tuning. It effectively transforms high dimensional features into low dimensional feature space to extract concept-specific features. It is implemented on the Further, all the parameters employed in the sparse features are manipulated. They are considered a loss and encoder function to compute the optimal features for effectively predicting the results. The encoder function maps the data points containing optimal features into latent representations based on disease characteristics. It is done from the patients' perspectives by the ReLu activation function based on generated feature characteristics.

A novel hierarchical feature selection method based on large margin nearest neighbor learning

The complexity of data has been increasing not only in the dimension of the data, but also in the interrelated class relationships. Pre-defined hierarchical structure of classes helps to model such relationships in a way, which divides the whole classification problem into a set of subtasks. Feature selection, known as a data preprocessing technique, aims to selecting useful feature to improve the performance of learning algorithms. However, the issue of close class relationships, especially under the same parent node in the hierarchical structure, is adverse to the feature selection. In this paper, we introduce a novel distance metric based feature selection method with large margin nearest neighbor strategy for the hierarchical classification problems. Different from other algorithms, our proposed method is to project the data into a low-dimensional feature space for learning correct metrics, and the weights of features will be indicated in the transformation matrix. A sparsity regularization and iterative optimization ensure that superior feature subsets can be selected. Experimental evaluation on several datasets with hierarchical class structure demonstrates that the proposed approach is comparable or better than some other state-of-the-art hierarchical feature selection methods.

Spatial feature-based convolutional neural network for PolSAR image classification

The deep learning algorithm has made great breakthroughs in optical image processing. Some deep learning algorithms require a large number of labeled samples for training. For PolSAR data sets, due to the influence of speckle noise and other factors, high-quality labeled data are limited. Therefore, it is meaningful to use deep learning algorithm to solve PolSAR classification problem in limited labeled dataset. This paper proposes a spatial feature-based convolutional neural network (SF-CNN). The network adopts a dual-branch CNN structure. Both of the two branches have the same structure and share parameters. SF-CNN can receive more than one sample as input. SF-CNN’s special structure can expand the original training set by combining different samples, and alleviate the problem of insufficient labeled training data in PolSAR image classification tasks. When training, SF-CNN maps high-dimensional PolSAR image to low-dimensional feature space. In low-dimensional feature space, SF-CNN enhances the ability of network to extract discriminative features by maximizing or minimizing the distance between feature centers of different classes. In order to dig up the relationship between the samples, the test sample features are compared with every training sample feature when testing. Finally, labels of test samples are determined by the comparison result. The result of SF-CNN in PolSAR image classification task is better than that of standard CNN.

A Novel Deep Auto-Encoder Based Linguistics Clustering Model for Social Text

The wide adoption of media and social media has increased the amount of digital content to an enormous level. Natural language processing (NLP) techniques provide an opportunity to extract and explore meaningful information from a large amount of text. Among natural languages, Urdu is one of the widely used languages worldwide for spoken and written communications. Due to its wide adopt-ability, digital content in the Urdu language is increasing briskly, especially with social media and online NEWS feeds. Government agencies and advertisers must filter and understand the content to analyze the trends and cohorts in their interest and national prerogative. Clustering is considered a baseline and one of the first steps in natural language understanding. There are many state-of-the-art clustering techniques specifically for English, French, and Arabic, but no significant research has been conducted in Urdu language processing. Doing it for short text segments is challenging because of limited features and the absence of meaningful language discourse and nuance. Many rule-based NLP techniques are adopted to overcome these issues, relying on human-designed features and rules. Therefore, these methods do not promise remarkable results. Alongside NLP, deep learning techniques are pretty efficient in capturing contextual information with minimal noise compared to other traditional methods. By taking on this challenging job, we develop a deep learning-based technique for Urdu short text clustering for the very first time without a human-designed feature. In this paper, we propose a method of short text clustering using a deep neural network that automatically learns feature representations and clustering assignments simultaneously. This method learns clustering objectives by converting the high dimensional feature space to a low dimensional feature space. Our experiments on the Urdu NEWS headlines dataset show remarkable results compared to state-of-the-art methods.

Spectral clustering of single cells using Siamese nerual network combined with improved affinity matrix.

Limitations of bulk sequencing techniques on cell heterogeneity and diversity analysis have been pushed with the development of single-cell RNA-sequencing (scRNA-seq). To detect clusters of cells is a key step in the analysis of scRNA-seq. However, the high-dimensionality of scRNA-seq data and the imbalances in the number of different subcellular types are ubiquitous in real scRNA-seq data sets, which poses a huge challenge to the single-cell-type detection.We propose a meta-learning-based model, SiaClust, which is the combination of Siamese Convolutional Neural Network (CNN) and improved spectral clustering, to achieve scRNA-seq cell type detection. To be specific, with the help of the constrained Sigmoid kernel, the raw high-dimensionality data is mapped to a low-dimensional space, and the Siamese CNN learns the differences between the cell types in the low-dimensional feature space. The similarity matrix learned by Siamese CNN is used in combination with improved spectral clustering and t-distribution Stochastic Neighbor Embedding (t-SNE) for visualization. SiaClust highlights the differences between cell types by comparing the similarity of the samples, whereas blurring the differences within the cell types is better in processing high-dimensional and imbalanced data. SiaClust significantly improves clustering accuracy by using data generated by nine different species and tissues through different scNA-seq protocols for extensive evaluation, as well as analogies to state-of-the-art single-cell clustering models. More importantly, SiaClust accurately locates the exact site of dropout gene, and is more flexible with data size and cell type.

High fidelity deep learning-based MRI reconstruction with instance-wise discriminative feature matching loss.

To improve reconstruction fidelity of fine structures and textures in deep learning- (DL) based reconstructions. A novel patch-based Unsupervised Feature Loss (UFLoss) is proposed and incorporated into the training of DL-based reconstruction frameworks in order to preserve perceptual similarity and high-order statistics. The UFLoss provides instance-level discrimination by mapping similar instances to similar low-dimensional feature vectors and is trained without any human annotation. By adding an additional loss function on the low-dimensional feature space during training, the reconstruction frameworks from under-sampled or corrupted data can reproduce more realistic images that are closer to the original with finer textures, sharper edges, and improved overall image quality. The performance of the proposed UFLoss is demonstrated on unrolled networks for accelerated two- (2D) and three-dimensional (3D) knee MRI reconstruction with retrospective under-sampling. Quantitative metrics including normalized root mean squared error (NRMSE), structural similarity index (SSIM), and our proposed UFLoss were used to evaluate the performance of the proposed method and compare it with others. In vivo experiments indicate that adding the UFLoss encourages sharper edges and more faithful contrasts compared to traditional and learning-based methods with pure loss. More detailed textures can be seen in both 2D and 3D knee MR images. Quantitative results indicate that reconstruction with UFLoss can provide comparable NRMSE and a higher SSIM while achieving a much lower UFLoss value. We present UFLoss, a patch-based unsupervised learned feature loss, which allows the training of DL-based reconstruction to obtain more detailed texture, finer features, and sharper edges with higher overall image quality under DL-based reconstruction frameworks. (Code available at: https://github.com/mikgroup/UFLoss).

Explanation and Use of Uncertainty Quantified by Bayesian Neural Network Classifiers for Breast Histopathology Images.

Despite the promise of Convolutional neural network (CNN) based classification models for histopathological images, it is infeasible to quantify its uncertainties. Moreover, CNNs may suffer from overfitting when the data is biased. We show that Bayesian-CNN can overcome these limitations by regularizing automatically and by quantifying the uncertainty. We have developed a novel technique to utilize the uncertainties provided by the Bayesian-CNN that significantly improves the performance on a large fraction of the test data (about 6% improvement in accuracy on 77% of test data). Further, we provide a novel explanation for the uncertainty by projecting the data into a low dimensional space through a nonlinear dimensionality reduction technique. This dimensionality reduction enables interpretation of the test data through visualization and reveals the structure of the data in a low dimensional feature space. We show that the Bayesian-CNN can perform much better than the state-of-the-art transfer learning CNN (TL-CNN) by reducing the false negative and false positive by 11% and 7.7% respectively for the present data set. It achieves this performance with only 1.86 million parameters as compared to 134.33 million for TL-CNN. Besides, we modify the Bayesian-CNN by introducing a stochastic adaptive activation function. The modified Bayesian-CNN performs slightly better than Bayesian-CNN on all performance metrics and significantly reduces the number of false negatives and false positives (3% reduction for both). We also show that these results are statistically significant by performing McNemar's statistical significance test. This work shows the advantages of Bayesian-CNN against the state-of-the-art, explains and utilizes the uncertainties for histopathological images. It should find applications in various medical image classifications.

Accelerated sparse nonnegative matrix factorization for unsupervised feature learning

Sparse Nonnegative Matrix Factorization (SNMF) is a fundamental unsupervised representation learning technique, and it represents low-dimensional features of a data set and lends itself to a clustering interpretation. However, the model and algorithm of SNMF have some shortcomings. In this work, we created a clustering method by improving the SNMF model and its Alternating Direction Multiplier Method acceleration algorithm. A novel, fast and closed-form iterative solution is proposed for SNMF with implicit sparse constraints which are L1 and L2 norms of the coefficient and basis matrixes, respectively. A low-dimensional feature space is also proposed as result of the closed-form iteration formats of each sub-problem obtained by variable splitting. In addition, the convergence points of the presented iterative algorithms are stationary points of the model. Finally, numerical experiments show that the improved algorithm is comparable to the sate-of-the-art methods in data clustering.

Degradation Prediction of PEMFCs Using Stacked Echo State Network Based on Genetic Algorithm Optimization

Durability is considered as one of the main technical obstacles to the large-scale commercialization of proton exchange membrane fuel cells (PEMFCs), which can be effectively improved through degradation prediction techniques. This article proposes a stacked echo state network (ESN) based on the genetic algorithm (GA) to predict the future degradation trend of PEMFCs. By alternately using the projection layer and the encoding layer, the proposed method can make full use of the temporal kernel property of the ESN to encode the multiscale and multilevel dynamics of the stack voltage, thereby obtaining more robust generalization performance and higher accuracy than the existing methods. Specifically, a stack voltage time series of PEMFCs is projected into the high-dimensional echo state space of the reservoir. Then, an autoencoder projects the echo state representation into the low-dimensional feature space. After that, the GA is utilized to optimize the hyperparameters of the developed model. Based on two open-source datasets of PEMFCs with different accelerated test conditions, this article systematically tested the proposed degradation prediction methods based on different model structures. Test results demonstrate that the proposed method is superior to traditional prediction methods in terms of accuracy and generalization performance.

Age Classification in Forensic Medicine Using Machine Learning Techniques.

The aim of the study was to assess the capabilities of age determination (age group) at death using classification techniques by histomorphometric characteristics of osseous and cartilaginous tissue aging.Materials and MethodsThe study material was a database containing the findings of morphometric researches of osseous and cartilaginous tissue histologic specimens from 294 categorized male corpses aged 10–93 years. For data analysis and classification we used modern machine learning methods: k-NN, SVM, logistic regression, CatBoost, SGD, naive Bayes, random forest, nonlinear dimensionality reduction methods (t-SNE and uMAP), and recursive feature elimination for feature selection.ResultsThe used techniques (algorithms) provided effective representation of a complex data set (76 histomorphometric features), allowing to reveal the cluster structure inside the low dimensional feature space, thus fitting the classifier becomes even more reasonable. During feature selection, we estimated their importance for age group classification and studied the relationship between classification quality and the number of features inside the feature space. Data pre-processing made it possible to get rid of noise and keep most informative features, thereby accelerating a learning process and improving the classification quality. Data projection showed more well-defined cluster structure in the space of selected features. The accuracy of establishing certain groups was equal to 90%. It proves high efficiency of machine learning techniques used for forensic age diagnostics based on histomorphometric findings.

Modified LPP based on Riemannian metric for feature extraction and fault detection

Dimensionality reduction methods based on manifold learning are widely adopted for industrial process monitoring. However, in many situations, these methods fail to preserve manifold intrinsic features in low-dimensional space, resulting in reduced process monitoring efficacy. To overcome this problem, a modified locality preserving projection (MLPP) based on the Riemannian metric is put forward. First, the Riemannian metric, which embodies a manifold’s geometric information, is estimated from process data. Then, the low dimensional embedding coordinates obtained from LPP are supplemented with an estimate of the Riemannian metric. Finally, a process monitoring model is developed, and kernel density estimation is utilized to approximate confidence bounds for T2 and SPE statistics. The proposed MLPP method is applied to the feature extraction of Twin-Peaks dataset, fault detection of hot strip mill, steam turbine system and Tennessee Eastman processes. The effectiveness of MLPP method is compared with both the manifold learning and deep learning approaches. In addition, the proposed method is evaluated under various noisy conditions. The average fault detection rate of 98.9%, 99.6% and 84.4% in hot strip mill, steam turbine system and Tennessee Eastman processes, respectively, are higher than the existing methods. Quantitative results indicate the superiority of the proposed MLPP method.

Machine Learning Methods for the Prediction of Micromagnetic Magnetization Dynamics

Machine learning (ML) entered the field of computational micromagnetics only recently. The main objective of these new approaches is the automatization of solutions of parameter-dependent problems in micromagnetism such as fast response curve estimation modeled by the Landau-Lifschitz-Gilbert (LLG) equation. Data-driven models for the solution of time- and parameter-dependent partial differential equations require high dimensional training data-structures. ML in this case is by no means a straight-forward trivial task, it needs algorithmic and mathematical innovation. Our work introduces theoretical and computational conceptions of certain kernel and neural network based dimensionality reduction approaches for efficient prediction of solutions via the notion of low-dimensional feature space integration. We introduce efficient treatment of kernel ridge regression and kernel principal component analysis via low-rank approximation. A second line follows neural network (NN) autoencoders as nonlinear data-dependent dimensional reduction for the training data with focus on accurate latent space variable description suitable for a feature space integration scheme. We verify and compare numerically by means of a NIST standard problem. The low-rank kernel method approach is fast and surprisingly accurate, while the NN scheme can even exceed this level of accuracy at the expense of significantly higher costs.

A self-supervised domain-general learning framework for human ventral stream representation

Anterior regions of the ventral visual stream encode substantial information about object categories. Are top-down category-level forces critical for arriving at this representation, or can this representation be formed purely through domain-general learning of natural image structure? Here we present a fully self-supervised model which learns to represent individual images, rather than categories, such that views of the same image are embedded nearby in a low-dimensional feature space, distinctly from other recently encountered views. We find that category information implicitly emerges in the local similarity structure of this feature space. Further, these models learn hierarchical features which capture the structure of brain responses across the human ventral visual stream, on par with category-supervised models. These results provide computational support for a domain-general framework guiding the formation of visual representation, where the proximate goal is not explicitly about category information, but is instead to learn unique, compressed descriptions of the visual world.