Dimensional Feature Space Research Articles

Endpoint carbon content and temperature prediction model in BOF steelmaking based on posterior probability and intra-cluster feature weight online dynamic feature selection

Abstract A posterior probability and intra-cluster feature weight online dynamic feature selection algorithm is proposed to address the issues of high dimensionality and high volatility of data in the basic oxygen furnace (BOF) steelmaking production process. First, a genetic algorithm with fixed feature space dimensions is introduced, which narrows the solution space by predefining the number of selected features, thereby enhancing the stability of feature selection. Second, the posterior probability of samples and intra-cluster feature weights are used to weigh and calculate the feature importance of the current sample, obtaining the optimal features that align with the current operating conditions. Finally, the dynamically selected features are used in a regression model to predict the carbon content and temperature of the BOF steelmaking process data. Simulations of actual BOF steelmaking process data showed that the prediction accuracy was 86% within a carbon content error range of 0.02, and 88% within a temperature error range of 10°C.

IBhb-Lys: Identify lysine β-hydroxybutyrylation sites using autoencoder feature representation and fuzzy SVM algorithm

Lysine β-hydroxybutyrylation (Kbhb) is newly discovered β-hydroxybutyrylate-derived histone modification which has been associated with the pathogenesis of many human diseases. To further elucidate the biological significance and molecular mechanism of Kbhb, it is necessary to accurately identify the Kbhb sites from protein sequences. In this study, a novel computational model named iBhb-Lys is developed for the identification of Kbhb sites. Four types of features are combined to encode each Kbhb site as a 3266-dimensional feature vector. And the autoencoder network is used to reduce the dimensionality of feature space, due to the high dimensionality of the combined features. In addition, to effectively reduce the influence of noise and outlier on classification, a new fuzzy support vector machine algorithm is proposed by incorporating the density around the sample into the fuzzy membership function. As illustrated by independent test, the AUC value of iBhb-Lys has increased by 2.22 % compared to the existing predictor KbhbXG. Feature analysis shows that some amino acid composition features, such as the occurrence frequency of leucine and histidine residues around Kbhb sites, contribute profoundly to the identification of Kbhb sites. The conclusions drawn in this study may provide useful reference for studying the molecular mechanism of Kbhb.

Research on high voltage circuit breaker multi-sensor signal feature selection method based on GA-Kmeans algorithm

Abstract Currently, in the research on fault information of high-voltage circuit breakers (HVCBs) based on multi-sensor data, issues such as shallow feature assessment and unclear feature selection rules exist, leading to a decrease in fault identification accuracy due to redundant characteristics. To address this, this paper proposes a novel feature selection method based on GA-Kmeans. This method encodes the original feature space into binary format and employs the clustering accuracy of the Kmeans model as the fitness function to obtain a high-quality feature subset that maximally represents and distinguishes faults. By combining different feature selection methods with fault diagnosis models, six typical application scenarios are constructed. Results indicate that compared to traditional methods such as Relief-F, KPCA, GA-SVM for feature selection, and SVM for fault diagnosis, the proposed GA-Kmeans method reduces the dimensionality of the original feature space and employs the Kmeans clustering algorithm as the diagnostic model, achieving a final diagnostic accuracy of 95.29%. This method outperforms others, with a 37.24% higher diagnostic accuracy than SVM under the original feature space, and a decrease of 47.90% in standard deviation. This validates the necessity of feature selection and the superiority of the proposed method, providing a reliable and stable diagnostic basis for subsequent mechanical fault diagnosis of HVCBs.

Nonlinear classification and regression in underwater sonar applications using convex-complex neural networks

Recently, the sonar signal processing community has seen increased interest in neural networks for nonlinear classification and regression. While innovative neural network architectures have achieved remarkable results in other fields such as computer vision and natural language processing, their application to sonar data presents challenges. The multiple nonlinearities in these networks make it difficult to interpret results. Recent works such as Grad-CAM and Layer-wise Relevance Propagation attempt to empirically visualize what features are being selected by the network, but there has been little theoretical work to explain why neural networks perform so well. However, recent work by Pilanci et al. has shown that two-layer neural networks with a ReLU activation function can be reformulated as a convex optimization that can be solved optimally in polynomial time. Their results rely on the observation that ReLU neural networks are piece-wise linear models that lift the original data into a higher dimensional feature space. Using a valid activation function, we show that this Convex Neural Network framework can be extended to applications where the input data is complex-valued. We evaluate our algorithm on simulated sonar data. [This talk will present research funded by DoD Navy (NEEC) Grant number N001742010016 and the ONR grant numbers N000142112420 and N000142312503.]

Multi-population differential evolution approach for feature selection with mutual information ranking

Feature selection is a crucial aspect of data preprocessing because of the significant effect of redundant features on classification performance and the extensive computational resources required. Evolutionary algorithm-based feature-selection methods have shown remarkable results in determining the optimal feature subset. To further enhance classification performance, this paper proposes a novel multi-population differential evolution approach for feature selection with mutual information ranking (MI-MPODE). Firstly, population preprocessing guided by mutual information is employed to reduce the dimensionality of the initial feature space. Then, the feature subset obtained from mutual information serves as the initial population for MI-MPODE. MI-MPODE incorporates a novel multi-population information-sharing mechanism, with common individuals from three layers contributing to inter-subpopulation information sharing. Additionally, an individual enhancement strategy is proposed to handle the variations of individuals in the population, and a Lens imaging opposition-based learning method is adopted to improve the algorithm’s optimization capability. MI-MPODE is compared with several state-of-the-art evolutionary algorithm-based feature selection methods through experimental comparisons. The results show that MI-MPODE outperforms all comparison algorithms on more than half of the datasets, with a significant reduction in the number of features used, demonstrating a significant advantage over competitors.

An autoencoder model based on one-dimensional neural network for epileptic EEG anomaly detection

We propose an autoencoder model based on a one-dimensional convolutional neural network (1DCNN) as the feature extraction network for efficient detection of epileptic EEG anomalies. The local information of normal EEG signals was captured by utilizing the local feature extraction ability of 1DCNN for training of an autoencoder to learn the expression of normal EEG data in low dimensional feature space. With the difference between the input and output as the anomaly score, the threshold was determined by the optimal equilibrium point of the ROC curve, and the EEG signals exceeding the threshold were diagnosed as the seizure data. The performance of the 1DCNN-AE epilepsy detection model was evaluated using the publicly available CHB-MIT scalp EEG dataset and TUH scalp EEG dataset. The AUC of the 1DCNN-AE model reached 0.890 of CHB-MIT and 0.686 of TUH under the average level of patients, and the epilepsy detection rate reached 0.974 and 0.893, and these results were better than the latest epilepsy anomaly detection models LSTM-VAE and GRU-VAE. The 1DCNN model had a parameter quantity of 58.5M, which was at the same level with LSTM-VAE (47.4 M) and GRU-VAE (36.9 M) but with much smaller FLOPs (0.377 G) than LSTM-VAE (21.6 G) and GRU-VAE (16.2 G). The autoencoder model based one-dimensional convolutional neural network can effectively detect abnormal EEG signals in epileptic seizure.

Analytical learning classifier based on predefined evenly-distributed class centroids

For the pattern recognition, most classification models are solved iteratively, except for Linear LDA, KLDA and ELM etc. In this paper, a nonlinear classification network model based on predefined evenly-distributed class centroids (PEDCC) is proposed. Its analytical solution can be obtained and has good interpretability. Using the characteristics of maximizing the inter-class distance of PEDCC and derivative weighted minimum mean square error loss function to minimize the intra-class distance, we can not only realize the effective nonlinearity of the network, but also obtain the analytical solution of the network weight. Then, the sample is classified based on GDA. In order to further improve the performance of classification, PCA is used to reduces the dimensionality of the original sample, meanwhile, the CReLU activation function are adopted to enhances the expression ability of the features. The network transforms the samples into the higher dimensional feature space through the weighted minimum mean square error, so as to find a better separation hyperplane. In experiments, the feasibility of the network structure is verified from pure linear 𝑾, 𝑾+Tanh, and PCA+𝑾+Tanh respectively on many small data sets and large data sets, and compared with SVM and ELM in terms of training speed and recognition rate. The results show that, in general, this model has advantages on small data set both in recognition accuracy and training speed, while it has advantages in training speed on large data sets. Finally, by introducing a multi-stage network structure based on the latent feature norm, the classifier network can further significantly improve the classification performance, the recognition rate of small data sets is effectively improved and much higher than that of existing methods, while the recognition rate of large data sets is similar to that of SVM.

DMMGNet: A discrimination mapping and memory bank mean guidance-based network for high-performance few-shot industrial anomaly detection

For deep learning-based industrial anomaly detection, it is still challenging to get adequate images for model training and achieve cold start for cross-product migration, restricting their practical application in real industrial production. Herein, an innovative few-shot anomaly detection network DMMGNet based on discrimination mapping and memory bank mean guidance strategies are demonstrated, which is trained by a new two-branch data augmentation technique. By separating the features stored in memory bank from the features used for training, the two-branch data augmentation method can significantly improve the robustness of few-shot model training and reduce the redundance of memory bank. In the elaborately designed discrimination mapping module, new negative samples are generated by adding dynamic Gaussian noise to normal samples along the channel dimension in feature space to solve the problem of sample imbalance. Meanwhile, the discrimination mapping module also helps to map the feature distribution of positive samples to the target domain more efficiently and reduce the deviation of feature domain, conducive to a more precise separation of positive and negative samples. In addition, a novel mean guidance approach with an optimized loss function is developed to guide the positive sample feature mapping by specifying the local feature space center to form a clear feature domain contour and enhance the detection accuracy. The multiple experimental results validate that our DMMGNet outperforms the most advanced anomaly detection counterparts on image-level AUROC, showing an increase by 0.3–3 % on both MVTec AD and MPDD benchmarks under several few-shot scenarios.

HEnsem_DTIs: A heterogeneous ensemble learning model for drug-target interactions prediction

Drug discovery is the process by which a drug is discovered. Drug-target interactions prediction is a major part of drug discovery. Unfortunately, producing new drugs is time-consuming and expensive; Because it requires a lot of human and laboratory resources. Recently, predictions have been made using computational methods to solve these problems and prevent blindly examining all interactions. Various experiences using computational methods show that no single algorithm can be suitable for all applications; Hence, ensemble learning is expressed. Although various ensemble methods have been proposed, it is still not easy to find a suitable ensemble method for a particular dataset. In general, the existing algorithms in aggregation and combination method are selected manually based on experience. Reinforcement learning can be one way to meet this challenge. High-dimensional feature space and class imbalance are among the challenges of drug-target interactions prediction. This paper proposes HEnsem_DTIs, a heterogeneous ensemble model, for predicting drug-target interactions using dimensionality reduction and concepts of recommender systems to address these challenges. HEnsem_DTIs is configured with reinforcement learning. Dimensionality reduction is applied to handle the challenge of high-dimensional feature space and recommender systems to improve under-sampling and solve the class imbalance challenge. Six datasets are used to evaluate the proposed model; Results of the evaluation on datasets show that HEnsem_DTIs works better than other models in this field. Results of evaluation of the proposed model on the first dataset using 10-fold cross-validation experiments show the amount of sensitivity 0.896, specificity 0.954, GM 0.924, AUC 0.930 and AUPR 0.935.

Hybrid genetic optimization for quantum feature map design

Kernel methods are an import class of techniques in machine learning. To be effective, good feature maps are crucial for mapping non-linearly separable input data into a higher dimensional (feature) space, thus allowing the data to be linearly separable in feature space. Previous work has shown that quantum feature map design can be automated for a given dataset using NSGA-II, a genetic algorithm, while both minimizing circuit size and maximizing classification accuracy. However, the evaluation of the accuracy achieved by a candidate feature map is costly. In this work, we demonstrate the suitability of kernel-target alignment as a substitute for accuracy in genetic algorithm-based quantum feature map design. Kernel-target alignment is faster to evaluate than accuracy and does not require some data points to be reserved for its evaluation. To further accelerate the evaluation of genetic fitness, we provide a method to approximate kernel-target alignment. To improve kernel-target alignment and root mean squared error, the final trainable parameters of the generated circuits are further trained using COBYLA to determine whether a hybrid approach applying conventional circuit parameter training can easily complement the genetic structure optimization approach. A total of eight new approaches are compared to the original across nine varied binary classification problems from the UCI machine learning repository, showing that kernel-target alignment and its approximation produce feature map circuits enabling comparable accuracy to the previous work but with larger margins on training data (in excess of 20% larger) that improve further with circuit parameter training.

A multi-stage sub-structural damage localization approach using multi-label radial basis function neural network and auto-regressive model parameters

This paper proposes a Structural Health Monitoring (SHM) approach to detect localized damage by employing a Multi-Label Radial Basis Function Neural Network (ML-RBFNN). The proposed research methodology aims to identify structural damage more efficiently. This involves dividing the main structure into smaller sub-structures known as zones instead of employing a global search for damage across the entire structure. Each zone is further divided into smaller groups, including sub-zones, nodes, or elements. To reduce the complexities associated with feature extraction, auto-regressive (AR) model parameters extracted from raw time series are considered damage-sensitive features. Considering the simultaneous effects of multiple sensors can increase the number of inputs to the networks, making it challenging to reduce the complexity of the dataset. To address this problem, the principal component analysis (PCA) method is utilized to decrease the dimension of feature space. The effectiveness of the proposed approach is evaluated through a comprehensive comparative analysis against conventional single-stage damage detection methods, with a focus on computational costs and accuracy. Furthermore, the noise-sensitivity characteristics of the proposed method are explored, providing insights into their robustness under various conditions. The validation of the approach is carried out through numerical modeling using the ASCE benchmark structure and experimental data collected from the Qatar University Grandstand Simulator. The application of ML-RBFNN with AR parameters as features yielded a remarkable accuracy exceeding 98% in diagnosing damage occurrence and location for both multi-stage and single-stage SHM methods. Moreover, our proposed approach demonstrates enhanced computational efficiency compared to the single-stage method.

Detection and Type Recognition of SAR Artificial Modulation Targets Based on Multi-Scale Amplitude-Phase Features

With respect to the detection and recognition of an Artificial Modulation Target (AMT) with different modulated types, the state-of-the-art methods generally suffer the deficiencies of overfitting and insufficient generalization of existing neural network solutions. To address these problems, this paper proposes a multi-scale amplitude-phase feature discrimination method for AMTs in SAR images. First, a multi-type modulated AMT Dataset is generated (AMT Detection and Modulation Type Recognition Dataset, ADMTR Dataset), wherein the factors of jamming position, jamming-to-signal ratio (JSR), and the modulated parameter are considered to enhance the generalization. Second, a Multi-Input Multi-Output Fusion Wavelet Neural Network (MIMOFWTNN) is established, which not only uses the amplitude information of the scene but also adequately makes use of the phase and high-frequency information. This empowers us to detect the AMT in a higher dimensional feature space such that the type recognition can be implemented with more certainty. Analysis and discussions conducted on comparison experiments and ablation experiments demonstrate that the proposed network can achieve an average accuracy of 96.96% on the cross-validation set and a correct rate of 99.0% on the completely independent test set, which outperforms the compared methods.

A family of recognition algorithms based on the construction of k-dimensional threshold rules

In this article, the authors proposed a family of recognition algorithms based on k-dimensional threshold rules and focused on the object recognition presented in the feature space of high dimension. These recognition algorithms are based on the concept of forming a set of k-dimensional preferred threshold rules that define an extremal algorithm within the proposed family of algorithms. A structural description of the developed family of recognition algorithms is given, presented as a sequence of computational procedures. The main ones are the procedures for determining a set of representative features; discriminant functions for each class; normalizing transformations; groups of highly correlated discriminant functions; basic discriminant functions; a set of preferred discriminant functions; a generalized recognition operator based on preferred discriminant functions. The results of experimental studies are presented; they showed that an increase in the accuracy of recognition results is provided (in comparison with known analogs) when using the indicated family of recognition algorithms. The main advantage of the proposed recognition algorithms is not only an increase in accuracy but also a significant reduction in arithmetic operations in object recognition. This property of the algorithms makes it possible to create various software recognition systems that operate in real time.

3D clustering of gene expression data from systemic autoinflammatory diseases using self-organizing maps (Clust3D)

Background and objectiveSystemic autoinflammatory diseases (SAIDs) are characterized by widespread inflammation, but for most of them there is a lack of specific biomarkers for accurate diagnosis. Although a number of machine learning algorithms have been used to analyze SAID datasets, aiding in the discovery of novel biomarkers, there is a growing recognition of the importance of SAID timeseries clustering, as it can capture the temporal dynamics of gene expression patterns. MethodologyThis paper proposes a novel clustering methodology to efficiently associate three-dimensional data. The algorithm utilizes competitive learning to create a self-organizing neural network and adjust neuron positions in time-dependent and high dimensional feature space in order to assign them as clustering centers. The quantitative evaluation of the clustering was based on well-known clustering indices. Furthermore, a differential expression analysis and classification pipeline was employed to assess the capability of the proposed methodology to extract more accurate pathway-specific genes from its clusters. For that, a comparative analysis was also conducted against a heuristic timeseries clustering method. ResultsThe proposed methodology achieved better overall clustering indices scores and classification metrics using genes derived from its clusters. Notable cases include a threefold increase in the Calinski-Harabasz clustering index, a twofold improvement in the Davies–Bouldin clustering index and a ∼60% increase in the classification specificity score. ConclusionA novel clustering methodology was developed and applied on several gene expression timeseries datasets from systemic autoinflammatory diseases, and its ability to efficiently produce well separated clusters compared to existing heuristic methods was demonstrated.

Gauge-Optimal Approximate Learning for Small Data Classification.

Small data learning problems are characterized by a significant discrepancy between the limited number of response variable observations and the large feature space dimension. In this setting, the common learning tools struggle to identify the features important for the classification task from those that bear no relevant information and cannot derive an appropriate learning rule that allows discriminating among different classes. As a potential solution to this problem, here we exploit the idea of reducing and rotating the feature space in a lower-dimensional gauge and propose the gauge-optimal approximate learning (GOAL) algorithm, which provides an analytically tractable joint solution to the dimension reduction, feature segmentation, and classification problems for small data learning problems. We prove that the optimal solution of the GOAL algorithm consists in piecewise-linear functions in the Euclidean space and that it can be approximated through a monotonically convergent algorithm that presents-under the assumption of a discrete segmentation of the feature space-a closed-form solution for each optimization substep and an overall linear iteration cost scaling. The GOAL algorithm has been compared to other state-of-the-art machine learning tools on both synthetic data and challenging real-world applications from climate science and bioinformatics (i.e., prediction of the El Niño Southern Oscillation and inference of epigenetically induced gene-activity networks from limited experimental data). The experimental results show that the proposed algorithm outperforms the reported best competitors for these problems in both learning performance and computational cost.

Machine learning decoding of single neurons in the thalamus for speech brain-machine interfaces

Objective. Our goal is to decode firing patterns of single neurons in the left ventralis intermediate nucleus (Vim) of the thalamus, related to speech production, perception, and imagery. For realistic speech brain-machine interfaces (BMIs), we aim to characterize the amount of thalamic neurons necessary for high accuracy decoding. Approach. We intraoperatively recorded single neuron activity in the left Vim of eight neurosurgical patients undergoing implantation of deep brain stimulator or RF lesioning during production, perception and imagery of the five monophthongal vowel sounds. We utilized the Spade decoder, a machine learning algorithm that dynamically learns specific features of firing patterns and is based on sparse decomposition of the high dimensional feature space. Main results. Spade outperformed all algorithms compared with, for all three aspects of speech: production, perception and imagery, and obtained accuracies of 100%, 96%, and 92%, respectively (chance level: 20%) based on pooling together neurons across all patients. The accuracy was logarithmic in the amount of neurons for all three aspects of speech. Regardless of the amount of units employed, production gained highest accuracies, whereas perception and imagery equated with each other. Significance. Our research renders single neuron activity in the left Vim a promising source of inputs to BMIs for restoration of speech faculties for locked-in patients or patients with anarthria or dysarthria to allow them to communicate again. Our characterization of how many neurons are necessary to achieve a certain decoding accuracy is of utmost importance for planning BMI implantation.

Abstract 2024: Elucidating The Role Of Shape In Aortic Disease State Pathology With Machine Learning

Maximum aortic diameter has been the mainstay metric for aortic disease management and surgical intervention for decades, specifically acute aortic dissection and aneurysm. While this size metric has been effective at separating patient populations into groupings associated with required intervention and monitoring for eventual intervention respectively, size alone ignores the role of complex geometric shape and how it may be further indicative of disease state. Analyses of aortic shape in addition to size have been discussed at length in literature, however the representations and methods to convey shape are siloed and often highly subjective. Machine learning methodologies offer a means to explore high dimensional feature spaces and can identify a sparse set of metrics which result in a division of patient responses parametrized by physical and interpretable variables. Additionally, by introducing a modeling workflow which contains engineered parameters designed to describe aortic shape, in addition to traditional size metrics, the exploration of shape dependence on intervention outcomes becomes more comprehensive across size-shape parameter spaces. The engineered parameters presented primarily incorporate estimations of integrated Gaussian surface curvature into features that contain both the magnitude and spatial variation in shape of the aortas. The models discussed accurately classify over two hundred aortic disease patients spanning aneurysm, dissection, and other modalities with accuracies above 90% and performance across multiple modeling methodologies is discussed. The classification success of the geometrically informed feature space suggests the underlying dependence on pathology shape on the success of the intervention outcome. The most clinically effective models will preserve the wealth of information in modern medical diagnostics and imaging into descriptive, and physically interpretable, parameters to aid in clinical decision making.

A novel feature selection method based on quantum support vector machine

Feature selection is critical in machine learning to reduce dimensionality and improve model accuracy and efficiency. The exponential growth in feature space dimensionality for modern datasets directly results in ambiguous samples and redundant features, which can severely degrade classification accuracy. Quantum machine learning offers potential advantages for addressing this challenge. In this paper, we propose a novel method, quantum support vector machine feature selection (QSVMF), integrating quantum support vector machines with multi-objective genetic algorithm. QSVMF optimizes multiple simultaneous objectives: maximizing classification accuracy, minimizing selected features and quantum circuit costs, and reducing feature covariance. We apply QSVMF for feature selection on a breast cancer dataset, comparing the performance of QSVMF against classical approaches with the selected features. Experimental results show that QSVMF achieves superior performance. Furthermore, the Pareto front solutions of QSVMF enable analysis of accuracy versus feature set size trade-offs, identifying extremely sparse yet accurate feature subsets. We contextualize the biological relevance of the selected features in terms of known breast cancer biomarkers. This work highlights the potential of quantum-based feature selection to enhance machine learning efficiency and performance on complex real-world data.

Accelerated Double-Sketching Subspace Newton

This paper proposes a second-order stochastic algorithm called Accelerated Double-Sketching Subspace Newton (ADSSN) to solve large-scale optimization problems with high dimensional feature spaces and substantial sample sizes. The proposed ADSSN has two computational superiority. First, ADSSN achieves a fast local convergence rate by exploiting Nesterov’s acceleration technique. Second, by taking full advantage of the double sketching strategy, ADSSN provides a lower computational cost for each iteration than competitive approaches. Moreover, these advantages hold for actually all sketching techniques, which enables practitioners to design custom sketching methods for specific applications. Finally, numerical experiments are carried out to demonstrate the efficiency of ADSSN compared with accelerated gradient descent and two single sketching counterparts.

How to Overcome Curse-of-Dimensionality for Out-of-Distribution Detection?

Machine learning models deployed in the wild can be challenged by out-of-distribution (OOD) data from unknown classes. Recent advances in OOD detection rely on distance measures to distinguish samples that are relatively far away from the in-distribution (ID) data. Despite the promise, distance-based methods can suffer from the curse-of-dimensionality problem, which limits the efficacy in high dimensional feature space. To combat this problem, we propose a novel framework, Subspace Nearest Neighbor (SNN), for OOD detection. In training, our method regularizes the model and its feature representation by leveraging the most relevant subset of dimensions (i.e. subspace). The subspace learning yields highly distinguishable distance measures between ID and OOD data. We provide comprehensive experiments and ablations to validate the efficacy of SNN. Compared to the current best distance-based method, SNN reduces the average FPR95 by 15.96% on the CIFAR-100 benchmark.