High-dimensional Feature Space Research Articles

A novel hybrid model for lung and colon cancer detection using pre-trained deep learning and KELM

Cancer poses a significant threat to life due to its aggressive nature, high potential for metastasis, and heterogeneity. Globally, both the men and women are mostly affected by lung and colon (LC) cancer. Early and accurate detection is essential to enhance the well-being of patients facing LC cancer. The traditional techniques face challenges such as low accuracy and high execution time. This work combines the pre-trained deep learning (DL) models like ResNet-50, InceptionV3, and DenseNet with Kernel Extreme Learning Machine (KELM) for the accurate and fast diagnosis of LC cancer using histopathology images. The pre-trained DL models provide a strong foundation for feature extraction, capturing intricate patterns indicative of cancerous tissues. Feature fusion combines the complementary strengths of multiple pre-trained models, enhancing the method's ability to capture diverse features and improve classification accuracy. KELM efficiently handles the high-dimensional feature space generated by the DL models, ensuring fast and accurate classification. Mutation Boosted Dwarf Mongoose Optimization Algorithm (MB-DMOA) further optimizes the model parameters, mitigating the risk of getting stuck in local optima and facilitating better convergence towards the global optimum. By leveraging the advantages of each technique and integrating them into a cohesive framework, the proposed approach achieves superior performance in diagnosing LC cancer, ultimately improving patient outcomes. The proposed study attains accuracy (98.9%), F1-score (97.6%), specificity (96.5%), precision (96.7%) and sensitivity (95.8%). The results illuminate the efficacy of this approach within therapeutic environments.

Enhancing Fault Identification, Classification and Location Accuracy in Transmission Lines

This research paper presents a proposed system for fault identification, classification and location in transmission lines using a Support Vector Machine (SVM)-based technique in conjunction with a Positive Sequence Analyzer. The objective is to develop an accurate and reliable method for identifying, classifying and locating different fault types in transmission lines. The proposed system leverages the capabilities of SVMs in handling high-dimensional feature spaces and the fault signature extraction capabilities of the Positive Sequence Analyzer. Experimental evaluations are conducted to assess the performance and effectiveness of the proposed system, comparing it with existing fault identification and classification methods. The results demonstrate the superior performance and robustness of the SVM-based technique utilizing the Positive Sequence Analyzer, providing a valuable contribution to fault management and system reliability in transmission line networks.

3D clustering of gene expression data from systemic autoinflammatory diseases using self-organizing maps (Clust3D)

Background and objectiveSystemic autoinflammatory diseases (SAIDs) are characterized by widespread inflammation, but for most of them there is a lack of specific biomarkers for accurate diagnosis. Although a number of machine learning algorithms have been used to analyze SAID datasets, aiding in the discovery of novel biomarkers, there is a growing recognition of the importance of SAID timeseries clustering, as it can capture the temporal dynamics of gene expression patterns. MethodologyThis paper proposes a novel clustering methodology to efficiently associate three-dimensional data. The algorithm utilizes competitive learning to create a self-organizing neural network and adjust neuron positions in time-dependent and high dimensional feature space in order to assign them as clustering centers. The quantitative evaluation of the clustering was based on well-known clustering indices. Furthermore, a differential expression analysis and classification pipeline was employed to assess the capability of the proposed methodology to extract more accurate pathway-specific genes from its clusters. For that, a comparative analysis was also conducted against a heuristic timeseries clustering method. ResultsThe proposed methodology achieved better overall clustering indices scores and classification metrics using genes derived from its clusters. Notable cases include a threefold increase in the Calinski-Harabasz clustering index, a twofold improvement in the Davies–Bouldin clustering index and a ∼60% increase in the classification specificity score. ConclusionA novel clustering methodology was developed and applied on several gene expression timeseries datasets from systemic autoinflammatory diseases, and its ability to efficiently produce well separated clusters compared to existing heuristic methods was demonstrated.

Machine learning decoding of single neurons in the thalamus for speech brain-machine interfaces

Objective. Our goal is to decode firing patterns of single neurons in the left ventralis intermediate nucleus (Vim) of the thalamus, related to speech production, perception, and imagery. For realistic speech brain-machine interfaces (BMIs), we aim to characterize the amount of thalamic neurons necessary for high accuracy decoding. Approach. We intraoperatively recorded single neuron activity in the left Vim of eight neurosurgical patients undergoing implantation of deep brain stimulator or RF lesioning during production, perception and imagery of the five monophthongal vowel sounds. We utilized the Spade decoder, a machine learning algorithm that dynamically learns specific features of firing patterns and is based on sparse decomposition of the high dimensional feature space. Main results. Spade outperformed all algorithms compared with, for all three aspects of speech: production, perception and imagery, and obtained accuracies of 100%, 96%, and 92%, respectively (chance level: 20%) based on pooling together neurons across all patients. The accuracy was logarithmic in the amount of neurons for all three aspects of speech. Regardless of the amount of units employed, production gained highest accuracies, whereas perception and imagery equated with each other. Significance. Our research renders single neuron activity in the left Vim a promising source of inputs to BMIs for restoration of speech faculties for locked-in patients or patients with anarthria or dysarthria to allow them to communicate again. Our characterization of how many neurons are necessary to achieve a certain decoding accuracy is of utmost importance for planning BMI implantation.

Age-Invariant Face Recognition Using FaceNet and Multi-task Cascaded Convolutional Networks (MTCNN)

In an era where face recognition technology serves as a cornerstone for security, surveillance, and human-computer interaction, the persistence of age-related variations in facial appearance presents a significant challenge. This research introduces an innovative solution that amalgamates the capabilities of the FaceNet deep learning model with Multi-task Cascaded Convolutional Networks (MTCNN) to achieve precise and robust face recognition across diverse age groups. Leveraging FaceNet's prowess in extracting distinctive features from facial images and mapping them to a high-dimensional feature space for efficient face matching, our system incorporates MTCNN as a pre-processing step to accurately detect and align faces, effectively mitigating age-related changes in facial geometry. Crucially, this novel approach obviates the need for age-specific databases and age group categorization, rendering it a versatile and practical solution for age-invariant face recognition. Rigorous experimentation on benchmark datasets underscores the system's resilience and accuracy across a broad spectrum of age groups, culminating in state-of-the-art results in age-invariant face recognition. The implications of this proposed approach are profound, promising to fortify security measures and enhance user experiences in applications such as access control, personal identification, and customer service, while ensuring dependable and accurate face recognition, irrespective of an individual's age. Key Words: Face recognition, face aging, generative adversarial networks

Method of feature extraction of abnormal communication signal in network based on nonlinear technology

Abstract Accurate extraction of abnormal communication signal features in the network is the basis to ensure the completion of network communication. Therefore, a method of feature extraction of abnormal communication signal in network based on nonlinear technology is proposed. In this method, wavelet transform is adopted to decompose the abnormal network communication signals in the high- and low-frequency bands. According to the distribution characteristics of noise and signal in the frequency band, the corresponding parameters are selected for phase space reconstruction and nonlinear dimension reduction of local tangent space mainstream shape recognition algorithm, and the decomposition coefficients of wavelet packet after noise reduction are reconstructed to realize the nonlinear noise reduction of abnormal signal; the denoised abnormal communication signal in network is mapped to the high-dimensional feature space. The principal component is analyzed in accordance with the nonlinear function in the mapped feature space, and the nonlinear function is solved by self-organizing neural network to output the principal component extraction result. According to test results, this method has a significant signal noise reduction effect, results are more than 92% for different abnormal communication signals, and the features of abnormal signals are accurately extracted.

Multi-scale learnable key-channel attention network for point cloud classification and segmentation

The 3D point cloud, a common format of 3D data, finds extensive applications in fields like remote sensing, surveying, robotics, and more. Addressing the challenges posed by insufficient weighted feature information and the offset between weighted results and task expectations in the attention mechanism, this paper proposes a Multi-scale Learnable Key-channel Attention Network (MLKNet). First, we introduce a feature feedback-repair module to mitigate the impact of information loss in the feature embedding process. This module aims to fully embed the original data into a high-dimensional feature space, ensuring a rich supply of feature information for subsequent transformer modules. Second, an efficient hierarchical local feature encoder extract and aggregate local features from point clouds at various scales, thereby significantly enhancing the model's capability to represent geometric structures. Third, a novel learnable key-channel attention module allows tasks to influence the feature selection and weighting process directly, make the highlighted features as close to task expectations as possible, effectively enhancing the network's perception of global semantic information. Our method was benchmarked on various tasks where we achieved overall accuracy (OA) of 92.3% on the ModelNet40 classification task and achieved instance mean intersection over union (ins. mIoU) of 87.6% on the ShapeNet-part segmentation task. The results indicate the superior performance of our method.

Deep learning and genetic algorithm-based ensemble model for feature selection and classification of breast ultrasound images

Feature extraction and selection are important techniques in the classification of medical images. Extraction of key features and selection of relevant features are the preliminary processes that are essential for identifying the shape of an object or diagnosis of a tumor in images. In this study, we conduct a thorough comparison of deep neural networks' performance. The comparison of the networks infers MobileNet as optimal for feature extraction from medical images, where it has minimal parameters and high validation accuracy. For feature selection, we employ the Genetic Algorithm (GA) because of its proficiency in handling high-dimensional and complex feature space. GA's iterative process aligns well with the unique characteristics of breast ultrasound (BUS) images, which enhances its efficacy in selecting salient features of BUS images. An ensemble model, capitalizing on the collective decision-making capabilities of multiple classifiers based on a weighted voting scheme for classification, is proposed. Empirical evaluations are conducted using two publicly available BUS image datasets, BUSI and UDIAT. The proposed model demonstrates a notable improvement of approximately 4% and 9% in accuracy for the BUSI and UDIAT, respectively. The improved diagnostic accuracy in breast abnormality identification allows for early abnormality diagnosis. This improves treatment outcomes for breast cancer patients and highlights the practical value of the proposed method for improving BUS image categorization.

Accelerated Double-Sketching Subspace Newton

This paper proposes a second-order stochastic algorithm called Accelerated Double-Sketching Subspace Newton (ADSSN) to solve large-scale optimization problems with high dimensional feature spaces and substantial sample sizes. The proposed ADSSN has two computational superiority. First, ADSSN achieves a fast local convergence rate by exploiting Nesterov’s acceleration technique. Second, by taking full advantage of the double sketching strategy, ADSSN provides a lower computational cost for each iteration than competitive approaches. Moreover, these advantages hold for actually all sketching techniques, which enables practitioners to design custom sketching methods for specific applications. Finally, numerical experiments are carried out to demonstrate the efficiency of ADSSN compared with accelerated gradient descent and two single sketching counterparts.

Feature selection for multi-label learning based on variable-degree multi-granulation decision-theoretic rough sets

Multi-label learning (MLL) suffers from the high-dimensional feature space teeming with irrelevant and redundant features. To tackle this, several multi-label feature selection (MLFS) algorithms have emerged as vital preprocessing steps. Nonetheless, existing MLFS methods have their shortcomings. Primarily, while they excel at harnessing label-feature relationships, they often struggle to leverage inter-feature information effectively. Secondly, numerous MLFS approaches overlook the uncertainty in the boundary domain, despite its critical role in identifying high-quality features. To address these issues, this paper introduces a novel MLFS algorithm, named VMFS. It innovatively integrates multi-granulation rough sets with three-way decision, leveraging multi-granularity decision-theoretic rough sets (MGDRS) with variable degrees for optimal performance. Initially, we construct coarse decision (RDC), fine decision (RDF), and uncertainty decision (RDU) functions for each object based on MGDRS with variable degrees. These decision functions then quantify the dependence of attribute subsets, considering both deterministic and uncertain aspects. Finally, we employ the dependency to assess attribute importance and rank them accordingly. Our proposed method has undergone rigorous evaluation on various standard multi-label datasets, demonstrating its superiority. Experimental results consistently show that VMFS significantly outperforms other algorithms on most datasets, underscoring its effectiveness and reliability in multi-label learning tasks.

How to Overcome Curse-of-Dimensionality for Out-of-Distribution Detection?

Machine learning models deployed in the wild can be challenged by out-of-distribution (OOD) data from unknown classes. Recent advances in OOD detection rely on distance measures to distinguish samples that are relatively far away from the in-distribution (ID) data. Despite the promise, distance-based methods can suffer from the curse-of-dimensionality problem, which limits the efficacy in high dimensional feature space. To combat this problem, we propose a novel framework, Subspace Nearest Neighbor (SNN), for OOD detection. In training, our method regularizes the model and its feature representation by leveraging the most relevant subset of dimensions (i.e. subspace). The subspace learning yields highly distinguishable distance measures between ID and OOD data. We provide comprehensive experiments and ablations to validate the efficacy of SNN. Compared to the current best distance-based method, SNN reduces the average FPR95 by 15.96% on the CIFAR-100 benchmark.

Research on the principle, performance, and application of UCB algorithm in multi arm slot machine problems

As Internet technology continues to evolve, recommender systems have become an integral part of daily life. However, traditional methods are increasingly falling short of meeting evolving user expectations. Utilizing survey data from the MovieLens dataset, a comparative approach was employed to investigate the efficacy, performance, and applicability of the UCB (Upper Confidence Bound) algorithm in addressing the multi-armed bandit problem. The study reveals that the UCB algorithm significantly impacts the cumulative regret value, indicating its robust performance in the multi-armed bandit setting. Furthermore, LinUCBan enhanced version of the UCB algorithmexhibits exceptional overall performance. The algorithm's efficiency is not just limited to the regret value but extends to handling high-dimensional feature spaces and delivering personalized recommendations. Unlike traditional UCB algorithms, LinUCB adapts more fluidly to high-dimensional environments by leveraging a linear model to simulate the reward function associated with each arm. This adaptability makes LinUCB particularly effective for complex, feature-rich recommendation scenarios. The performance of the UCB algorithm is also contingent upon parameter selection, making this an important factor to consider in practical implementations. Overall, both UCB and its modified version, LinUCB, present compelling solutions for the challenges faced by modern recommender systems.

A hybrid feature selection algorithm combining information gain and grouping particle swarm optimization for cancer diagnosis.

Cancer diagnosis based on machine learning has become a popular application direction. Support vector machine (SVM), as a classical machine learning algorithm, has been widely used in cancer diagnosis because of its advantages in high-dimensional and small sample data. However, due to the high-dimensional feature space and high feature redundancy of gene expression data, SVM faces the problem of poor classification effect when dealing with such data. Based on this, this paper proposes a hybrid feature selection algorithm combining information gain and grouping particle swarm optimization (IG-GPSO). The algorithm firstly calculates the information gain values of the features and ranks them in descending order according to the value. Then, ranked features are grouped according to the information index, so that the features in the group are close, and the features outside the group are sparse. Finally, grouped features are searched using grouping PSO and evaluated according to in-group and out-group. Experimental results show that the average accuracy (ACC) of the SVM on the feature subset selected by the IG-GPSO is 98.50%, which is significantly better than the traditional feature selection algorithm. Compared with KNN, the classification effect of the feature subset selected by the IG-GPSO is still optimal. In addition, the results of multiple comparison tests show that the feature selection effect of the IG-GPSO is significantly better than that of traditional feature selection algorithms. The feature subset selected by IG-GPSO not only has the best classification effect, but also has the least feature scale (FS). More importantly, the IG-GPSO significantly improves the ACC of SVM in cancer diagnostic.

Credit Card Deception Recognition Using Random Forest Machine Learning Algorithm

INTRODUCTION: The credit card deception poses a global threat, resulting in significant monetary losses and identity theft. Detecting fraudulent transactions promptly is crucial for mitigating these losses. Machine learning algorithms, specifically the random forest algorithm, show promise in addressing this issue.
 OBJECTIVES: This research paper presents a comprehensive study of numerous machine learning algorithms for credit card deception recognition, focusing on the random forest algorithm.
 METHODS: To tackle the increasing fraud challenges and the need for more effective detection systems, we develop an advanced credit card deception detection system utilizing machine learning algorithms. We evaluate our system's performance using precision, recall, & F1-score metrics. Additionally, we provide various insights into the key features for fraud detection, empowering financial institutions to enhance their detection systems. The paper follows a structured approach.
 RESULTS: We review existing work on credit card fraud detection, detail the dataset and pre-processing steps, present the random forest algorithm and its application to fraud detection, compare its performance against other algorithms, discuss fraud detection challenges, and propose effective solutions.
 CONCLUSION: Finally, we conclude the research paper and suggest potential areas for future research. Our experiments demonstrate that the random forest algorithm surpasses other machine learning algorithms in accuracy, precision, recall, & F1-score. Moreover, the system effectively addresses challenges like imbalanced data and high-dimensional feature spaces. Our findings offer valuable insights into the most relevant features for fraud detection empowering financial organizations to improve their fraud detection capabilities.

A Novel Framework for Forest Above-Ground Biomass Inversion Using Multi-Source Remote Sensing and Deep Learning

The estimation of forest above-ground biomass (AGB) can be significantly improved by leveraging remote sensing (RS) and deep learning (DL) techniques. In this process, it is crucial to obtain appropriate RS features and develop a suitable model. However, traditional methods such as random forest (RF) feature selection often fail to adequately consider the complex relationships within high-dimensional RS feature spaces. Moreover, challenges related to parameter selection and overfitting inherent in DL models may compromise the accuracy of AGB estimation. Therefore, this study proposes a novel framework based on freely available Sentinel-1 synthetic aperture radar (SAR) and Sentinel-2 optical data. Firstly, we designed new indices through the formula analogous with vegetation index calculation to integrate multidimensional spectral and structural information. Then, leveraging the simplicity of computational principles, a pigeon-inspired optimization algorithm (PIO) was introduced into a bi-directional long short-term memory neural network (PIO-BiLSTM), which achieved the set objective function through repeated iteration and validation to obtain the optimal model parameters. Finally, to verify the framework’s effect, we conducted experiments in two different tree species and compared another seven classical optimization algorithms and machine learning models. The results indicated that the new indices significantly improved the inversion accuracy of all models in both categories, and the PIO-BiLSTM model achieved the highest accuracy (Category-1: R2 = 0.8055, MAE = 8.8475 Mg·ha−1, RMSE = 12.2876 Mg·ha−1, relative RMSE = 18.1715%; Category-2: R2 = 0.7956, MAE = 1.7103 Mg·ha−1, RMSE = 2.2887 Mg·ha−1, relative RMSE = 9.3000%). Compared with existing methods, the proposed framework greatly reduced the labor costs in parameter selection, and its potential uncertainty also decreased by up to 9.0%. Furthermore, the proposed method has a strong generalization ability and is independent of tree species, indicating its great potential for future forest AGB inversion in wider regions with diverse forest types.

High-Dimensional Ensemble Learning Classification: An Ensemble Learning Classification Algorithm Based on High-Dimensional Feature Space Reconstruction

When performing classification tasks on high-dimensional data, traditional machine learning algorithms often fail to filter out valid information in the features adequately, leading to low levels of classification accuracy. Therefore, this paper explores the high-dimensional data from both the data feature dimension and the model ensemble dimension. We propose a high-dimensional ensemble learning classification algorithm focusing on feature space reconstruction and classifier ensemble, called the HDELC algorithm. First, the algorithm considers feature space reconstruction and then generates a feature space reconstruction matrix. It effectively achieves feature selection and reconstruction for high-dimensional data. An optimal feature space is generated for the subsequent ensemble of the classifier, which enhances the representativeness of the feature space. Second, we recursively determine the number of classifiers and the number of feature subspaces in the ensemble model. Different classifiers in the ensemble system are assigned mutually exclusive non-intersecting feature subspaces for model training. The experimental results show that the HDELC algorithm has advantages compared with most high-dimensional datasets due to its more efficient feature space ensemble capability and relatively reliable ensemble operation performance. The HDELC algorithm makes it possible to solve the classification problem for high-dimensional data effectively and has vital research and application value.

In defense of local descriptor-based few-shot object detection.

State-of-the-art image object detection computational models require an intensive parameter fine-tuning stage (using deep convolution network, etc). with tens or hundreds of training examples. In contrast, human intelligence can robustly learn a new concept from just a few instances (i.e., few-shot detection). The distinctive perception mechanisms between these two families of systems enlighten us to revisit classical handcraft local descriptors (e.g., SIFT, HOG, etc.) as well as non-parametric visual models, which innately require no learning/training phase. Herein, we claim that the inferior performance of these local descriptors mainly results from a lack of global structure sense. To address this issue, we refine local descriptors with spatial contextual attention of neighbor affinities and then embed the local descriptors into discriminative subspace guided by Kernel-InfoNCE loss. Differing from conventional quantization of local descriptors in high-dimensional feature space or isometric dimension reduction, we actually seek a brain-inspired few-shot feature representation for the object manifold, which combines data-independent primitive representation and semantic context learning and thus helps with generalization. The obtained embeddings as pattern vectors/tensors permit us an accelerated but non-parametric visual similarity computation as the decision rule for final detection. Our approach to few-shot object detection is nearly learning-free, and experiments on remote sensing imageries (approximate 2-D affine space) confirm the efficacy of our model.

Koopman fault‐tolerant model predictive control

AbstractThis paper introduces a novel data‐driven approach to develop a fault‐tolerant model predictive controller (MPC) for non‐linear systems. By adopting a Koopman operator‐theoretic perspective, the proposed method leverages historical data from the system to construct a data‐driven model that captures the non‐linear behaviour and fault characteristics. The fault influence is addressed through an online estimation of a time‐varying Koopman predictor, which allows for adjusting the MPC control law to counteract the fault effects. This estimation is performed in a higher dimensional Koopman feature space, where the dynamics behave linearly. As a result, the non‐linear fault‐tolerant MPC optimization problem can be replaced with a more practical and feasible linear time‐varying one using the approximated Koopman predictor. Moreover, by incorporating the online update procedure, the time‐varying Koopman predictor can represent the dynamics of the faulty system. Hence, the controller can adapt and compensate for the faults in real‐time, integrating the fault diagnosis module in the MPC framework and eliminating the need for a separate fault detection unit. Finally, the efficacy of the proposed approach is demonstrated through case study results, which highlight the ability of the controller to mitigate faults and maintain desired system behaviour.

A Partial-Label U-Net Learning Method for Compound-Fault Diagnosis With Fault- Sample Class Imbalance

In the operation process of the rotating machinery, compound faults have various combination forms and are difficult to reproduce, which results in the scarcity of training samples of various compound faults. In this paper, a method of partial-label learning and classification fusion is proposed to investigate the above problems. Wavelet packet transformation (WPT) and dimensionless parameterization (DP) are utilized to extract features in high-dimensional space. A label-specific feature learning architecture is proposed to make up for the shortage of handcrafted features. An improved focal loss is exploited for learning prominent compound-fault characteristics and imbalanced fault samples. Basic probability assignments (BPA) about fault classes are generated from each partial classifier and combined to obtain a final inference about fault types. Experimental results of single-fault and compound-fault classification illustrate the effectiveness of the proposed method.

A Novel High-Dimensional Kernel Joint Non-Negative Matrix Factorization With Multimodal Information for Lung Cancer Study.

Judging and identifying biological activities and biomarkers inside tissues from imaging features of diseases is challenging, so correlating pathological image data with genes inside organisms is of great significance for clinical diagnosis. This paper proposes a high-dimensional kernel non-negative matrix factorization (NMF) method based on muti-modal information fusion. This algorithm can project RNA gene expression data and pathological images (WSI) into a common feature space, where the heterogeneous variables with the largest coefficient in the same projection direction form a co-module. In addition, the miRNA-mRNA and miRNA-lncRNA interaction networks in the ceRNA network are added to the algorithm as a priori information to explore the relationship between the images and the internal activities of the gene. Furthermore, the radial basis kernel function is used to calculate the feature proportion between different kinds of genes mapped in the high-dimensional feature space and projected into the common feature space to explore the gene interaction in the high-dimensional situation. The original feature matrix is regularized to improve biological correlation, and the feature factors are sparse by orthogonal constraints to reduce redundancy. Experimental results show that the proposed NMF method is better than the traditional NMF method in stability, decomposition accuracy, and robustness. Through data analysis applied to lung cancer, genes related to tissue morphology are found, such as COL7A1, CENPF and BIRC5. In addition, gene pairs with a correlation degree exceeding 0.8 are found, and potential biomarkers of significant correlation with survival are obtained such as CAPN8. It has potential application value for the clinical diagnosis of lung cancer.