High-dimensional Feature Space Research Articles

Innovative Visualization Approach for Biomechanical Time Series in Stroke Diagnosis Using Explainable Machine Learning Methods: A Proof-of-Concept Study

Stroke remains a predominant cause of mortality and disability worldwide. The endeavor to diagnose stroke through biomechanical time-series data coupled with Artificial Intelligence (AI) poses a formidable challenge, especially amidst constrained participant numbers. The challenge escalates when dealing with small datasets, a common scenario in preliminary medical research. While recent advances have ushered in few-shot learning algorithms adept at handling sparse data, this paper pioneers a distinctive methodology involving a visualization-centric approach to navigating the small-data challenge in diagnosing stroke survivors based on gait-analysis-derived biomechanical data. Employing Siamese neural networks (SNNs), our method transforms a biomechanical time series into visually intuitive images, facilitating a unique analytical lens. The kinematic data encapsulated comprise a spectrum of gait metrics, including movements of the ankle, knee, hip, and center of mass in three dimensions for both paretic and non-paretic legs. Following the visual transformation, the SNN serves as a potent feature extractor, mapping the data into a high-dimensional feature space conducive to classification. The extracted features are subsequently fed into various machine learning (ML) models like support vector machines (SVMs), Random Forest (RF), or neural networks (NN) for classification. In pursuit of heightened interpretability, a cornerstone in medical AI applications, we employ the Grad-CAM (Class Activation Map) tool to visually highlight the critical regions influencing the model’s decision. Our methodology, though exploratory, showcases a promising avenue for leveraging visualized biomechanical data in stroke diagnosis, achieving a perfect classification rate in our preliminary dataset. The visual inspection of generated images elucidates a clear separation of classes (100%), underscoring the potential of this visualization-driven approach in the realm of small data. This proof-of-concept study accentuates the novelty of visual data transformation in enhancing both interpretability and performance in stroke diagnosis using limited data, laying a robust foundation for future research in larger-scale evaluations.

Self-Attentive Local Aggregation Learning With Prototype Guided Regularization for Point Cloud Semantic Segmentation of High-Speed Railways

Point cloud semantic segmentation for railway infrastructures is an essential step towards establishing railway digital twins. Deep learning-based methods have shown great potential in this field compared to traditional methods that rely on hand-crafted features. However, deep learning-based methods for railway point clouds still face typical challenges that need to be addressed. In this regard, we propose a novel learning framework named SALAProNet, which consists of a set of effective and concise modular solutions. The first challenge addressed is the massive data scale of railway point clouds, which makes it difficult to directly process large-scale point clouds due to memory limitations. To solve this problem, we adapt efficient random sampling in the network and propose the Self-Attentive Aggregation (SAA) module based on an attention mechanism to greatly expand the receptive field, which covers the unsampled points and successfully retains information in a high-dimensional feature space. The second challenge is fine-grained segmentation, where we propose the Local Geometry Embedding (LGE) module to embed local geometry. With the help of context information provided by SAA, the network can perform fine-grained segmentation for railway infrastructures. The third challenge is the insufficient generalization ability of the network, where we propose a Prototype Guided Regularization (PGR) method to guide the network to segment the point cloud among railways with different construction standards. This method enhances the network’s interpretability and improves its generalization ability. We have validated our proposed framework through experiments on different datasets, and it outperforms state-of-the-art approaches.

Support vector machine in ultrahigh-dimensional feature space

Classification and feature selection play an important role in knowledge discovery in high-dimensional data. Although penalized Support Vector Machine (SVM) is among the most powerful methods for classification and automatic feature selection in high-dimensional feature space, it is not directly applicable to ultrahigh-dimensional cases, wherein the number of features far exceeds the sample size. In this paper, we suggest an efficient two-step method for simultaneous classification and identifying important features in the setting of ultrahigh-dimensional models. Specifically, we first develop an independence screening procedure to reduce the dimensionality of the feature space to a moderate scale, and then penalized support vector machine is applied to the dimension-reduced feature space to select important features further and estimate the coefficients, via a (penalized) model fit. Implementation of the suggested two-step method is not limited by the dimensionality of the models and entails much less computational cost. Numerical examples and a real data analysis are used to demonstrate the finite sample performance of our proposal.

A Hybrid Multiwavelet Transform with Grey Wolf Optimization Used for an Efficient Classification of Documents

In machine learning, feature selection is crucial to increase performance and shorten the model's learning time. It seeks to discover the pertinent predictors from high-dimensional feature space. However, a tremendous increase in the feature dimension space poses a severe obstacle to feature selection techniques. Study process to address this difficulty, the authors suggest a hybrid feature selection method consisting of the Multiwavelet transform and Gray Wolf optimization. The proposed approach minimizes the overall downsides while cherry picking the benefits of both directions. This notable wavelet transform development employs both wavelet and vector scaling functions. Additionally, multiwavelets have orthogonality, symmetry, compact support, and significant vanishing moments. One of the most advanced areas of study of artificial intelligence is optimization algorithms. Grey Wolf Optimization (GWO) here produced artificial techniques that yielded good performance results and were more responsive to current needs. Keywords — About four key words or phrases in order of importance, separated by commas, used to compile the subject index for the last issue for the year.

Improving the accuracy of sentiment analysis using a linguistic rule-based feature selection method in tourism reviews

Sentiment Analysis technique involves extracting the relevant information from Unstructured User Reviews (UUR) dataset fetched from online and classifying them into appropriate positive and negative comments for making decisions. In UUR, data may be in noisy state, irrelevant features exist which creates high dimensional feature space. To design an effective sentiment learning model, users are required to extract the most relevant sentiment features from UUR. To overcome the issue, we proposed a Linguistic rule based feature selection method for extracting and selecting the sentiment features for Sentiment Analysis as it improves the predictive performance of classification algorithms. The proposed novel feature selection method involves identifying the various sentiment features in the review dataset by using filtering methods such as POS tags, n-grams. In the ensemble model, where the Random Forest classification algorithm is trained for textual sentiment classification, the chosen sentiment feature sets are used. Finally, we test our approach using the real-time review dataset that was collected from a multitude of sources, and the results demonstrate prediction accuracy that is superior to that of existing Sentiment analysis techniques.

Industrial process monitoring with density-based kernel structure preserving projections model

Practice shows that the influence of latent geometric structures and nonlinear relationships in the industrial data on process monitoring cannot be ignored. Traditional methods focus on the distance information of a pair of points, while we think that the density information of more points can better reflect the geometry shape of the data. Based on this idea, this paper further combines density information with distance information to preserve data latent structure information, a nonlinear process monitoring method to extract global-local spatial geometric structure information is proposed, called density-based kernel structure preserving projections (DKSPP). The principle of this method is: First, search for the neighbors and non-neighbors of a sample in the high-dimensional feature space after nonlinear mapping. Then, the spatial distance information and density information among the data points are comprehensively constructed, and the projection points after dimensionality reduction are constrained to achieve global-local structure preservation. Finally, the obtained projections are used to set anomaly detection indices, and a DKSPP-based anomaly detection model is implemented to monitor processes. The experimental results of a benchmark dataset and an actual industrial application demonstrate the high detection rates and performance of the proposed method.

A novel semi-supervised fault detection and isolation method for battery system of electric vehicles

The detection and isolation of early and minor faults in vehicle battery systems is vital to safe driving and improving power utilization. This paper proposes a data-driven model to achieve accurate, early, and economical fault detection and isolation. The model is based on kernel principal component analysis (KPCA), which maps complex nonlinear data from the input space into a high-dimensional feature space to gain a detection model with good performance. To overcome the difficulty of hyperparameter selection, KPCA is trained using Bayesian Optimization (BO) iterations with a small amount of labeled data and a large amount of unlabeled data. This step can obtain the optimal hyperparameter to greatly improve the model fault detection capability, which is beneficial for detecting both early faults and minor faults. In addition, a unified contribution graph based on the partial differentiation of KPCA was adopted to build a reasonable isolation scheme. The semi-supervised model of KPCA based on Bayesian Optimization and contribution graph is developed to reveal the relationship between fault and variable. Finally, the proposed method is fully tested on four fault datasets and the results prove the excellent detection capability in the early stage of faults compared with other methods and the accurate fault isolation capability from the occurrence to the end of the fault.

ERPCA-GFCM: An Enhanced DDoS Detection Model on Internet of Things

This research paper introduced an Enhanced Distributed Denial-of-Service (DDoS) Detection model specifically designed for IoT devices. Given the prevalence of DDoS attacks targeting IoT devices, which involve overwhelming a system with malicious traffic to disrupt its normal functioning, the proposed model aimed to enhance the security and resilience of IoT networks. To address this, the proposed model integrated multiple techniques to improve detection and classification accuracy. The first technique, ER-Relief algorithm, is a feature selection method made to address the presence of noise and outliers in the dataset by minimizing a loss function based on the empirical sum of margins. To further enhance the model's performance, Principal Component Analysis (PCA) was utilized for dimensionality reduction. PCA transforms the original high-dimensional feature space into a lower-dimensional space while preserving the most critical information. To achieve better clustering results, the model incorporates the Global Fuzzy C-means algorithm. This algorithm addressed the issue of sensitivity to initial conditions, which lead to suboptimal clustering results. By incorporating fuzzy logic principles, Global Fuzzy C-means assigning data points to multiple clusters with varying degrees of membership, providing a more nuanced representation of the underlying data structure. Lastly, the Random Forest algorithm was employed for training and testing the model. The model was then tested on the CICDDoS2019 dataset, which contains three (3) types of DDoS attacks, namely DNS, UDP, and MSSQL attacks. Based on the evaluation results, the proposed model achieved an impressive accuracy of 97.92%, recall of 97.92%, F1-score of 97.90%, and precision of 97.93%. These metrics highlighted the model's effectiveness, showcasing its ability to accurately detect and classify various types of DDoS attacks with high precision and recall. This research contributes to the advancement of network security by providing a robust and reliable solution for combating DDoS attacks.

Comparative Analysis of Firefly and Modified Firefly Algorithms in Multimodal Biometric Authentication Systems

Abstract: This research focuses on addressing the challenges of high-dimensional feature spaces and selecting significant features in multimodal biometric systems. Feature-level fusion poses persistent issues that require extensive investigation due to its potential for enhancing biometric recognition accuracy. This study proposes the integration of meta-heuristic optimization techniques into the feature selection phase of a multimodal biometric system, prior to the classification phase, to identify the most relevant features from two distinct biological traits. To determine the authorization status of an individual, the fused feature vectors are inputted into a Support Vector Machine (SVM) classifier. The study leverages the physiological biometrics of the face and iris to validate the findings of previous research, highlighting the exceptional accuracy of iris recognition and the natural acceptability of face recognition for identity verification purposes.

Mental State Detection From Tweets By Machine Learning

The world over, mental illness is a serious issue. Many people use the social media that may affect their mental health positively, but often result in negative sentiments. This research aims to determine an individual's mental state based on their social media behavior on Twitter. We analysed a dataset including 170000 real tweets by using natural language processing and machine learning techniques. Decision tree, support vector machine, and recurrent neural network (RNN) were used for classifying twitter users, to detect if they are in positive or negative mental state. These models were compared to determine which approach provides more accurate detection of a positive/negative mental state. Then, the RNN yielded the highest accuracy 0.76 among the models, with the precision, recall, and the F_1 score being 0.75, 0.74, and 0.75, respectively. The truncated singular value decomposition was also utilised to visualise the high-dimensional feature space of the data.

GTR-GA: Harnessing the power of graph-based neural networks and genetic algorithms for text augmentation

Text augmentation is a popular technique in natural language processing (NLP) that has been shown to improve the performance of various downstream tasks. The goal of text augmentation is to generate additional training data from existing data, thereby increasing the amount of data available for training machine learning models. In this paper, we propose a novel approach to text augmentation that combines the power of graph-based neural networks and genetic algorithms. Our proposed scheme, called GTR-GA, is designed to generate diverse and high-quality augmented text data by exploring the high-dimensional feature space of text data using graph-based neural networks and genetic algorithms. Our approach utilizes a graph attention network (GAT) based model, called HetGAPN, to learn node representations in a heterogeneous graph representing text data. We introduce the concept of node aggregation and bidirectional attention in HetGAPN to better capture the relationships between different features of the input text data. We then use a genetic algorithm to generate a diverse set of candidate embeddings that explore different parts of the input space. Our approach uses perplexity as the objective function to evaluate the quality of the augmented text data generated by each candidate embedding. The most fit candidate embeddings are selected as parents, and crossover and mutation operators are used to generate a new population of candidate embeddings. Our experimental results show that GTR-GA is capable of generating high-quality augmented text data that improves the performance of various downstream NLP tasks, such as sentiment analysis and text classification. The proposed scheme can be applied to a wide range of NLP tasks and can help overcome the data scarcity problem that is often encountered in NLP.

Hybrid Filter-Genetic Feature Selection Method For Arabic Sentiment Analysis

The dramatic increase in user comments describing their feelings about products, services, and events brings sentiment analysis to the forefront as a way to monitor public opinion about products and events. Feature selection is an important subtask of sentiment analysis, which aims to improve the performance of learning algorithms and reduce the dimensionality of a problem. Feature selection is an important subtask of sentiment analysis, as it can improve the performance of learning algorithms while reducing the dimensionality of a problem. Moreover, the high-dimensional feature spaces caused by the morphological richness of Arabic motivate further research in this area. In this paper, a hybrid filter-based and genetic feature selection algorithm is proposed using four machine learning algorithms, namely decision tree, Naive-Bayes, K-NN and meta-ensemble methods. The performance of the proposed algorithm is compared with the performance of baseline models. A wide range of experiments are conducted on two standard Arabic datasets. The experimental results clearly show that the improved methods outperform the other baseline models for Arabic sentiment analysis. The results show that the improved models outperform traditional approaches in terms of classification accuracy, with a 5% increase in the macro average of F1.

A subregion-based survival prediction framework for GBM via multi-sequence MRI space optimization and clustering-based feature bundling and construction

Aiming at accurate survival prediction of Glioblastoma (GBM) patients following radiation therapy, we developed a subregion-based survival prediction framework via a novel feature construction method on multi-sequence MRIs. The proposed method consists of two main steps: (1) a feature space optimization algorithm to determine the most appropriate matching relation derived between multi-sequence MRIs and tumor subregions, for using multimodal image data more reasonable; (2) a clustering-based feature bundling and construction algorithm to compress the high-dimensional extracted radiomic features and construct a smaller but effective set of features, for accurate prediction model construction. For each tumor subregion, a total of 680 radiomic features were extracted from one MRI sequence using Pyradiomics. Additional 71 geometric features and clinical information were collected resulting in an extreme high-dimensional feature space of 8231 to train and evaluate the survival prediction at 1 year, and the more challenging overall survival prediction. The framework was developed based on 98 GBM patients from the BraTS 2020 dataset under five-fold cross-validation, and tested on an external cohort of 19 GBM patients randomly selected from the same dataset. Finally, we identified the best matching relationship between each subregion and its corresponding MRI sequence, a subset of 235 features (out of 8231 features) were generated by the proposed feature bundling and construction framework. The subregion-based survival prediction framework achieved AUCs of 0.998 and 0.983 on the training and independent test cohort respectively for 1 year survival prediction, compared to AUCs of 0.940 and 0.923 for survival prediction using the 8231 initial extracted features for training and validation cohorts respectively. Finally, we further constructed an effective stacking structure ensemble regressor to predict the overall survival with the C-index of 0.872. The proposed subregion-based survival prediction framework allow us to better stratified patients towards personalized treatment of GBM.

Parametric Model for Coaxial Cavity Filter with Combined KCCA and MLSSVR

Aiming at the problems of poor data effectiveness, low modeling accuracy, and weak generalization in the tuning process of microwave cavity filters, a parametric model for coaxial cavity filter using kernel canonical correlation analysis (KCCA) and multioutput least squares support vector regression (MLSSVR) is proposed in this study. First, the low-dimensional tuning data is mapped to the high-dimensional feature space by kernel canonical correlation analysis, and the nonlinear feature vectors are fused by the kernel function; second, the multioutput least squares support vector regression algorithm is used for parametric modeling to solve the problems of low accuracy and poor prediction performance; third, the support vector of the parameter model is optimized by the differential evolution whale algorithm (DWA) to improve the convergence and generalization ability of the model in actual tuning. Finally, the tuning experiments of two cavity filters with different topologies are carried out. The experimental results show that the proposed method has an obvious improvement in generalization performance and prediction accuracy compared with the traditional methods.

Multichannel Approach for Sentiment Analysis Using Stack of Neural Network with Lexicon Based Padding and Attention Mechanism

Abstract Sentiment analysis (SA) has been an important focus of study in the fields of computational linguistics and data analysis for a decade. Recently, promising results have been achieved when applying DNN models to sentiment analysis tasks. Long short-term memory (LSTM) models, as well as its derivatives like gated recurrent unit (GRU), are becoming increasingly popular in neural architecture used for sentiment analysis. Using these models in the feature extraction layer of a DNN results in a high dimensional feature space, despite the fact that the models can handle sequences of arbitrary length. Another problem with these models is that they weight each feature equally. Natural language processing (NLP) makes use of word embeddings created with word2vec. For many NLP jobs, deep neural networks have become the method of choice. Traditional deep networks are not dependable in storing contextual information, so dealing with sequential data like text and sound was a nightmare for such networks. This research proposes multichannel word embedding and employing stack of neural networks with lexicon-based padding and attention mechanism (MCSNNLA) method for SA. Using convolution neural network (CNN), Bi-LSTM, and the attention process in mind, this approach to sentiment analysis is described. One embedding layer, two convolution layers with max-pooling, one LSTM layer, and two fully connected (FC) layers make up the proposed technique, which is tailored for sentence-level SA. To address the shortcomings of prior SA models for product reviews, the MCSNNLA model integrates the aforementioned sentiment lexicon with deep learning technologies. The MCSNNLA model combines the strengths of emotion lexicons with those of deep learning. To begin, the reviews are processed with the sentiment lexicon in order to enhance the sentiment features. The experimental findings show that the model has the potential to greatly improve text SA performance.

Overlapping group screening for binary cancer classification with TCGA high-dimensional genomic data.

Precision medicine has been a global trend of medical development, wherein cancer diagnosis plays an important role. With accurate diagnosis of cancer, we can provide patients with appropriate medical treatments for improving patients' survival. Since disease developments involve complex interplay among multiple factors such as gene-gene interactions, cancer classifications based on microarray gene expression profiling data are expected to be effective, and hence, have attracted extensive attention in computational biology and medicine. However, when using genomic data to build a diagnostic model, there exist several problems to be overcome, including the high-dimensional feature space and feature contamination. In this paper, we propose using the overlapping group screening (OGS) approach to build an accurate cancer diagnosis model and predict the probability of a patient falling into some disease classification category in the logistic regression framework. This new proposal integrates gene pathway information into the procedure for identifying genes and gene-gene interactions associated with the classification of cancer outcome groups. We conduct a series of simulation studies to compare the predictive accuracy of our proposed method for cancer diagnosis with some existing machine learning methods, and find the better performances of the former method. We apply the proposed method to the genomic data of The Cancer Genome Atlas related to lung adenocarcinoma (LUAD), liver hepatocellular carcinoma (LHC), and thyroid carcinoma (THCA), to establish accurate cancer diagnosis models.

Hybrid metaheuristic method of ABC kernel filtering for nonlinear acoustic echo cancellation

This research work uses nonlinear kernel adaptive filtering to eliminate echo, a type of voice that is undesirable and communicated over the microphone. Adaptive filters enable the detection of time-varying potentials and the tracking of the signal’s dynamic changes. In this work, a hybrid metaheuristic technique that combines the Artificial Bee Colony (ABC) with the Kernel Adaptive Improved Proportionate and Normalized Least Mean Square (KIPNLMS) filter is developed to significantly boost echo cancellation efficiency. The unique adaptive filter based on sub-filter and proportionate adaptation is combined in the advanced technique, which presents an improved Non-linear Acoustic Echo Cancellation (NAEC) framework. An essential part of kernel techniques is that they translate the information on to a high-dimensional feature space, where linear filtering is performed. In the initial space, the arithmetic operations are carried out by evaluating inner products among pairs of input patterns, also known as kernels. Along with the ABC step size regulation technique, kernel-based adaptive filtering improves Signal to Noise Ratio (SNR) and accelerates convergence. The algorithm's effectiveness is assessed through simulation using signal Echo Return Loss Enhancement (ERLE). The performance of Kernel LMS, Kernal Normalized LMS (KNLMS), Kernal Proportionate NLMS, and KIPNLMS algorithms are compared. In contrast to typical kernel adaptive filters, the suggested ABC-KIPNLMS increases the ERLE more than 10 dB compared to other algorithms.

Leaky FinFET for Reservoir Computing with Temporal Signal Processing.

Reservoir computing can greatly reduce the hardware and training costs of recurrent neural networks with temporal data processing. To implement reservoir computing in a hardware form, physical reservoirs transforming sequential inputs into a high-dimensional feature space are necessary. In this work, a physical reservoir with a leaky fin-shaped field-effect transistor (L-FinFET) is demonstrated by the positive use of a short-term memory property arising from the absence of an energy barrier to suppress the tunneling current. Nevertheless, the L-FinFET reservoir does not lose its multiple memory states. The L-FinFET reservoir consumes very low power when encoding temporal inputs because the gate serves as an enabler of the write operation, even in the off-state, due to its physical insulation from the channel. In addition, the small footprint area arising from the scalability of the FinFET due to its multiple-gate structure is advantageous for reducing the chip size. After the experimental proof of 4-bit reservoir operations with 16 states for temporal signal processing, handwritten digits in the Modified National Institute of Standards and Technology dataset are classified by reservoir computing.

Normal Sinkhorn Distance: A novel metric for evaluating generated signals and its application in mechanical fault diagnosis

Nowadays, leveraging data augmentation-based methods to address the data-shortage problem in diagnosis field becomes fairly prevailing, while assessing the quality of the generated data receives little attention. Typically, the data quality is evaluated by straightforward employing some existing shallow functions or simple classification models, which have several disadvantages. In this paper, the limitations of existing techniques are comprehensively summarized and illustrated by pathological examples. Accordingly, a novel metric method called Normal Sinkhorn Distance (NSD) is developed to evaluate the quality of vibration data. Firstly, the raw vibration data are directly transformed into time–frequency images with a tensor format and stored to ensure the following computation efficiency. Then an off-the-shelf pretrained Inception model is employed to capture their high-order data structure and amplitude-wise dependence. Finally, the NSD metric is constructed and adopted to measure the similarity between the real data and generated data in a high-dimensional feature space. Extensive results indicate that the proposed NSD provides a more reliable indication for practitioners than the existing evaluation measures on the data quality.

TMBserval: a statistical explainable learning model reveals weighted tumor mutation burden better categorizing therapeutic benefits.

A high tumor mutation burden (TMB) is known to drive the response to immune checkpoint inhibitors (ICI) and is associated with favorable prognoses. However, because it is a one-dimensional numerical representation of non-synonymous genetic alterations, TMB suffers from clinical challenges due to its equal quantification. Since not all mutations elicit the same antitumor rejection, the effect on immunity of neoantigens encoded by different types or locations of somatic mutations may vary. In addition, other typical genomic features, including complex structural variants, are not captured by the conventional TMB metric. Given the diversity of cancer subtypes and the complexity of treatment regimens, this paper proposes that tumor mutations capable of causing various degrees of immunogenicity should be calculated separately. TMB should therefore, be segmented into more exact, higher dimensional feature vectors to exhaustively measure the foreignness of tumors. We systematically reviewed patients' multifaceted efficacy based on a refined TMB metric, investigated the association between multidimensional mutations and integrative immunotherapy outcomes, and developed a convergent categorical decision-making framework, TMBserval (Statistical Explainable machine learning with Regression-based VALidation). TMBserval integrates a multiple-instance learning concept with statistics to create a statistically interpretable model that addresses the broad interdependencies between multidimensional mutation burdens and decision endpoints. TMBserval is a pan-cancer-oriented many-to-many nonlinear regression model with discrimination and calibration power. Simulations and experimental analyses using data from 137 actual patients both demonstrated that our method could discriminate between patient groups in a high-dimensional feature space, thereby rationally expanding the beneficiary population of immunotherapy.