Unsupervised Feature Extraction Research Articles

RDCSAE-RKRVFLN: A unified deep learning framework for robust and accurate DOA estimation

This paper introduces an innovative unified deep learning (DL) model, “reduced deep convolutional stack autoencoder (RDCSAE)-robust kernel-based random vector functional link network (RKRVFLN)” (RDCSAE-RKRVFLN), for addressing practical challenges like phase and gain uncertainty, mutual coupling effects, and element position errors in direction of arrival (DOA) estimation. The RDCSAE utilizes deep convolutional neural network (DCNN) and stack autoencoder (SAE) for robust feature extraction. The RKRVFLN integrates a concise SAE output with kernel-based learning. Thus, the integration of RDCSAE (suitable for unsupervised feature extraction) and RKRVFLN (for supervised learning) in the developed model results in efficient use of limited training data. Performance metrics of this method include among others the accuracy (separation between sources), root mean square error (RMSE), signal-to-noise ratio (SNR), etc. The method performs better than existing data driven methods in terms of computational efficiency, faster learning speed, enhanced model generalization, higher accuracy, and shorter event regression time and empirical methods. In the presence of above mentioned four perturbations, this method successfully resolves multiple sources with a resolution of 0.3° with a RMSE of 0.00376. For real time application, the paper implements RDCSAE-RKRVFLN on a high-speed reconfigurable Xilinx Virtex-5 field-programmable gate array (FPGA) realizing a digital DOA estimator for one source. The hardware estimates the DOA with a maximum error of 0.01953125 which follows closely the simulation results indicating the efficiency and feasibility of the method.

Systematic comparison of unsupervised feature extraction methods for enhancing discrimination accuracy of brown rice flour adulteration in LIBS

Laser-induced breakdown spectroscopy (LIBS) is widely applied in various critical fields due to its technical advantages. However, with the explosive growth in spectral data volume, the challenges of high-dimension and redundant information of spectra are becoming increasingly prominent. Therefore, dimensionality reduction of spectra is crucial for LIBS. In this study, to enhance the discrimination accuracy of brown rice flour and its adulterants, a systematic comparative analysis of eight unsupervised feature extraction methods was conducted, including principal component analysis (PCA), non-negative matrix factorization (NMF), independent component analysis (ICA), multidimensional scaling (MDS), kernel PCA (KPCA), locally linear embedding (LLE), Laplacian eigenmaps (LE), and isometric mapping (ISOMAP). The results indicate that linear feature extraction methods (PCA, NMF, ICA, and MDS) show a more obvious improvement in the analytical performance of the classification models, compared to nonlinear feature extraction methods (KPCA, LLE, LE, and ISOMAP). Among these methods, the commonly used PCA has the best improvement effect. The accuracy, macro precision, macro recall, and macro F1-score of the classification models improve from 92.73%, 94.13%, 92.92%, and 0.93 to 95.52%, 95.67%, 95.63%, and 0.96. Therefore, this study can provide experience and guidance for selecting appropriate unsupervised feature extraction methods in subsequent research, contributing to the further application and development of LIBS.

Prediction Model of Ancient Village Pottery Building Microspace Design Style by Integrating Machine Learning

Microspace design styles on pottery fragments offer a captivating window into the artistic heritage and cultural practices of past societies. These intricate details and recurring patterns hold valuable clues about rituals, beliefs, and artistic expressions. However, existing approaches struggle to capture the relationships between design elements within a single piece. This limitation hinders the ability to capture the holistic meaning conveyed by the microspace design style. To overcome this limitation, this research proposes a novel approach called MoANN-DSOA for predicting microspace design styles based on ancient village pottery. MoANN-DSOA utilizes a Mosaic Attention Neural Network (MoANN) to analyze both the visual image and the relationships between design elements. This allows for a more detailed understanding of the artistic message encoded within each fragment. Additionally, a Dove Swarm Optimization Algorithm (DSOA) optimizes the MoANN architecture, potentially enhancing the accuracy of capturing intricate details. The proposed MoANN-DSOA method attains 7.78%, 27.89% and 33.335% higher accuracy and 14%, 20.81% and 32.36% higher F-Score compared to the existing methods like Categorization and Retrieval of Painted Pottery with CNNs (CNN), Utilizing CNN-VGG16-VGG19 Approach for Distinguishing Surface Treatments in Wheel-Thrown Pottery (CNN-VGG16-VGG19) and Unsupervised Feature Extraction of Ceramic Profiles using a Deep Variational Convolutional Autoencoder (DVCA) respectively. By this, the proposed MoANN-DSOA methodology paves the way for efficient information extraction from pottery image, unlocking deeper understanding and facilitating the construction of more informed historical narratives.

SAFE: Unsupervised image feature extraction using self‐attention based feature extraction network

AbstractThe ability to extract high‐quality features from data is critical for machine learning applications. With the development of deep learning, various methods have been developed for image feature extraction, and unsupervised techniques have gained popularity due to their ability to operate without response variables. Autoencoders with encoder–decoder architectures are a common example of such techniques, but they are limited by a lack of proportional relationship between model reconstruction and encoder feature extraction performance. If the decoder is composed of multiple layers and mapping to a higher dimension is easier, the feature extraction performance of the encoder is likely to decrease. However, previous research has not adequately addressed this limitation. This study identifies the limitations of conventional unsupervised feature extraction techniques that utilize the encoder–decoder architecture, and proposes a novel feature extraction technique called SAFE, which utilizes a self‐attention mechanism to eliminate decoder effects and improve the performance of encoder. To validate the effectiveness of the proposed model, we conducted experiments using diverse datasets (MNIST, Fashion MNIST, SVHN, and WM811K). The results of the experiments demonstrated that our proposed method exhibited, on average, 2%–10% higher performance in terms of accuracy and F‐measure compared to the existing feature extraction techniques in the classification problem. While our research has limitations, specifically in its applicability only to the selection of image features, future studies should be undertaken to explore its potential application in various fields.

Unsupervised feature extraction based on uncorrelated approach

In high-dimensional spaces, mathematically driven data processing methods have recently attracted a lot of attention. We consider the situation when information is obtained by sampling a probability distribution with support on or close to a sub-manifold of Euclidean space. In this paper, we provide an innovative unsupervised learning method called Uncorrelated Neighborhood Preserving Embedding (UNPE) which identifies the underlying manifold structure of a data set and preserves the neighborhood structure of the data set. We provide a concrete formulation with UNPE, an iterative technique to demonstrate the usefulness of the framework, which has been confirmed by experimental findings on datasets Coil20, Pie, Tox, and Prostate-GE that uses three different parameters viz., F-score, NMI, and accuracy. It is observed that performance is better than the LPP algorithm by 1%, PCA by 2%, and more than 2% of LLE and LE algorithms.

Identification of liquor adulteration by Raman spectroscopy method based on ICNAFS

The health of consumers can be impacted by the additives placed into the liquor. To address the issues of poor accuracy, low reliability, and complex operational procedures in identifying adulteration in existing liquor, an improved convex non-negative matrix factorization (ICNAFS) with an adaptive graph constraint for unsupervised feature extraction is proposed in this paper, with the goal of achieving rapid identification of adulteration in liquor by Raman spectroscopy through dimensionality reduction. For the sake to streamline the calculation process for effective feature extraction and increase the accuracy of the analyzed model, the proposed ICNAFS method incorporates two fundamental models, such as ridge regression and convex non-negative matrix factorization (NMF). In particular, dimensionality reduction of the original spectrum is initially conducted using Principal Component Analysis (PCA), Sequential Projection Algorithm (SPA), Convex Non-Negative Matrix Factorization with an Adaptive Graph Constraint (CNAFS), and ICNAFS respectively. k-means is subsequently employed to merge the four models for clustering analysis. The results suggest that the accuracy of the presented ICNAFS-assisted k-means model is higher than the other techniques, with a clustering accuracy of 98.67%, exhibiting a 4% improvement over the existing CNAFS, through examination of 150 sets of tainted liquor data from five categories of samples. This demonstrates the potency of the proposed ICNAFS-assisted k-means clustering model in conjunction with Raman spectroscopy as a method for detecting tainted liquor.

Residual life prediction of multi-layer composite coatings based on vector quantised and attention-variational AutoEncoder network

ABSTRACT The quality of multi-layer composite coatings (MLCC) directly affects the reliability of equipment. Ultrasonic C-scan has advantages of non-destructive and high-precision. The accurate determination of residual life (RL) from C-scan images of coatings is hindered by the difficulty of feature extraction. To address this issue, a RL prediction method for MLCC based on the Vector Quantised and Attention-Variational AutoEncoder (VQA-VAE) network is proposed. First, by incorporating attention mechanism and residual module, the VQA-VAE network can perform unsupervised feature extraction on C-scan images effectively. Then, the features are fused by two-dimensional entropy weighting method to get the health index (HI), which can be used to establish the RL prediction model. To verify the effectiveness of this method, C-scan is performed on MLCC by ultrasonic microscope to obtain C-scan images, and RL of MLCC is predicted. The experiment results demonstrate that this method achieves higher accuracy in predicting RL of MLCC. In a word, this is a non-destructive, high-efficiency, high-precision, and large-scale measurement technique for MLCC.

Graph Representation Learning-Based Fixed-Length Clinical Feature Vector Generation from Heterogeneous Medical Records.

Transformation of patient data extracted from a database into fixed-length numerical vectors requires expertise in topical medical knowledge as well as data manipulation-thus, manual feature design is labor-intensive. In this study, we propose a machine learning-based method to for this purpose applicable to electronic medical data recorded during hospitalization, which utilizes unsupervised feature extraction based on graph embedding. Unsupervised learning is performed on a heterogeneous graph using Graph2Vec, and the inclusion of clinically useful data in the obtained embedding representation is evaluated by predicting readmission within 30 days of discharge based on it. The embedded representations are observed to improve predictive performance significantly as the information contained in the graph increases, indicating the suitability of the proposed method for feature design corresponding to clinical information.

An end-to-end intrusion detection system with IoT dataset using deep learning with unsupervised feature extraction

An end-to-end intrusion detection system with IoT dataset using deep learning with unsupervised feature extraction

Drug repositioning using multiple gene expression profiles

Drug repositioning using multiple gene expression profiles Chuo University’s Professor Y-h. Taguchi places focus on drug repositioning using multiple gene expression profiles. In my previous manuscripts (1-3), I introduced our studies of in silico drug repositioning using gene expression profiles. Nevertheless, in these studies, we could use the single gene expression profile to perform in silico drug repositioning. In this manuscript, the last one in this series, I introduce our recent study (4) in which we could use multiple gene expression profiles. Please see my previous article for more details about the in silico drug repositioning. (5)In the studies (1-3) introduced previously, I used tensor decomposition (TD) based unsupervised feature extraction (FE) (6,7), which was a mathematical/computational method for data processing and was recently implemented in Bioconductor packages. (8)

Genetic algorithm-based semisupervised convolutional neural network for real-time monitoring of Escherichia coli fermentation of recombinant protein production using a Raman sensor.

As a non-destructive sensing technique, Raman spectroscopy is often combined with regression models for real-time detection of key components in microbial cultivation processes. However, achieving accurate model predictions often requires a large amount of offline measurement data for training, which is both time-consuming and labor-intensive. In order to overcome the limitations of traditional models that rely on large datasets and complex spectral preprocessing, in addition to the difficulty of training models with limited samples, we have explored a genetic algorithm-based semi-supervised convolutional neural network (GA-SCNN). GA-SCNN integrates unsupervised process spectral labeling, feature extraction, regression prediction, and transfer learning. Using only an extremely small number of offline samples of the target protein, this framework can accurately predict protein concentration, which represents a significant challenge for other models. The effectiveness of the framework has been validated in a system of Escherichia coli expressing recombinant ProA5M protein. By utilizing the labeling technique of this framework, the available dataset for glucose, lactate, ammonium ions, and optical density at 600 nm (OD600) has been expanded from 52 samples to 1302 samples. Furthermore, by introducing a small component of offline detection data for recombinant proteins into the OD600 model through transfer learning, a model for target protein detection has been retrained, providing a new direction for the development of associated models. Comparative analysis with traditional algorithms demonstrates that the GA-SCNN framework exhibits good adaptability when there is no complex spectral preprocessing. Cross-validation results confirm the robustness and high accuracy of the framework, with the predicted values of the model highly consistent with the offline measurement results.

A Deep Stochastic Adaptive Fourier Decomposition Network for Hyperspectral Image Classification.

Deep learning-based hyperspectral image (HSI) classification methods have recently shown excellent performance, however, there are two shortcomings that need to be addressed. One is that deep network training requires a large number of labeled images, and the other is that deep network needs to learn a large number of parameters. They are also general problems of deep networks, especially in applications that require professional techniques to acquire and label images, such as HSI and medical images. In this paper, we propose a deep network architecture (SAFDNet) based on the stochastic adaptive Fourier decomposition (SAFD) theory. SAFD has powerful unsupervised feature extraction capabilities, so the entire deep network only requires a small number of annotated images to train the classifier. In addition, we use fewer convolution kernels in the entire deep network, which greatly reduces the number of deep network parameters. SAFD is a newly developed signal processing tool with solid mathematical foundation, which is used to construct the unsupervised deep feature extraction mechanism of SAFDNet. Experimental results on three popular HSI classification datasets show that our proposed SAFDNet outperforms other compared state-of-the-art deep learning methods in HSI classification.

Unsupervised Feature Extraction from Raw Data for Gesture Recognition with Wearable Ultra Low-Power Ultrasound.

Wearable ultrasound is a novel sensing approach that shows promise in multiple application domains, and specifically in hand gesture recognition. In fact, ultrasound enables to collect information from deep musculoskeletal structures at high spatiotemporal resolution and high signal-to-noise ratio, making it a perfect candidate to complement surface electromyography for improved accuracy performance and on-the-edge classification. However, existing wearable solutions for ultrasound-based gesture recognition are not sufficiently low-power for continuous, long-term operation. On top of that, practical hardware limitations of wearable ultrasound devices (limited power budget, reduced wireless throughput, restricted computational power) set the need for the compressed size of models for feature extraction and classification. To overcome these limitations, this paper presents a novel end-to-end approach for feature extraction from raw musculoskeletal ultrasound data suited for edge-computing, coupled with an armband for hand gesture recognition based on a truly wearable (12 cm2, 9 g), ultra-low power (16 mW) ultrasound probe. The proposed approach uses a 1D convolutional autoencoder to compress raw ultrasound data by 20× while preserving the main amplitude features of the envelope signal. The latent features of the autoencoder are used to train an XGBoost classifier for hand gesture recognition on datasets collected with a custom US armband, considering armband removal/repositioning in between sessions. Our approach achieves a classification accuracy of 96%. Furthermore, the proposed unsupervised feature extraction approach offers generalization capabilities for inter-subject use, as demonstrated by testing the pre-trained Encoder on a different subject and conducting post-training analysis, revealing that the operations performed by the Encoder are subject-independent. The autoencoder is also quantized to 8-bit integers and deployed on an ultra-low-power wearable ultrasound probe along with the XGBoost classifier, allowing for a gesture recognition rate ≥ 25 Hz and leading to 21% lower power consumption (at 30 FPS) compared to the conventional approach (raw data transmission and remote processing).

Seismic response prediction of a damped structure based on data-driven machine learning methods

Damping technology has been widely used because of its good vibration control effect. However, due to the strong nonlinearity of the added dampers, accurately predicting the seismic response of damped structures remains a challenge. This study investigates the application of interpretable machine learning (ML)-based and deep learning-based approaches to the prediction of the maximum inter-storey displacement of a damped structure. A comprehensive database consisting of 13,855 structural responses to ground motions was collected. Seven traditional interpretable ML algorithms including random forest and extreme gradient boosting (XGBoost), a convolutional neural network based on large receptive field, and seismic wave transformer (SWT) model based on a transformer network were developed. The predictions show that the error of the SWT based on unsupervised feature extraction is reduced by 50.90% compared with that of the optimal XGBoost in ensemble learning. Although the SWT has the highest global accuracy, XGBoost is found to have a smaller error when the structure is in a linear state with peak ground acceleration as partition index, so an aggregation model (AM)-based structural response prediction method was also proposed. The accuracy of the AM improved by 27.95% compared with that of the SWT. In contrast with other ML models, the proposed AM is more advantageous in terms of computational efficiency and accuracy.

An Adaptive Parallel Feature Learning and Hybrid Feature Fusion-Based Deep Learning Approach for Machining Condition Monitoring.

The rapid development of information and communication technologies has facilitated machining condition monitoring toward a data-driven paradigm, of which the Industrial Internet of Things (IIoT) serves as the fundamental basis to acquire data from physical equipment with sensing technologies as well as to learn the relationship between the system condition and the collected condition monitoring data. However, most data-driven methods suffer from using a single-domain space, ignoring the importance of the learned features, and failing to incorporate the handcrafted features assisted by domain knowledge. To solve these limitations, a novel deep learning approach is proposed for machining condition monitoring in the IIoT environment, which consists of three phases, including: 1) the unsupervised parallel feature extraction; 2) adaptive feature importance weighting; and 3) hybrid feature fusion. First, separate sparse autoencoders are utilized to conduct the unsupervised parallel feature extraction, which enables to learn abstract feature representation from multiple domain spaces simultaneously. Then, an attention module is designed for the adaptive feature importance weighting, which can assign higher weights to those critical features accordingly. Moreover, a hybrid feature fusion is deployed to complement the automatic feature learning and further yield better model performance by fusing the handcrafted features assisted by domain knowledge. Finally, a real-life case study and extensive experiments have been conducted to show the effectiveness and superiority of the proposed approach.

Application of kernel principal component analysis for optical vector atomic magnetometry

Vector atomic magnetometers that incorporate electromagnetically induced transparency (EIT) allow for precision measurements of magnetic fields that are sensitive to the directionality of the observed field by virtue of fundamental physics. However, a practical methodology of accurately recovering the longitudinal angle of the local field through observations of EIT spectra has not been established. In this work, we address this problem of angle determination with an unsupervised machine learning algorithm utilizing nonlinear dimensionality reduction. The proposed algorithm was developed to interface with spectroscopic measurements from an EIT-based atomic rubidium magnetometer and uses kernel principal component analysis (KPCA) as an unsupervised feature extraction tool. The resulting KPCA features allow each EIT spectrum measurement to be represented by a single coordinate in a new reduced dimensional feature space, thereby streamlining the process of angle determination. A supervised support vector regression (SVR) machine was implemented to model the resulting relationship between the KPCA projections and field direction. If the magnetometer is configured so that the azimuthal angle of the field is defined with a polarization lock, the KPCA-SVR algorithm is capable of predicting the longitudinal angle of the local magnetic field within 1 degree of accuracy and the magnitude of the absolute field with a resolution of 70 nT. The combined scalar and angular sensitivity of this method make the KPCA-enabled EIT magnetometer competitive with conventional vector magnetometry methods. © 2023. All rights reserved.

Cost-Efficient Deep Learning Model for Colorectal Cancer Diagnosis: Leveraging Unsupervised Learning Composite Network

Despite this, the mortality rate associated with CRC is among the highest of all malignancies. Even while prompt and accurate diagnosis can significantly improve patient outcomes, traditional diagnostic procedures are usually expensive, invasive, and time-consuming. This is despite the fact that rapid diagnosis can significantly improve patient outcomes. Deep learning has emerged as a potential avenue for the diagnosis of cancer, despite the prohibitive nature of the costs associated with developing robust models in terms of both computational and financial resources. In this article, the authors offer a deep learning model that is optimized for the diagnosis of colorectal cancer at a low cost. The model was trained using an unsupervised learning composite network (ULCN). Unsupervised learning is utilized by the ULCN in order to pre-process enormous amounts of unlabeled data in order to cut down on the necessary volume of labelled data as well as the associated cost. Our model was able to capitalize on the underlying structures and patterns in the data, which allowed it to perform far better than other methods available at the time on a test set of histopathological images. The unsupervised feature extraction phase significantly reduced the amount of time required for training, hence showcasing the efficiency of the model. The findings indicate that by combining unsupervised learning into deep learning systems, it may be possible to create a diagnostic tool that is more easily accessible and scalable in clinical settings.

Abnormal Condition Detection Method of Industrial Processes Based on the Cascaded Bagging-PCA and CNN Classification Network

The accurate detection of abnormal working conditions is very important for the safe and stable operation of production process in process industry. Considering that normal data can be easily obtained in industry, unsupervised learning is one of the important methods of anomaly detection. Different from the experience setting of unsupervised anomaly detection index, supervised learning can set anomaly detection index automatically. But it is mostly used in the research of fault classification. In this paper, a new Cascaded Bagging-PCA and CNN Classification Network (CBPCA-CNN) was proposed to realize supervised anomaly detection. The proposed CBPCA-CNN method had the advantages of unsupervised feature extraction and supervised classification decision, which was helpful to improve the detection accuracy. The validation results on the standard data set of the TE process showed that the average accuracy of CBPCA-CNN method was 97.67%, which was higher than the compared methods. This paper verified the feasibility of using supervised learning method to solve anomaly detection.

Semi-Supervised Seizure Prediction Model Combining Generative Adversarial Networks and Long Short-Term Memory Networks

In recent years, significant progress has been made in seizure prediction using machine learning methods. However, fully supervised learning methods often rely on a large amount of labeled data, which can be costly and time-consuming. Unsupervised learning overcomes these drawbacks but can suffer from issues such as unstable training and reduced prediction accuracy. In this paper, we propose a semi-supervised seizure prediction model called WGAN-GP-Bi-LSTM. Specifically, we utilize the Wasserstein Generative Adversarial Network with Gradient Penalty (WGAN-GP) as the feature learning model, using the Earth Mover’s distance and gradient penalty to guide the unsupervised training process and train a high-order feature extractor. Meanwhile, we built a prediction model based on the Bidirectional Long Short-Term Memory Network (Bi-LSTM), which enhances seizure prediction performance by incorporating the high-order time-frequency features of the brain signals. An independent, publicly available dataset, CHB-MIT, was applied to train and validate the model’s performance. The results showed that the model achieved an average AUC of 90.08%, an average sensitivity of 82.84%, and an average specificity of 85.97%. A comparison with previous research demonstrates that our proposed method outperforms traditional adversarial network models and optimizes unsupervised feature extraction for seizure prediction.

Multi-scale nonlinear edge-based three-phase model for unsupervised hyperspectral feature extraction

Multi-scale nonlinear edge-based three-phase model for unsupervised hyperspectral feature extraction