Feature Learning Model Research Articles

Medium- and Long-Term Distributed Photovoltaic Power Prediction Based on Multiple Time Series Feature and Multiple-Model Fusion

Distributed photovoltaic power stations have advantages such as local direct power supply and reduced transmission energy consumption, and whose demands are constantly being developed. Conducting research on medium- and long-term distributed photovoltaic prediction will have significant value for applications such as the electricity trade market, power grid operation, and the planning of new power stations. Due to characteristics such as long time dependence, disperse power stations, and strong randomness, making accurate and stable predictions becomes very difficult. In this research, we propose a multiple time series feature and multiple-model fusion-based ensemble learning model for medium- and long-term distributed photovoltaic power prediction (M2E-DPV). Considering the wave influence and the differences in distributions in different areas of photovoltaic power, multiple feature combinations are designed to increase feature expression ability and adaptability. Based on the boost ensemble learning model, trained on a single model of different time scale features, the optimal scoring strategy is used for multiple model fusion in the rolling prediction process, and finally, time-segmented probabilistic correction is performed. The experiment results show the effectiveness of the M2E-DPV under multiple feature combinations and multi-model fusion strategies. The average MAPE, R2, and ACC indicators are 0.15, 0.96, and 0.91, respectively. Compared with other methods, there is a significant improvement, indicating that the prediction ability of the model framework proposed in this paper is advanced and robust.

Class‐wised domain decoupling‐guided adversarial feature learning for cross‐age face recognition

AbstractHow to extract age‐independent identity features from face images has long been the major challenge in cross‐age face recognition task. In this letter, a class‐wised domain decoupling‐guided adversarial feature learning network to extract age‐independent face features is proposed. In this model, an adversarial feature learning module is designed to decompose identity features and age features. Considering that artificially designed age interference suppression strategies are difficult to fit the complex relationship between identity features and age features, an adversarial training strategy incorporated with an attention mechanism in the adversarial feature learning module is introduced to suppress age interference adaptively. Besides, under the idea of maximizing the distribution difference of identity feature domain and age feature domain, a class‐wised domain decoupling module is further designed to guide the model to extract age‐independent identity features. The proposed model has been validated on the well‐known public datasets CALFW and CACD‐VS, where it achieved remarkable recognition accuracy rates of 94.5% and 99.5%, respectively. This represents improvements of 4.2% and 0.1% over the state‐of‐the‐art single‐task‐based models, clearly showcasing the effectiveness of the proposed class‐wised domain decoupling‐guided adversarial feature learning model.

Efficient EEG Feature Learning Model Combining Random Convolutional Kernel with Wavelet Scattering for Seizure Detection.

Automatic seizure detection has significant value in epilepsy diagnosis and treatment. Although a variety of deep learning models have been proposed to automatically learn electroencephalography (EEG) features for seizure detection, the generalization performance and computational burden of such deep models remain the bottleneck of practical application. In this study, a novel lightweight model based on random convolutional kernel transform (ROCKET) is developed for EEG feature learning for seizure detection. Specifically, random convolutional kernels are embedded into the structure of a wavelet scattering network instead of original wavelet transform convolutions. Then the significant EEG features are selected from the scattering coefficients and convolutional outputs by analysis of variance (ANOVA) and minimum redundancy-maximum relevance (MRMR) methods. This model not only preserves the merits of the fast-training process from ROCKET, but also provides insight into seizure detection by retaining only the helpful channels. The extreme gradient boosting (XGboost) classifier was combined with this EEG feature learning model to build a comprehensive seizure detection system that achieved promising epoch-based results, with over 90% of both sensitivity and specificity on the scalp and intracranial EEG databases. The experimental comparisons showed that the proposed method outperformed other state-of-the-art methods for cross-patient and patient-specific seizure detection.

Psoriatic Disease Type Prediction and Analysis Using Deep Feature Learning Model

Psoriatic Disease Type Prediction and Analysis Using Deep Feature Learning Model

Multimodal self-supervised learning for remote sensing data land cover classification

Deep learning has revolutionized the remote sensing image processing techniques over the past few years. Nevertheless, annotating high-quality samples is difficult and time-consuming, which limits the performance of deep neural networks because of insufficient supervision information. Aiming to solve this contradiction, we investigate the multimodal self-supervised learning (MultiSSL) paradigm for pre-training and classification of remote sensing image. Specifically, the proposed self-supervised feature learning model consists of asymmetric encoder–decoder structure, in which deep unified encoder learns high-level key information characterizing multimodal remote sensing data and task-specific lightweight decoders are developed to reconstruct original data. To further enhance feature extraction capability, the cross-attention layers are utilized to exchange information contained in heterogeneous characteristics, thus learning more complementary information from multimodal remote sensing data. In fine-tuning stage, the pre-trained encoder and cross-attention layer serve as feature extractor, and leaned characteristics are combined with corresponding spectral information for land cover classification through a lightweight classifier. The self-supervised pre-training model can learn high-level key features from unlabeled samples, thereby utilizing the feature extraction capability of deep neural networks while reducing their dependence on annotated samples. Compared with existing classification paradigms, the proposed multimodal self-supervised pre-training and fine-tuning scheme achieves superior performance for remote sensing image land cover classification.

An LSTM-based adversarial variational autoencoder framework for self-supervised neural decoding of behavioral choices

Objective.This paper presents data-driven solutions to address two challenges in the problem of linking neural data and behavior: (1) unsupervised analysis of behavioral data and automatic label generation from behavioral observations, and (2) extraction of subject-invariant features for the development of generalized neural decoding models. Approach. For behavioral analysis and label generation, an unsupervised method, which employs an autoencoder to transform behavioral data into a cluster-friendly feature space is presented. The model iteratively refines the assigned clusters with soft clustering assignment loss, and gradually improves the learned feature representations. To address subject variability in decoding neural activity, adversarial learning in combination with a long short-term memory-based adversarial variational autoencoder (LSTM-AVAE) model is employed. By using an adversary network to constrain the latent representations, the model captures shared information among subjects’ neural activity, making it proper for cross-subject transfer learning. Main results. The proposed approach is evaluated using cortical recordings of Thy1-GCaMP6s transgenic mice obtained via widefield calcium imaging during a motivational licking behavioral experiment. The results show that the proposed model achieves an accuracy of 89.7% in cross-subject neural decoding, outperforming other well-known autoencoder-based feature learning models. These findings suggest that incorporating an adversary network eliminates subject dependency in representations, leading to improved cross-subject transfer learning performance, while also demonstrating the effectiveness of LSTM-based models in capturing the temporal dependencies within neural data. Significance. Results demonstrate the feasibility of the proposed framework in unsupervised clustering and label generation of behavioral data, as well as achieving high accuracy in cross-subject neural decoding, indicating its potentials for relating neural activity to behavior.

Representational and sequential feature learning model for fault classification in power system

ABSTRACT The classification of three-phase faults in electrical power systems is essential. The vast majority of fault classification techniques are focused on machine learning framework that requires pre-processing algorithms and rely highly on feature engineering. In this paper, a hybrid Convolutional Neural Network and Long Short Term Memory (CNN-LSTM) deep learning model is designed and employed for fault classification. With this combination, both representational and sequential features are automatically extracted and effectively learned using lesser trainable parameters from the raw temporal data. The raw three-phase current and voltage data are used as input for the proposed model. The performance of the proposed model is tested using a 10-fold cross-validation technique on large simulated data produced from a standard IEEE 30-bus system. The efficacy of the proposed model is confirmed by comparing it with different standards and existing models from the literature. The impact of noisy data and load variation conditions are also studied and analysed on the model’s performance.

Efficient Feature Learning Model of Motor Imagery EEG Signals with L1-Norm and Weighted Fusion.

Brain-computer interface (BCI) for motor imagery is an advanced technology used in the field of medical rehabilitation. However, due to the poor accuracy of electroencephalogram feature classification, BCI systems often misrecognize user commands. Although many state-of-the-art feature selection methods aim to enhance classification accuracy, they usually overlook the interrelationships between individual features, indirectly impacting the accuracy of feature classification. To overcome this issue, we propose an adaptive feature learning model that employs a Riemannian geometric approach to generate a feature matrix from electroencephalogram signals, serving as the model's input. By integrating the enhanced adaptive L1 penalty and weighted fusion penalty into the sparse learning model, we select the most informative features from the matrix. Specifically, we measure the importance of features using mutual information and introduce an adaptive weight construction strategy to penalize regression coefficients corresponding to each variable adaptively. Moreover, the weighted fusion penalty balances weight differences among correlated variables, reducing the model's overreliance on specific variables and enhancing accuracy. The performance of the proposed method was validated on BCI Competition IV datasets IIa and IIb using the support vector machine. Experimental results demonstrate the effectiveness and superiority of the proposed model compared to the existing models.

Application of depth feature recognition technology in foreign object recognition in distribution network monitoring video

Foreign objects identification in the distribution network is an important link in the security of electric power, and is of great significance to the normal transportation of electric power. At present, a lot of equipment in the distribution network is in the open air environment, facing a large number of foreign interference. These foreign objects not only bring potential safety hazards to the distribution network, but also easily lead to short circuit, causing power supply difficulties within the region. Therefore, the research first constructs an optimized triplet feature learning model. On this basis, the HOG-SVM depth feature recognition model is proposed. In HOG-SVM, AM is introduced to improve recognition accuracy. In addition, the research enhances the night vision ability of the model by standardizing the features in the image region block. The results show that the AP of the model is stable at more than 90.54%, the average FPR is 2.21%, and the average FNR is 3.17%. The performance of HOG-SVM is significantly better than that of traditional SVM. It verifies the contribution of this research in the field of foreign object recognition and application value in ensuring the security of distribution network.

Prediction of Functional and Anatomic Progression in Lamellar Macular Holes

PurposeTo use artificial intelligence to identify imaging biomarkers for anatomical and functional progression of lamellar macular hole (LMH) and to elaborate a deep learning (DL) model based on optical coherence tomography (OCT) and OCT angiography (OCTA) for prediction of visual acuity (VA) loss in untreated LMHs. DesignMulticentric retrospective observational study ParticipantsPatients >18 years old diagnosed with idiopathic LMHs with availability of good quality OCT and OCTA acquisitions at baseline and a follow up>2 years were recruited. MethodsA DL model based on soft voting of two separate models (OCT and OCTA-based respectively) was trained for identification of cases with VA loss> 5 ETDRS letters (attributable to LMH progression only) during a 2-years follow-up. Biomarkers of anatomical and functional progression of LMH were evaluated with regression analysis, feature learning(support vector machine(SVM) model) and visualization maps. Main Outcome measuresEllipsoid zone(EZ) damage, volumetric tissue loss (TL), vitreopapillary adhesion (VPA), epiretinal proliferation, central macular thickness (CMT), parafoveal vessel density (VD) and vessel length density (VLD) of retinal capillary plexa, choriocapillaris (CC) flow deficit density (FDD). ResultsFunctionally progressing LMHs (VA-PROG group, 41/139 eyes (29.5%)) showed higher prevalence of EZ damage, higher volumetric TL, higher prevalence of VPA, lower superficial capillary plexus(SCP) VD and VLD and higher CC FDD compared to functionally stable LMHs (VA-STABLE group,98/139 eyes (70.5%)). The DL and SVM models showed 92.5% and 90.5% accuracy respectively. The best performing features in the SVM were EZ damage, TL, CC FDD and parafoveal SCP VD. Epiretinal proliferation and lower CMT were risk factors for anatomical progression only. ConclusionsDL can accurately predict functional progression of untreated LMHs over 2 years. The use of AI might improve our understanding of the natural course of retinal diseases. Integrity of CC and SCP might play an important role in the progression of LMHs.

DDFL: Dual-Domain Feature Learning for nighttime semantic segmentation

Nighttime semantic segmentation has been playing a critical role in intelligent transportation, building safety and urban management. However, nighttime scenes present some challenges such as complex structures, multiple light sources, uneven lighting and blurry image noise, which severely degrade the segmentation quality of nighttime images. To address these challenges, we propose a Dual-Domain Feature Learning (DDFL) model for nighttime semantic segmentation. Our approach introduces three innovative ideas. First, we establish an exposure correction module to address the impact of lighting differences on the model’s learning, so as to maximally restore the pixel distortion and blurry areas caused by artificial light in nighttime scenes. Second, we incorporate frequency domain information into the nighttime segmentation task to give the model stronger discrimination ability. Finally, we introduce a dual-domain fusion module to complement the information of learning from the spatial and frequency domains in a cross-fusion manner, enabling the network to perceive semantic information while preserving details. The proposed model was experimentally tested on the Nightcity, Nightcity+ and BDD100k datasets. Our results demonstrate that our model outperforms mainstream models, achieving mIoU scores of 56.73%, 57.41% and 28.97%, respectively, under different lighting, image exposure levels, and resolutions. These results show that our model is capable of segmenting nighttime scenes efficiently in a high-quality way.

Accurate plant species analysis for plant classification using convolutional neural network architecture

<span>Recently, plant identification has become an active trend due to encouraging results achieved in plant species detection and plant classification fields among numerous available plants using deep learning methods. Therefore, plant classification analysis is performed in this work to address the problem of accurate plant species detection in the presence of multiple leaves together, flowers, and noise. Thus, a convolutional neural network based deep feature learning and classification (CNN-DFLC) model is designed to analyze patterns of plant leaves and perform classification using generated fine-grained feature weights. The proposed CNN-DFLC model precisely estimates which the given image belongs to which plant species. Several layers and blocks are utilized to design the proposed CNN-DFLC model. Fine-grained feature weights are obtained using convolutional and pooling layers. The obtained feature maps in training are utilized to predict labels and model performance is tested on the Vietnam plant image (VPN-200) dataset. This dataset consists of a total number of 20,000 images and testing results are achieved in terms of classification accuracy, precision, recall, and other performance metrics. The mean classification accuracy obtained using the proposed CNN-DFLC model is 96.42% considering all 200 classes from the VPN-200 dataset.</span>

Multi-head self-attention mechanism-based global feature learning model for ASD diagnosis

Background and ObjectivesThe static functional connectivity (SFC) networks based on resting-state functional MRI (rs-fMRI) typically focus on local correlations between specific brain regions, neglecting the broader connections across the entire brain. This limitation can hinder the accurate diagnosis of neurological conditions such as Autism Spectrum Disorder (ASD). This study aimed to overcome this limitation and improve ASD dentification accuracy. MethodsWe propose a self-attention based ASD classification model. Employing sliding windows with longer window width, we locally sample the original data to increase the training sample size, thereby alleviating model overfitting. Subsequently, we introduce the multi-head self-attention mechanism, forming a deep model composed of stacked attention blocks. This ensure the capture of not only local correlations but also overall brain network features, significantly enhancing the classification accuracy of ASD. ResultsOur proposed model was evaluated on fMRI data from the ABIDE NYU site. Experimental results demonstrated an accuracy of 81.47%, a sensitivity of 83.8%, and a specificity of 80.16%. Compared to other methods in the literature, our approach exhibited superior accuracy. Furthermore, the experiments revealed that the biomarkers used by the model for classification are primarily distributed across brain regions such as the superior frontal gyrus, middle frontal gyrus, and hippocampus, aligning with previous research findings. ConclusionThe sliding window method effectively enriches the dataset and alleviates overfitting. Simultaneously, the suggested model, which relies on self-attention mechanisms, has the ability to effectively extract global information from brain regions, providing a viable method to improve the accuracy of ASD identification.

A rolling bearing fault diagnosis technique based on fined-grained multi-scale symbolic entropy and whale optimization algorithm-MSVM

As a critical and fragile rotary supporting component in mechanical equipment, fault diagnosis of rolling bearing has been a hot issue. A rolling bearing fault diagnosis technique based on fined-grained multi-scale symbolic entropy and whale optimization algorithm-multiclass support vector machine (abbreviated as FGMSE-WOA-MSVM) is proposed in this paper. Firstly, the vibration signals are decomposed with fine-grained multi-scale decomposition, and the symbolic entropy of the sub-signals at different analysis scales are extracted and constructed as the multi-dimension fault feature vector. In order to address the problem of sensitive parameters for MSVM model, whale optimization algorithm (abbreviated as WOA) is introduced to optimize the penalty factor and kernel function parameters to construct the optimal WOA-MSVM model. Finally, Instance analysis is carried out with bearing fault dataset from Jiangnan University to verify the parameters influence and the effectiveness on the unbalanced sample set. The results show that compared with different feature vector inputs and learning models such as k-Nearest Neighbor (abbreviated as KNN), Decision Tree (abbreviated as DT), Random Forest (RF), etc., the proposed technique can achieve an accuracy rate of 99.33%, besides, the computation speed is fast and the diagnosis efficiency is high which means its potential value for engineering application.

Lightweight Pig Face Feature Learning Evaluation and Application Based on Attention Mechanism and Two-Stage Transfer Learning

With the advancement of machine vision technology, pig face recognition has garnered significant attention as a key component in the establishment of precision breeding models. In order to explore non-contact individual pig recognition, this study proposes a lightweight pig face feature learning method based on attention mechanism and two-stage transfer learning. Using a combined approach of online and offline data augmentation, both the self-collected dataset from Shanxi Agricultural University's grazing station and public datasets underwent enhancements in terms of quantity and quality. YOLOv8 was employed for feature extraction and fusion of pig face images. The Coordinate Attention (CA) module was integrated into the YOLOv8 model to enhance the extraction of critical pig face features. Fine-tuning of the feature network was conducted to establish a pig face feature learning model based on two-stage transfer learning. The YOLOv8 model achieved a mean average precision (mAP) of 97.73% for pig face feature learning, surpassing lightweight models such as EfficientDet, SDD, YOLOv5, YOLOv7-tiny, and swin_transformer by 0.32, 1.23, 1.56, 0.43 and 0.14 percentage points, respectively. The YOLOv8-CA model’s mAP reached 98.03%, a 0.3 percentage point improvement from before its addition. Furthermore, the mAP of the two-stage transfer learning-based pig face feature learning model was 95.73%, exceeding the backbone network and pre-trained weight models by 10.92 and 3.13 percentage points, respectively. The lightweight pig face feature learning method, based on attention mechanism and two-stage transfer learning, effectively captures unique pig features. This approach serves as a valuable reference for achieving non-contact individual pig recognition in precision breeding.

Worst-Case Discriminative Feature Learning via Max-Min Ratio Analysis.

We propose a novel discriminative feature learning method via Max-Min Ratio Analysis (MMRA) for exclusively dealing with the long-standing "worst-case class separation" problem. Existing technologies simply consider maximizing the minimal pairwise distance on all class pairs in the low-dimensional subspace, which is unable to separate overlapped classes entirely especially when the distribution of samples within same class is diverging. We propose a new criterion, i.e., Max-Min Ratio Analysis (MMRA) that focuses on maximizing the minimal ratio value of between-class and within-class scatter to extremely enlarge the separability on the overlapped pairwise classes. Furthermore, we develop two novel discriminative feature learning models for dimensionality reduction and metric learning based on our MMRA criterion. However, solving such a non-smooth non-convex max-min ratio problem is challenging. As an important theoretical contribution in this paper, we systematically derive an alternative iterative algorithm based on a general max-min ratio optimization framework to solve a general max-min ratio problem with rigorous proofs of convergence. More importantly, we also present another solver based on bisection search strategy to solve the SDP problem efficiently. To evaluate the effectiveness of proposed methods, we conduct extensive pattern classification and image retrieval experiments on several artificial datasets and real-world ScRNA-seq datasets, and experimental results demonstrate the effectiveness of proposed methods.

Incipient fault detection enhancement based on spatial-temporal multi-mode siamese feature contrast learning for industrial dynamic process

Incipient faults are characterized by low-amplitude, unclear fault features, which are susceptible to unknown disturbances, leading to unsatisfactory detection performance. In this paper, an incipient fault detection enhancement method based on siamese spatial-temporal multi-mode feature contrast learning method is proposed. Firstly, we design a novel siamese spatial-temporal multi-mode convolutional neural network model consisting of two weight-shared spatial-temporal multi-mode convolutional neural networks and one feature discrimination measure operator, which are then used to extract the spatial-temporal multi-mode features of two datasets and to measure the distance between them. Then, an incipient fault feature discrimination intensification training strategy is developed to enhance the incipient fault detection performance. Specifically, this strategy intends to maximize the feature distance between the normal data and the incipient fault data, as well as that between different incipient faults, while minimizing the feature distance between the normal data and between the same incipient faults. Moreover, due to the long-term slow change characteristic of the incipient fault, the multi-head self-attention Long Short-Term Memory is presented as a dynamic feature learning model to further lopsidedly learn the incipient fault temporal long-term dependency according to attention weights utilizing the scaled dot-product multi-head self-attention mechanism. Finally, the performance of the proposed method is demonstrated on two industrial cases.

Semi-Supervised Seizure Prediction Model Combining Generative Adversarial Networks and Long Short-Term Memory Networks

In recent years, significant progress has been made in seizure prediction using machine learning methods. However, fully supervised learning methods often rely on a large amount of labeled data, which can be costly and time-consuming. Unsupervised learning overcomes these drawbacks but can suffer from issues such as unstable training and reduced prediction accuracy. In this paper, we propose a semi-supervised seizure prediction model called WGAN-GP-Bi-LSTM. Specifically, we utilize the Wasserstein Generative Adversarial Network with Gradient Penalty (WGAN-GP) as the feature learning model, using the Earth Mover’s distance and gradient penalty to guide the unsupervised training process and train a high-order feature extractor. Meanwhile, we built a prediction model based on the Bidirectional Long Short-Term Memory Network (Bi-LSTM), which enhances seizure prediction performance by incorporating the high-order time-frequency features of the brain signals. An independent, publicly available dataset, CHB-MIT, was applied to train and validate the model’s performance. The results showed that the model achieved an average AUC of 90.08%, an average sensitivity of 82.84%, and an average specificity of 85.97%. A comparison with previous research demonstrates that our proposed method outperforms traditional adversarial network models and optimizes unsupervised feature extraction for seizure prediction.

Compound fault separation and diagnosis method for online rolling bearings based on RSEUnet and 1DCNN

Compound fault signals interfere with one another, resulting in an inconspicuous feature extraction that requires sophisticated signal processing techniques and expert experience. However, good online diagnostic methods are not available to carry out this process. This paper proposes a method based on Residual Connection and Squeeze-and-Excitation Unet (RSEUnet) and one-dimensional convolutional neural network (1DCNN). The process includes fault separation and diagnosis. First, the feature extraction module of the RSEUnet network introduces an attention mechanism and a residual connection that adaptively assigns various weights to different channels. This model is used to train the maps of the fault signal after time-frequency transformation. Ideal binary masks with excellent performance are the training targets to complete the intelligent separation of compound faults. Second, the 1DCNN is used as a feature learning model to efficiently learn the features of single faults from time-domain signals. An embedded system consisting of a Jetson Nano and a signal acquisition circuit is then built to perform online diagnosis. The test is carried out on the fault experimental platform. Results show that the method has an accuracy of 99.71%, making it highly suitable for the diagnosis of bearing compound faults.

Emotion Recognition Using Hierarchical Spatiotemporal Electroencephalogram Information from Local to Global Brain Regions.

To understand human emotional states, local activities in various regions of the cerebral cortex and the interactions among different brain regions must be considered. This paper proposes a hierarchical emotional context feature learning model that improves multichannel electroencephalography (EEG)-based emotion recognition by learning spatiotemporal EEG features from a local brain region to a global brain region. The proposed method comprises a regional brain-level encoding module, a global brain-level encoding module, and a classifier. First, multichannel EEG signals grouped into nine regions based on the functional role of the brain are input into a regional brain-level encoding module to learn local spatiotemporal information. Subsequently, the global brain-level encoding module improved emotional classification performance by integrating local spatiotemporal information from various brain regions to learn the global context features of brain regions related to emotions. Next, we applied a two-layer bidirectional gated recurrent unit (BGRU) with self-attention to the regional brain-level module and a one-layer BGRU with self-attention to the global brain-level module. Experiments were conducted using three datasets to evaluate the EEG-based emotion recognition performance of the proposed method. The results proved that the proposed method achieves superior performance by reflecting the characteristics of multichannel EEG signals better than state-of-the-art methods.