Learning Framework Research Articles

Multi-scale structure-guided graph generation for multi-view semi-supervised classification

Graph convolutional network has emerged as a focal point in machine learning because of its robust graph processing capability. Most existing graph convolutional network-based approaches are designed for single-view data, yet in many practical scenarios, data is represented through multiple views. Moreover, due to the complexity of multiple views, normal graph generation methods can not mitigate redundancy to generate a high quality graph. Although the ability of graph convolutional network is undeniable, the quality of graph directly affects its performance. To tackle the aforementioned challenges, this paper proposes a multi-scale graph generation deep learning framework, called multi-scale semi-supervised graph generation based multi-view classification, consisting of two modules: edge sampling and path sampling. The former aims to generate an adjacency graph by selecting edges based on the maximum likelihood among graphs from different views. Meanwhile, the latter seeks to construct a adjacency graph according to the characteristics of paths within the graphs. Finally, the statistical technique is employed to extract commonality and generate a fused graph. Extensive experimental results robustly demonstrate the superior performance of our proposed framework, compared to other state-of-the-art multi-view semi-supervised approaches.

Bayesian deep learning: An enhanced AI framework for legal reasoning alignment

The integration of artificial intelligence into the field of law has penetrated the underlying logic of legal operations. Currently, legal AI systems face difficulties in representing legal knowledge, exhibit insufficient legal reasoning capabilities, have poor explainability, and are inefficient in handling causal inference and uncertainty. In legal practice, various legal reasoning methods (deductive reasoning, inductive reasoning, abductive reasoning, etc.) are often intertwined and used comprehensively. However, the reasoning modes employed by current legal AI systems are inadequate. Identifying AI models that are more suitable for legal reasoning is crucial for advancing the development of legal AI systems.Distinguished from the current high-profile large language models, we believe that Bayesian reasoning is highly compatible with legal reasoning, as it can perferm abductive reasoning, excel at causal inference, and admits the "defeasibility" of reasoning conclusions, which is consistent with the cognitive development pattern of legal professionals from apriori to posteriori. AI models based on Bayesian methods can also become the main technological support for legal AI systems. Bayesian neural networks have advantages in uncertainty modeling, avoiding overfitting, and explainability. Legal AI systems based on Bayesian deep learning frameworks can combine the advantages of deep learning and probabilistic graphical models, facilitating the exchange and supplementation of information between perception tasks and reasoning tasks. In this paper, we take perpetrator prediction systems and legal judegment prediction systems as examples to discuss the construction and basic operation modes of the Bayesian deep learning framework. Bayesian deep learning can enhance reasoning ability, improve the explainability of models, and make the reasoning process more transparent and visualizable. Furthermore, Bayesian deep learning framework is well-suited for human-machine collaborative tasks, enabling the complementary strengths of humans and machines.

Transfer learning for high-dimensional linear regression via the elastic net

In this paper, the high-dimensional linear regression problem is explored via the Elastic Net under the transfer learning framework. Within this framework, potentially related source datasets are leveraged to enhance estimation or prediction beyond what can be achieved solely with the target data. When transferable sources are known, an oracle transfer learning algorithm is proposed based on the Elastic Net. Additionally, the ℓ1/ℓ2 estimation error bounds for the corresponding estimator are established. When the transferable sources are unknown, a novel procedure for detecting transferable sources via the Elastic Net is also proposed, with its selection consistency demonstrated under regular conditions. This method transforms the source detection problem into a variable selection problem in high-dimensional space and always gets results that are consistent with the true outcomes. The performance of these methods is further demonstrated through a variety of numerical examples. Finally, our approach is applied to analyze several real datasets for illustrative purposes.

A machine learning framework for seismic risk assessment of industrial equipment

The paper aims to propose a novel machine learning framework for seismic risk assessment of industrial facilities. In this respect, a compound artificial neural network model is employed, which is based on two different artificial neural network models in series. The first artificial neural network is a regression model employed to generate samples of a vector-valued intensity measure. The second one is a classification model that is used to predict structural damage, starting from the outcomes of the first artificial neural network model. The datasets used for training and validation of the two artificial neural networks are based on hazard-consistent accelerograms and numerical analyses that are performed with an efficient finite element model of the structure. The methodology entails a preliminary feature selection phase for the identification of the aforementioned vector-valued of intensity measures that better classifies the damage/no-damage condition of the structure. This phase is implemented through the principal component analysis method. Subsequently, the Metropolis–Hastings algorithm is used to generate samples of a selected intensity measure, feeding the first ANN model. In turn, the chosen features are used as input parameters of the second ANN model to generate samples of damage/no-damage events. Using the two ANN in series, the mean annual frequency of exceeding a specific limit state is derived. The proposed framework is validated using a typical multi-storey steel frame, focusing on the seismic risk assessment of a vertical storage tank located at the first floor of the primary structure. The proposed method exhibits some clear advantages of combining numerical models with ANN techniques, mainly related to: a reduced computational time; the avoidance of any prior information on the probabilistic model of fragility curves; and the use of model-driven data.

DeepForest-HTP: A Novel Deep Forest Approach for Predicting Antihypertensive Peptides

Hypertension is a major preventable risk factor for cardiovascular disease, affecting over 1.5 billion adults worldwide. Antihypertensive peptides (AHTPs) have gained attention as a natural therapeutic option with minimal side effects. This study proposes a Deep Forest-based machine learning framework for AHTP prediction, leveraging a multi-granularity cascade structure to enhance classification accuracy. We integrated data from BIOPEP-UWM and three previously used datasets, totaling 2000 peptide sequences, and introduced novel feature extraction methods to build a comprehensive dataset for model training.This study represents the first application of Deep Forest for AHTP identification, demonstrating substantial classification performance advantages over traditional methods (e.g., SVM, CNN, and XGBoost) as well as recent mainstream prediction models (Ensemble-AHTPpred, CNN-SVM Ensemble, and mAHTPred). Requiring no complex manual feature engineering, the model adapts flexibly to various data needs, offering a novel perspective for efficient AHTP prediction and promising utility in hypertension management.On the benchmark dataset, the model achieved high accuracy, sensitivity, and AUC, providing a robust tool for identifying safe and effective AHTPs. However, future efforts should incorporate larger and more diverse independent validation datasets to further improve the model and enhance its generalizability. Additionally, the model's predictive accuracy relies on known AHTP targets and sequence features, potentially limiting its ability to detect AHTPs with uncharacterized or atypical properties.

Trustworthy Bayesian Deep Learning Framework for Uncertainty Quantification and Confidence Calibration: Application in Machinery Fault Diagnosis

Trustworthy Bayesian Deep Learning Framework for Uncertainty Quantification and Confidence Calibration: Application in Machinery Fault Diagnosis

Thermal Stratification Prediction in Reactor System Based on CFD Simulations Accelerated by A Data-driven Coarse-grid Turbulence model

Thermal stratification in large enclosures is an integral phenomenon to nuclear reactor system safety. Currently, the effective model for thermal stratification utilizes a multi-scale method that integrates 1-D system-level and 3-D CFD code, which offers thermal stratification details while supplying system-level data across various domains. Nonetheless, harmonizing two codes that operate on different spatial and temporal scales presents a significant challenge, with high-resolution CFD simulations requiring substantial computational resources. This study introduced a data-driven coarse-grid turbulence model based on local flow characteristics at a significantly coarser scale targeting improved efficiency and accuracy in reactor safety analysis concerning thermal stratification. A machine learning framework has been introduced to expedite the RANS-solving process by coupling of OpenFOAM and TensorFlow, which entails training a deep neural network with fine-grid CFD-generated data to predict turbulent eddy viscosity. The feasibility of the developed data-driven turbulence model was proven through the SUPERCAVNA experimental facility problem validation.

HirMTL: Hierarchical Multi-Task Learning for dense scene understanding

In the realm of artificial intelligence, simultaneous multi-task learning is crucial, particularly for dense scene understanding. To address this, we introduce HirMTL, a novel hierarchical multi-task learning framework designed to enhance dense scene analysis. HirMTL is adept at facilitating interaction at the scale level, ensuring task-adaptive multi-scale feature fusion, and fostering task-level feature interchange. It leverages the inherent correlations between tasks to create a synergistic learning environment. Initially, HirMTL enables concurrent sharing and fine-tuning of features at the single-scale level. This is further extended by the Task-Adaptive Fusion module (TAF), which intelligently blends features across scales, specifically attuned to each task’s unique requirements. Complementing this, the Asymmetric Information Comparison Module (AICM) skillfully differentiates and processes both shared and unique features, significantly refining task-specific performance with enhanced accuracy. Our extensive experiments on various dense prediction tasks validate HirMTL’s exceptional capabilities, showcasing its superiority over existing multi-task learning models and underscoring the benefits of its hierarchical approach.

Variable speed limit control strategy considering traffic flow lane assignment in mixed-vehicle driving environment

To accurately predict traffic flow and optimize the operations of freeway bottleneck areas in a mixed-vehicle driving environment, this paper proposes a traffic prediction model and a variable speed limit (VSL) cooperative control strategy. Firstly, a lane-level short-term traffic prediction model, physics informed Transformer and cell transmission model (PIT-CTM), is constructed by combining the Transformer neural network and lane-level cell transmission model (CTM) based on the physics-informed deep learning framework. On this basis, the accuracy and transferability of PIT-CTM are analysed. Secondly, a lane assignment decision model is presented, which enables the dynamic planning of the optimal traffic distribution across lanes. Furthermore, a lane-level VSL control model is constructed based on the model predictive control (MPC) framework. The model induces vehicles to change lanes earlier by setting the speed limit difference between lanes. By regulating the input flow in the bottleneck area of the freeway, it reduces conflicts between mainline vehicles and ramp vehicles. Finally, the feedback regulation between the lane assignment decision model and the lane-level VSL control model promotes the cooperative optimisation of the lateral and longitudinal flows and adapts the control strategy to the dynamic traffic characteristics. A three-lane freeway merging zone is selected, the numerical experiment is conducted and compared with differential lane-level VSL. The results show that the strategy can effectively optimise the mixed-vehicle traffic state and maintain better control performance under any connected and autonomous vehicle (CAV) penetration rates.

A unified Personalized Federated Learning framework ensuring Domain Generalization

Personalized Federated Learning (pFL) allows for the development of customized models for personalized information from multiple distributed domains. In real-world scenarios, some testing data may originate from new target domains (unseen domains) outside of the federated network, resulting in another learning task called Federated Domain Generalization (FedDG). In this paper, we aim to tackle the new problem, named Personalized Federated Domain Generalization (pFedDG), which not only protects the personalization but also obtains a general model for unseen target domains. We observe that pFL and FedDG objectives can conflict, posing challenges in addressing both objectives simultaneously. To sufficiently moderate the conflict, we develop a unified framework, named Personalized Federated Decoupled Representation (pFedDR), which decouples the two objectives using two separate loss functions simultaneously and uses an integrated predictor to serve both two learning tasks. Specifically, the framework decouples domain-sensitive layers linked to different representations and design an entropy increase loss to encourage the separation of two representations to achieve the pFedDG. Extensive experiments show that our pFedDR method achieves state-of-the-art performance for both tasks while incurring almost no increase in communication cost. Code is available at https://github.com/CSU-YL/pFedDR.

A deep-learning-based approach for simulating pedestrian turning flow

The applications of artificial intelligence technology in pedestrian and evacuation dynamics research have achieved gratifying progress recent years. Benefiting from the non-linear fitting ability of deep learning algorithm, such learning-based method might have better performance in modeling the individual micro behaviors comparing to traditional pedestrian and evacuation dynamics models. Hence, this paper proposes a deep-learning-based pedestrian dynamics model which can simulate the pedestrian flow in right-angled corridors. The training process is conducted with a deep learning framework composed of two functional layers namely Scene Perception layer (SP layer) and Motion Dynamic layer (MD layer). The input features of the SP layer and MD layer are obtained from a defined ‘sense field’ which captures information about walking facility structures and neighbors. Dataset generated from twelve groups of pedestrian turning flow experiments is used for data training. The initial experiments are further adopted to evaluate the model at both qualitative and quantitative level from pedestrian motion data simulated by trained model. Qualitatively, the simulation results align with the corresponding experiments in terms of fundamental diagrams in different measurement areas and the headway distance-velocity relationship, demonstrating realistic motion characteristics, proper reactions to changeable walking facility structures and collision avoidance tendencies of agents driven by our model. For quantitative evaluation of the model precision, two indicators respectively calculate the duration and trajectory disparities are introduced and both of them yield relatively small values. Moreover, eight groups of external experiments completely independent from training data are introduced to validate the generalization ability of our model, the simulation results are found to match reality well without prior knowledge. The proposed framework presented is a success trial for simulating pedestrian turning flow and have the potential to be adapted to different scenarios. Outcomes presented will be of beneficial guidance for different engineering application such as performance-based fire design and crowd management.

A Practical Multiobjective Learning Framework for Optimal Hardware-Software Co-Design of Control-on-a-Chip Systems

A Practical Multiobjective Learning Framework for Optimal Hardware-Software Co-Design of Control-on-a-Chip Systems

Robust multi-stage progressive autoencoder for hyperspectral anomaly detection

Recently, Autoencoders (AEs) have demonstrated remarkable performance in the field of hyperspectral anomaly detection, owing to their powerful capability in handling high-dimensional data. However, they often overlook the inherent global distribution characteristics and long-range dependencies in hyperspectral images (HSI). This oversight makes it challenging to accurately characterize and describe boundaries between different backgrounds and anomalies in complex HSI, thereby affecting detection accuracy. To address this issue, a robust multi-stage progressive autoencoder for hyperspectral anomaly detection (RMSAD) is proposed. Initially, a progressive multi-stage learning framework based on convolutional autoencoders is employed. This framework incrementally reveals and integrates deep contextual features along with their long-range dependencies in HSI, aiming to accurately characterize the background and anomalies. Subsequently, an innovative multi-scale fusion strategy is introduced at the intersections of each stage, reinforcing the learning and representation of background and global spatial details across multiple stages. Finally, by collectively extracting abnormal spatial information across stages, effectively reducing the tendency of autoencoders to reconstruct anomalies. This ensures the efficient restoration and replication of global textural details in HSI. The experimental results on the six HSI datasets demonstrate that the proposed RMSAD is superior to other state-of-the-art methods.

Recurrence Plot-Aided Deep Learning Framework for Condition Monitoring of Transformer Insulation Using Fourier Transform Infrared Spectroscopy

Recurrence Plot-Aided Deep Learning Framework for Condition Monitoring of Transformer Insulation Using Fourier Transform Infrared Spectroscopy

Accurate Prediction of Essential Proteins Using Ensemble Machine Learning

Abstract Essential proteins are crucial for biological processes and can be identified through both experimental and computational methods. While experimental approaches are highly accurate, they often demand extensive time and resources. To address these challenges, we present a computational ensemble learning framework designed to identify essential proteins more efficiently. Our method begins by using node2vec to transform proteins in the Protein-Protein Interaction (PPI) network into continuous, low-dimensional vectors. We also extract a range of features from protein sequences, including graph-theory-based, information-based, compositional, and physiochemical attributes. Additionally, we leverage deep learning techniques to analyze high-dimensional Position-Specific Scoring Matrices (PSSMs) and capture evolutionary information. We then combine these features for classification using various machine learning algorithms. To enhance performance, we integrate the outputs of these algorithms through ensemble methods such as voting, weighted averaging, and stacking. This approach effectively addresses data imbalances and improves both robustness and accuracy. Our ensemble learning framework achieved an AUC of 0.948 and an accuracy of 0.9252, outperforming other computational methods. These results demonstrate the effectiveness of our approach in accurately identifying essential proteins and highlight its superior feature extraction capabilities.

NebulaFL: Self-Organizing Efficient Multilayer Federated Learning Framework With Adaptive Load Tuning in Heterogeneous Edge Systems

NebulaFL: Self-Organizing Efficient Multilayer Federated Learning Framework With Adaptive Load Tuning in Heterogeneous Edge Systems

A genetic programming Rician noise reduction and explainable deep learning model for Alzheimer's diseases severity prediction.

Degradation of magnetic resonance imaging (MRI) remains a challenging issue, with noise being a key damaging component introduced due to a variety of environmental and mechanical factors. The aim of this research work is to addresses the issue of noise reduction and to predict Alzheimer's disease detection efficiently. First, we present a genetic programming (GP) technique for reducing Rician noise in MRI images to pre-process the dataset. To effectively reduce Rician noise, this GP approach combines a Feature Extraction component, GP Optimal Expression, and an Optimum Removal Estimation component. In the second phase, we design and develop an explainable Deep Learning framework. This framework uses a local data-driven interpretation technique based on SHAP values to investigate the relationship between the neural network's estimated AD diagnosis and the input MRI images. In addition, we handle class distribution by combining an oversampling strategy with a minority approach. Several assessment metrics are used to analyze the performance of our proposed model. The proposed method is tested on a variety of medical samples, and the results are compared to those obtained using other comparable approaches. We also test and compare our model to three cutting-edge models: DenseNet169, VGGNet15, and Inceptionv3. The empirical results show that our proposed model outperforms others, particularly in handling basic structures with limited spectral features, lower computational complexity, and less overfitting. This research worked addressed Rician noise issue in MRI images and predict AD severity prediction using explainable deep learning framework.

A two-stage deep learning architecture for detection global coastal and offshore submesoscale ocean eddy using SDGSAT-1 multispectral imagery

Submesoscale ocean eddies are essential oceanic phenomenon that control and influence the ocean energy cascade. Most existing eddy detection methods rely on low-resolution satellite altimeter data, which fail to capture submesoscale ocean features and oceanographic phenomena in shallow water. Introducing high-resolution multispectral data can alleviate these problems, yet it has been largely overlooked. A generalized and efficient deep learning architecture that combines developments in deep learning with Sustainable Development Goals Science Satellite 1 (SDGSAT-1) multispectral data from earth observations offers a potential pathway for more fine detection of ocean eddies. Considering that oceanic eddy exhibits spatially sparse characteristics on high-resolution remote sensing scenes, the oceanic eddy detection (OED) model suitable for global coastal and offshore regions is divided into two stages: eddy information judgment and eddy position determination. Correspondingly, SDGSAT-1 multispectral data from November 2021 to December 2022 were carried out to construct two submesoscale eddy datasets for training and testing each stage model. The union validation of multiple metrics demonstrates that the proposed OED model and its stage models achieve state-of-the-art (SOTA) performance, especially in optically complex coastal and offshore waters. We applied the model to real-world scenes captured by SDGSAT-1 in 2023, and found that the detected results were mainly located at the water depth below 200 m. The authenticity of the recognition results is validated using sea surface chlorophyll concentration, temperature, and topography data, indicating that the OED model has achieved remarkable effectiveness under various sea conditions. In addition, the temporal distributions and statistical characteristics of detected submesoscale eddies are analyzed over an extended period (November 2021 to November 2023). Finally, HISEA-2, Landsat-9, and Sentinel-2 served as testing grounds to validate the generalization of the proposed methodology, with experimental results demonstrating that the OED model possesses significant developmental potential for multi-source remote sensing data. This paper presents a comprehensive deep learning framework for the global-scale detection of submesoscale eddies and underscores the pivotal role of high-resolution multispectral imagery as an innovative data source for global coastal and offshore eddy identification.

Functional brain changes in Mexican women with fibromyalgia

Fibromyalgia (FM) is a chronic condition marked by widespread pain, fatigue, sleep problems, cognitive decline, and other symptoms. Despite extensive research, the pathophysiology of FM remains poorly understood, complicating diagnosis and treatment, which often relies on self-report questionnaires. This study explored structural and functional brain changes in women with FM, identified potential biomarkers, and examined their relationship with FM severity. MRI data from 33 female FM patients and 33 matched healthy controls were utilized, focusing on T1-weighted MRI and resting-state fMRI scans. Functional connectivity (FC) analysis was performed using a machine learning framework to differentiate FM patients from healthy controls and predict FM symptom severity. No significant differences were found in brain structural features, such as gray matter volume, white matter volume, deformation-based morphometry, and cortical thickness. However, significant differences in FC were observed between FM patients and healthy controls, particularly in the default mode network (DMN), somatomotor network (SMN), visual network (VIS), and dorsal attention network (DAN). The FC metrics were significantly associated with FM severity. Our prediction model differentiated FM patients from healthy controls with an area under the curve of 0.65. FC measures accurately estimated FM symptom severities with a significant correlation (r = 0.45, p = 0.007). Functional connections in the DMN, VIS, and DAN were crucial in determining FM severity. These findings suggest that integrating brain FC measurements could serve as valuable biomarkers for identifying FM patients from healthy individuals and predicting FM symptom severity, improving diagnostic accuracy and facilitating the development of targeted therapeutic strategies.

A soft scanning electron microscopy for efficient segmentation of alloy microstructures based on a new self-supervised pre-training deep learning network

To provide an on-site metallographic segmentation using only optical microscopy images, sSEM-Net, a soft scanning electron microscopy network, is developed based on a self-supervised pre-training deep learning framework. During model training, only a sparse collection of SEM images is necessary for annotation assistance. By integrating CNN and Transformer, sSEM-Net efficiently utilizes global context information while mitigating data dependency and computational resource constraints. Using only readily available optical microscopy images as input, sSEM-Net achieves metallographic segmentation comparable to SEM images, catering to rapid and cost-effective industrial needs. This methodology leverages non-destructive inspection attributes, catering to rapid and cost-sensitive industrial requirements. The efficacy of the proposed sSEM-Net is demonstrated through metallographic structure analysis of TC4 titanium alloy, with potential extensions to other alloy types.