Unique Machine Learning Research Articles

Machine Learning Approaches for Diagnostics and Prognostics of Industrial Systems Using Industrial Open Source Data

In the field of Prognostics and Health Management (PHM), recent years have witnessed a significant surge in the application of machine learning (ML). Despite this growth, the field grapples with a lack of unified guidelines and systematic approaches for effectively implementing these ML techniques and comprehensive analysis regarding industrial open-source data across varied scenarios. To address these gaps, this paper provides a comprehensive review of ML approaches for diagnostics and prognostics of industrial systems using opensource datasets from PHM Data Challenge Competitions held between 2018 and 2023 by PHM Society and IEEE Reliability Society and summarizes a unified ML framework. This review systematically categorizes and scrutinizes the problems, challenges, methodologies, and advancements demonstrated in these competitions, highlighting the evolving role of both conventional machine learning and deep learning in tackling complex industrial tasks related to detection, diagnosis, assessment, and prognosis. Moreover, this paper delves into the common challenges in PHM data challenge competitions by emphasizing data-related and model-related issues and evaluating the limitations of these competitions. The potential solutions to address these challenges are also summarized. Finally, we identify key themes and potential directions for future research, providing opportunities and prospects for next-generation ML-PHM development in PHM domain.

A unified moment tensor potential for silicon, oxygen, and silica

Si and its oxides have been extensively explored in theoretical research due to their technological importance. Simultaneously describing interatomic interactions within both Si and SiO2 without the use of ab initio methods is considered challenging, given the charge transfers involved. Herein, this challenge is overcome by developing a unified machine learning interatomic potentials describing the Si/SiO2/O system, based on the moment tensor potential (MTP) framework. This MTP is trained using a comprehensive database generated using density functional theory simulations, encompassing diverse crystal structures, point defects, extended defects, and disordered structure. Extensive testing of the MTP is performed, indicating it can describe static and dynamic features of very diverse Si, O, and SiO2 atomic structures with a degree of fidelity approaching that of DFT.

Unified machine learning approach for predicting CFST column axial load capacity

Unified machine learning approach for predicting CFST column axial load capacity

A Unified Machine Learning Through Focus Resist 3-D Structure Model

A Unified Machine Learning Through Focus Resist 3-D Structure Model

Towards a Unified Architecture Powering Scalable Learning Models with IoT Data Streams, Blockchain, and Open Data

The huge amount of data produced by the Internet of Things need to be validated and curated to be prepared for the selection of relevant data in order to prototype models, train them, and serve the model. On the other side, blockchains and open data are also important data sources that need to be integrated into the proposed integrative models. It is difficult to find a sufficiently versatile and agnostic architecture based on the main machine learning frameworks that facilitate model development and allow continuous training to continuously improve them from the data streams. The paper describes the conceptualization, implementation, and testing of a new architecture that proposes a use case agnostic processing chain. The proposed architecture is mainly built around the Apache Submarine, an unified Machine Learning platform that facilitates the training and deployment of algorithms. Here, Internet of Things data are collected and formatted at the edge level. They are then processed and validated at the fog level. On the other hand, open data and blockchain data via Blockchain Access Layer are directly processed at the cloud level. Finally, the data are preprocessed to feed scalable machine learning algorithms.

Predicting Air Quality from Measured and Forecast Meteorological Data: A Case Study in Southern Italy

A great deal of attention has been devoted to the analysis of particulate matter (PM) concentrations in various scenarios because of their negative effects on human health. Here, we investigate how meteorological conditions can affect PM concentrations in the peculiar case of the district of the city of Lecce in the Apulia region (Southern Italy), which is characterized by the highest tumor rate of the whole region despite the absence of nearby heavy industries. We present a unified machine learning framework which combines air quality and meteorological data, either measured on ground or forecast. Our findings show that the concentrations of PM10, PM2.5, NO2 and CO are significantly associated with the meteorological conditions and suggest that it is possible to predict air quality using either ground weather observations or weather forecasts.

Melting of MgSiO3 determined by machine learning potentials

The melting temperature of MgSiO${}_{3}$ is crucial in controlling the interior structures and dynamics of Earth and super-Earths. Here, the authors propose an iterative learning scheme that combines enhanced sampling, feature selection, and deep learning, and develop a unified machine learning potential of $a\phantom{\rule{0}{0ex}}b$ $i\phantom{\rule{0}{0ex}}n\phantom{\rule{0}{0ex}}i\phantom{\rule{0}{0ex}}t\phantom{\rule{0}{0ex}}i\phantom{\rule{0}{0ex}}o$ quality. This is valid over a wide pressure-temperature range to determine the melting temperature of MgSiO${}_{3}$. Modeling based on these results shows that heat flux from the core to the mantle is favorable of generating magnetic fields.

Graph-guided Bayesian matrix completion for ocean sound speed field reconstruction.

Reconstructing ocean sound speed field (SSF) from limited and noisy measurements/estimates is crucial for many ocean acoustic applications, including underwater tomography, target localization/tracking, and communications. Classical reconstruction methods include deterministic approaches (e.g., spline interpolation) and geostatistical methods (e.g., kriging). They exhibit a strong link to linear regression and Gaussian process regression in machine learning (ML) literature, by uniformly viewing them as supervised regression models that learn the mapping from the geographical locations to the sound speed outputs. From a unified ML perspective, theoretical analysis indicates that classical reconstruction methods have several drawbacks, such as the sensitivity to noises and high computational cost. To overcome these drawbacks, inspired by the recent thriving development of graph machine learning, we introduce graph-guided Bayesian low-rank matrix completions (LRMCs) for fine-scale and accurate ocean SSF reconstruction. In particular, a more general graph-guided LRMC model is proposed that encompasses the state-of-the-art one as a special case. The proposed model and the associated inference algorithm simultaneously exploit the global (low-rankness) and local (graph structure) information of ocean sound speed data, thus striking an outstanding balance of reconstruction accuracy and computational complexity. Numerical results using real-life ocean SSF data have demonstrated the encouraging performances of the proposed approaches.

A real-time electrical load forecasting and unsupervised anomaly detection framework

We propose a unified machine learning (ML) framework for simultaneously performing electrical load forecasting and unsupervised anomaly detection in real time. For load forecasting, we introduce a training data generator (TDG) and a look-back optimizer (LBO) to enhance the performance of a baseline ML-driven prediction approach. The proposed TDG selects a subset of the entire historical dataset of power consumption to derive a relatively small training dataset that is used for forecasting using ML. It is designed to operate on raw meter data, without requiring any input data conditioning or additional information (e.g., weather, building occupancy), thus providing a data-efficient training approach. The proposed LBO optimizes prediction performance by tracking the accuracy of prior forecasts and automatically adjusting the framework to improve prediction outcomes. For the problem of anomaly detection, the proposed framework is unsupervised and determines abnormality by comparing fluctuations in the current load with power consumption changes in the training dataset. Therefore, in addition to not requiring any data-labeling, it circumvents the problem of imbalanced classes associated with ML-based anomaly detection methods. The proposed framework is evaluated using a real-world power consumption dataset from a large US-based university campus with considerable power usage complexity. Twelve different evaluation metrics are used to comprehensively assess the effectiveness of our approach. Our evaluation results show that compared with alternative methods, the proposed framework consistently achieves superior outcomes with regard to both load forecasting and anomaly detection, representing a promising approach to electrical load forecasting and anomaly detection in practice.

Machine Learning-Based Desmoplastic Signatures Predict Patient Outcomes in Pancreatic Ductal Adenocarcinoma

Introduction: Pancreatic ductal adenocarcinoma (PDAC) is characterized by desmoplasia, a fibrotic matrix response with unclear impact on prognosis. We profiled clinical PDAC using two novel technologies: a machine learning algorithm to quantify matrix patterning in 436 patients and a spatial protein/cell phenotyping algorithm using CO-Detection by indEXing (CODEX). We integrated these modalities with patient metadata to define a unified, machine learning-based histologic signature for PDAC prognosis. Methods: Trichrome-stained clinical specimens were analyzed by an unsupervised algorithm for ultrastructural analysis (>13,000 images, 147 local/global fiber features), which were processed by uniform manifold approximation and projection (UMAP) and fitted to a minimum spanning tree-based pseudotime model. A custom 31-plex CODEX panel was used to visualize tumor- and stroma-associated cells by UMAP of protein expression. Cell interactions were quantified based on spatial co-localization (k=20 nearest neighbors). Binary classifiers for Poor/Good survivorship and recurrence (two- and one-year thresholds, respectively) were trained in MATLAB. Results: Matrix patterning comprised of two architectural groups (L,R), with group-R patients experiencing earlier median death and relapse by 461d and 95d, respectively. Poorer outcomes were primarily driven by three survival-negative interaction “nodes”: Type-I stromal fibroblasts, endothelial cells, and proliferating tumor cells, collectively forming two cell-pattern signatures (A,B) in which signature-B patients experienced earlier median death by 858d. A unified machine learning model successfully classified binary survivorship and recurrence with area under the curve of 0.943-0.946. Conclusion: Fibrotic architecture and cell patterning are powerful independent predictors for PDAC outcomes, with specific desmoplasia-associated cell phenotypes largely driving differential prognoses.

A Research on Cross-Regional Debris Flow Susceptibility Mapping Based on Transfer Learning

Debris flow susceptibility mapping (DFSM), which has proven to be one of the most effective tools for risk management, faces a variety of problems. To realize the rational use of debris flow sample resources and improve the modeling efficiency, a unified model based on transfer learning was established for cross-regional DFSM. First, samples with 10 features collected from two debris flow-prone areas were separately used to perform factor prediction ability analysis (FPAA) based on the information gain ratio (IGR) method and then develop traditional machine learning models based on random forests (RF). Secondly, two feature matrices representing different areas were projected into a common latent feature space to obtain two new feature matrices. Then, the samples with new features were used together for FPAA and developing a unified machine learning model. Finally, the performance of the models was obtained and compared based on the area under curves (AUC) and some statistical results. All the conditioning factors played different roles in debris flow prediction in the two study areas, based on which two traditional models and a unified model were established. The unified model based on feature transferring realized efficient cross-regional modeling, solved the unconvincing problem of limited sample modeling, and enabled more accurate identification of some debris flow samples.

A Novel Framework for Trustworthy Privacy Preserving Machine Learning Model for Industrial IoT Systems Using Blockchain Techniques

Industrial Internet of Things (IIoT) is changing many driving enterprises like transportation, mining, horticulture, energy and medical care. Machine Learning calculations are utilized for getting stages for IT frameworks. The IoT network unit hubs typically asset in a strange manner by making them more responsible to digital assaults. IIoT frameworks requests various situations in genuine one among them is giving security and the causes that encompass them in true viewpoints. It incorporates a system called PriModChain causes security and reliability on IIoT information by joining differential protection, Ethereum block chain and unified Machine learning. Consequently, security will be compromised and we use PriMod chain for giving protection and different compliances and created utilizing Python with attachment programming on essential PC.

Revisiting the Risk Factors for Endometriosis: A Machine Learning Approach.

Endometriosis is a condition characterized by implants of endometrial tissues into extrauterine sites, mostly within the pelvic peritoneum. The prevalence of endometriosis is under-diagnosed and is estimated to account for 5–10% of all women of reproductive age. The goal of this study was to develop a model for endometriosis based on the UK-biobank (UKB) and re-assess the contribution of known risk factors to endometriosis. We partitioned the data into those diagnosed with endometriosis (5924; ICD-10: N80) and a control group (142,723). We included over 1000 variables from the UKB covering personal information about female health, lifestyle, self-reported data, genetic variants, and medical history prior to endometriosis diagnosis. We applied machine learning algorithms to train an endometriosis prediction model. The optimal prediction was achieved with the gradient boosting algorithms of CatBoost for the data-combined model with an area under the ROC curve (ROC-AUC) of 0.81. The same results were obtained for women from a mixed ethnicity population of the UKB (7112; ICD-10: N80). We discovered that, prior to being diagnosed with endometriosis, affected women had significantly more ICD-10 diagnoses than the average unaffected woman. We used SHAP, an explainable AI tool, to estimate the marginal impact of a feature, given all other features. The informative features ranked by SHAP values included irritable bowel syndrome (IBS) and the length of the menstrual cycle. We conclude that the rich population-based retrospective data from the UKB are valuable for developing unified machine learning endometriosis models despite the limitations of missing data, noisy medical input, and participant age. The informative features of the model may improve clinical utility for endometriosis diagnosis.

Towards unified machine learning characterization of lithium-ion battery degradation across multiple levels: A critical review

Lithium-ion battery (LIB) degradation is often characterized at three distinct levels: mechanisms, modes, and metrics. Recent trends in diagnostics and prognostics have been heavily influenced by machine learning (ML). This review not only provides a unique multi-level perspective on characterizing LIB degradation, but also highlights the role of ML in achieving higher accuracies with accelerated computation times. We survey the state-of-the-art in degradation research and show that existing techniques lay the foundations for a unified ML method – a single tool for characterizing degradation at multiple levels. This could inform optimal management of lithium-ion systems, thus extending lifetimes and reducing costs. We propose a framework for the hypothesized technique using pulse injection, digital-twinning, and neural networks, and identify the challenges and future trends in degradation research.

Vine water status mapping with multispectral UAV imagery and machine learning

Optimizing water management has become one of the biggest challenges for grapevine growers in California, especially during drought conditions. Monitoring grapevine water status and stress level across the whole vineyard is an essential step for precision irrigation management of vineyards to conserve water. We developed a unified machine learning model to map leaf water potential ({psi }_{mathrm{leaf}}), by combining high-resolution multispectral remote sensing imagery and weather data. We conducted six unmanned aerial vehicle (UAV) flights with a five-band multispectral camera from 2018 to 2020 over three commercial vineyards, concurrently with ground measurements of sampled vines. Using vegetation indices from the orthomosaiced UAV imagery and weather data as predictors, the random forest (RF) full model captured 77% of {psi }_{mathrm{leaf}} variance, with a root mean square error (RMSE) of 0.123 MPa, and a mean absolute error (MAE) of 0.100 MPa, based on the validation datasets. Air temperature, vapor pressure deficit, and red edge indices such as the normalized difference red edge index (NDRE) were found as the most important variables in estimating {psi }_{mathrm{leaf}} across space and time. The reduced RF models excluding weather and red edge indices explained 52–48% of {psi }_{mathrm{leaf}} variance, respectively. Maps of the estimated {psi }_{mathrm{leaf}} from the RF full model captured well the patterns of both within- and cross-field spatial variability and the temporal change of vine water status, consistent with irrigation management and patterns observed from the ground sampling. Our results demonstrated the utility of UAV-based aerial multispectral imaging for supplementing and scaling up the traditional point-based ground sampling of {psi }_{mathrm{leaf}}. The pre-trained machine learning model, driven by UAV imagery and weather data, provides a cost-effective and scalable tool to facilitate data-driven precision irrigation management at individual vine levels in vineyards.

Predicting spatio-temporal man-made slope failures induced by rainfall in Hong Kong using machine learning techniques

Rain-induced man-made slope failures pose great threats to public safety as most man-made slopes are formed in densely populated areas. A critical step in managing landslide risks is to predict the time, locations and consequences of slope failures in future rainstorms. Based on comprehensive databases of in-service man-made slopes, rainstorms and landslides in Hong Kong during the past 35 years, a spatio-temporal landslide forecasting model for man-made slopes is developed in this study within a unified machine learning framework. With a storm-based data integration strategy and multiclass classification on landslide scales, the framework incorporates landslide time and consequences in landslide susceptibility mapping to successfully achieve spatio-temporal landslide forecasting. The machine learning-based landslide forecasting model is validated against historical landslide incidents both temporally and spatially and through a case study of the June 2008 storm; the model significantly outperforms the prevailing statistical rainfall–landslide correlations in terms of prediction accuracy. The model can predict the real-time evolution of probabilities, scales and spatial distribution of landslides during the progression of a rainstorm, which can never be achieved by statistical methods. It can serve as an essential module for state-of-the-art landslide risk assessment and early warning.

Feature Blending: An Approach toward Generalized Machine Learning Models for Property Prediction.

From studying the atomic structure and chemical behavior to the discovery of new materials and investigating properties of existing materials, machine learning (ML) has been employed in realms that are arduous to probe experimentally. While numerous highly accurate models, specifically for property prediction, have been reported in the literature, there has been a lack of a generalized framework. Herein we propose a novel feature selection approach that enables the development of a unified ML model for property prediction for several classes of materials. It involves an ingenious blending of selected features from various classes of data such that the resultant feature set equips the model with global data descriptors capturing both class-specific as well as global traits. We took accurate band gaps of three distinct classes of 2D materials as our target property to develop the proposed feature blending approach. Using Gaussian process regression (GPR) with the blended features, the ML model developed here resulted in an average root-mean-squared error of 0.12 eV for unseen data belonging to any of the participating classes. The feature blending approach proposed here can be extended to additional classes of materials and also to predict other properties.

Machine learning aided supercritical water gasification for H2-rich syngas production with process optimization and catalyst screening

Hydrogen production from wet organic wastes through supercritical water gasification (SCWG) promotes sustainable development. However, it is always time-consuming and expensive to achieve optimal SCWG conditions and suitable catalysts for different wastes to produce H2-rich syngas. Herein, we developed a unified machine learning (ML) framework to predict syngas composition from SCWG processes inclusive of non-catalytic and catalytic systems. The neural network (NN) model which was the core of the framework exhibited generalizable and satisfactory accuracy (R2 > 0.85) for all systems evaluated. The model also aided in the screening of catalyst and provided optimal conditions for accelerating experiments to produce H2-rich syngas by maximizing H2 yield and minimizing CO2 yield. ML model-based exploration found that SCWG temperature and solid content in the feedstock were the two most important factors affecting syngas composition. ML-based optimization suggested that Fe-based catalyst exhibited a greater potential to promote SCWG of wet wastes under optimal operational conditions. Besides, a web-based graphic user interface was developed by embedding the developed NN model for free access to the SCWG scientific community.

Capacity prediction of cold-formed stainless steel tubular columns using machine learning methods

Current methods of designing cold-formed stainless-steel tubular columns provide a series of design formulae for columns to adapt to various grades of materials and failure modes. In this study, machine learning algorithms were used to establish a unified capacity prediction method. Test data on cold-formed stainless-steel tubular columns were collected from the literature, and Pearson correlation analysis was performed to establish the relations among the design parameters. Seven classical machine learning algorithms were introduced and developed to predict column capacity in a variety of cases used for the analysis. The results show that the random forest performs the best, followed by another gradient boosted tree-based ensemble algorithm, XGBoost. The non-dimensional slenderness of the member and of the cross-section (denoted as comprehensive parameters) are the two key parameters for capacity prediction in the current design methods, and are used as the feature input to acquire the base machine learning model. More models are evaluated by adding one or more material and geometric variables into the feature input. The analysis results show that the performance of the random forest algorithm can be significantly improved by considering the comprehensive parameters together with the member slenderness, ratio of tensile strength to yield strength, and strain hardening exponent. The predictions of the machine learning-based method were compared with those of the current design method in Eurocode. It is concluded that the proposed unified machine learning model provides more accurate results owing to the consideration of the diversities of both material properties and member failure modes.

LncLocator 2.0: a cell-line-specific subcellular localization predictor for long non-coding RNAs with interpretable deep learning.

Long non-coding RNAs (lncRNAs) are generally expressed in a tissue-specific way, and subcellular localizations of lncRNAs depend on the tissues or cell lines that they are expressed. Previous computational methods for predicting subcellular localizations of lncRNAs do not take this characteristic into account, they train a unified machine learning model for pooled lncRNAs from all available cell lines. It is of importance to develop a cell-line-specific computational method to predict lncRNA locations in different cell lines. In this study, we present an updated cell-line-specific predictor lncLocator 2.0, which trains an end-to-end deep model per cell line, for predicting lncRNA subcellular localization from sequences. We first construct benchmark datasets of lncRNA subcellular localizations for 15 cell lines. Then we learn word embeddings using natural language models, and these learned embeddings are fed into convolutional neural network, long short-term memory and multilayer perceptron to classify subcellular localizations. lncLocator 2.0 achieves varying effectiveness for different cell lines and demonstrates the necessity of training cell-line-specific models. Furthermore, we adopt Integrated Gradients to explain the proposed model in lncLocator 2.0, and find some potential patterns that determine the subcellular localizations of lncRNAs, suggesting that the subcellular localization of lncRNAs is linked to some specific nucleotides. The lncLocator 2.0 is available at www.csbio.sjtu.edu.cn/bioinf/lncLocator2 and the source code can be found at https://github.com/Yang-J-LIN/lncLocator2.