Reliable Model Research Articles

Multimodal Fused Deep Learning for Drug Property Prediction: Integrating Chemical Language and Molecular Graph

Accurately predicting molecular properties is a challenging but essential task in drug discovery. Recently, many mono-modal deep learning methods have been successfully applied to molecular property prediction. However, mono-modal learning is inherently limited as it relies solely on a single modality of molecular representation, which restricts a comprehensive understanding of drug molecules. To overcome the limitations, we propose a multimodal fused deep learning (MMFDL) model to leverage information from different molecular representations. Specifically, we construct a triple-modal learning model by employing Transformer-Encoder, BiGRU, and graph convolutional network (GCN) to process three modalities of information from chemical language and molecular graph: SMILES-encoded vectors, ECFP fingerprints, and molecular graphs, respectively. We evaluate the proposed triple-modal model using five fusion approaches on six molecule datasets including Delaney, Llinas2020, Lipophilicity, SAMPL, BACE, and pKa from DataWarrior. The results show that the MMFDL model achieves the highest Pearson coefficients, and stable distribution of Pearson coefficients in a random splitting test, outperforming mono-modal models in accuracy and reliability. Furthermore, we validate the generalization ability of our model in the prediction of binding constants for protein-ligand complex molecules and assess the resilience capability against noise. Through analysis of feature distributions in chemical space and the assigned contribution of each modal model, we demonstrate that the MMFDL model shows the ability to acquire complementary information by using proper models and suitable fusion approaches. By leveraging diverse sources of bioinformatics information, multimodal deep learning models hold the potential for successful drug discovery.

Deep Learning Ensemble for Flood Probability Analysis

Predicting flood events is complex due to uncertainties from limited gauge data, high data and computational demands of traditional physical models, and challenges in spatial and temporal scaling. This research innovatively uses only three remotely sensed and computed factors: rainfall, runoff and temperature. We also employ three deep learning models—Feedforward Neural Network (FNN), Convolutional Neural Network (CNN), and Long Short-Term Memory (LSTM)—along with a deep neural network ensemble (DNNE) using synthetic data to predict future flood probabilities, utilizing the Savitzky–Golay filter for smoothing. Using a hydrometeorological dataset from 1993–2022 for the Nile River basin, six flood predictors were derived. The FNN and LSTM models exhibited high accuracy and stable loss, indicating minimal overfitting, while the CNN showed slight overfitting. Performance metrics revealed that FNN achieved 99.63% accuracy and 0.999886 ROC AUC, CNN had 95.42% accuracy and 0.893218 ROC AUC, and LSTM excelled with 99.82% accuracy and 0.999967 ROC AUC. The DNNE outperformed individual models in reliability and consistency. Runoff and rainfall were the most influential predictors, while temperature had minimal impact.

Damage-driven framework for reliability assessment of steam turbine rotors operating under flexible conditions

The high-temperature rotating structures (HTRS), e.g., steam turbine rotors, often operate in extremely harsh environments with a flexible load condition during peak shaving of power system. In this work, a damage-driven framework for reliability assessment is developed in terms of the cumulative damage-damage threshold interference (CD-DT) principle, in which the cumulative damage and damage threshold are regarded as two random parameters to address uncertainties. The CD-DT principle is founded on the engineering damage theory and incorporates physics-of-failure into the probabilistic modeling of high-temperature structural reliability. Probabilistic damage analysis, correlation analysis of weak sites, system-level reliability analysis, and sensitivity analysis have been encompassed in this framework. Three numerical examples are used to verify the effectiveness and applicability of the proposed framework. Application to steam turbine rotor involving multiple weak sites with multi-damage modes illustrate the implementation procedures of the framework. Results show that the reliability-based design life of rotor decreases with the increases of start-stop frequency, the implementation of a two-shift operation would pose a threat to meeting the safety requirement of a 30-year design life. Furthermore, sensitivity analysis highlights the critical influences of initial rotor temperature and speed rising rate on rotor reliability, providing insights for operational maintenance and reliability optimization.

Time-dependent mechanical behavior analysis using Weibull models of cellulose hollow fiber membranes produced by green solvent

The mechanical behavior of sustainable (“green”), bio-based cellulose hollow fiber membranes (hereafter, hollow fiber) produced using green solvent, which is potential for desalination and solvent-resistant nanofiltration, can be evaluated either in dry or water-conditioned (wet) states. However, the effect of conditioning time on the mechanical behavior of hollow fibers has not been conclusive. In this work, an extensive evaluation involving an experimental campaign and rigorous application of statistical models for cellulose hollow fibers produced using green solvent (1-ethyl-3-methylimidazolium diethyl phosphate or [EMIM][DEP]) via a dry-jet spinning technique is reported. First, we evaluated the microstructures and separation performance (water permeation, solute rejection) of hollow fibers made of different cellulose concentrations (10–15 wt%) in both pristine and conditioned states. Then, the strength, stiffness, and ductility of hollow fiber samples of different gauge lengths (25 and 50 mm) were measured by a standard tensile test device. Here, the effect of conditioning time on mechanical properties was evaluated after the hollow fiber samples were water-immersed at different times (from 30 s to 2 months). The strength distribution of conditioned hollow fibers was modeled and analyzed using two-parameter and three-parameter Weibull models. Our results showed that the cellulose content of 15 wt% in the hollow fibers produced a denser microstructure, providing the lowest molecular weight cut-off (MWCO). The cellulose content and gauge length slightly modified the mechanical properties of hollow fibers when they were evaluated in pristine, dry conditions. The mechanical properties of hollow fibers were drastically reduced after 10 min of water immersion, beyond which their properties were stabilized. Weibull modeling of mechanical reliability showed that two- or three-parameter Weibull could estimate the failure probability of hollow fibers quite well. Nonetheless, the three-parameter Weibull is better than the two-parameter one in terms of accuracy in determining the threshold strength located at the “lower-end tail.” Our analysis provides a rigorous demonstration of Weibull modeling in evaluating the time-dependent mechanical behavior of ductile, cellulose-based hollow fiber membranes, which is essential to support the design and development of future hollow fiber membranes in desalination industries.

Accurate identification of salt domes using deep learning techniques: Transformers, generative artificial intelligence and liquid state machines

AbstractAcross various global regions abundant in oil and natural gas reserves, the presence of substantial sub‐surface salt deposits holds significant relevance. Accurate identification of salt domes becomes crucial for enterprises engaged in oil and gas exploration. Our research introduces a precise method for the automatic detection of salt domes, leveraging advanced deep learning architectures such as U‐net, transformers, artificial intelligence generative models and liquid state machines. In comparison with state‐of‐the‐art techniques, our model demonstrates superior performance, achieving a stable and validated intersection over the union metric, indicating high accuracy and robustness. Furthermore, the Dice similarity coefficient attaining underscores the model's proficiency in closely aligning with ground truth across diverse scenarios. This evaluation, conducted on 1000 seismic images, reveals that our proposed architecture is not only comparable but often surpasses existing segmentation models in effectiveness and reliability.

Statistical modeling and assessment of storage reliability of 771 silicone seals for electrical connectors

Statistical modeling and assessment of storage reliability of 771 silicone seals for electrical connectors

A Visual Environment for Data Driven Protein Modeling and Validation.

In structural biology, validation and verification of new atomic models are crucial and necessary steps which limit the production of reliable molecular models for publications and databases. An atomic model is the result of meticulous modeling and matching and is evaluated using a variety of metrics that provide clues to improve and refine the model so it fits our understanding of molecules and physical constraints. In cryo electron microscopy (cryo-EM) the validation is also part of an iterative modeling process in which there is a need to judge the quality of the model during the creation phase. A shortcoming is that the process and results of the validation are rarely communicated using visual metaphors. This work presents a visual framework for molecular validation. The framework was developed in close collaboration with domain experts in a participatory design process. Its core is a novel visual representation based on 2D heatmaps that shows all available validation metrics in a linear fashion, presenting a global overview of the atomic model and provide domain experts with interactive analysis tools. Additional information stemming from the underlying data, such as a variety of local quality measures, is used to guide the user's attention toward regions of higher relevance. Linked with the heatmap is a three-dimensional molecular visualization providing the spatial context of the structures and chosen metrics. Additional views of statistical properties of the structure are included in the visual framework. We demonstrate the utility of the framework and its visual guidance with examples from cryo-EM.

STUDY OF TECHNOLOGY FOR THE RELIABILITY AND SURVIVABILITY MODELLING OF ONBOARD CONTROL SYSTEM OF SMALL SPACECRAFT OPERATING IN COMPLEX MODES

The technology of system modeling of reliability and survivability of the onboard control system (OCS) of the spacecraft is presented in the study of various options for the activation of operating modes. The technology of spacecraft functioning based on the concept of digital twins currently takes a leading position. That’s concept allows you to create realistic virtual copies of spacecraft, to simulate not only the objects themselves, but also the processes of their design and operation in various conditions of a priori uncertainty. Such conditions, first of all, should include the destructive effects of an aggressive external environment (outer space) and the multi-mode nature (sequence and intensity of the modes involved) of the functioning of onboard spacecraft systems. The implementation of the requirements of multi-purpose and multi-mode control in these conditions is closely related to the study of the reliability and survivability of such objects from the standpoint of considering their structural construction. The proposed article discusses an approach to assessing the reliability and survivability of onboard systems of small spacecraft (SS), which is based on the concept of a parametric genome structure, taking into account the multi-mode operation in an aggregated form.

Models of osmotic stress as a tool for proteomics and metabolomics of legume seeds

Drought poses a significant challenge to the sustainable development of modern agriculture and to the achievement of high crop yields. Water deficit causes osmotic stress and triggers plant physiological responses characterized by reduced water potential, diminished stomatal conductance, and decreased photosynthetic efficiency. Long-term adaptation to osmotic stress entails intricate metabolic rearrangements, leading to the accumulation of osmoprotectants, activation of antioxidant systems, and increased biosyntheses of stress-protective proteins. The severity and duration of drought, along with plant genotype and developmental stage, influence the plant response to stress, consequently affecting crop yield and quality. Particularly in the context of legumes, which are crucial for human and animal nutrition, understanding adaptive strategies to water deficit is essential for the cultivation of drought-resistant genotypes, primarily because these crops predominantly thrive in semi-arid regions. Proteomics and metabolomics approaches, in turn, serve as valuable tools, offering critical insights into the molecular dynamics governing plant responses to drought stress. Furthermore, the use of reliable drought simulation models is imperative for the effective evaluation of legume response to water scarcity, aiding the cultivation of drought-tolerant varieties. This review highlights the perspectives of utilizing different osmotic stress models to investigate proteome and metabolome alteration within seeds of food legumes.

A Federated Learning-Based Industrial Health Prognostics for Heterogeneous Edge Devices Using Matched Feature Extraction

Data-driven industrial health prognostics require rich training data to develop accurate and reliable predictive models. However, stringent data privacy laws and the abundance of edge industrial data necessitate decentralized data utilization. Thus, the industrial health prognostics field is well suited to significantly benefit from federated learning (FL), a decentralized and privacy-preserving learning technique. However, FL-based health prognostics tasks have hardly been investigated due to the complexities of meaningfully aggregating model parameters trained from heterogeneous data to form a high performing federated model. Specifically, data heterogeneity among edge devices, stemming from dissimilar degradation mechanisms and unequal dataset sizes, poses a critical statistical challenge for developing accurate federated models. We propose a pioneering FL-based health prognostic model with a feature similarity-matched parameter aggregation algorithm to discriminatingly learn from heterogeneous edge data. The algorithm searches across the heterogeneous locally trained models and matches neurons with probabilistically similar feature extraction functions first, before selectively averaging them to form the federated model parameters. As the algorithm only averages similar neurons, as opposed to conventional naive averaging of coordinate-wise neurons, the distinct feature extractors of local models are carried over with less dilution to the resultant federated model. Using both cyclic degradation data of Li-ion batteries and non-cyclic data of turbofan engines, we demonstrate that the proposed method yields accuracy improvements as high as 44.5\% and 39.3\% for state-of-health estimation and remaining useful life estimation, respectively.

Marginal Subspace Learning With Group Low-Rank for Unsupervised Domain Adaptation.

Unsupervised domain adaptation is intended to construct a reliable model for the unlabeled target samples using the well-labeled but differently distributed source samples. To tackle the domain shift issue, learning domain-invariant feature representations across domains is important, and most of the existing methods have concentrated on this goal. However, these methods rarely take into consideration the group discriminability of the feature representation, which is detrimental to the final recognition. Therefore, this article proposes a novel unsupervised domain adaptation method, named marginal subspace learning with group low-rank (MSL-GLR), to extract both domain-invariant and discriminative feature representations. Specifically, MSL-GLR uses the retargeting strategy to relax the regression matrix, such that the regression values would be forced to satisfy a margin maximization criterion for the requirement of correct classification. Moreover, MSL-GLR imposes a class-induced low-rank constraint, which enables the samples of each class to be located in their respective subspace. In this way, the distance between samples from the same class can be decreased and the discriminant ability of the projection is greatly improved. Furthermore, with the help of alternating direction method of multipliers (ADMM), an efficient algorithm is presented to solve the resulting optimization problem. Finally, the effectiveness of the proposed MSL-GLR is demonstrated by comprehensive evaluations on multiple domain adaptation benchmark datasets.

Integrated analysis of reliability, power, and performance for IoT devices and servers

The Internet of Things (IoT) is currently widely used in various sectors and spaces. IoT devices are becoming small yet powerful servers and perform server-like functions. Reliability is a critical aspect in both IoT devices and servers, as they work together to create a robust and dependable IoT ecosystem. Power and performance are two other major considerations of an IoT system. Modeling, analysis, evaluation, and optimization of reliability, power, and performance for IoT devices and servers are major components in IoT systems development and deployment. In this paper, we conduct an integrated study of reliability, power, and performance for IoT devices and servers by mathematically rigorous modeling and analysis. The contributions of the paper can be summarized as follows. We establish a continuous-time Markov chain (CTMC) model that incorporates server failure rate, server repair rate, task arrival rate, and task processing rate. Using such an analytical model, we can calculate the server availability, the average task response time, and the average power consumption. We point out that there is an optimal server speed that minimizes the power-time product and a combined cost-performance metric of power, performance, and reliability. We show the impact of server reliability on response time, power consumption, server utilization, and the power-performance tradeoff. To the best of the author’s knowledge, this is the first paper that takes a combined approach to modeling and analysis of reliability, power, and performance for IoT devices and servers. It has been noticed that there has been little such theoretically solid investigation in the existing literature. Therefore, this paper has made tangible contributions and significant advances in the joint understanding of reliability, power, performance, and their interplay in IoT devices and servers quantitatively and mathematically.

Precise Modeling and Analysis of Aviation Power System Reliability via the Aviation Power System Reliability Probability Network Model

This study addresses the challenges of accurately analyzing the reliability of aviation power systems (APS) using traditional models by introducing the Aviation Power System Reliability Probability Network Model (APS-RPNM). The model directly transforms the system architecture into an equivalent probability network, aiming to develop a precise reliability model that captures system functions and fault logic. By classifying APS components into five distinct structural patterns and mapping them to corresponding nodes in the APS-RPNM, the model is successfully constructed. Specifically, None-Input-to-Multiple-Output components are transformed into two-state nodes, while Multiple-Input-to-None-Output, Single-Input-to-Multiple-Output, and Multiple-Input-to-Single-Output components are mapped to three-state nodes. For Multiple-Input-to-Multiple-Output components, a novel approach employing multiple two-state sub-nodes is adopted to capture their complex functional logic. A case study comparing the performance of the APS-RPNM with the traditional minimal path set method in reliability analysis was conducted. The results demonstrate that the APS-RPNM not only simplifies the model construction process and eliminates errors stemming from subjective engineering judgments but also enables the efficient computation of power supply reliability for all load points in a single inference by integrating all of the components. This significantly improves computational efficiency and system dependency analysis capabilities, highlighting the APS-RPNM’s tremendous potential in optimizing the reliability design of APS.

Effectiveness of large language models in automated evaluation of argumentative essays: finetuning vs. zero-shot prompting

To address the long-standing challenge facing traditional automated writing evaluation (AWE) systems in assessing higher-order thinking, this study built an AWE system for scoring argumentative essays by finetuning the GPT-3.5 Large Language Model. The system’s effectiveness was compared with that of the non-finetuned GPT-3.5 and GPT-4 base models via zero-shot prompting, which involves applying the model to perform tasks without any prior specific training or examples on those tasks. The dataset used was the TOEFL Public Writing Dataset provided by Education Testing Service (ETS), containing 480 argumentative essays with ground truth scores under two essay prompts. Three finetuned models were generated: two finetuned exclusively on either prompt and one on both. All finetuned and base models were used to score the remaining essays after finetuning and their scoring effectiveness was compared with ground truth scores, i.e., benchmark scores assigned by ETS-trained human raters. The impact of the variety of finetuning prompts and the robustness of finetuned models were also explored. Results showed a 100% consistency of all models in two scoring sessions. More importantly, the finetuned models significantly outperformed the base models in accuracy and reliability. The best-performing model, finetuned on prompt 1, showed an RMSE of 0.57, a percentage agreement (score discrepancy ≤ 0.5) of 84.72% and a QWK of 0.78. Further, the model finetuned on both prompts did not exhibit enhanced performance, and the two models finetuned on one prompt remained robust when scoring essays from the alternative prompt. These results suggest (1) task-specific finetuning for AWE is beneficial; (2) finetuning does not require a large variety of essay prompts; and (3) fine-tuned models are robust to unseen essays.

Reliability Modeling and Parameter Estimation for High-Speed Train Wheels Subject to Multi-Dimensional Degradation Processes Considering Mutual Dependency

Reliability Modeling and Parameter Estimation for High-Speed Train Wheels Subject to Multi-Dimensional Degradation Processes Considering Mutual Dependency

MAD-Former: A Traceable Interpretability Model for Alzheimer's Disease Recognition Based on Multi-Patch Attention.

The integration of structural magnetic resonance imaging (sMRI) and deep learning techniques is one of the important research directions for the automatic diagnosis of Alzheimer's disease (AD). Despite the satisfactory performance achieved by existing voxel-based models based on convolutional neural networks (CNNs), such models only handle AD-related brain atrophy at a single spatial scale and lack spatial localization of abnormal brain regions based on model interpretability. To address the above limitations, we propose a traceable interpretability model for AD recognition based on multi-patch attention (MAD-Former). MAD-Former consists of two parts: recognition and interpretability. In the recognition part, we design a 3D brain feature extraction network to extract local features, followed by constructing a dual-branch attention structure with different patch sizes to achieve global feature extraction, forming a multi-scale spatial feature extraction framework. Meanwhile, we propose an important attention similarity position loss function to assist in model decision-making. The interpretability part proposes a traceable method that can obtain a 3D ROI space through attention-based selection and receptive field tracing. This space encompasses key brain tissues that influence model decisions. Experimental results reveal the significant role of brain tissues such as the Fusiform Gyrus (FuG) in AD recognition. MAD-Former achieves outstanding performance in different tasks on ADNI and OASIS datasets, demonstrating reliable model interpretability.

Multi-objective placement and sizing of energy hubs in energy networks considering generation and consumption uncertainties

This paper presents the placement and sizing of energy hubs (EHs) in electricity, gas, and heating networks. EH is a coordinator framework for various power sources, storage devices, and responsive loads. For simultaneous modeling of economic, operation, reliability, and flexibility indices, the proposed scheme is expressed as a three-objective optimization in the form of Pareto optimization based on the sum of weighted functions. The objective functions of this problem respectively minimize the planning cost of EHs (equal to the total cost of construction of hubs and their expected operating cost), the expected energy loss of the mentioned networks, and the expected energy not-supplied (EENS) of these networks in the case of an N − 1 event. The problem is constrained by power flow equations and operation and reliability constraints of these network together with the EH planning and operation model, and flexibility constraints of the EHs. Then, to achieve unique optimal solution in the shortest possible time, a linear approximation model is extracted for the proposed scheme. Moreover, scenario-based stochastic programming (SBSP) is employed to model uncertainties of load, energy cost, renewable power, and accessibility of the mentioned network equipment. Finally, the obtained numerical results indicate the capability of the proposed scheme in enhancing the economic and flexibility situation of EHs and improving the reliability and operation status of energy networks along with achieving optimal planning and operation for EHs.

Comparative analysis of machine learning techniques for predicting drilling rate of penetration (ROP) in geothermal wells: A case study of FORGE site

Enhanced Geothermal Systems (EGS) present a compelling means to unlock the considerable yet largely untapped thermal energy within the earth's crust globally. The EGS drilling cost constitutes a substantial portion, up to 60 %, of the overall expenses. Consequently, streamlining and optimizing the drilling processes for these systems hold immense economic significance, with the potential to substantially lower costs, advance the utilization of geothermal energy, and contribute significantly to the reduction of carbon emissions. This research aims to enhance EGS drilling operations by seeking ways of reducing drilling cost.In this research, we harnessed the predictive power of 10 state-of-the-art machine learning (ML) algorithms to anticipate a crucial drilling parameter: ROP. Using the FORGE dataset, we developed a code tailored for the intensive preprocessing of drilling data. This code offers many options, including various noise-removing techniques and scaling approaches. However, the primary focus of our work extends to quantifying uncertainties intrinsic to the predictions of the 10 algorithms employed. To achieve this, a comprehensive approach involved subjecting each algorithm to 40 runs, utilizing the best model from the tuning process.The results show that unaddressed uncertainties may lead to unstable model behavior, where small changes in the input data or random initialization result in significant prediction variations. Thus, models that do not account for uncertainty may be overfit to the noise in the training data, leading to poor generalization to new, unseen data. Remarkably, the results highlight the superiority of the Extreme Tree, Light Gradient Boost, Random Forest, and Gradient Boosting algorithms, showcasing mean absolute error (MAE) and R2 values within the ranges of 3.5–4.5 and 0.9–0.95, respectively. Conversely, artificial neural networks and support vector machine algorithms demonstrate comparatively lower performance, with MAE ranging from 4.7 to 6.3 and R2 from 0.85 to 0.91.Although the results presented in this work are only based on the drilling parameters from one well of the FORGE site and do not include rock properties or geological parameters, however, the proposed nuanced understanding of algorithmic performance is valuable for refining predictive models in geothermal drilling applications, ensuring robustness and reliability in the face of diverse operational scenarios. Navigating a successful workflow, adeptly processing data, and meticulously quantifying uncertainties linked to each predictive algorithm emerge as formidable yet indispensable tasks. Our research not only sheds light on these complexities but also paves the way for a strategic optimization approach based on data-driven ROP models. Therefore, the potential for substantial savings in real-time drilling operations becomes apparent, unlocking a new realm of possibilities for the geothermal energy sector.

Prediction of Liaoning province steel import and export trade based on deep learning models

AbstractIn the field of deep learning, time series forecasting, particularly for economic and trade data, is a critical area of research. This study introduces a hybrid of auto regressive integrated moving average and gated recurrent unit (ARIMA‐GRU) to enhance the prediction of steel import and export trade in Liaoning Province, addressing the limitations of traditional time series methods. Traditional models like ARIMA excel with linear data but often struggle with non‐linear patterns and long‐term dependencies. The ARIMA‐GRU model combines ARIMA's linear data analysis with GRU's proficiency in non‐linear pattern recognition, effectively capturing complex dynamics in economic datasets. Our experiments show that this hybrid approach surpasses traditional models in accuracy and reliability for forecasting steel trade, providing valuable insights for economic planning and strategic decision‐making. This innovative approach not only advances the field of economic forecasting but also demonstrates the potential of integrating deep learning techniques in complex data analysis.

Modeling acoustic wavefields from moving sources in the presence of a time-varying free surface

Marine vibrators are increasingly being recognized as a viable alternative to seismic airguns for ocean-bottom acquisition due to their ability to generate more low-frequency content and their less adverse impact on marine wildlife. However, their use introduces processing challenges, such as the Doppler effect and time-dependent source-receiver offsets, which are negligible in conventional airgun acquisition. In addition, the time-varying nature of the sea surface during the multisecond acquisition time introduces further challenges for processing and inversion. To accurately account for source motion and time-varying sea-surface effects in seismic data processing, we develop a reliable and robust numerical modeling tool. We use a mimetic finite-difference method in a generalized coordinate system to model the full acoustic wavefield triggered by a moving source in the presence of a time-varying sea surface. Our approach uses a time- and space-dependent coordinate transformation, which tracks the source movement and conforms to the irregular time-varying sea surface, to map an irregular physical domain in Cartesian coordinates to a regular computational domain in generalized coordinates. We formulate this coordinate transformation such that the coordinate systems conformally match below the ocean-bottom level. Numerical examples demonstrate that this approach is accurate and stable, even for an unrealistically exaggerated sea state. This computational tool is not limited to modeling but also could be used to develop advanced processing techniques for marine vibrator data, such as imaging and inversion.