Neural Network Based Machine Learning Research Articles

Unraveling impacts of polycrystalline microstructures on ionic conductivity of ceramic electrolytes by computational homogenization and machine learning

The ionic conductivity at the grain boundaries (GBs) in oxide ceramics is typically several orders of magnitude lower than that within the grain interior. This detrimental GB effect is the main bottleneck for designing high-performance ceramic electrolytes intended for use in solid-state lithium-ion batteries, fuel cells, and electrolyzer cells. The macroscopic ionic conductivity in oxide ceramics is essentially governed by the underlying polycrystalline microstructures where GBs and grain morphology go hand in hand. This provides the possibility to enhance the ion conductivity by microstructure engineering. To this end, a thorough understanding of microstructure–property correlation is highly desirable. In this work, we investigate numerous polycrystalline microstructure samples with varying grain and grain boundary features. Their macroscopic ionic conductivities are numerically evaluated by the finite element homogenization method, whereby the GB resistance is explicitly regarded. The influence of different microstructural features on the effective ionic conductivity is systematically studied. The microstructure–property relationships are revealed. Additionally, a graph neural network-based machine learning model is constructed and trained. It can accurately predict the effective ionic conductivity for a given polycrystalline microstructure. This work provides crucial quantitative guidelines for optimizing the ionic conducting performance of oxide ceramics by tailoring microstructures.

Monitoring of water volume in a porous reservoir using seismic data: A 3D simulation study

A potential framework to estimate the volume of water stored in a porous storage reservoir from seismic data is neural networks. In this study, the man-made groundwater reservoir is modeled as a coupled poroviscoelastic–viscoelastic medium, and the underlying wave propagation problem is solved using a three-dimensional discontinuous Galerkin method coupled with an Adams–Bashforth time stepping scheme. The wave problem solver is used to generate databases for the neural network-based machine learning model to estimate the water volume. In the numerical examples, we investigate a deconvolution-based approach to normalize the effect from the source wavelet in addition to the network's tolerance for noise levels. We also apply the SHapley Additive exPlanations method to obtain greater insight into which part of the input data contributes the most to the water volume estimation. The numerical results demonstrate the capacity of the fully connected neural network to estimate the amount of water stored in the porous storage reservoir.

Machine learning driven performance enhancement of perovskite solar cells with CNT as both hole transport layer and back contact

We propose a machine learning-based approach to enhance the performance of perovskite solar cells (PSCs) using carbon nanotubes as both hole transport layer (HTL) and back contact. Carbon-based PSCs offer low fabrication costs, long-term mechanical stability, high charge transport, and broad wavelength transparency. Our study estimates and enhances power conversion efficiency (PCE) of carbon-based PSCs through machine learning. PSC with single-walled carbon nanotube (SWCNT)-based HTL and back contact is investigated in simulations using SCAPS-1D and achieved 14.24% PCE, closely aligning with experimental data. Additionally, we explore the impact of various easily tunable fabrication parameters, such as the band gap and electron affinity of SWCNT in HTL, HTL thickness, active layer thickness, electron transport layer (ETL) thickness, HTL dopant concentration, absorber defect density, and ETL dopant concentration, on PCE. We generate a dataset of 20,480 samples to train classical and neural network-based machine learning models to predict performance parameters of PSCs, including short-circuit current density (JSC), open-circuit voltage (VOC), fill factor (FF), and PCE (η). The current density vs. voltage (J-V) characteristics curve is well-fitted by the predicted JSC, VOC, and FF, as evidenced by an average root mean squared error (RMSE) of 0.03 and a goodness of fit (R2) value of 0.99. The fully connected artificial neural network performs best, with RMSE of 0.05% and R2 of 0.99998 for predicting PCE. Genetic algorithm is used to optimize PCE, yielding a maximum PCE of 20.92%. Furthermore, an analysis of feature dimension and dataset size on prediction performance guides future modeling approaches.

Artificial neural network assisted numerical analysis on performance enhancement of Sb2(S,Se)3 solar cell with SnS as HTL

This paper presents the design and investigation of a highly efficient photovoltaic (PV) device comprising antimony sulfide-selenide (Sb2(S,Se)3) absorber utilizing the SCAPS-1D simulator. Initially, the validity of this simulation process is justified by reproducing the experimental structure, and a comparison is made between the experimental and simulation outcomes. Then we have inserted tin sulfide (SnS) as a hole transport layer (HTL) and zinc sulfide (ZnS) as a buffer at the back and front parts of the suggested new heterojunction structure. In addition, to explore the role of intrinsic parameters on performances of the designed device, an artificial neural network-based machine learning (ML) algorithm is applied. It is revealed that the proposed SnS HTL and ZnS buffer form better band alignment with the promising Sb2(S,Se)3 absorber than others. Thus, the efficiency improves by diminishing recombination loss in the proposed Sb2(S,Se)3 solar device. The PV parameters are also thoroughly assessed by varying crucial factors including absorber thickness, acceptor concentration, defects in absorber, defects at SnS/Sb2(S,Se)3 and Sb2(S,Se)3/ZnS interfaces, temperature, work function, and cell resistances. To achieve the best performances, the thicknesses of HTL, absorber, and buffer are optimized to be 0.1 μm, 1.0 μm, and 0.05 μm, respectively. The simulation predicts an efficiency of 28.20 % accompanied by open-circuit voltage (Voc) of 0.94 V, short-circuit current density (Jsc) of 34.65 mA/cm2, and fill-factor (FF) of 86.22 %. Implementation of ML algorithms uncovers that the predicted performance for both training and testing data is correlated with the actual PCE generated by the SCAPS-1D. The optimized physical parameters derived in this simulation will be very useful for researchers to establish an environmentally friendly, cost-effective, and highly efficient Sb2(S,Se)3-based thin-film PV devices.

A Comprehensive Review on Fractional Order Control

Fractional order control is one of the most important enablers of advanced closed loop industrial automation systems. This paper presents a detailed bibliometric study of fractional order control related research published during the past two decades (2000–2022) in the Web of Science (WoS) and Scopus indexed sources. The WoS analysis included yearly publications, publication types and top contributing publishing sources, organizations, researchers, countries and research areas. VoS viewer based full counting methodology was used to analyze citation, co-citation and co-authorship based collaborations among authors, organizations, countries, sources and documents. Co-occurrence, timeline and burst detection analyses of keywords and published articles were also carried out to unravel significant fractional control research trends in the last two decades. The Scopus analysis firstly included annual publication counts, subject areas, document types and top contributing authors, sources, nations, institutions and funding agencies. Secondly, dedicated Scopus searches were carried out for hardware implementation, non integer control and patents. Results indicate that the fractional order controller research has been primarily focused in engineering domain, followed by mathematics, physics, automation control systems and computer science. Materials, mechanics and telecommunications are some of the lesser explored areas. There is a lot of scope of fractional order control research related to patents, hardware implementation, non integer control, anomalous diffusion, chaos and task analysis. Future investigations will also include neural networks based machine learning in fractional order control solutions. Key Words: Fractional order controller, FOPID, Fractional calculus

A machine learning model for predicting the lymph node metastasis of early gastric cancer not meeting the endoscopic curability criteria

BackgroundWe developed a machine learning (ML) model to predict the risk of lymph node metastasis (LNM) in patients with early gastric cancer (EGC) who did not meet the existing Japanese endoscopic curability criteria and compared its performance with that of the most common clinical risk scoring system, the eCura system.MethodsWe used data from 4,042 consecutive patients with EGC from 21 institutions who underwent endoscopic submucosal dissection (ESD) and/or surgery between 2010 and 2021. All resected EGCs were histologically confirmed not to satisfy the current Japanese endoscopic curability criteria. Of all patients, 3,506 constituted the training cohort to develop the neural network-based ML model, and 536 constituted the validation cohort. The performance of our ML model, as measured by the area under the receiver operating characteristic curve (AUC), was compared with that of the eCura system in the validation cohort.ResultsLNM rates were 14% (503/3,506) and 7% (39/536) in the training and validation cohorts, respectively. The ML model identified patients with LNM with an AUC of 0.83 (95% confidence interval, 0.76–0.89) in the validation cohort, while the eCura system identified patients with LNM with an AUC of 0.77 (95% confidence interval, 0.70–0.85) (P = 0.006, DeLong’s test).ConclusionsOur ML model performed better than the eCura system for predicting LNM risk in patients with EGC who did not meet the existing Japanese endoscopic curability criteria.Mini-abstractWe developed a neural network-based machine learning model that predicts the risk of lymph node metastasis in patients with early gastric cancer who did not meet the endoscopic curability criteria.

Recognizing Sign Language using Machine Learning and Deep Learning Models

Individuals with hearing impairments communicate mostly through sign language. Our goal was to create an American Sign Language recognition dataset and utilize it in a neural network-based machine learning model that can interpret hand gestures and positions into natural language. In our study, we incorporated the SVM, CNN and Resnet-18 models to enhance predictability when interpreting ASL signs through this new dataset, which includes provisions such as lighting and distance limitations. Our research also features comparison results between all the other models implemented under invariant conditions versus those using our proposed CNN model. As demonstrated by its high levels of precision at 95.10% despite changes encountered during testing procedures like varying data sets or scene configurations where losses are minimal (0.545), there exists great potential for future applications in image recognition systems requiring deep learning techniques. Furthermore, these advancements may lead to significant improvements within various fields related explicitly to speech-language therapy sessions designed specifically around helping people overcome challenges associated with deafness while building bridges towards improved social integration opportunities.

Pore fractal characteristics and ANN model of polyacrylonitrile fiber reinforced concrete under the synergistic effects of freeze-thaw and erosion

The degradation of the concrete pore structure is a major concern among scholars due to its negative impact on durability. This study conducted a salt freezing test on polyacrylonitrile fiber concrete (PANFRC) using a composite freeze-thaw medium consisting of Na2SO4, MgSO4, and NaCl (5 % + 5% + 3.5 %). The pore volume fractal dimension was determined by analyzing the T2 spectrum obtained through nuclear magnetic resonance technology. The study explored the influence of different types of pores on frost resistance and developed an artificial neural network-based machine learning model to predict macroscopic frost resistance using the pore volume fractal dimension. The experimental findings indicate that plain concrete exhibits a density loss of 4.67 % upon failure, whereas the density loss in PANFRC is reduced to less than 3 %. The frost resistance durability index is observed to improve as the fiber mass fraction increases within specific ranges. Notably, when the fiber content surpasses 1.2 kg/m³, the durability factor (DF) can be effectively doubled. Drawing from the concept of freeze-thaw cycles, a novel frost-thaw erosion damage model was developed, which incorporates the influence of fiber content under the combined action of freeze-thaw cycles and erosion. The analysis of pore dimensions reveals that the fractal characteristics become more pronounced with the increase in pore size, although this does not extend to the micro-pore range (i.e., those smaller than 0.1 μm). In comparison to other machine learning algorithms, the ANN demonstrates its superiority by establishing a meso-macro transfer prediction model with enhanced generalization capabilities.

Real-time signal detection for Cyclotron Radiation Emission Spectroscopy measurements using antenna arrays

Cyclotron Radiation Emission Spectroscopy (CRES) is a technique for precision measurement of the energies of charged particles, which is being developed by the Project 8 Collaboration to measure the neutrino mass using tritium beta-decay spectroscopy. Project 8 seeks to use the CRES technique to measure the neutrino mass with a sensitivity of 40 meV, requiring a large supply of tritium atoms stored in a multi-cubic meter detector volume. Antenna arrays are one potential technology compatible with an experiment of this scale, but the capability of an antenna-based CRES experiment to measure the neutrino mass depends on the efficiency of the signal detection algorithms. In this paper, we develop efficiency models for three signal detection algorithms and compare them using simulations from a prototype antenna-based CRES experiment as a case-study. The algorithms include a power threshold, a matched filter template bank, and a neural network based machine learning approach, which are analyzed in terms of their average detection efficiency and relative computational cost. It is found that significant improvements in detection efficiency and, therefore, neutrino mass sensitivity are achievable, with only a moderate increase in computation cost, by utilizing either the matched filter or machine learning approach in place of a power threshold, which is the baseline signal detection algorithm used in previous CRES experiments by Project 8.

Computational linguistics at the crossroads: A comprehensive review of NLP advancements

New NLP breakthroughs have put Computational Linguistics at a crossroads. NLP's past, present, and future are covered. This review explains computational linguistics' creation with a brief history of linguistics and computer science. Early solutions processed and understood natural language using rule-based systems using manually constructed linguistic rules. Over time, these tactics became increasingly problematic as language became more complex and obscure. Statistical approaches transformed operations. Neural network-based machine learning methods are leading the area because they can learn complicated patterns and representations from large text collections. A data-driven model revolution in natural language processing enhanced language modelling, machine translation, and sentiment analysis. Next, NLP improvements for several tasks and applications are evaluated. Language understanding models that capture semantic nuances and contextual relationships use deep learning frameworks. Word embeddings and transformer-based architectures like GPT and BERT perform well on benchmark datasets for text classification, question answering, and named item identification. The paper also shows how NLP interacts with computer vision, voice processing, and other domains to show the merits and cons of cross-disciplinary research. Multimodal techniques that combine text, graphics, and audio may increase natural language processing and interpretation. The review discusses NLP's effects on prejudice, justice, and privacy. Responsible development and implementation are needed when NLP technology becomes widespread due to algorithmic bias and data privacy concerns. NLP research directions and concerns are reviewed. Existing models may meet standards but fail in practice.

A Comparative Study on Various ANN Optimization Algorithms for Magnetorheological Elastomer Carbonyl Iron Particle Concentration Estimation

Estimation particle composition such as particle shape, size, and concentration are crucial prior to the fabrication process of magnetorheological elastomer (MRE) to avoid process repetition due to inaccurate formulation. Currently, most of MRE prediction model were purposely used to predict the rheological properties such as shear stress and dynamic modulus, known as forward model. Nonetheless, very few studies have been reported to be capable able of predicting particle composition particularly in MR materials, which known as inverse model. Therefore, this paper proposed a carbonyl iron particle (CIP) concentration based MRE prediction model using neural network algorithm. Neural network-based machine learning model is more approachable compared to conventional mathematical modelling approach due to easily identify trends and pattern while handling multi-variety data. Various optimization algorithms have been employed such as Adam, RMSprop, SGD, AdaGrad, and Nadam throughout the modelling process. As the results, given shear strain amplitude, magnetic flux density, storage modulus, and loss factor as model input, SGD gave the maximum prediction accuracy with 0.95 and 3.038 MPa of R2 and RMSE, respectively. Hence, this model can be the basis to the MRE material and devices development particularly as the tool to reduce costing and time consuming.

Early prediction of ventricular peritoneal shunt dependency in aneurysmal subarachnoid haemorrhage patients by recurrent neural network-based machine learning using routine intensive care unit data.

Aneurysmal subarachnoid haemorrhage (aSAH) can lead to complications such as acute hydrocephalic congestion. Treatment of this acute condition often includes establishing an external ventricular drainage (EVD). However, chronic hydrocephalus develops in some patients, who then require placement of a permanent ventriculoperitoneal (VP) shunt. The aim of this study was to employ recurrent neural network (RNN)-based machine learning techniques to identify patients who require VP shunt placement at an early stage. This retrospective single-centre study included all patients who were diagnosed with aSAH and treated in the intensive care unit (ICU) between November 2010 and May 2020 (n = 602). More than 120 parameters were analysed, including routine neurocritical care data, vital signs and blood gas analyses. Various machine learning techniques, including RNNs and gradient boosting machines, were evaluated for their ability to predict VP shunt dependency. VP-shunt dependency could be predicted using an RNN after just one day of ICU stay, with an AUC-ROC of 0.77 (CI: 0.75-0.79). The accuracy of the prediction improved after four days of observation (Day 4: AUC-ROC 0.81, CI: 0.79-0.84). At that point, the accuracy of the prediction was 76% (CI: 75.98-83.09%), with a sensitivity of 85% (CI: 83-88%) and a specificity of 74% (CI: 71-78%). RNN-based machine learning has the potential to predict VP shunt dependency on Day 4 after ictus in aSAH patients using routine data collected in the ICU. The use of machine learning may allow early identification of patients with specific therapeutic needs and accelerate the execution of required procedures.

Ecurrent neural network decoding of rotated surface codes based on distributed strategy

Abstract Quantum error correction is a crucial technology for realizing quantum computers. These computers achieve fault-tolerant quantum computing by detecting and correcting errors using decoding algorithms. Quantum error correction using neural network-based machine learning methods is a promising approach that is adapted to physical systems without the need to build noise models. In this paper, we use a distributed decoding strategy, which effectively alleviates the problem of exponential growth of the training set required for neural networks as the code distance of quantum error-correcting codes increases. Our decoding algorithm is based on renormalization group decoding and recurrent neural network decoder. The recurrent neural network is trained through the Resnet architecture to improve its decoding accuracy. Then we tested the decoding performance of our distributed strategy decoder, recurrent neural network decoder and the classic minimum weight perfect matching(MWPM) decoder for rotated surface codes with different code distances under the circuit noise model, the thresholds of this three decoders are about 0.0052, 0.0051, and 0.0049 respectively. Our results demonstrate that the distributed strategy decoder outperforms the other two decoders, achieving approximately a 5% improvement in decoding efficiency compared to the MWPM decoder and approximately a 2% improvement compared to the recurrent neural network decoder.

An artificial neural network-based numerical estimation of the boiling pressure drop of different refrigerants flowing in smooth and micro-fin tubes

Abstract In thermal engineering implementations, heat exchangers need to have improved thermal capabilities and be smaller to save energy. Surface adjustments on tube heat exchanger walls may improve heat transfer using new manufacturing technologies. Since quantifying enhanced tube features is quite difficult due to the intricacy of fluid flow and heat transfer processes, numerical methods are preferred to create efficient heat exchangers. Recently, machine learning algorithms have been able to analyze flow and heat transfer in improved tubes. Machine learning methods may increase heat exchanger efficiency estimates using data. In this study, the boiling pressure drop of different refrigerants in smooth and micro-fin tubes is predicted using an artificial neural network-based machine learning approach. Two different numerical models are built based on the operating conditions, geometric specifications, and dimensionless numbers employed in the two-phase flows. A dataset including 812 data points representing the flow of R12, R125, R134a, R22, R32, R32/R134a, R407c, and R410a through smooth and micro-fin pipes is used to evaluate feed-forward and backward propagation multi-layer perceptron networks. The findings demonstrate that the neural networks have an average error margin of 10 percent when predicting the pressure drop of the refrigerant flow in both smooth and micro-fin tubes. The calculated R-values for the artificial neural network’s supplementary performance factors are found above 0.99 for all models. According to the results, margins of deviations of 0.3 percent and 0.05 percent are obtained for the tested tubes in Model 1, while deviations of 0.79 percent and 0.32 percent are found for them in Model 2.

Optimization Assisted by Neural Network-Based Machine Learning in Electromagnetic Applications

We introduce an optimization assisted by a neural network (ONN) predictor to the electromagnetic community. ONN belongs to the class of the surrogate model-based optimization approaches, and approximates the objective function using a non-linear approximator – neural network. We provide a comprehensive description of the ONN algorithm and the details of its mathematical formulations. We apply ONN to optimize three popular benchmark functions and compare its performance with some commonly used optimization algorithms, namely particle swarm optimization, genetic algorithm, and Bayesian optimization. For the first time, we demonstrate ONN’s applicability and effectiveness in antenna design problems by optimizing a six-element Yagi-Uda antenna and by solving a challenging ten-dimensional dual-band slotted patch antenna constrained optimization problem. To achieve this, ONN is linked with a full-wave electromagnetic simulation solver through an application user interface. The optimized slotted patch is fabricated and measured to demonstrate how ONN can be part of the full antenna design process. Our empirical results indicate that ONN requires less objective function evaluations to reach the same qualitative point and reaches better optimal points for the same number of iterations for the studied benchmark functions and antenna optimization problems compared to the aforementioned baseline optimization algorithms.

Intelligent Beta-Based Polynomial Approximation of Activation Functions for a Robust Data Encryption System

Deep neural network-based machine learning algorithms are widely used within different sectors and produce excellent results. However, their use requires access to private, often confidential, and sensitive information (financial, medical, etc). This requires precise measures and particular attention to data security and confidentiality. In this paper, we propose a new solution to this problem by integrating a proposed Convolutional Neural Network (CNN) model on encrypted data within the constraints of homomorphic encryption techniques. Specifically, we focus on the approximate activation functions ReLU, Sigmoid, and Tanh, which appear to be the key functions of CNNs. We start by developing new low-degree polynomials, which are essential for successful Homomorphic Encryption (HE). The activation functions will be replaced by these polynomials, which are based on the Beta function and its primitive. To make certain that the data is contained within a given range, the next step is to build a new CNN model using batch normalization. Finally, our methodology and the effectiveness of the proposed strategy are evaluated using Mnist and Cifar10. The experimental results support the proposed approach’s efficiency.

Fractional order control: A bibliometric analysis (2000–2022)

Fractional order control is one of the most important enablers of advanced closed loop industrial automation systems. This paper presents a detailed bibliometric study of fractional order control related research published during the past two decades (2000–2022) in the Web of Science (WoS) and Scopus indexed sources. The WoS analysis included yearly publications, publication types and top contributing publishing sources, organizations, researchers, countries and research areas. VoS viewer based full counting methodology was used to analyze citation, co-citation and co-authorship based collaborations among authors, organizations, countries, sources and documents. Co-occurrence, timeline and burst detection analyses of keywords and published articles were also carried out to unravel significant fractional control research trends in the last two decades. The Scopus analysis firstly included annual publication counts, subject areas, document types and top contributing authors, sources, nations, institutions and funding agencies. Secondly, dedicated Scopus searches were carried out for hardware implementation, non integer control and patents. Results indicate that the fractional order controller research has been primarily focused in engineering domain, followed by mathematics, physics, automation control systems and computer science. Materials, mechanics and telecommunications are some of the lesser explored areas. There is a lot of scope of fractional order control research related to patents, hardware implementation, non integer control, anomalous diffusion, chaos and task analysis. Future investigations will also include neural networks based machine learning in fractional order control solutions.

RoseNNa: A performant, portable library for neural network inference with application to computational fluid dynamics

The rise of neural network-based machine learning ushered in high-level libraries, including TensorFlow and PyTorch, to support their functionality. Computational fluid dynamics (CFD) researchers have benefited from this trend and produced powerful neural networks that promise shorter simulation times. For example, multilayer perceptrons (MLPs) and Long Short Term Memory (LSTM) recurrent-based (RNN) architectures can represent sub-grid physical effects, like turbulence. Implementing neural networks in CFD solvers is challenging because the programming languages used for machine learning and CFD are mostly non-overlapping, We present the roseNNa library, which bridges the gap between neural network inference and CFD. RoseNNa is a non-invasive, lightweight (1000 lines), and performant tool for neural network inference, with focus on the smaller networks used to augment PDE solvers, like those of CFD, which are typically written in C/C++ or Fortran. RoseNNa accomplishes this by automatically converting trained models from typical neural network training packages into a high-performance Fortran library with C and Fortran APIs. This reduces the effort needed to access trained neural networks and maintains performance in the PDE solvers that CFD researchers build and rely upon. Results show that RoseNNa reliably outperforms PyTorch (Python) and libtorch (C++) on MLPs and LSTM RNNs with less than 100 hidden layers and 100 neurons per layer, even after removing the overhead cost of API calls. Speedups range from a factor of about 10 and 2 faster than these established libraries for the smaller and larger ends of the neural network size ranges tested. Program summaryProgram Title: RoseNNaCPC Library link to program files:https://doi.org/10.17632/srbrfx8k74.1Developer's repository link:https://github.com/comp-physics/roseNNaLicensing provisions: MITProgramming language: Fortran90 and PythonNature of problem: Neural network applications in computational fluid dynamics exhibit high promise. However, they are often not deployed on HPC systems due to limited deep-learning support in Fortran or C/C++. Previous attempts to solve this issue rely on significant manual intervention. This leaves researchers with no practical solution for conveniently porting their trained models for use in performant HPC codebases.Solution method: RoseNNa converts trained neural networks from popular machine learning frameworks to Fortran/C code for inference, requiring minimal manual intervention. Metaprogramming enables automatic code creation, and ONNX ensures flexible machine learning library support.

Predicting Major Bleeding in Patients with Venous Thromboembolism on Extended Anticoagulation Therapy Using Follow-up Data and Long Short-Term Memory Based Recurrent Neural Network

Background: It is often challenging for physicians to decide on the duration of anticoagulation treatment for patients with venous thromboembolism (VTE), as they need to weigh the risk of recurrent thrombosis and bleeding at the same time. Several clinical models have been developed to help physicians identify patients with high risk of bleeding. However, these tools use only the baseline clinical information and are not able to incorporate the clinical conditions and events that occur over time that may influence the risk of bleeding. Therefore, capturing the patterns and relationships in the continuously changing time series of clinical data could be a more powerful approach to developing predictor models than just relying on the baseline clinical information. Nonetheless, creating predictive models from follow-up clinical information is challenging given that time series clinical data are non-uniform, high-dimensional, multivariate observations. Here, we present the first attempt at creating a model that uses the time series follow-up information to predict major bleeding over time using a Recurrent Neural Network (RNN) with Long Short-Term Memory (LSTM) cells. Methods: 2542 patients diagnosed with VTE were enrolled in a prospective cohort study over 8 years. In addition to recording their clinical information at baseline, 6-month follow-up interviews were conducted using a standard script to monitor bleeding status and record clinical information. Major bleeding was defined by the International Society on Thrombosis and Haemostasis, with suspected bleeding events classified by an independent adjudication committee. Overall, 118 patients had major bleeding - a 4.6% incidence rate. The median and mode of the clinical variables were used to impute missing numerical and categorical values, respectively, and for patients who had no follow-up information, for whom bleeding occurred before the first follow-up, an artificial follow-up data point was generated from their corresponding baseline data. Thereafter, the data was divided into two stratified sets: 70% for training, and 30% for testing. Five supervised neural network-based machine learning models with different architectures were trained on the baseline dataset, or the follow-up dataset, or both to predict major bleeding. After training, these machine learning models were tested on the testing set and compared to the conventional clinical models, modified to make them compatible with the available predictor variables in our dataset, including the CHAP, the HAS-BLED, the VTE-BLEED, the RIETE, the ACCP, and the OBRI, which only use the baseline information. Results: Overall, the models that used the follow-up information had a higher area under the Receiver Operating Curve (AUROC) or c-statistic compared to the other models that only relied on the baseline dataset. In particular, the LSTM RNN model was able to achieve AUROC of 81.3% that is more than 10% higher compared to the best performing clinical model. We discovered that the LSTM RNN model mostly relied on features such as number of concomitant medications, years since baseline visit, use of specific antibiotics or antiplatelet agents, and presence of new hypertension to predict bleeding from the follow-up dataset. Furthermore, half of the bleeding events occurred within the first year after patients' baseline visits - a trend reflected in the predictions made by LSTM RNN model. Finally, the models that used both the baseline and the follow-up datasets showed different results depending on their architectures; that is, the simpler ensemble model achieved AUROC of 82.5% while the more complex model had AUROC of 70.8% due to overfitting. Conclusion: We have shown that using time series follow-up data can improve bleeding risk prediction in patients with VTE who are on extended anticoagulant therapy compared to just using the baseline data, and clinicians might benefit from using such an approach. Furthermore, our results indicate that LSTM RNN is a suitable architecture to model routine clinical follow-up data. Finally, we believe using time series data could improve the performance of the other clinical models that are currently based on one-time baseline measurements.

A graph neural network-based machine learning model for sentiment polarity and behavior identification of COVID patients

A graph neural network-based machine learning model for sentiment polarity and behavior identification of COVID patients