Abstract 4370225: Optimizing nursing team assessment with computer vision and machine learning: A feasible approach for interstage video analysis

Short-term runoff prediction with GRU and LSTM networks without requiring time step optimization during sample generation

Cryptocurrencies created by Nakamoto in 2009 have gained significant interest due to their potential for high returns. However, the cryptocurrency market's unpredictability makes it challenging to forecast prices accurately. To tackle this issue, a deep learning model has been developed that utilizes Long Short-Term Memory (LSTM) neural networks and Convolutional Neural Networks (CNNs) to predict cryptocurrency prices. LSTMs, a type of recurrent neural network, are well-suited for analyzing time series data and have been successful in various prediction applications. Additionally, CNNs, primarily used for image analysis tasks, can be employed to extract relevant patterns and characteristics from input data in Bitcoin price prediction applications. This study contributes to the existing related works on cryptocurrency price prediction by exploring various predictive models and techniques, which involve a machine learning model, deep learning model, time series analysis, and as well as a hybrid model that combines deep learning methods to predict cryptocurrency prices as well as enhance the accuracy and reliability of the price predictions. To ensure accurate predictions in this study, a trustworthy dataset from investing.com was sought. The dataset, sourced from investing.com, consists of 1826 time series data samples. The dataset covers the time frame from January 1, 2018, to December 31, 2022, providing data for a period of 5 years. Subsequently, pre-processing was conducted on the dataset to guarantee the quality of the input. As a result of absent values and concerns regarding the dataset's obsolescence, an alternative dataset was sourced to avoid these issues. The performance of the LSTM and CNN models was evaluated using root mean squared error (RMSE), mean squared error (MSE), mean absolute error (MAE) and R-squared (R2). It was observed that they outperformed each other to a certain degree in short-term forecasts compared to long-term predictions, where the R2Â values for LSTM range from 0.973 to 0.986, while for CNNs, they range from 0.972 to 0.988 for 1 day, 3 days and 7 days windows length. Nevertheless, the LSTM model demonstrated the most favorable performance with the lowest error rate. The RMSE values for the LSTM model ranged from 1203.97 to 1645.36, whereas the RMSE values for the CNNs model ranged from 1107.77 to 1670.93. As a result, the LSTM model exhibited a lower error rate in RMSE and achieved the highest accuracy in R2Â compared to the CNNs model. Considering these comparative outcomes, the LSTM model can be deemed as the most suitable model for this specific case

In hydrology, the use of machine learning (ML) has gained traction due to its ability to provide alternative or complementary approaches to traditional process-based modelling. These models identify numerical patterns in time series data without needing to solve conservation equations. This flexibility enables hydrological calculations in areas where data sources are incomplete or non-existent.Studies benchmarking ML models (SVM, RNN, CNN) against process-based models have shown that ML models deliver promising results with lower computational cost and less information about the physical processes they are modelling. Consequently, they can effectively utilize spatially discretized physical data on a large scale.This study designs a Long Short-Term Memory (LSTM) neural network to learn sequential relationships between atmospheric, climatic and geographic features and daily streamflow data from 39 headwater gauging stations in the northern Ebro river basin. LSTM models include an internal state that can store information and learn long-term dependencies, enabling them to model sequential data effectively. However, the numerical patterns identified by LSTM models do not inherently respect universal physical laws, such as the conservation of mass.To address the limitation, the Mass-Conserving LSTM (MC-LSTM) model has been employed and compared with the standard LSTM model. The MC-LSTM model introduces a modified cell structure that adheres to conservation laws by extending the learning bias to model the redistribution of mass.This analysis highlights not only the high accuracy of LSTM models in predictive hydrologic modelling but also the critical importance of integrating physics-based features to enable ML models to effectively capture the hydrological dynamics of the basin.Acknowledgments: This work is funded by the European Research Council (ERC) through the Horizon Europe 2021 Starting Grant program under REA grant agreement number 101039181-SED@HEAD.

Wave runup prediction is necessary for offshore structure designs and early warnings. Data-driven methods based on machine learning have inspired reduced-order solutions for wave–structure interaction problems. This study provides the quantification of deep learning algorithms’ potential for wave runup prediction. Two prominent deep learning models were utilized to predict the wave runups along the fore column of semisubmersible under head seas. The long short-term memory (LSTM) and the temporal convolutional networks (TCNs) were comprehensively compared based on the datasets from a model test carried out in the deep ocean basin. The LSTM and TCN model structures were optimized to compare prediction accuracy and computational complexity reasonably. The results reveal that (1) both developed TCN and LSTM models had a satisfied prediction accuracy of over 90%. Their predictions were extended to 10 s into the future with accuracies over 80% and 45%, respectively. (2) With the noise-extended datasets, the LSTM model was robust to noises, while the TCN model showed better prediction performance on the extreme wave runup events. (3) The incident wave and dominant rotation provided the major information for wave runup prediction. TCN and LSTM models’ prediction accuracies were 91.5% and 89.3% based on the simplified input tensors composed of incident wave and pitch. The comparison showed the great potential of the TCN model to predict the nonlinear wave runup with less time and memory costs. The input tensors’ design and optimization based on physical understanding also play a significant role in the prediction performance.

This study aims to develop and optimize a Long Short-Term Memory (LSTM) model to reduce overfitting in text classification using the Kaggle IMDB movie review dataset. Overfitting is a common problem in machine learning that causes the model to overfit to the training data, thus degrading its performance on the test data. In this study, various optimization techniques such as regularization, dropout, and careful training methods are applied to improve the generalization of the LSTM model. This study shows that overfitting reduction techniques, such as dropout and the use of the RMSProp optimizer, significantly improve the performance of the Long Short-Term Memory (LSTM) model in IMDB movie review text classification. The optimized LSTM model achieves an accuracy of 83.45%, an increase of 2.07% compared to the standard model which has an accuracy of 81.38%. The precision of the optimized model increases to 89.65%, compared to 84.46% in the standard model, although the recall is slightly lower (75.69% compared to 76.91%). The F1-score of the optimized model is also higher, which is 82.07% compared to 80.53% in the standard model. The experimental results show that the techniques successfully improve the accuracy and reliability of the text classification model, with better performance on the test data. This research makes a significant contribution to understanding and overfitting in deep learning models in the context of natural language processing, and offers insights into best practices in applying LSTM models to text classification.

BackgroundHigh flow nasal cannula (HFNC) provides noninvasive respiratory support for children who are critically ill who may tolerate it more readily than other noninvasive ventilation (NIV) techniques such as bilevel positive airway pressure and continuous positive airway pressure. Moreover, HFNC may preclude the need for mechanical ventilation (intubation). Nevertheless, NIV or intubation may ultimately be necessary for certain patients. Timely prediction of HFNC failure can provide an indication for increasing respiratory support.ObjectiveThe aim of this study is to develop and compare machine learning (ML) models to predict HFNC failure.MethodsA retrospective study was conducted using the Virtual Pediatric Intensive Care Unit database of electronic medical records of patients admitted to a tertiary pediatric intensive care unit between January 2010 and February 2020. Patients aged <19 years, without apnea, and receiving HFNC treatment were included. A long short-term memory (LSTM) model using 517 variables (vital signs, laboratory data, and other clinical parameters) was trained to generate a continuous prediction of HFNC failure, defined as escalation to NIV or intubation within 24 hours of HFNC initiation. For comparison, 7 other models were trained: a logistic regression (LR) using the same 517 variables, another LR using only 14 variables, and 5 additional LSTM-based models using the same 517 variables as the first LSTM model and incorporating additional ML techniques (transfer learning, input perseveration, and ensembling). Performance was assessed using the area under the receiver operating characteristic (AUROC) curve at various times following HFNC initiation. The sensitivity, specificity, and positive and negative predictive values of predictions at 2 hours after HFNC initiation were also evaluated. These metrics were also computed for a cohort with primarily respiratory diagnoses.ResultsA total of 834 HFNC trials (455 [54.6%] training, 173 [20.7%] validation, and 206 [24.7%] test) met the inclusion criteria, of which 175 (21%; training: 103/455, 22.6%; validation: 30/173, 17.3%; test: 42/206, 20.4%) escalated to NIV or intubation. The LSTM models trained with transfer learning generally performed better than the LR models, with the best LSTM model achieving an AUROC of 0.78 versus 0.66 for the 14-variable LR and 0.71 for the 517-variable LR 2 hours after initiation. All models except for the 14-variable LR achieved higher AUROCs in the respiratory cohort than in the general intensive care unit population.ConclusionsML models trained using electronic medical record data were able to identify children at risk of HFNC failure within 24 hours of initiation. LSTM models that incorporated transfer learning, input data perseveration, and ensembling showed improved performance compared with the LR and standard LSTM models.

Falls in the elderly is a major public health concern due to its high prevalence, serious consequences and heavy burden on the society. Many falls in older people happen within a very short time, which makes it difficult to predict a fall before it occurs and then to provide protection for the person who is falling. The primary objective of this study was to develop deep neural networks for predicting a fall during its initiation and descending but before the body impacts to the ground so that a safety mechanism can be enabled to prevent fall-related injuries. We divided the falling process into three stages (non-fall, pre-impact fall and fall) and developed deep neutral networks to perform three-class classification. Three deep learning models, convolutional neural network (CNN), long short term memory (LSTM), and a novel hybrid model integrating both convolution and long short term memory (ConvLSTM) were proposed and evaluated on a large public dataset of various falls and activities of daily living (ADL) acquired with wearable inertial sensors (accelerometer and gyroscope). Fivefold cross validation results showed that the hybrid ConvLSTM model had mean sensitivities of 93.15, 93.78, and 96.00% for non-fall, pre-impact fall and fall, respectively, which were higher than both LSTM (except the fall class) and CNN models. ConvLSTM model also showed higher specificities for all three classes (96.59, 94.49, and 98.69%) than LSTM and CNN models. In addition, latency test on a microcontroller unit showed that ConvLSTM model had a short latency of 1.06 ms, which was much lower than LSTM model (3.15 ms) and comparable with CNN model (0.77 ms). High prediction accuracy (especially for pre-impact fall) and low latency on the microboard indicated that the proposed hybrid ConvLSTM model outperformed both LSTM and CNN models. These findings suggest that our proposed novel hybrid ConvLSTM model has great potential to be embedded into wearable inertial sensor-based systems to predict pre-impact fall in real-time so that protective devices could be triggered in time to prevent fall-related injuries for older people.

Credit card (CC) fraud has been a persistent problem and has affected financial organizations. Traditional machine learning (ML) algorithms are ineffective owing to the increased attack space, and techniques such as long short-term memory (LSTM) have shown promising results in detecting CC fraud patterns. However, owing to the black box nature of the LSTM model, the decision-making process could be improved. Thus, in this paper, we propose a scheme, RaKShA, which presents explainable artificial intelligence (XAI) to help understand and interpret the behavior of black box models. XAI is formally used to interpret these black box models; however, we used XAI to extract essential features from the CC fraud dataset, consequently improving the performance of the LSTM model. The XAI was integrated with LSTM to form an explainable LSTM (X-LSTM) model. The proposed approach takes preprocessed data and feeds it to the XAI model, which computes the variable importance plot for the dataset, which simplifies the feature selection. Then, the data are presented to the LSTM model, and the output classification is stored in a smart contract (SC), ensuring no tampering with the results. The final data are stored on the blockchain (BC), which forms trusted and chronological ledger entries. We have considered two open-source CC datasets. We obtain an accuracy of 99.8% with our proposed X-LSTM model over 50 epochs compared to 85% without XAI (simple LSTM model). We present the gas fee requirements, IPFS bandwidth, and the fraud detection contract specification in blockchain metrics. The proposed results indicate the practical viability of our scheme in real-financial CC spending and lending setups.

Multilabel text classification is a task of categorizing text into one or more categories. Like other machine learning, multilabel classification performance is limited when there is small labeled data and leads to the difficulty of capturing semantic relationships. In this case, it requires a multi-label text classification technique that can group four labels from news articles. Deep Learning is a proposed method for solving problems in multi-label text classification techniques. By comparing the seven proposed Long Short-Term Memory (LSTM) models with large-scale datasets by dividing 4 LSTM models with 1 layer, 2 layer and 3-layer LSTM and Bidirectional LSTM to show that LSTM can achieve good performance in multi-label text classification. The results show that the evaluation of the performance of the 2-layer LSTM model in the training process obtained an accuracy of 96 with the highest testing accuracy of all models at 94.3. The performance results for model 3 with 1-layer LSTM obtained the average value of precision, recall, and f1-score equal to the 94 training process accuracy. This states that model 3 with 1-layer LSTM both training and testing process is better. The comparison among seven proposed LSTM models shows that model 3 with 1 layer LSTM is the best model.

Drought prediction is crucial for mitigating risks and designing measures to alleviate its impact. Machine learning models have been widely applied in the field of drought prediction in recent years. This study concentrated on predicting meteorological droughts in southwest China, a region prone to frequent and severe droughts, particularly in areas with sparse meteorological station coverage. The long short-term memory (LSTM) predictive model, which is a deep learning model, was constructed by calculating standardized precipitation evapotranspiration index (SPEI) values based on 144 weather station observations from 1980 to 2020. The 5-fold cross-validation method was used for the hyperparameter optimization of the model. The LSTM model underwent comprehensive assessment and validation through multiple methods. This included the use of several accuracy assessment indicators and a comparison of results. The comparison covered different drought characteristics among the LSTM predictive model, the benchmark random forest (RF) predictive model, the historical drought situations, and the calculated SPEI values based on observations from 144 weather stations. The results showed that the training results of the LSTM predictive model basically agreed with the SPEI values calculated from weather station observations. The model-predicted variation trend of SPEI values for 2020 was similar to the variation in SPEI values calculated based on weather station observations. On the test set, the coefficient of determination (R 2), the root mean square error, the explained variance score, the Nash–Sutcliffe efficiency, and the Kling–Gupta efficiency were 0.757, 0.210, 0.802, 0.761, and 0.212, respectively. The total consistency rate of the drought grade was 59.26%. The spatial correlation distribution of SPEI values between LSTM model prediction and calculation from meteorological stations in 2020 was more than 0.5 for most regions. The correlation coefficients exceeded 0.6 in western Tibet and Chengdu Plains. Compared to the RF model, the LSTM model excelled in all five performance evaluation metrics and demonstrated a higher overall consistency rate for drought categories. The Kruskal–Wallis test for both the LSTM and RF models all indicated no significant difference in the distributions between the predicted and observed data. Scatter plots revealed that the prediction accuracy for both models in 2020 was suboptimal, with the SPEI showing a comparatively narrow range of values. Nonetheless, the LSTM model significantly outperformed the RF model in terms of prediction accuracy. In summary, the LSTM model demonstrated good overall performance, accuracy, and applicability. It has the potential to enhance dynamic drought prediction in regions with complex terrain, diverse climatic factors, and sparse weather station networks.

Objectives: Atmospheric Ozone plays an important role in global climate change, human health and environmental conditions. Accurate and timely prediction of variation in Total Column Ozone (TCO) concentration is very important for both climatology and environment. In this context, the present study aims to utilize the advancement of Artificial Intelligence in Machine learning and a conventional method to develop models for the prediction of TCO concentration using historical time series data over a tropical region, India. Methods: Long Short-Term Memory (LSTM) deep learning networks are very useful in classifying, processing and making predictions based on time series data. In this work, Multiple Linear Regression (MLR) model, Long Short-Term Memory (LSTM) models are developed for forecasting TCO over a tropical region. The Statistical Parameters such as Root Mean Square Error (RMSE), Mean Absolute Error (MAE) and Mean Absolute Percentage Error (MAPE) were used to analyze and evaluate the performance of the proposed models. Findings: In the present study the TCO concentration varied between 217.4 DU and 288.6 DU. The performances of MLR and LSTM model were compared. From the results, it is evident that the values of RMSE, MAE and MAPE of the MLR model are 3.807924, 2.93766, 1.15 respectively whereas the values for LSTM model are 3.074492, 2.574, 1.01 respectively. LSTM was found to be the accurate model for the study data set and the predicted TCO concentration has a very good correlation with the actual observations and it does not rely on the meteorological parameters. Novelty: The LSTM models are usually employed in the field of machine translation, speech recognition and for the prediction of traffic levels, stock levels and air pollutants. This work utilizes the fast numerical machine learning computing library Tensorflow and a high level neural network library Keras that runs on top of Tensorflow to develop a LSTM network model for the prediction of the concentration levels of TCO. Keywords: Total Column Ozone; Regression; Forecasting; Climate Change; Long Short-Term Memory

Increasing incidents of phishing attacks tempt a significant challenge for cybersecurity personals. Phishing is a deceitful venture with an intention to steal confidential information of an organization or an individual. Many works have been performed to build anti-phishing solutions over the years, but attackers are coming with new manoeuvres from time to time. Many of the existing techniques are experimented based on limited set of URLs and dependent on other software to collect domain related information of the URLs. In this paper, with an aim to build a more accurate and effective phishing attack detection system, we used the concept of ensemble learning using Long Short-Term Memory (LSTM) models. We proposed ensemble of LSTM models using bagging approach and stacking approach. For performing classification using LSTM method, no separate feature extraction is done. Ensemble models are built integrating the predictions of multiple LSTM models. Performances of proposed ensemble LSTM methods are compared with five different machine learning classification methods. To implement these machine learning algorithms, different URL based lexical features are extracted. Mutual Information based feature selection algorithm is used to select more relevant features to perform classifications. Both the bagging and the stacking approaches of ensemble learning using LSTM models outperform other machine learning techniques. The results are compared with other anti-phishing solutions implemented using deep learning methods. Our approaches have proved to be the more accurate one with a low false positive rate of less than 0.15% performed comparatively on a larger dataset.

Precise rock lithology identification from well logs is critical for reservoir characterization and field development. Traditional knowledge-based lithology interpretation is highly dependent on the interpreter’s experience and judgment, which could lead to erroneous decision making or biased prediction. To reduce human involvement and improve interpretation efficiency and consistency, a knowledge-constrained long short-term memory (LSTM) network solution is introduced. In this study, LSTM networks are applied with different constrains to obtain the mapping relations and validate the knowledge-constrained LSTM model accordingly. The entire workflow mainly includes input logging data preprocessing, different constrain validations during the LSTM model training, and validation processes. This study covers and compares the direct LSTM model without constrains, rectangular constrain LSTM (RCLSTM), and Gaussian window weighted constrain LSTM (GWLSTM). In particular, GWLSTM applies a sample cluster as input instead of single sample points. The weight of the sample point is controlled by a distance-correlated Gaussian window, which means the closer to the predicting point, the greater the impact on the prediction. LSTM, RCLSTM, and GWLSTM models are tested on a field data set of five wells in a typical sandstone gas reservoir. Two wells are used to train the network, while the other three wells are used for network assessment. The test results demonstrate that by applying LSTM networks to establish the mapping between the logging curves (e.g., CNL, DT, DEN, GR, and RD) and rock lithology, rock lithologies in target formation can be predicted from well logs. Moreover, the lithology predictions by the GWLSTM model are more accurate than those of conventional LSTM and RCLSTM models, especially for thin layers. In conclusion, GWLSTM networks improve lithology identification accuracy by taking stratigraphic sequences into consideration. And the Gaussian window constrains are more effective than rectangular window constrains for thin layer predictions. Lastly, GWLSTM doesn’t require a large training data set, which makes it advantageous for reservoirs with limited wells.

Streamflow plays a vital role in water resource management and environmental impact assessment. This study is a novel application of the Long Short-Term Memory (LSTM) model, a type of recurrent neural network, for real-time streamflow prediction in the Upper Humber River Watershed in western Newfoundland. It also compares the performance of the LSTM model with the physically based SWAT model. The LSTM model was optimized by tuning hyperparameters and adjusting the window size to balance capturing historical data and ensuring prediction stability. Using single input variables such as daily average temperature or precipitation, the LSTM achieved a high Nash-Sutcliffe Efficiency (NSE) of 0.95. In comparison, the results show that the LSTM model delivers a more competitive performance, achieving an NSE of 0.95 versus SWAT’s 0.77, and a percent bias (PBIAS) of 0.62 compared to SWAT’s 8.26. Unlike SWAT, the LSTM model does not overestimate high flows and excels in predicting low flows. Additionally, the LSTM successfully predicted daily streamflow using real-time data. Despite challenges in interpretability and generalizability, the LSTM model demonstrated strong performance, particularly during extreme events, making it a valuable tool for streamflow prediction in cold climates where accurate forecasts are crucial for effective water management. This study highlights the potential of the LSTM model’s application to water resource management.

BackgroundAdministrative costs for billing and insurance-related activities in the United States are substantial. One critical cause of the high overhead of administrative costs is medical billing errors. With advanced deep learning techniques, developing advanced models to predict hospital and professional billing codes has become feasible. These models can be used for administrative cost reduction and billing process improvements.ObjectiveIn this study, we aim to develop an automated anesthesiology current procedural terminology (CPT) prediction system that translates manually entered surgical procedure text into standard forms using neural machine translation (NMT) techniques. The standard forms are calculated using similarity scores to predict the most appropriate CPT codes. Although this system aims to enhance medical billing coding accuracy to reduce administrative costs, we compare its performance with that of previously developed machine learning algorithms.MethodsWe collected and analyzed all operative procedures performed at Michigan Medicine between January 2017 and June 2019 (2.5 years). The first 2 years of data were used to train and validate the existing models and compare the results from the NMT-based model. Data from 2019 (6-month follow-up period) were then used to measure the accuracy of the CPT code prediction. Three experimental settings were designed with different data types to evaluate the models. Experiment 1 used the surgical procedure text entered manually in the electronic health record. Experiment 2 used preprocessing of the procedure text. Experiment 3 used preprocessing of the combined procedure text and preoperative diagnoses. The NMT-based model was compared with the support vector machine (SVM) and long short-term memory (LSTM) models.ResultsThe NMT model yielded the highest top-1 accuracy in experiments 1 and 2 at 81.64% and 81.71% compared with the SVM model (81.19% and 81.27%, respectively) and the LSTM model (80.96% and 81.07%, respectively). The SVM model yielded the highest top-1 accuracy of 84.30% in experiment 3, followed by the LSTM model (83.70%) and the NMT model (82.80%). In experiment 3, the addition of preoperative diagnoses showed 3.7%, 3.2%, and 1.3% increases in the SVM, LSTM, and NMT models in top-1 accuracy over those in experiment 2, respectively. For top-3 accuracy, the SVM, LSTM, and NMT models achieved 95.64%, 95.72%, and 95.60% for experiment 1, 95.75%, 95.67%, and 95.69% for experiment 2, and 95.88%, 95.93%, and 95.06% for experiment 3, respectively.ConclusionsThis study demonstrates the feasibility of creating an automated anesthesiology CPT classification system based on NMT techniques using surgical procedure text and preoperative diagnosis. Our results show that the performance of the NMT-based CPT prediction system is equivalent to that of the SVM and LSTM prediction models. Importantly, we found that including preoperative diagnoses improved the accuracy of using the procedure text alone.