Evaluating Contextual Embedding Models for Multi-Label PICO Classification in Heart Disease: Addressing the Intervention - Comparison Bottleneck

Introduction: Forecasting electrical energy demand is crucial for predicting future energy consumption patterns, which aids in effective energy management and distribution. Various forecasting methods have been developed, yet this study explores univariate time series analysis using Bidirectional Long Short-Term Memory (BiLSTM) and a hybrid Convolutional Neural Network–Long Short-Term Memory (CNN-LSTM) model. These deep learning techniques are designed to capture both temporal dependencies and spatial patterns, improving predictive performance in energy forecasting. Objectives: This study aims to evaluate the forecasting performance of deep learning models in univariate time series energy demand prediction. Specifically, it seeks to: Implement and compare the forecasting performance of Bidirectional LSTM and hybrid CNN-LSTM models using a publicly available dataset from Transmission Service Operators (TSO). Preprocess the dataset using appropriate data preparation techniques, such as normalization, handling missing values, and feature selection, before training the models. Assess predictive accuracy by evaluating both models using key performance metrics, including Mean Absolute Error (MAE), Mean Absolute Percentage Error (MAPE), Root Mean Squared Error (RMSE), and R-Squared (R²). Methods: The dataset used in this study was obtained from a public portal for Transmission Service Operators (TSO). Before training, the data underwent preprocessing techniques such as normalization, handling missing values, and feature selection to improve model performance. Two deep learning models—BiLSTM and CNN-LSTM—were implemented and trained on the dataset. The performance of each model was evaluated using four key metrics: Mean Absolute Error (MAE) – measures the average magnitude of errors, Mean Absolute Percentage Error (MAPE) – represents error as a percentage of actual values, Root Mean Squared Error (RMSE) – penalizes larger errors more heavily than MAE, R-Squared (R²) – indicates how well predictions align with actual data. Results: Experimental findings reveal that the hybrid CNN-LSTM model outperformed the BiLSTM model across all evaluation metrics. The CNN-LSTM model achieved a lower MAE of 499.08 compared to 780.56 in BiLSTM, a lower MAPE of 1.80% versus 2.52%, and a reduced RMSE of 671.37 compared to 1,042.20. Additionally, the CNN-LSTM model obtained a slightly higher R² score of 0.97 compared to 0.94 in BiLSTM, indicating a better fit for the data. Conclusion: The results demonstrate that integrating CNN with LSTM significantly improves predictive accuracy in univariate time series energy demand forecasting. The CNN component enhances feature extraction, allowing the LSTM layers to capture complex temporal dependencies more effectively. Consequently, the hybrid CNN-LSTM model emerges as a more robust approach compared to BiLSTM alone, making it a valuable tool for accurate energy demand forecasting. Further research can explore additional deep learning architectures or hybrid models to optimize forecasting performance further.

BackgroundExtracting primary care information in terms of Patient/Problem, Intervention, Comparison and Outcome, known as PICO elements, is difficult as the volume of medical information expands and the health semantics is complex to capture it from unstructured information. The combination of the machine learning methods (MLMs) with rule based methods (RBMs) could facilitate and improve the PICO extraction. This paper studies the PICO elements extraction methods. The goal is to combine the MLMs with the RBMs to extract PICO elements in medical papers to facilitate answering clinical questions formulated with the PICO framework.MethodsFirst, we analyze the aspects of the MLM model that influence the quality of the PICO elements extraction. Secondly, we combine the MLM approach with the RBMs in order to improve the PICO elements retrieval process. To conduct our experiments, we use a corpus of 1000 abstracts.ResultsWe obtain an F-score of 80% for P element, 64% for the I element and 92% for the O element. Given the nature of the used training corpus where P and I elements represent respectively only 6.5 and 5.8% of total sentences, the results are competitive with previously published ones.ConclusionsOur study of the PICO element extraction shows that the task is very challenging. The MLMs tend to have an acceptable precision rate but they have a low recall rate when the corpus is not representative. The RBMs backed up the MLMs to increase the recall rate and consequently the combination of the two methods gave better results.

In this paper, the primary focus is of Slot Tagging of Gujarat Dialogue, which enables the Gujarati language communication between human and machine, allowing machines to perform given task and provide desired output. The accuracy of tagging entirely depends on bifurcation of slots and word embedding. It is also very challenging for a researcher to do proper slot tagging as dialogue and speech differs from human to human, which makes the slot tagging methodology more complex. Various deep learning models are available for slot tagging for the researchers, however, in the instant paper it mainly focuses on Long Short-Term Memory (LSTM), Convolutional Neural Network - Long Short-Term Memory (CNN-LSTM) and Long Short-Term Memory – Conditional Random Field (LSTM-CRF), Bidirectional Long Short-Term Memory (BiLSTM), Convolutional Neural Network - Bidirectional Long Short-Term Memory (CNN-BiLSTM) and Bidirectional Long Short-Term Memory – Conditional Random Field (BiLSTM-CRF). While comparing the above models with each other, it is observed that BiLSTM models performs better than LSTM models by a variation ~2% of its F1-measure, as it contains an additional layer which formulates the word string to traverse from backward to forward. Within BiLSTM models, BiLSTM-CRF has outperformed other two Bi-LSTM models. Its F1-measure is better than CNN-BiLSTM by 1.2% and BiLSTM by 2.4%.KeywordsSpoken Language Understanding (SLU)Long Short-Term Memory (LSTM)Slot taggingBidirectional Long Short-Term Memory (BiLSTM)Convolutional Neural Network - Bidirectional Long Short-Term Memory (CNN-BiLSTM)Bidirectional Long Short-Term Memory (BiLSTM-CRF)

In recent years, the application of deep learning methods has become increasingly popular, especially for big data, because big data has a very large data size and needs to be predicted accurately. One of the big data is the document text data of cancer clinical trials. Clinical trials are studies of human participation in helping people's safety and health. The aim of this paper is to classify cancer clinical texts from a public data set. The proposed algorithms are Bidirectional Long Short Term Memory (BiLSTM) and Word Embedding Features (WE). This study has contributed to a new classification model for documenting clinical trials and increasing the classification performance evaluation. In this study, two experiments work are conducted, namely experimental work BiLSTM without WE, and experimental work BiLSTM using WE. The experimental results for BiLSTM without WE were accuracy = 86.2; precision = 85.5; recall = 87.3; and F-1 score = 86.4. meanwhile the experiment results for BiLSTM using WE stated that the evaluation score showed outstanding performance in text classification, especially in clinical trial texts with accuracy = 92,3; precision = 92.2; recall = 92.9; and F-1 score = 92.5.

PICO element detection in medical text without metadata: Are first sentences enough?

With the rise of Artificial Intelligence (AI), machine learning techniques are now conquering the research field of Prognostics and Health Management (PHM). Classic deployable prognostic models manipulate large amount of machinery historical data to map the degradation process based on inherent features. Nowadays one of the major challenges in prognostics research is the data deficit problem when historical data is not available or accessible, in enough quantity and variety. In the frames of Transfer Learning, the domain adaptation technique aims to build a model with strong generalization ability which can be transferred to datasets with different distributions. In this paper, a Domain Adversarial Neural Network (DANN) model is combined with a Bidirectional Long Short- Term Memory (Bi-LSTM) neural network for the estimation of the Remaining Useful Life (RUL) of rolling element bearings. The unsupervised domain adaptation is fulfilled using a labelled bearing degradation dataset as the source domain data and an unlabelled dataset captured under different operation conditions as the target domain data for the Bi-LSTM DANN. The proposed method achieves promising results, applied on real bearing vibration data captured on run-to-failure tests, with high prediction accuracy of the bearing RUL compared to un-adapted methods.

This study analyzes consumer review sentiments of Ayam Goreng Nelongso on Google Maps using the Bidirectional Long Short-Term Memory (Bi-LSTM) method. The data used consists of 4,450 reviews, with varying training and testing data ratios ranging from 90:10 to 10:90. The evaluation results show that the Bi-LSTM model performs excellently, with an average accuracy of 98.33%, precision of 99.44%, recall of 99.44%, and F1-score of 99.44%. These findings demonstrate that Bi-LSTM can reliably and consistently identify positive, negative, and neutral sentiments in consumer review data, providing valuable insights for improving the services and product quality of the MSME Ayam Goreng Nelongso.Keywords: Sentiment Analysis, Ayam Goreng Nelongso, Google Maps, Bidirectional Long Short-Term Memory, Bi-LSTM. ABSTRAKPenelitian ini menganalisis sentimen ulasan konsumen terhadap Ayam Goreng Nelongso di Google Maps menggunakan metode Bidirectional Long Short-Term Memory (Bi-LSTM). Data yang digunakan mencakup 4.450 ulasan dengan rasio data latih dan uji bervariasi, mulai dari 90:10 hingga 10:90. Hasil evaluasi menunjukkan bahwa model Bi-LSTM memiliki performa sangat baik dengan rata-rata akurasi 98,33%, presisi 99,44%, recall 99,44%, dan F1-score 99,44%. Temuan ini menunjukkan bahwa Bi-LSTM mampu secara andal dan konsisten mengidentifikasi sentimen positif, negatif, dan netral pada data ulasan konsumen, memberikan wawasan yang bermanfaat untuk peningkatan layanan dan kualitas produk UMKM Ayam Goreng Nelongso.Kata Kunci: Analisis Sentimen, Ayam Goreng Nelongso, Google Maps, Bidirectional Long Short-Term Memory, Bi-LSTM.

Lately, IoT (Internet of Things) based m-healthcare application is rising to give real-time servicing in the present global lifestyle. Cloud-based healthcare architecture provides best results than traditional approaches. Currently, Integrating IoT devices in the medical environment plays a crucial role in handling a massive amount of medical data. Thus, researcher workers aimed to automate the procedure of diagnosis and detecting diseases with the help of could computing techniques. Furthermore, deep learning and machine learning techniques used in the healthcare field allow healthcare professionals to focus, monitor, highlight and diagnose the region of the problem and present the accurate and required solution in a short duration. Therefore, this paper presents Artificial Bee Colony Optimization with Ensemble Deep Learning based Disease Diagnosis (ABCO-EDLDD) in the IoT atmosphere. The proposed ABCO-EDLDD procedure effectively identifies the existence of diseases in the IoT atmosphere. At the initial stage, the ABCO-EDLDD technique transforms the input data gathered by the IoT devices in different ways. Next, the ABCO algorithm is utilized for the optimal selection of feature subsets. For disease classification, an ensemble of DL models such as Gated Recurrent Unit (GRU), Long Short Term Memory (LSTM), and Bidirectional Long Short Term Memory (BiLSTM) model. Finally, the RMSProp optimizer is used for the optimum tuning of the DL models. The experimental evaluation of the ABCO-EDLDD algorithm takes place by implementing two medical datasets: the HAPT dataset and the heart disease dataset. The experimental results reported the improved performance of the ABCO-EDLDD procedure over other current techniques.

The advancement of information technology provides convenience, but it also brings about problems. One area affected by this is the election process in Indonesia, which has seen a rise in fake news often used to discredit political opponents. Fake news misleads the public into believing incorrect information related to the election. To address this issue, a system is needed to detect fake news in the 2024 election to help the public differentiate between true and false information. This system is developed using an artificial intelligence and deep learning approach trained to do text classification on fake news detection. The training data consists of 1999 entries obtained from the Global Fact-Check Database from turnbackhoax.id, detik.com, and cnnindonesia.com. The machine learning model is built using the Bidirectional Long Short-Term Memory (BI-LSTM) algorithm, which is suitable for processing text data. This study compares two types of feature representations: TF-IDF and contextual embeddings with the IndoBERT model. The study results in the best model for text classification with an accuracy of 92% and a loss of 42.92%, achieved by the model using TF-IDF feature representation. The implementation of this system aims to enhance the integrity of the election process by minimizing the spread of misinformation. Future work will focus on refining the model and expanding the dataset to include more diverse sources for improved accuracy and robustness.

As the stock market is an important part of the national economy, more and more investors have begun to pay attention to the methods to improve the return on investment and effectively avoid certain risks. Many factors affect the trend of the stock market, and the relevant information has the nature of time series. This paper proposes a composite model CNN‐BiSLSTM to predict the closing price of the stock. Bidirectional special long short‐term memory (BiSLSTM) improved on bidirectional long short‐term memory (BiLSTM) adds 1 − tanh(x) function in the output gate which makes the model better predict the stock price. The model extracts advanced features that influence stock price through convolutional neural network (CNN), and predicts the stock closing price through BiSLSTM after the data processed by CNN. To verify the effectiveness of the model, the historical data of the Shenzhen Component Index from July 1, 1991, to October 30, 2020, are used to train and test the CNN‐BiSLSTM. CNN‐BiSLSTM is compared with multilayer perceptron (MLP), recurrent neural network (RNN), long short‐term memory (LSTM), BiLSTM, CNN‐LSTM, and CNN‐BiLSTM. The experimental results show that the mean absolute error (MAE), root‐mean‐squared error (RMSE), and R‐square (R2) evaluation indicators of the CNN‐BiSLSTM are all optimal. Therefore, CNN‐BiSLSTM can accurately predict the closing price of the Shenzhen Component Index of the next trading day, which can be used as a reference for the majority of investors to effectively avoid certain risks.

Using the full PICO model as a search tool for systematic reviews resulted in lower recall for some PICO elements

In cybersecurity, the rise of fileless malware poses a significant challenge to endpoint security. Traditional detection methods often fail against these sophisticated attacks, necessitating advanced techniques like deep learning models. This study highlights the limitations of Bi-Directional Long Short-Term Memory (BLSTM) models in dynamic malware analysis and proposes enhancements through Convolutional Long Short-Term Memory (ConvLSTM) architecture. BLSTM models process input sequences in forward and backward directions, combining the results into one output. While this dual-layer approach improves analysis, it is time-consuming, potentially increasing the risk of fileless malware attacks. A key limitation of BLSTM is the lack of parameter sharing between forward and backward directions. This reduces its ability to capture spatial and temporal features simultaneously, hindering effectiveness in detecting fileless malware. To address this, the ConvLSTM model consolidates feature extraction within a single LSTM cell layer. ConvLSTM breaks down samples into subsequence and uses timesteps for additional feature extraction, enabling spatial-temporal data analysis and improving malware prediction accuracy. The model was tested using a dynamic malware dataset. Unlike traditional LSTM, ConvLSTM integrates convolutional layers, allowing parameter sharing across both spatial and temporal dimensions. This reduces computational complexity and improves model performance in handling multidimensional data. The research re-simulated prior work with BLSTM using the same malware dataset. The Spyder app ran the event simulator and the ConvLSTM model's results replaced BLSTM's using identical parameters. Time, accuracy and loss were the main performance metrics. ConvLSTM outperformed BLSTM, achieving 98% detection accuracy compared to BLSTM's 90%. It also significantly reduced processing time, averaging 10 seconds, while BLSTM took 22 seconds. ConvLSTM experienced lower losses, averaging 10% per epoch versus BLSTM's 20%. In conclusion, ConvLSTM offers superior performance over BLSTM in fileless malware detection. Its enhanced computational efficiency and ability to quickly mitigate threats make it a robust solution for fortifying endpoint security against evolving cyber threats. ConvLSTM holds potential in strengthening defence mechanisms against sophisticated malware attacks, providing a proactive approach to safeguarding networks and data.

In the modern environment, it is quite difficult to detect Heart Disease (HD) through early stage symptoms. Heart disease is the cause of mortality if the diagnosis is delayed including death occurs to prevent these issues. This paper proposes an innovative approach for the detection of heart disease from HD dataset, incorporating advanced techniques. Initially, data cleaning and data normalization is applied to the HD to eliminate or to find the duplicate values. Next, the images are segmented using K means clustering, allowing for the separation of relevant regions associated with HD. Features are then optimized from the segmented HD using the Dung Beetle Optimization (DBO) algorithm, capturing optimized values for further analysis. Finally, a Bidirectional Long Short Term Memory (BiLSTM) framework is proposed as a hybrid model to enhance the classification of HD images, leveraging both forward and backward temporal information to overcome challenges in disease analysis. The assessment of proposed work using python software reveals that the proposed framework with BiLSTM classifier ranks with improved accuracy of 91.56% when compared to the other techniques.

In recent years, the penetration of solar power at residential and utility levels has progressed exponentially. However, due to its stochastic nature, the prediction of solar global horizontal irradiance (GHI) with higher accuracy is a challenging task; but, vital for grid management: planning, scheduling & balancing. Therefore, this paper proposes an ensemble model using the extended scope of wavelet transform (WT) and bidirectional long short term memory (BiLSTM) deep learning network to forecast 24-h ahead solar GHI. The WT decomposes the input time series data into different finite intrinsic model functions (IMF) to extract the statistical features of input time series. Further, the study reduces the number of IMF series by combining the wavelet decomposed components (D1-D6) series on the basis of comprehensive experimental analysis with an aim to improve the forecasting accuracy. Next, the trained standalone BiLSTM networks are allocated to each IMF sub-series to execute the forecasting. Finally, the forecasted values of each sub-series from BiLSTM networks are reconstructed to deliver the final solar GHI forecast. The study performed monthly solar GHI forecasting for one year dataset using one month moving window mechanism for the location of Ahmedabad, Gujarat, India. For the performance comparison, the naïve predictor as a benchmark model, standalone long short term memory (LSTM), gated recurrent unit (GRU), BiLSTM and two other wavelet-based BiLSTM models are also simulated. From the results, it is observed that the proposed model outperforms other models in terms of root mean square error (RMSE) & mean absolute percentage error (MAPE), coefficient of determination (R2) and forecast skill (FS). The proposed model reduces the monthly average RMSE by range from 26.04–58.89%, 5.17–31.35%, 23.26–56.06% & 21.08–57% in comparison with benchmark, standalone BiLSTM, GRU & LSTM networks respectively. On the other hand, the monthly average MAPE is reduced by range from 9 to 51.18%, 12.59–28.14%, 30.43–59.19% & 26.54–58.92% in comparison to benchmark, standalone BiLSTM, GRU & LSTM respectively. Further, the proposed model obtained the value of R2 equal to 0.94 and forecast skill (%) of 47% with reference to the benchmark model.

Biomedical event extraction is an important branch of biomedical information extraction. Trigger extraction is the most essential sub-task in event extraction, which has been widely concerned. Existing trigger extraction studies are mostly based on conventional machine learning or neural networks. But they neglect the ambiguity of word representations and the insufficient feature extraction by shallow hidden layers. In this paper, trigger extraction is treated as a sequence labeling problem. We introduce the language model to dynamically compute contextualized word representations and propose a multi-layer residual bidirectional long short-term memory (BiLSTM) architecture. First, we concatenate contextualized word embedding, pretrained word embedding and character-level embedding as the feature representations, which effectively solves the tokens’ ambiguity in biomedical corpora. Then, the designed BiLSTM block with residual connection and gated multi-layer perceptron is adopted to extract features iteratively. This architecture improves the ability of our model to capture information and avoids gradient exploding or vanishing. Finally, we combine the multi-layer residual BiLSTM with CRF layer to obtain more reasonable label sequences. Comparing with other state-of-the-art methods, the proposed model achieves the competitive performance (F1-score: 80.74%) on the biomedical multi-level event extraction (MLEE) corpus without any manual participation and feature engineering.