Embedding Technique Research Articles

Swift Detection of XSS Attacks: Enhancing XSS Attack Detection by Leveraging Hybrid Semantic Embeddings and AI Techniques

AbstractCross-Site Scripting (XSS) attacks continue to be a significant threat to web application security, necessitating robust detection mechanisms to safeguard user data and ensure system integrity. In this study, we present a novel approach for detecting XSS attacks that harnesses the combined capabilities of the Universal Sentence Encoder (USE) and Word2Vec embeddings as a feature extractor, aiming to enhance the performance of machine learning and deep learning techniques. By leveraging the semantic understanding of sentences offered by USE and the word-level representations from Word2Vec, we obtain a comprehensive feature representation for XSS attack payloads. Our proposed approach aims to capture both fine-grained word meanings and broader sentence contexts, leading to enhanced feature extraction and improved model performance. We conducted extensive experiments utilizing machine learning and deep learning architectures to evaluate the effectiveness of our approach. The obtained results demonstrate that our combined embeddings approach outperforms traditional methods, achieving superior accuracy, precision, recall, ROC, and F1-score in detecting XSS attacks. This study not only advances XSS attack detection but also highlights the potential of state-of-the-art natural language processing techniques in web security applications. Our findings offer valuable insights for the development of more robust and effective security measures against XSS attacks.

Determining community happiness index with transformers and attention-based deep learning

In the current digital era, evaluating the quality of people's lives and their happiness index is closely related to their expressions and opinions on Twitter social media. Measuring population welfare goes beyond monetary aspects, focusing more on subjective well-being, and sentiment analysis helps evaluate people's perceptions of happiness aspects. Aspect-based sentiment analysis (ABSA) effectively identifies sentiments on predetermined aspects. The previous study has used Word-to-Vector (Word2Vec) and long short-term memory (LSTM) methods with or without attention mechanism (AM) to solve ABSA cases. However, the problem with the previous study is that Word2Vec has the disadvantage of being unable to handle the context of words in a sentence. Therefore, this study will address the problem with bidirectional encoder representations from transformers (BERT), which has the advantage of performing bidirectional training. Bayesian optimization as a hyperparameter tuning technique is used to find the best combination of parameters during the training process. Here we show that BERT-LSTM-AM outperforms the Word2Vec-LSTM-AM model in predicting aspect and sentiment. Furthermore, we found that BERT is the best state-of-the-art embedding technique for representing words in a sentence. Our results demonstrate how BERT as an embedding technique can significantly improve the model performance over Word2Vec.

Autobalanced Multitask Node Embedding Framework for Intelligent Education.

Recently, online education has become popular. Many e-learning platforms have been launched with various intelligent services aimed at improving the learning efficiency and effectiveness of learners. Graphs are used to describe the pairwise relations between entities, and the node embedding technique is the foundation of many intelligent services, which have received increasing attention from researchers. However, the graph in the intelligent education scenario has three noteworthy properties, namely, heterogeneity, evolution, and lopsidedness, which makes it challenging to implement ecumenical node embedding methods on it. In this article, an autobalanced multitask node embedding model is proposed, named MNE, and applied to the interaction graph, settling a few actual tasks in intelligent education. More specifically, MNE builds two purpose-built self-supervised node embedding learning tasks for heterogeneous evolutive graphs. Edge-specific reconstruction tasks are built according to the semantic information and properties of the heterogeneous edges, and an evolutive weight regression task is designed, aiding the model to perceive the evolution of learners' implicit cognitive states. Then, both aleatoric and epistemic uncertainty quantification techniques are introduced, achieving both task- and node-level weight estimation and instructing subtask autobalancing. Experimental results on real-world datasets indicate that the proposed model outperforms the state-of-the-art graph embedding methods on two assessment tasks and demonstrates the validity of the proposed multitask framework and subtask balancing mechanism. Our implementations are available at https://github.com/ccnu-mathits/MNE4HEN.

Sentiment analysis of student feedback using attention-based RNN and transformer embedding

Sentiment analysis systems aim to assess people’s opinions across various domains by collecting and categorizing feedback and reviews. In our study, researchers put forward a sentiment analysis system that leverages three distinct embedding techniques: automatic, global vectors (GloVe) for word representation, and bidirectional encoder representations from transformers (BERT). This system features an attention layer, with the best model chosen through rigorous comparisons. In developing the sentiment analysis model, we employed a hybrid dataset comprising students’ feedback and comments. This dataset comprises 3,820 comments, including 2,773 from formal evaluations and 1,047 generated by ChatGPT and prompting engineering. Our main motivation for integrating generative AI was to balance both positive and negative comments. We also explored recurrent neural network (RNN), gated recurrent unit (GRU), long short-term memory (LSTM), and bidirectional long short-term memory (Bi-LSTM), with and without pre-trained GloVe embedding. These techniques produced F-scores ranging from 67% to 69%. On the other hand, the sentiment model based on BERT, particularly its KERAS implementation, achieved higher F-scores ranging from 83% to 87%. The Bi-LSTM architecture outperformed other models and the inclusion of an attention layer further enhanced the performance, resulting in F-scores of 89% and 88% from the Bi-LSTM-BERT sentiment models, respectively.

An API Recommendation Method for Querying Mobile Computing Problems

Nowadays mobile computing has penetrated into our lives and work. Major companies have invested a lot of material and financial resources in the development of mobile terminals. For developers, querying the right API is extremely important. However, finding the right APIs can be time-consuming and laborious. In this paper, to solve the problems developers may face in the actual development process and improve the development efficiency, we propose RAMC (Recommendation APIs for Mobile Computing), an API recommendation approach leveraging word embedding technique and the information crawling from Stack Overflow posts and Java core packages, to recommend appropriate APIs for developers. Furthermore, RAMC also provides developers with label words, similar questions and relevant code. To evaluate the effectiveness of RAMC, we decided to analyze our system by simulating an instance. By testing a problem encountered during mobile computing development, the RAMC can effectively output the much-related API and tags for developers.

CVs Classification Using Neural Network Approaches Combined with BERT and Gensim: CVs of Moroccan Engineering Students

Deep learning (DL)-oriented document processing is widely used in different fields for extraction, recognition, and classification processes from raw corpus of data. The article examines the application of deep learning approaches, based on different neural network methods, including Gated Recurrent Unit (GRU), long short-term memory (LSTM), and convolutional neural networks (CNNs). The compared models were combined with two different word embedding techniques, namely: Bidirectional Encoder Representations from Transformers (BERT) and Gensim Word2Vec. The models are designed to evaluate the performance of architectures based on neural network techniques for the classification of CVs of Moroccan engineering students at ENSAK (National School of Applied Sciences of Kenitra, Ibn Tofail University). The used dataset included CVs collected from engineering students at ENSAK in 2023 for a project on the employability of Moroccan engineers in which new approaches were applied, especially machine learning, deep learning, and big data. Accordingly, 867 resumes were collected from five specialties of study (Electrical Engineering (ELE), Networks and Systems Telecommunications (NST), Computer Engineering (CE), Automotive Mechatronics Engineering (AutoMec), Industrial Engineering (Indus)). The results showed that the proposed models based on the BERT embedding approach had more accuracy compared to models based on the Gensim Word2Vec embedding approach. Accordingly, the CNN-GRU/BERT model achieved slightly better accuracy with 0.9351 compared to other hybrid models. On the other hand, single learning models also have good metrics, especially based on BERT embedding architectures, where CNN has the best accuracy with 0.9188.

Deep parallel contextual analysis framework based emotion prediction in community wellness communications on social media

Understanding public emotion on social media about community wellness is crucial for enhancing health awareness and guiding policy-making. In order to more fully mine the deep contextual semantical information of short texts and further enhance the effectiveness of emotion prediction in social media, we propose the Deep Parallel Contextual Analysis Framework (DPCAF) in the community wellness domain, specifically addressing the challenges of limited text length and available semantical features in social media text. Specifically, at the embedding layer, we first utilize two different word embedding techniques to generate high-quality vector representations, aiming to achieve more comprehensive semantical capture, stronger generalization ability, and more robust model performance. Subsequently, in the deep contextual layer, the obtained representations are fused with POS and locational representations, and processed through a deep parallel layer composed of Convolutional Neural Networks and Bidirectional Long Short-Term Memory Network. An attention model is then used to further extract semantical features of social media texts. Finally, these deep parallel contextual representations are post-integrated for emotion prediction. Experiments on a dataset collected from social media regarding community wellness demonstrate that compared to benchmark models, DPCAF achieves at least a 4.81 % increase in Precision, a 3.44 % increase in Recall, and a 10.81 % increase in F1-score. Relative to the most advanced models, DPCAF shows a minimum improvement of 2.65 % in Precision, 3.02 % in Recall, and 2.53 % in F1-score.

Towards Media Monitoring: Detecting Known and Emerging Topics through Multilingual and Crosslingual Text Classification

This study aims to address challenges in media monitoring by enhancing closed-set topic classification in multilingual contexts (where both training and testing occur in several languages) and crosslingual contexts (where training is in English and testing spans all languages). To achieve this goal, we utilized a dataset from the European Media Monitoring webpage, which includes approximately 15,000 article titles across 18 topics in 58 different languages spanning a period of nine months from May 2022 to March 2023. Our research conducted comprehensive comparative analyses of nine approaches, encompassing a spectrum of embedding techniques (word, sentence, and contextual representations) and classifiers (trainable/fine-tunable, memory-based, and generative). Our findings reveal that the LaBSE+FFNN approach achieved the best performance, reaching macro-averaged F1-scores of 0.944 ± 0.015 and 0.946 ± 0.019 in both multilingual and crosslingual scenarios. LaBSE+FFNN’s similar performance in multilingual and crosslingual scenarios eliminates the need for machine translation into English. We also tackled the open-set topic classification problem by training a binary classifier capable of distinguishing between known and new topics with the average loss of ∼0.0017 ± 0.0002. Various feature types were investigated, reaffirming the robustness of LaBSE vectorization. The experiments demonstrate that, depending on the topic, new topics can be identified with accuracies above ∼0.796 and of ∼0.9 on average. Both closed-set and open-set topic classification modules, along with additional mechanisms for clustering new topics to organize and label them, are integrated into our media monitoring system, which is now used by our real client.

Identification of patients' smoking status using an explainable AI approach: a Danish electronic health records case study.

Smoking is a critical risk factor responsible for over eight million annual deaths worldwide. It is essential to obtain information on smoking habits to advance research and implement preventive measures such as screening of high-risk individuals. In most countries, including Denmark, smoking habits are not systematically recorded and at best documented within unstructured free-text segments of electronic health records (EHRs). This would require researchers and clinicians to manually navigate through extensive amounts of unstructured data, which is one of the main reasons that smoking habits are rarely integrated into larger studies. Our aim is to develop machine learning models to classify patients' smoking status from their EHRs. This study proposes an efficient natural language processing (NLP) pipeline capable of classifying patients' smoking status and providing explanations for the decisions. The proposed NLP pipeline comprises four distinct components, which are; (1) considering preprocessing techniques to address abbreviations, punctuation, and other textual irregularities, (2) four cutting-edge feature extraction techniques, i.e. Embedding, BERT, Word2Vec, and Count Vectorizer, employed to extract the optimal features, (3) utilization of a Stacking-based Ensemble (SE) model and a Convolutional Long Short-Term Memory Neural Network (CNN-LSTM) for the identification of smoking status, and (4) application of a local interpretable model-agnostic explanation to explain the decisions rendered by the detection models. The EHRs of 23,132 patients with suspected lung cancer were collected from the Region of Southern Denmark during the period 1/1/2009-31/12/2018. A medical professional annotated the data into 'Smoker' and 'Non-Smoker' with further classifications as 'Active-Smoker', 'Former-Smoker', and 'Never-Smoker'. Subsequently, the annotated dataset was used for the development of binary and multiclass classification models. An extensive comparison was conducted of the detection performance across various model architectures. The results of experimental validation confirm the consistency among the models. However, for binary classification, BERT method with CNN-LSTM architecture outperformed other models by achieving precision, recall, and F1-scores between 97% and 99% for both Never-Smokers and Active-Smokers. In multiclass classification, the Embedding technique with CNN-LSTM architecture yielded the most favorable results in class-specific evaluations, with equal performance measures of 97% for Never-Smoker and measures in the range of 86 to 89% for Active-Smoker and 91-92% for Never-Smoker. Our proposed NLP pipeline achieved a high level of classification performance. In addition, we presented the explanation of the decision made by the best performing detection model. Future work will expand the model's capabilities to analyze longer notes and a broader range of categories to maximize its utility in further research and screening applications.

Memory Corrections to Markovian Langevin Dynamics.

Analysis of non-Markovian systems and memory-induced phenomena poses an everlasting challenge in the realm of physics. As a paradigmatic example, we consider a classical Brownian particle of mass M subjected to an external force and exposed to correlated thermal fluctuations. We show that the recently developed approach to this system, in which its non-Markovian dynamics given by the Generalized Langevin Equation is approximated by its memoryless counterpart but with the effective particle mass M∗<M, can be derived within the Markovian embedding technique. Using this method, we calculate the first- and the second-order memory correction to Markovian dynamics of the Brownian particle for the memory kernel represented as the Prony series. The second one lowers the effective mass of the system further and improves the precision of the approximation. Our work opens the door for the derivation of higher-order memory corrections to Markovian Langevin dynamics.

A novel self-supervised sentiment classification approach using semantic labeling based on contextual embeddings

AbstractSentiment Analysis (SA) is a domain or context-oriented task since the sentiment words convey different sentiments in various domains. As a result, the domain-independent lexicons cannot correctly recognize the sentiment of domain-dependent words. To address this problem, this paper proposes a novel self-supervised SA method based on semantic similarity, contextual embedding, and Deep Learning Techniques. It introduces a new Pseudo-label generator that estimates the pseudo-labels of samples using semantic similarity between the samples and their sentiment words. It proposes two new concepts to calculate semantic similarity: The Soft-Cosine Similarity of a sample with its Positive words (SCSP) and the Soft-Cosine Similarity of a document with its Negative words (SCSN). Then, the Pseudo-label generator uses these concepts and the number of sentiment words to estimate the label of each sample. Later on, a novel method is proposed to find the samples with highly accurate pseudo-labels. Finally, a hybrid classifier, composed of a Convolutional Neural Network (CNN) and a Gated Recurrent Unit (GRU), is trained using these highly accurate pseudo-labeled data to predict the label of unseen data. The comparison of the proposed method with the lexicons and other similar existing methods demonstrates that the proposed method outperforms them in terms of accuracy, precision, recall, and F1 score.

On Embedding Implementations in Text Ranking and Classification Employing Graphs

This paper aims to enhance the Graphical Set-based model (GSB) for ranking and classification tasks by incorporating node and word embeddings. The model integrates a textual graph representation with a set-based model for information retrieval. Initially, each document in a collection is transformed into a graph representation. The proposed enhancement involves augmenting the edges of these graphs with embeddings, which can be pretrained or generated using Word2Vec and GloVe models. Additionally, an alternative aspect of our proposed model consists of the Node2Vec embedding technique, which is applied to a graph created at the collection level through the extension of the set-based model, providing edges based on the graph’s structural information. Core decomposition is utilized as a method for pruning the graph. As a byproduct of our information retrieval model, we explore text classification techniques based on our approach. Node2Vec embeddings are generated by our graphs and are applied in order to represent the different documents in our collections that have undergone various preprocessing methods. We compare the graph-based embeddings with the Doc2Vec and Word2Vec representations to elaborate on whether our approach can be implemented on topic classification problems. For that reason, we then train popular classifiers on the document embeddings obtained from each model.

Hadamard Error-Correcting Codes and Their Application in Digital Watermarking.

In communication technologies such as digital watermarking, wireless sensor networks (WSNs), and visual light communication (VLC), error-correcting codes are crucial. The Enhanced Hadamard Error-Correcting Code (EHC), which is based on 2D Hadamard Basis Images, is a novel error correction technique that is presented in this study. This technique is used to evaluate the effectiveness of the video watermarking scheme. Even with highly sophisticated embedding techniques, watermarks usually fail to resist such comprehensive attacks because of the extraordinarily high compression rate of approximately 1:200 that is frequently employed in video dissemination. It can only be used in conjunction with a sufficient error-correcting coding method. This study compares the efficacy of the well-known Reed-Solomon Code with this novel technique, the Enhanced Hadamard Error-Correcting Code (EHC), in maintaining watermarks in embedded videos. The main idea behind this newly created multidimensional Enhanced Hadamard Error-Correcting Code is to use a 1D Hadamard decoding approach on the 2D base pictures after they have been transformed into a collection of one-dimensional rows. Following that, the image is rebuilt, allowing for a more effective 2D decoding procedure. Using this technique, it is possible to exceed the theoretical error-correcting capacity threshold of ⌊dmin-12⌋ bits, where dmin is the Hamming distance. It may be possible to achieve better results by converting the 2D EHC into a 3D format. The new Enhanced Hadamard Code is used in a video watermarking coding scheme to show its viability and efficacy. The original video is broken down using a multi-level interframe wavelet transform during the video watermarking embedding process. Low-pass filtering is applied to the video stream in order to extract a certain frequency range. The watermark is subsequently incorporated using this filtered section. Either the Reed-Solomon Correcting Code or the Enhanced Hadamard Code is used to protect the watermarks. The experimental results show that EHC far outperforms the RS Code and is very resilient against severe MPEG compression.

Ultrastructural features of primary cancer and its metastases

Mortality from metastatic lesions accounts for a significant proportion of oncological practice, and the issues of tumor chemoresistance and micrometastasis formation with the development of early recurrence are also important, which requires research and solution of these problems. Methods: The study used sectional material from the Pathology Department for ultrastructural examination of primary tumors and metastases, employing fixation, embedding, sectioning, staining, and electron microscopy imaging techniques.

High-performance computing in healthcare:an automatic literature analysis perspective

The adoption of high-performance computing (HPC) in healthcare has gained significant attention in recent years, driving advancements in medical research and clinical practice. Exploring the literature on HPC implementation in healthcare is valuable for decision-makers as it provides insights into potential areas for further investigation and investment. However, manually analyzing the vast number of scholarly articles is a challenging and time-consuming task. Fortunately, topic modeling techniques offer the capacity to process extensive volumes of scientific literature, identifying key trends within the field. This paper presents an automatic literature analysis framework based on a state-of-art vector-based topic modeling algorithm with multiple embedding techniques, unveiling the research trends surrounding HPC utilization in healthcare. The proposed pipeline consists of four phases: paper extraction, data preprocessing, topic modeling and outlier detection, followed by visualization. It enables the automatic extraction of meaningful topics, exploration of their interrelationships, and identification of emerging research directions in an intuitive manner. The findings highlight the transition of HPC adoption in healthcare from traditional numerical simulation and surgical visualization to emerging topics such as drug discovery, AI-driven medical image analysis, and genomic analysis, as well as correlations and interdisciplinary connections among application domains.

Deep learning path-like collective variable for enhanced sampling molecular dynamics.

Several enhanced sampling techniques rely on the definition of collective variables to effectively explore free energy landscapes. The existing variables that describe the progression along a reactive pathway offer an elegant solution but face a number of limitations. In this paper, we address these challenges by introducing a new path-like collective variable called the "deep-locally non-linear-embedding," which is inspired by principles of the locally linear embedding technique and is trained on a reactive trajectory. The variable mimics the ideal reaction coordinate by automatically generating a non-linear combination of features through a differentiable generalized autoencoder that combines a neural network with a continuous k-nearest neighbor selection. Among the key advantages of this method is its capability to automatically choose the metric for searching neighbors and to learn the path from state A to state B without the need to handpick landmarks a priori. We demonstrate the effectiveness of DeepLNE by showing that the progression along the path variable closely approximates the ideal reaction coordinate in toy models, such as the Müller-Brown potential and alanine dipeptide. Then, we use it in the molecular dynamics simulations of an RNA tetraloop, where we highlight its capability to accelerate transitions and estimate the free energy of folding.

Semantic matching in GUI test reuse

Reusing test cases across apps that share similar functionalities reduces both the effort required to produce useful test cases and the time to offer reliable apps to the market. The main approaches to reuse test cases across apps combine different semantic matching and test generation algorithms to migrate test cases across Android apps. In this paper we define a general framework to evaluate the impact and effectiveness of different choices of semantic matching with Test Reuse approaches on migrating test cases across Android apps. We offer a thorough comparative evaluation of the many possible choices for the components of test migration processes. We propose an approach that combines the most effective choices for each component of the test migration process to obtain an effective approach. We report the results of an experimental evaluation on 8,099 GUI events from 337 test configurations. The results attest the prominent impact of semantic matching on test reuse. They indicate that sentence level perform better than word level embedding techniques. They surprisingly suggest a negligible impact of the corpus of documents used for building the word embedding model for the Semantic Matching Algorithm. They provide evidence that semantic matching of events of selected types perform better than semantic matching of events of all types. They show that the effectiveness of overall Test Reuse approach depends on the characteristics of the test suites and apps. The replication package that we make publicly available online (https://star.inf.usi.ch/#/software-data/11) allows researchers and practitioners to refine the results with additional experiments and evaluate other choices for test reuse components.

ADVANCMENTS IN TEXT SUMMARIZATION AND EXTRACTIVE QUESTION- ANSWERING : A MACHINE LEARNING APPROACH

In the era of social media platforms, the rapid expansion of data mining in the fields of information retrieval and natural language processing emphasizes the crucial need for automated text summarization. At the current time, pretrained word embedding techniques and sequence to sequence models can be effectively repurposed in the realm of social network summarization to efficiently condense significant information with strong encoding capabilities. However, dealing with the challenge of extended text dependency and efficiently utilizing latent topic mapping presents an increasingly significant obstacle for these computational models. In this document, we propose a topic- focused approach for both extractive and abstractive summarization, integrating Question Answering features by employing BERT and Pegasus pretrained models. A comprehensive analysis of the architectural complexities of the pre-existing models utilized in the creation of our model has been presented. The evaluation of our Text Summarization &amp; Question Answering Model was conducted meticulously on a variety of datasets including Multi news, XSum, and CNN/Daily Mail, with experimental results indicating its achievement of state-of-the-art performance levels based on ROGUE scores. Following this, the verification of our results through human evaluation confirms that the summaries generated by our model are in alignment with the performance standards set by humans across various datasets. Furthermore, we have integrated an Extractive Question Answering task into our model, proposing an architecture leveraging BERT and conducting comparative analyses against alternative language models to gauge its efficacy. Keywords—Natural Language Processing (NLP), Text Summarization, Question &amp; Answering (Q&amp;A), Transformer Models, Pegasus, BERT (Bidirectional Encoder Representations from Transformers), Neural Topic Modeling, Extractive Summarization, Abstractive Summarization, Semantic Representation, Topic Embedding

Prediction of coronary artery bypass graft outcomes using a single surgical note: An artificial intelligence-based prediction model study.

Healthcare providers currently calculate risk of the composite outcome of morbidity or mortality associated with a coronary artery bypass grafting (CABG) surgery through manual input of variables into a logistic regression-based risk calculator. This study indicates that automated artificial intelligence (AI)-based techniques can instead calculate risk. Specifically, we present novel numerical embedding techniques that enable NLP (natural language processing) models to achieve higher performance than the risk calculator using a single preoperative surgical note. The most recent preoperative surgical consult notes of 1,738 patients who received an isolated CABG from July 1, 2014 to November 1, 2022 at a single institution were analyzed. The primary outcome was the Society of Thoracic Surgeons defined composite outcome of morbidity or mortality (MM). We tested three numerical-embedding techniques on the widely used TextCNN classification model: 1a) Basic embedding, treat numbers as word tokens; 1b) Basic embedding with a dataloader that Replaces out-of-context (ROOC) numbers with a tag, where context is defined as within a number of tokens of specified keywords; 2) ScaleNum, an embedding technique that scales in-context numbers via a learned sigmoid-linear-log function; and 3) AttnToNum, a ScaleNum-derivative that updates the ScaleNum embeddings via multi-headed attention applied to local context. Predictive performance was measured via area under the receiver operating characteristic curve (AUC) on holdout sets from 10 random-split experiments. For eXplainable-AI (X-AI), we calculate SHapley Additive exPlanation (SHAP) values at an ngram resolution (SHAP-N). While the analyses focus on TextCNN, we execute an analogous performance pipeline with a long short-term memory (LSTM) model to test if the numerical embedding advantage is robust to model architecture. A total of 567 (32.6%) patients had MM following CABG. The embedding performances are as follows with the TextCNN architecture: 1a) Basic, mean AUC 0.788 [95% CI (confidence interval): 0.768-0.809]; 1b) ROOC, 0.801 [CI: 0.788-0.815]; 2) ScaleNum, 0.808 [CI: 0.785-0.821]; and 3) AttnToNum, 0.821 [CI: 0.806-0.834]. The LSTM architecture produced a similar trend. Permutation tests indicate that AttnToNum outperforms the other embedding techniques, though not statistically significant verse ScaleNum (p-value of .07). SHAP-N analyses indicate that the model learns to associate low blood urine nitrate (BUN) and creatinine values with survival. A correlation analysis of the attention-updated numerical embeddings indicates that AttnToNum learns to incorporate both number magnitude and local context to derive semantic similarities. This research presents both quantitative and clinical novel contributions. Quantitatively, we contribute two new embedding techniques: AttnToNum and ScaleNum. Both can embed strictly positive and bounded numerical values, and both surpass basic embeddings in predictive performance. The results suggest AttnToNum outperforms ScaleNum. With regards to clinical research, we show that AI methods can predict outcomes after CABG using a single preoperative note at a performance that matches or surpasses the current risk calculator. These findings reveal the potential role of NLP in automated registry reporting and quality improvement.

Qualitative and quantitative detection of microplastics in soil based on LIF technology combined with OOA-ELM/SPA-PLS

Traditional methods for detecting soil microplastics include chemical digestion, density separation, staining, etc. These methods often only provide qualitative analysis of microplastics and are difficult to accurately quantify. To address this issue, laser induced fluorescence (LIF) technology is introduced, which can achieve high sensitivity, non-destructive, and rapid detection, and improve the accuracy and efficiency of qualitative and quantitative analysis of soil microplastics. Therefore, based on LIF technology, a qualitative prediction model for optimizing Extreme Learning Machine (ELM) using Osprey Optimization Algorithm (OOA) and a quantitative prediction model combining Continuous Projection Algorithm (SPA) with Partial Least Squares Algorithm (PLS) were established. Qualitative analysis used eight types of microplastics, with AS, PA66, ABS, PS, POM, PBT, PET, and PVC as experimental samples. Initially, spectral data is obtained using LIF technology and the raw spectral data is pre-processed using polynomial smoothing algorithm (SG) and moving average (MA) methods. Subsequently, factor analysis (FA) and local linear embedding (LLE) techniques are applied to dimensionality reduction and then input into the ELM model. At the same time, the ELM parameter is optimized by using OOA algorithm. In the quantitative analysis, PA66 powder and soil are mixed in different proportions (10:0, 8:2, 5:5, 2:8, 0:10), SG and MA pre-processing are performed on the spectral data obtained by LIF technology, and SPA is used to select the spectral characteristic wavelength for the original data and the pre-processed spectral data, and then a linear regression model is established combined with PLS. In order to facilitate the comparison of data, the experimental results are reserved to 5 decimal places, the experimental results indicate that LIF technology is faster than traditional methods in extracting raw data. The convergence of the OOA-optimized ELM model is better than that of the PSO-optimized ELM model. Among all the models of qualitative analysis, SG-LLE-OOA-ELM had the best prediction effect, with mean square error (MSE) of 0.00834, coefficient of determination (R2) of 0.99841, root mean square error (RMSE) of 0.09134, and mean absolute error (MAE) of 0.07038. Among the models of quantitative analysis, the best effect is SG-SPA-PLS model with R2 of 0.99951.