Sentence Encoder Research Articles

Improvement for deep learning for suicide and depression identification with unsupervised label correction

Abstract. Our study try to address the challenge of accurately identifying depression and suicidal ideation on social media platforms by introducing an enhanced novel methodology for unsupervised feature selection and label correction. Utilizing advanced word embedding models like BERT and the Universal Sentence Encoder, we transform textual content into dense numerical vectors that capture the nuanced emotional context of online discussions. Our approach enhances these embeddings with a deep neural network (DNN) to extract distinctive features, reducing the dimensionality of the data through Principal Component Analysis (PCA). For label correction, we employ clustering techniques including OPTICS, K-medoids, and hierarchical clustering, which are robust against noisy data points. We then train classifiers using CNN, DNN, logistic regression, and random forest algorithms, evaluated with metrics such as accuracy, precision, recall, F1 score, and AUC. This methodology improves the accuracy of classifying depressive and suicidal sentiments to some extent, assisting to utilize the vast data available on social media to advance mental health diagnostics and interventions.

A Chinese Nested Named Entity Recognition Model for Chicken Disease Based on Multiple Fine-Grained Feature Fusion and Efficient Global Pointer

Extracting entities from large volumes of chicken epidemic texts is crucial for knowledge sharing, integration, and application. However, named entity recognition (NER) encounters significant challenges in this domain, particularly due to the prevalence of nested entities and domain-specific named entities, coupled with a scarcity of labeled data. To address these challenges, we compiled a corpus from 50 books on chicken diseases, covering 28 different disease types. Utilizing this corpus, we constructed the CDNER dataset and developed a nested NER model, MFGFF-BiLSTM-EGP. This model integrates the multiple fine-grained feature fusion (MFGFF) module with a BiLSTM neural network and employs an efficient global pointer (EGP) to predict the entity location encoding. In the MFGFF module, we designed three encoders: the character encoder, word encoder, and sentence encoder. This design effectively captured fine-grained features and improved the recognition accuracy of nested entities. Experimental results showed that the model performed robustly, with F1 scores of 91.98%, 73.32%, and 82.54% on the CDNER, CMeEE V2, and CLUENER datasets, respectively, outperforming other commonly used NER models. Specifically, on the CDNER dataset, the model achieved an F1 score of 79.68% for nested entity recognition. This research not only advances the development of a knowledge graph and intelligent question-answering system for chicken diseases, but also provides a viable solution for extracting disease information that can be applied to other livestock species.

Towards a cyberbullying detection approach: fine-tuned contrastive self-supervised learning for data augmentation

AbstractCyberbullying on social media platforms is pervasive and challenging to detect due to linguistic subtleties and the need for extensive data annotation. We introduce a Deep Contrastive Self-Supervised Learning (DCSSL) model that integrates a Natural Language Inference (NLI) dataset, a fine-tuned sentence encoder, and data augmentation to enhance the understanding of cyberbullying's nuanced semantics and offensiveness. The DCSSL model effectively captures contextual dependencies and the varied semantic implications inherent in cyberbullying instances, addressing the limitations of manual data annotation processes when compared against established models such as BERT and Bi-LSTM. Our proposed model registers a significant improvement, achieving a macro average F1 score of 0.9231 on cyberbullying datasets, highlighting its applicability in environments where manual annotation is impractical or unavailable.

Extracting Key-phrase Embedding using Deep Average Network and Maximal Marginal Relevance to Enhance Information Retrieval

Background: Automatic keyphrase extraction (AKE) is essential to many NLP and information retrieval tasks. Extracting high-quality key phrases is difficult due to technological advancements and the exponential growth ‎of ‎textual data and digital sources. Unsupervised keyphrase extraction with cheap computing cost that relies ‎on ‎heuristic notions of phrase importance such as embedding similarities but their development necessitates in-depth subject expertise. Materials and Methods: This paper presents a method to obtain a semantic understanding of the query ‎and index documents by using ‎the embedding technique(universal Sentence ‎encoder (USE) ) while keeping the most informative ‎using Maximal Marginal Relevance (MMR) and then scoring(an ‎inverted index) the most documents relevant to the query ‎vector to improve the ‎performance of IR systems.‎ Results: The proposed retrieval model implement on the (Fire2011) dataset. The final ‎stage was evaluating the results of the baseline and the results (indexing and ranking) by using mean average precision (MAP). The ‎result of the baseline was 0.61, while the result ‎inverted index was 0.6277519 .‎ Conclusions: In this paper, we have discussed document retrieval using keep key phrases that ‎have ‏informativeness ‎properties by ‎using maximal ‎marginal ‎relevance, since if we extract a fixed number of top ‎keyphrases, ‎redundancy ‎hinders the diversification of the ‎extracted keyphrases.

Automatic metaphor processing in developmental dyslexia

IntroductionPrevious research found metaphor impairments with dyslexia; however, it is unclear if difficulties are due to initial activation of the metaphorical meaning or to subsequent discourse integration processes. The study examines the presence of early automatic processing of metaphors in adults with developmental dyslexia, considering the role of executive functions and metaphor familiarity. MethodsUsing a sentence recall task and a semantic judgment task from the Metaphor Interference Effect (MIE) paradigm, we evaluated two early stages of metaphor comprehension, namely the generation of the figurative meaning and the suppression of the literal meaning. High and low familiar metaphors, and their scrambled counterparts, were aurally presented to participants, who were asked to judge whether sentences were literally true or literally false. Afterwards, they were provided ten minutes to recall the sentences they heard to verify the depth of processing for each type of stimulus. A total of 26 participants with dyslexia were included in the experimental group, and 31 in the control group. ResultsIndividuals with dyslexia showed a MIE and an accuracy rate that are similar to participants without dyslexia. Inhibition correlated with the MIE size only for high familiar metaphors, and working memory seemed to play no role in the process. In the recall task, both groups demonstrated a better encoding of the metaphorical sentences compared to scrambled metaphors, but participants with dyslexia recalled less metaphors than did the control group, showing that metaphors are no exception to the limitations in sentence retrieval typically found in dyslexia. ConclusionsOur findings suggest that individuals with dyslexia are comparable to participants without dyslexia in their ability to automatically compute metaphorical meanings. Thus, difficulties in metaphor comprehension in people with dyslexia that have been detected in previous studies might depend on meaning construction in context rather than online semantic processing.

Swift Detection of XSS Attacks: Enhancing XSS Attack Detection by Leveraging Hybrid Semantic Embeddings and AI Techniques

AbstractCross-Site Scripting (XSS) attacks continue to be a significant threat to web application security, necessitating robust detection mechanisms to safeguard user data and ensure system integrity. In this study, we present a novel approach for detecting XSS attacks that harnesses the combined capabilities of the Universal Sentence Encoder (USE) and Word2Vec embeddings as a feature extractor, aiming to enhance the performance of machine learning and deep learning techniques. By leveraging the semantic understanding of sentences offered by USE and the word-level representations from Word2Vec, we obtain a comprehensive feature representation for XSS attack payloads. Our proposed approach aims to capture both fine-grained word meanings and broader sentence contexts, leading to enhanced feature extraction and improved model performance. We conducted extensive experiments utilizing machine learning and deep learning architectures to evaluate the effectiveness of our approach. The obtained results demonstrate that our combined embeddings approach outperforms traditional methods, achieving superior accuracy, precision, recall, ROC, and F1-score in detecting XSS attacks. This study not only advances XSS attack detection but also highlights the potential of state-of-the-art natural language processing techniques in web security applications. Our findings offer valuable insights for the development of more robust and effective security measures against XSS attacks.

Bias Detection in Media: An NLP-Based Approach using Corpus Statistics and Sentence Embeddings

In this paper, we implement a Natural Language Processing (NLP) solution for binary classification to categorize a sentence as biased or unbiased. Detecting bias is a challenge in the media today, but can be utilized to help readers identify which sources portray bias. The general approach to classifying a sentence as biased or unbiased involves representing words and sentences using probability or pretrained vectorization models. Our final model only contained probabilistic data about the connection between words, sentences, and each class. We used Pointwise Mutual Information (PMI) and Term Frequency Inverse Document Frequency (TF-IDF) as heuristics for finding the relationship between sentences and the biased and unbiased classes. We also leveraged Google’s Universal Sentence Encodings (USE) to capture the meaning of the sentences. Our results revealed a possible limitation in USE’s training data in terms of bias detection. Through topic analysis, we were able to uncover insights surrounding which topics are characterized by minimal bias. We were able to use these discoveries to contextualize the model’s performance.

Semantically Enriched Cross-Lingual Sentence Embeddings for Crisis-related Social Media Texts

Tasks such as semantic search and clustering on crisis-related social media texts enhance our comprehension of crisis discourse, aiding decision-making and targeted interventions. Pre-trained language models have advanced performance in crisis informatics, but their contextual embeddings lack semantic meaningfulness. Although the CrisisTransformers family includes a sentence encoder to address the semanticity issue, it remains monolingual, processing only English texts. Furthermore, employing separate models for different languages leads to embeddings in distinct vector spaces, introducing challenges when comparing semantic similarities between multi-lingual texts. Therefore, we propose multi-lingual sentence encoders (CT-XLMR-SE and CT-mBERT-SE) that embed crisis-related social media texts for over 50 languages, such that texts with similar meanings are in close proximity within the same vector space, irrespective of language diversity. Results in sentence encoding and sentence matching tasks are promising, suggesting these models could serve as robust baselines when embedding multi-lingual crisis-related social media texts. The models are publicly available at: https://huggingface.co/crisistransformers.

CrisisTransformers: Pre-trained language models and sentence encoders for crisis-related social media texts

Social media platforms play an essential role in crisis communication, but analysing crisis-related social media texts is challenging due to their informal nature. Transformer-based pre-trained models like BERT and RoBERTa have shown success in various NLP tasks, but they are not tailored for crisis-related texts. Furthermore, general-purpose sentence encoders are used to generate sentence embeddings, regardless of the textual complexities in crisis-related texts. Advances in applications like text classification, semantic search, and clustering contribute to the effective processing of crisis-related texts, which is essential for emergency responders to gain a comprehensive view of a crisis event, whether historical or real-time. To address these gaps in crisis informatics literature, this study introduces CrisisTransformers, an ensemble of pre-trained language models and sentence encoders trained on an extensive corpus of over 15 billion word tokens from tweets associated with more than 30 crisis events, including disease outbreaks, natural disasters, conflicts, and other critical incidents. We evaluate existing models and CrisisTransformers on 18 crisis-specific public datasets. Our pre-trained models outperform strong baselines across all datasets in classification tasks, and our best-performing sentence encoder improves the state-of-the-art by 17.43% in sentence encoding tasks. Additionally, we investigate the impact of model initialization on convergence and evaluate the significance of domain-specific models in generating semantically meaningful sentence embeddings. The models are publicly available at: https://huggingface.co/crisistransformers.

A Sentence-Embedding-Based Dashboard to Support Teacher Analysis of Learner Concept Maps

Concept mapping is a valuable method to represent a domain of knowledge, also with the aim of supporting educational needs. Students are called upon to construct their own knowledge through a meaningful learning process, linking new concepts to concepts they have already learned, i.e., connecting new knowledge to knowledge they already possess. Moreover, the particular graphic form of a concept map makes it easy for the teacher to construct and interpret both. Consequently, for an educator, the ability to assess concept maps offered by students, facilitated by an automated system, can prove invaluable. This becomes even more apparent in educational settings where there is a large number of students, such as in Massive Open Online Courses. Here, we propose two new measures devised to evaluate the similarity between concept maps based on two deep-learning embedding models: InferSent and Universal Sentence Encoder. An experimental evaluation with a sample of teachers confirms the validity of one such deep-learning model as the baseline of the new similarity measure. Subsequently, we present a proof-of-concept dashboard where the measures are used to encode a concept map in a 2D space point, with the aim of helping teachers monitor students’ concept-mapping activity.

Code Comments: A Way of Identifying Similarities in the Source Code

This study investigates whether analyzing the code comments available in the source code can effectively reveal functional similarities within software. The authors explore how both machine-readable comments (such as linter instructions) and human-readable comments (in natural language) can contribute towards measuring the code similarity. For the former, the work is relying on computing the cosine similarity over the one-hot encoded representation of the machine-readable comments, while for the latter, the focus is on detecting similarities in English comments, using threshold-based computations against the similarity measurements obtained using models based on Levenshtein distances (for form-based matches), Word2Vec (for contextual word representations), as well as deep learning models, such as Sentence Transformers or Universal Sentence Encoder (for semantic similarity). For evaluation, this research has analyzed the similarities between different source code versions of the open-source code editor, VSCode, based on existing ESlint-specific directives, as well as applying natural language processing techniques on incremental releases of Kubernetes, an open-source system for automating containerized application management. The experiments outlines the potential for detecting code similarities solely based on comments, and observations indicate that models like Universal Sentence Encoder are providing a favorable balance between recall and precision. This research is integrated into Project Martial, an open-source project for automatic assistance in detecting plagiarism in software.

Revisiting clustering for efficient unsupervised dialogue structure induction

In the development of a task-oriented dialogue system, defining the dialogue structure is a time-consuming task. Hence, several works have looked into automatically inferring it from data, e.g., actual conversations between a customer and a support agent. To recover such dialogue structure, recent methods based on discrete variational models learn to jointly encode and cluster utterances in dialogue states, but (i) represent utterances by only considering preceding dialogue context, and (ii) are slow to train since they are optimized with a compute-expensive decoding objective. We revisit and improve upon an existing efficient pipeline approach, commonly adopted as a baseline, that first encodes utterances and then clusters them with k-means to induce the dialogue structure. However, the existing approach represents utterances as bag-of-words or skip-thought vectors, which have been shown to perform poorly in semantic similarity tasks, and without considering dialogue context. We therefore first investigate the use of more powerful transformer-based encoders for encoding utterances. Next, we propose ellodar, a method for learning representations that capture both preceding and subsequent dialogue context, inspired by word-to-vec training strategies. ellodar is efficient since representations are learned directly in the encoding space by finetuning just a single linear layer on top of a frozen sentence encoder with a vector-to-vector regression training objective. Extensive experiments on representative datasets for dialogue structure induction (SimDial, Schema Guided Dialogues, DSTC2, and CamRest676) demonstrate that in terms of effectiveness to induce the correct dialogue structure, (i) clustering utterances represented by transformed-based encoders improves recent joint models by 13%–32% on standard cluster metrics, and (ii) clustering ellodar’s representations yields additional improvements ranging from +20% to +26%, with speedups of ×\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\ imes $$\\end{document} 10\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\ extbf{10}$$\\end{document}–104\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\ extbf{10}^{\ extbf{4}}$$\\end{document} compared to the recent joint models.

Unveiling Business Activity Patterns of Digital Transformation through K-Means Clustering with Universal Sentence Encoder in Transport and Logistics Sectors

Digital transformation brings several key benefits to transportation and logistics firms, such as operational efficiency, improved customer experience, and supply chain visibility. This study seeks to identify and categorize business activities within the transportation and logistics sector, focusing on themes related to digital transformation. Additionally, it analyses the impact of firm size and sales performance. Data were sourced from 24 annual reports of four firms listed in the SET, covering the study period of 2017-2022. The study used context-free grammar-based verb patterns and a custom-made dictionary to extract business activities. The study used the Universal Sentence Encoder to represent the text and relied on the Dunn-Index and Silhouette score to determine the optimal number of clusters (both 8 clusters). Subsequently, K-means clustering was performed using these 8 clusters, leading to the categorization of eight digital transformation themes. Among these themes, profit management and regulation compliance are interesting discoveries, as they are rarely mentioned in previous studies. A total of 104 business activities were extracted as a result. The study also found that Online-to-Offline activity is an effective method for business model innovation, as well as a contemporary marketing tool.

Unsupervised Extractive Summarization with Learnable Length Control Strategies

Unsupervised extractive summarization is an important technique in information extraction and retrieval. Compared with supervised method, it does not require high-quality human-labelled summaries for training and thus can be easily applied for documents with different types, domains or languages. Most of existing unsupervised methods including TextRank and PACSUM rely on graph-based ranking on sentence centrality. However, this scorer can not be directly applied in end-to-end training, and the positional-related prior assumption is often needed for achieving good summaries. In addition, less attention is paid to length-controllable extractor, where users can decide to summarize texts under particular length constraint. This paper introduces an unsupervised extractive summarization model based on a siamese network, for which we develop a trainable bidirectional prediction objective between the selected summary and the original document. Different from the centrality-based ranking methods, our extractive scorer can be trained in an end-to-end manner, with no other requirement of positional assumption. In addition, we introduce a differentiable length control module by approximating 0-1 knapsack solver for end-to-end length-controllable extracting. Experiments show that our unsupervised method largely outperforms the centrality-based baseline using a same sentence encoder. In terms of length control ability, via our trainable knapsack module, the performance consistently outperforms the strong baseline without utilizing end-to-end training. Human evaluation further evidences that our method performs the best among baselines in terms of relevance and consistency.

Gated Recursive and Sequential Deep Hierarchical Encoding for Detecting Incongruent News Articles

With the increase in misinformation across digital platforms, incongruent news detection is becoming an important research problem. Earlier, researchers have exploited various feature engineering approaches and deep learning models with embedding to capture incongruity between news headlines and their respective bodies. Studies have broadly considered different combinations of bag-of-words-based features, sequential encoding, hierarchical encoding, headline-guided attention-based encoding, and so on, of the text in headlines and bodies. In this article, we focus on addressing two important limitations observed with hierarchical encoding and headline-guided attention-based encoding methods. The existing hierarchical encoding-based studies limit the hierarchical structure of the body of a news article to paragraph level only, undermining the importance of incorporating long-term dependence from word level to sentence, paragraph, and body. Furthermore, the existing headline-guided attention-based encoding focuses on contextually similar contents in the body of the headline, undermining the importance of incorporating contextually dissimilar contents. Motivated by the above observations, this article proposes a gated recursive and sequential deep hierarchical encoding (GraSHE) method for detecting incongruent news articles by extending the hierarchical structure of the news body from the body to the word level and incorporating incongruity weight. From various experimental setups over three publicly available benchmarks datasets, the experimental results indicate that the proposed model outperforms baseline models with bag-of-word-based features, sequential, hierarchical, and headline-guided attention-based encoding methods. To further validate the performance of the proposed model, we conduct several ablation studies. The following key observations can be made from the ablation study: 1) models with hierarchical encoding outperform models with nonhierarchical encoding; 2) recursive encoding of sentences boosts the performance of models as compared with sequential encoding of sentences within paragraphs; and 3) incongruent news article detection is domain-dependent. Incorporating explicit features further boosts the performance of proposed model and also decreases the domain dependence of models.

Novelty Evaluation using Sentence Embedding Models in Open-ended Cocreative Problem-solving

AbstractCollaborative creativity (cocreativity) is essential to generate original solutions for complex challenges faced in organisations. Effective cocreativity requires the orchestration of cognitive and social processes at a high level. Artificial Intelligence (AI) techniques, specifically deep learning sentence embedding models, have emerged as valuable tools for evaluating creativity and providing feedback to improve the cocreation process. This paper examines the implications of sentence embedding models for evaluating the novelty of open-ended ideas generated within the context of real-life project-based learning. We report a case study research design involving twenty-five secondary students, where a cocreative process was developed to solve a complex, open-ended problem. The novelty of the co-generated ideas was evaluated using eight pre-trained sentence embedding models and compared with experts’ evaluations. Correlation and regression analyses were performed to examine the reliability of the sentence embedding models in comparison to the experts’ scoring. Our findings disclose that sentence embedding models can solve the challenge of evaluating open-ended ideas generated during the cocreative process. Moreover, the results show that two-sentence embedding models significantly correlate better with experts- Universal Sentence Encoder Transformer (USE-T) and USE Deep Averaging Network (USE-DAN). These findings have a high pedagogical value as they successfully evaluate the novelty generated in a real problem-based environment that uses technology to promote key cocreative processes. Furthermore, the real-time evaluation facilitated by these models can have a strong pedagogical impact because it can provide valuable feedback to teachers and students, thereby optimising collaborative ideation processes and promoting effective cocreative teaching and learning methodologies.

Multi-aspect Understanding with Cooperative Graph Attention Networks for Medical Dialogue Information Extraction

Medical dialogue information extraction is an important but challenging task for Electronic Medical Records. Existing medical information extraction methods ignore the crucial information of sentence and multi-level dependency in dialogue, which limits their effectiveness for capturing essential medical information. To address these issues, we present a novel Multi-aspect Understanding with Cooperative Graph Attention Networks for Medical Dialogue Information Extraction to capture multi-aspect sentence information and multi-level dependency information from the dialogue. First, we propose the multi-aspect sentence encoder to capture various features from different perspectives. Second, we propose double graph attention networks to model the dependency features from intra-window and inter-window, respectively. Extensive experiments on a benchmark dataset have well-validated the effectiveness of the proposed method.

Generating Synthetic Resume Data with Large Language Models for Enhanced Job Description Classification

In this article, we investigate the potential of synthetic resumes as a means for the rapid generation of training data and their effectiveness in data augmentation, especially in categories marked by sparse samples. The widespread implementation of machine learning algorithms in natural language processing (NLP) has notably streamlined the resume classification process, delivering time and cost efficiencies for hiring organizations. However, the performance of these algorithms depends on the abundance of training data. While selecting the right model architecture is essential, it is also crucial to ensure the availability of a robust, well-curated dataset. For many categories in the job market, data sparsity remains a challenge. To deal with this challenge, we employed the OpenAI API to generate both structured and unstructured resumes tailored to specific criteria. These synthetically generated resumes were cleaned, preprocessed and then utilized to train two distinct models: a transformer model (BERT) and a feedforward neural network (FFNN) that incorporated Universal Sentence Encoder 4 (USE4) embeddings. While both models were evaluated on the multiclass classification task of resumes, when trained on an augmented dataset containing 60 percent real data (from Indeed website) and 40 percent synthetic data from ChatGPT, the transformer model presented exceptional accuracy. The FFNN, albeit predictably, achieved lower accuracy. These findings highlight the value of augmented real-world data with ChatGPT-generated synthetic resumes, especially in the context of limited training data. The suitability of the BERT model for such classification tasks further reinforces this narrative.

Comparing neural sentence encoders for topic segmentation across domains: not your typical text similarity task

Neural sentence encoders (NSE) are effective in many NLP tasks, including topic segmentation. However, no systematic comparison of their performance in topic segmentation has been performed. Here, we present such a comparison, using supervised and unsupervised segmentation models based on NSEs. We first compare results with baselines, showing that the use of NSEs does often provide improvements, except for specific domains such as news shows. We then compare over three different datasets a range of existing NSEs and a new NSE based on ad hoc pre-training strategy. We show that existing literature documenting general performance gains of NSEs does not always conform to the results obtained by the same NSEs in topic segmentation. If Transformers-based encoders do improve over previous approaches, fine-tuning in sentence similarity tasks or even on the same topic segmentation task we aim to solve does not always equate to better performance, as results vary across method being used and domains of application. We aim to explain this phenomenon and the relative poor performance of NSEs in news shows by considering how well different NSEs encode the underlying lexical cohesion of same-topic segments; to do so, we introduce a new metric, ARP. The results from this study suggest that good topic segmentation results do not always rely on good cohesion modelling on behalf of the segmenter and that is dependent upon what kind of text we are trying to segment. Also, it appears evident that traditional sentence encoders fail to create topically cohesive clusters of segments when used on conversational data. Overall, this work advances our understanding of the use of NSEs in topic segmentation and of the general factors determining the success (or failure) of a topic segmentation system. The new proposed metric can quantify the lexical cohesion of a multi-topic document under different sentence encoders and, as such, might have many different uses in future research, some of which we suggest in our conclusions.

A Systematic and Comparative Analysis of Semantic Search Algorithms

Users often struggle to discover the information they need online because of the massive volume of data that is readily available as well as being generated every day in the today’s digital age. Traditional keyword-based search engines may not be able to handle complex queries, which could result in irrelevant or insufficient search results. This issue can be solved by semantic search, which utilises machine learning and natural language processing to interpret the meaning and context of a user's query. In this paper we focus on analyzing the BM-25 algorithm, Mean of Word Vectors approach, Universal Sentence Encoder model, and Sentence-BERT model on the CISI Dataset for Semantic Search Task. The results indicate that, the Finetuned SBERT model performs the best.