Sentence Clustering Research Articles

Graphs in clusters: a hybrid approach to unsupervised extractive long document summarization using language models

Effective summarization of long documents is a challenging task. When addressing this challenge, Graph and Cluster-Based methods stand out as effective unsupervised solutions. Graph-Based Unsupervised methods are widely employed for summarization due to their success in identifying relationships within documents. Cluster-Based methods excel in minimizing redundancy by grouping similar content together before generating a concise summary. Therefore, this paper merges Cluster-Based and Graph-Based methods by applying language models for Unsupervised Extractive Summarization of long documents. The approach simultaneously extracts key information while minimizing redundancy. First, we use BERT-based sentence embeddings to create sentence clusters using k-means clustering and select the optimum number of clusters using the elbow method to ensure that sentences are categorized based on their semantic similarities. Then, the TextRank algorithm is employed within each cluster to rank sentences based on their importance and representativeness. Finally, the total similarity score of the graph is used to rank the clusters and eliminate less important sentence groups. Our method achieves comparable or better summary quality and reduced redundancy compared to both individual Cluster-Based and Graph-Based methods, as well as other supervised and Unsupervised baseline models across diverse datasets.

Experimental study on short-text clustering using transformer-based semantic similarity measure

Sentence clustering plays a central role in various text-processing activities and has received extensive attention for measuring semantic similarity between compared sentences. However, relatively little focus has been placed on evaluating clustering performance using available similarity measures that adopt low-dimensional continuous representations. Such representations are crucial in domains like sentence clustering, where traditional word co-occurrence representations often achieve poor results when clustering semantically similar sentences that share no common words. This article presents a new implementation that incorporates a sentence similarity measure based on the notion of embedding representation for evaluating the performance of three types of text clustering methods: partitional clustering, hierarchical clustering, and fuzzy clustering, on standard textual datasets. This measure derives its semantic information from pre-training models designed to simulate human knowledge about words in natural language. The article also compares the performance of the used similarity measure by training it on two state-of-the-art pre-training models to investigate which yields better results. We argue that the superior performance of the selected clustering methods stems from their more effective use of the semantic information offered by this embedding-based similarity measure. Furthermore, we use hierarchical clustering, the best-performing method, for a text summarization task and report the results. The implementation in this article demonstrates that incorporating the sentence embedding measure leads to significantly improved performance in both text clustering and text summarization tasks.

Discourse Coherence in English-Chinese Translation of Literary Texts: A Case Study on Tess of the D'Urbervilles Translated by Zhang Guruo

Discourse coherence refers to the mechanism by which words, sentences, and sentence clusters in a discourse are logically related to form a larger semantic structure, which is one of the most important factors of effective communication. Whether a discourse is coherent determines the extent to which the reader understands the discourse. This paper explores the theory of discourse translation and discourse coherence, compares and analyses the original text of Tess of the D'Urbervilles with Zhang Guruo's translated version in terms of semantic, contextual, logical, and linguistic coherence. Then it describes the strategies and methods used by the translators to ensure discourse coherence in translation, to gain a deeper understanding of the theory of discourse coherence and its application.

Unveiling Water Allocation Dynamics: A Text Analysis of 25 Years of Stakeholder Meetings

Abstract Managing water resources in regions with high climate variability and frequent extreme weather events poses challenges for policymakers. To facilitate water allocation in these cases, participatory and collaborative decision-making approaches have become common. However, the evaluation of these approaches is hindered by the lack of structured methods and data to understand them. To address this knowledge gap, we propose a novel methodology that leverages text data to identify key topics, conflicts, and influential actors that shape water allocation dynamics. Our methodology is tested using records of 1020 water basin committee meetings held between 1997 and 2021 across twelve basin committees in Ceará, Brazil – a region known for its extensive history of droughts that have impacted water governance. To uncover key water management issues discussed during these meetings, we employed a three-step topic modeling framework: (1) sentence embedding, (2) dimensionality reduction, and (3) sentence clustering. Furthermore, we used entity recognition, dependency parsing, and network graphs to identify powerful actors influencing these meetings and, ultimately, the decisions taken. Our findings revealed stakeholders’ heightened concern for urban water supply over agricultural demand during droughts. We found that “reservoir operation” was the most recurring topic, especially in basins where the strategic reservoirs are located. Discussions related to “climate information” became significantly more important over time, which indicates that water allocation decisions are increasingly based on the seasonal forecast and data on oceanic indices provided by the meteorology agency. Despite the presence of local users in the committees, governmental representatives dominated the discussions and were central in all river basins. In conclusion, our proposed approach harnesses existing text data to uncover spatiotemporal patterns related to participatory water allocation. This study opens new avenues for investigating water governance using text-based analysis.

Explainable text-based features in predictive models of crowdfunding campaigns

AbstractReward-Based Crowdfunding offers an opportunity for innovative ventures that would not be supported through traditional financing. A key problem for those seeking funding is understanding which features of a crowdfunding campaign will sway the decisions of a sufficient number of funders. Predictive models of fund-raising campaigns used in combination with Explainable AI methods promise to provide such insights. However, previous work on Explainable AI has largely focused on quantitative structured data. In this study, our aim is to construct explainable models of human decisions based on analysis of natural language text, thus contributing to a fast-growing body of research on the use of Explainable AI for text analytics. We propose a novel method to construct predictions based on text via semantic clustering of sentences, which, compared with traditional methods using individual words and phrases, allows complex meaning contained in the text to be operationalised. Using experimental evaluation, we compare our proposed method to keyword extraction and topic modelling, which have traditionally been used in similar applications. Our results demonstrate that the sentence clustering method produces features with significant predictive power, compared to keyword-based methods and topic models, but which are much easier to interpret for human raters. We furthermore conduct a SHAP analysis of the models incorporating sentence clusters, demonstrating concrete insights into the types of natural language content that influence the outcome of crowdfunding campaigns.

Optimizing Text Summarization with Sentence Clustering and Natural Language Processing

Optimizing Text Summarization with Sentence Clustering and Natural Language Processing

Advancing medical affair capabilities and insight generation through machine learning techniques

BackgroundPharmaceutical companies are increasingly leveraging machine learning techniques to optimize healthcare research, drug development, and medical affairs activities. AI (artificial intelligence) tools such as chatbots, virtual digital assistants, and research tools have been explored to varying degrees of maturity in industries such as consumer goods or software technology. However, there continues to be untapped opportunities within the pharmaceutical industry to employ these technologies for enhanced engagement and education with healthcare professionals (HCPs). Pharmacists, situated at the crossroads of clinical sciences and innovation, have the potential to elevate their role and significance within the pharmaceutical industry by developing and leveraging such technologies.MethodsTo address this, the python-coded tool, Medical Information (MI) Data Uses For AI Semantic Analysis (MUFASA), utilizes state-of-the-art Sentence Transformer library, clustering, and visualization techniques. MUFASA harnesses unsolicited MI data with AI technology, improving efficiency and providing actionable medical affairs intelligence for targeted content delivery to HCPs.ResultsMUFASA optimizes medical affairs activities through its distinctive features: semantic search, cluster analysis, and visualization. Its proficiency in understanding inquiries, as demonstrated through 3D vector mapping and clustering tests, enhances the efficiency of MI and Medical Science Liaison (MSL) case handling. It proves invaluable in training new staff, bolstering response uniformity, and mitigating compliance risks. Leveraging the HDBSCAN algorithm, MUFASA's cluster analysis uncovers deep insights and discerns actionable themes from large inquiry data sets. The visualization graphs, generated from semantic searches, support evidence-based decisions by tracking the effectiveness of initiatives and monitoring trend shifts. Collectively, MUFASA enriches strategic decision-making, cultivates actionable insights, and bolsters healthcare professional engagement.ConclusionThere are numerous opportunities for innovation within the intersection of healthcare and data science. Pharmaceutical manufacturers, with one of their medical affairs responsibilities being the collection of unsolicited inquiries, particularly from HCPs, stand poised to leverage machine learning capabilities to optimize its processes. The abundance of data generated by the growing effort to use it in meaningful ways presents an opportunity for pharmaceutical companies to harness machine learning techniques.

Graph-Based Extractive Text Summarization Sentence Scoring Scheme for Big Data Applications

The recent advancements in big data and natural language processing (NLP) have necessitated proficient text mining (TM) schemes that can interpret and analyze voluminous textual data. Text summarization (TS) acts as an essential pillar within recommendation engines. Despite the prevalent use of abstractive techniques in TS, an anticipated shift towards a graph-based extractive TS (ETS) scheme is becoming apparent. The models, although simpler and less resource-intensive, are key in assessing reviews and feedback on products or services. Nonetheless, current methodologies have not fully resolved concerns surrounding complexity, adaptability, and computational demands. Thus, we propose our scheme, GETS, utilizing a graph-based model to forge connections among words and sentences through statistical procedures. The structure encompasses a post-processing stage that includes graph-based sentence clustering. Employing the Apache Spark framework, the scheme is designed for parallel execution, making it adaptable to real-world applications. For evaluation, we selected 500 documents from the WikiHow and Opinosis datasets, categorized them into five classes, and applied the recall-oriented understudying gisting evaluation (ROUGE) parameters for comparison with measures ROUGE-1, 2, and L. The results include recall scores of 0.3942, 0.0952, and 0.3436 for ROUGE-1, 2, and L, respectively (when using the clustered approach). Through a juxtaposition with existing models such as BERTEXT (with 3-gram, 4-gram) and MATCHSUM, our scheme has demonstrated notable improvements, substantiating its applicability and effectiveness in real-world scenarios.

TEXT SUMMARIZATION IN MONGOLIAN LANGUAGE

Textual information in this new era, it is difficult to manually extract the summary of a large data different areas of social communication accumulates the enormous amounts of data. Therefore, it is important to develop methods for searching and absorbing relevant information, selecting important sentences, paragraphs from large texts, to summarize texts by finding topics of the text and frequency based clustering of sentences. In this paper, the author presents some ideas on using mathematical models in presenting the source text into a shorter version with semantics, graph-based approach for text summarization in Mongolian language.

Application and evaluation of sentence embedding and clustering methods in the context of concept hierarchy construction

Concept hierarchies, as part of knowledge representation methods, play an important role in supporting the exchange and sharing of information. We developed and automated a concept hierarchies construction process which includes several artificial intelligence techniques. When automating their construction from an existing, more or less structured, body of knowledge, the evaluation of the resulting concept hierarchy is an important step.We propose an approach for concept hierarchy construction (CHC) from short sentences, that makes use of methods like Sentence Embedding, Clustering, and Automatic Labeling to create a hierarchical representation consisting of three layers. Our major focus in this paper is not on the algorithms used but on their evaluation using manual clustering by experts and fuzzy sets.

Informing the development of an outcome set and banks of items to measure mobility among individuals with acquired brain injury using natural language processing.

The sheer number of measures evaluating mobility and inconsistencies in terminology make it challenging to extract potential core domains and items. Automating a portion of the data synthesis would allow us to cover a much larger volume of studies and databases in a smaller fraction of the time compared to the usual process. Thus, the objective of this study was to identify a comprehensive outcome set and develop preliminary banks of items of mobility among individuals with acquired brain injury (ABI) using Natural Language Processing (NLP). An umbrella review of 47 reviews evaluating the content of mobility measures among individuals with ABI was conducted. A search was performed on 5 databases between 2000 and 2020. Two independent reviewers retrieved copies of the measures and extracted mobility domains and items. A pre-trained BERT model (state-of-the-art model for NLP) provided vector representations for each sentence. Using the International Classification of Functioning, Disability, and Health Framework (ICF) ontology as a guide for clustering, a k-means algorithm was used to retrieve clusters of similar sentences from their embeddings. The resulting embedding clusters were evaluated using the Silhouette score and fine-tuned according to expert input. The study identified 246 mobility measures, including 474 domains and 2109 items. Encoding the clusters using the ICF ontology and expert knowledge helped in regrouping the items in a way that is more closely related to mobility terminology. Our best results identified banks of items that were used to create a 24 comprehensive outcome sets of mobility, including Upper Extremity Mobility, Emotional Function, Balance, Motor Control, Self-care, Social Life and Relationships, Cognition, Walking, Postural Transition, Recreation, and Leisure Activities, Activities of Daily Living, Physical Functioning, Communication, Work/Study, Climbing, Sensory Functions, General Health, Fatigue, Functional Independence, Pain, Alcohol and Drugs Use, Transportation, Sleeping, and Finances. The banks of items of mobility domains represent a first step toward establishing a comprehensive outcome set and a common language of mobility to develop the ontology. It enables researchers and healthcare professionals to begin exposing the content of mobility measures as a way to assess mobility comprehensively.

A natural language processing approach towards harmonisation of European medicinal product information.

Product information (PI) is a vital part of any medicinal product approved for use within the European Union and consists of a summary of products characteristics (SmPC) for healthcare professionals and package leaflet (PL) for patients, together with the product packaging. In this study, based on the English corpus of the EMA product information documents for all centrally approved medicinal products within the EU, a BERT sentence embedding model was used together with clustering and dimensional reduction techniques to identify sentence similarity clusters that could be candidates for standardization. A total of 1258 medicinal products were included in the study. From these, a total of 783 K sentences were extracted from SmPC and PL documents which were aggregated into a total of 284 and 129 semantic similarity clusters, respectively. The spread distribution among clusters shows separation into different cluster types. Examples of clusters with low spread include those with identical word embeddings due to current standardization, such as section headings and standard phrases. Others show minor linguistic variations, while the group with the largest variability contains variable wording but with significant semantic overlap. The sentence clusters identified could serve as candidates for further standardization of the PI. Moving from free text human wording to auto-generated text elements based on multiple-choice input for appropriate parts of the package leaflet and summary of product characteristics, could reduce both time and complexity for applicants as well as regulators, and ultimately provide patients and prescribers with documents that are easier to understand and better adapted for search availabilities.

Automatic Text Summarization using Document Clustering Named Entity Recognition

Due to the rapid development of internet technology, social media and popular research article databases have generated many open text information. This large amount of textual information leads to 'Big Data'. Textual information can be recorded repeatedly about an event or topic on different websites. Text summarization (TS) is an emerging research field that helps to produce summary from a single or multiple documents. The redundant information in the documents is difficult, hence part or all of the sentences may be omitted without changing the gist of the document. TS can be organized as an exposition to collect accents from its special position, rather than being semantic in nature. Non-ASCII characters and pronunciation, including tokenizing and lemmatization are involved in generating a summary. This research work has proposed an Entity Aware Text Summarization using Document Clustering (EASDC) technique to extract summary from multi-documents. Named Entity Recognition (NER) has a vital part in the proposed work. The topics and key terms are identified using the NER technique. Extracted entities are ranked with Zipf’s law and sentence clusters are formed using k-means clustering. Cosine similarity-based technique is used to eliminate the similar sentences from multi-documents and produce unique summary. The proposed EASDC technique is evaluated using CNN dataset and it shown an improvement of 1.6 percentage when compared with the baseline methods of Textrank and Lexrank.

Unsupervised Graph-Based Tibetan Multi-Document Summarization

Text summarization creates subset that represents the most important or relevant information in the original content, which effectively reduce information redundancy. Recently neural network method has achieved good results in the task of text summarization both in Chinese and English, but the research of text summarization in low-resource languages is still in the exploratory stage, especially in Tibetan. What’s more, there is no large-scale annotated corpus for text summarization. The lack of dataset severely limits the development of low-resource text summarization. In this case, unsupervised learning approaches are more appealing in low-resource languages as they do not require labeled data. In this paper, we propose an unsupervised graph-based Tibetan multi-document summarization method, which divides a large number of Tibetan news documents into topics and extracts the summarization of each topic. Summarization obtained by using traditional graph-based methods have high redundancy and the division of documents topics are not detailed enough. In terms of topic division, we adopt two level clustering methods converting original document into document-level and sentence-level graph, next we take both linguistic and deep representation into account and integrate external corpus into graph to obtain the sentence semantic clustering. Improve the shortcomings of the traditional K-Means clustering method and perform more detailed clustering of documents. Then model sentence clusters into graphs, finally remeasure sentence nodes based on the topic semantic information and the impact of topic features on sentences, higher topic relevance summary is extracted. In order to promote the development of Tibetan text summarization, and to meet the needs of relevant researchers for high-quality Tibetan text summarization datasets, this paper manually constructs a Tibetan summarization dataset and carries out relevant experiments. The experiment results show that our method can effectively improve the quality of summarization and our method is competitive to previous unsupervised methods.

SSC: Clustering Of Turkish Texts By Spectral Graph Partitioning

There is growing interest in studies on text classification as a result of the exponential increase in the amount of data available. Many studies have been conducted in the field of text clustering, using different approaches. This study introduces Spectral Sentence Clustering (SSC) for text clustering problems, which is an unsupervised method based on graph-partitioning. The study explains how the proposed model proposed can be used in natural language applications to successfully cluster texts. A spectral graph theory method is used to partition the graph into non-intersecting sub-graphs, and an unsupervised and efficient solution is offered for the text clustering problem by providing a physical representation of the texts. Finally, tests have been conducted demonstrating that SSC can be successfully used for text categorization. A clustering success rate of 97.08% was achieved in tests conducted using the TTC-3600 dataset, which contains open-access unstructured Turkish texts, classified into categories. The SSC model proposed performed better compared to a popular k-means clustering algorithm.

Health Information Needs of Young Chinese People Based on an Online Health Community: Topic and Statistical Analysis.

BackgroundThe internet has been widely accessible and well accepted by young people; however, there is a limited understanding of the internet usage patterns and characteristics on issues related to health problems. The contents posted on online health communities (OHCs) are valuable resources to learn about youth's health information needs.ObjectiveIn this study, we concurrently exploited statistical analysis and topic analysis of online health information needs to explore the distribution, impact factors, and topics of interest relevant to Chinese young people.MethodsWe collected 60,478 health-related data sets posted by young people from a well-known Chinese OHC named xywy.com. Descriptive statistical analysis and correlation analysis were applied to find the distribution and influence factors of the information needs of Chinese young people. Furthermore, a general 4-step topic mining strategy was presented for sparse short texts, which included sentence vectorization, dimension reduction, clustering, and keyword generation.ResultsIn the Chinese OHC, Chinese young people had a high demand for information in the areas of gynecology and obstetrics, internal medicine, dermatology, plastic surgery, and surgery, and they focused on topics such as treatment, symptoms, causes, pathology, and diet. Females accounted for 69.67% (42,136/60,478) and young adults accounted for 87.44% (52,882/60,478) of all data. Gender, age, and disease type all had a significant effect on young people's information needs and topic preferences (P<.001).ConclusionsWe conducted comprehensive analyses to discover the online health information needs of Chinese young people. The research findings are of great practical value to carry out health education and health knowledge dissemination inside and outside of schools according to the interests of youth, enable the innovation of information services in OHCs, and improve the health literacy of young people.

SOCIAL MORALITY EDUCATION IN THE REJANG’S CULTURE OF "SERAMBEAK"

The social morality education contained in Serambeak is a meeting of the sacred mission of Islamic education with the Rejang culture in terms maintaining the life of the Rejang community. The Rejang community lives on the basis of a rural life system that strongly adheres to a set of cultures and customs called Serambeak, which contain social morality education of the community. Thus, this qualitative research was conducted by means of the descriptive-analytic method in a way that the words, sentences, and discourses in the literary work of Serambeak were described, and were, then, analyzed by providing understanding and explanations of texts. Data interpretation was carried out resting upon the researcher’s and community’s perspectives through information shared by traditional and religious leaders. The data analysis technique was carried out by classifying the data based on tracing the sentence clusters in Serambeak through the character's speeches and their attitudes. The results of this study revealed the following: First, the use of Serambeak as a local culture with its literary beauty makes it easier for the Rejang community to receive the social morality education brought by Islam as a foreign religion. Second, the content of social morality education in Serambeak consists of the arrangement of life for action and demands for action. Third, the social morality education contained in Serambeak is an effort to erode the old life orders, attitudes, and behaviors of Rejang people.

Monitoring COVID-19 pandemic through the lens of social media using natural language processing and machine learning.

PurposeIt has been over a year since the first known case of coronavirus disease (COVID-19) emerged, yet the pandemic is far from over. To date, the coronavirus pandemic has infected over eighty million people and has killed more than 1.78 million worldwide. This study aims to explore “how useful is Reddit social media platform to surveil COVID-19 pandemic?” and “how do people’s concerns/behaviors change over the course of COVID-19 pandemic in North Carolina?”. The purpose of this study was to compare people’s thoughts, behavior changes, discussion topics, and the number of confirmed cases and deaths by applying natural language processing (NLP) to COVID-19 related data.MethodsIn this study, we collected COVID-19 related data from 18 subreddits of North Carolina from March to August 2020. Next, we applied methods from natural language processing and machine learning to analyze collected Reddit posts using feature engineering, topic modeling, custom named-entity recognition (NER), and BERT-based (Bidirectional Encoder Representations from Transformers) sentence clustering. Using these methods, we were able to glean people’s responses and their concerns about COVID-19 pandemic in North Carolina.ResultsWe observed a positive change in attitudes towards masks for residents in North Carolina. The high-frequency words in all subreddit corpora for each of the COVID-19 mitigation strategy categories are: Distancing (DIST)—“social distance/distancing”, “lockdown”, and “work from home”; Disinfection (DIT)—“(hand) sanitizer/soap”, “hygiene”, and "wipe"; Personal Protective Equipment (PPE)—“mask/facemask(s)/face shield”, “n95(s)/kn95”, and “cloth/gown”; Symptoms (SYM)—“death”, “flu/influenza”, and “cough/coughed”; Testing (TEST)—“cases”, “(antibody) test”, and “test results (positive/negative)”.ConclusionThe findings in our study show that the use of Reddit data to monitor COVID-19 pandemic in North Carolina (NC) was effective. The study shows the utility of NLP methods (e.g. cosine similarity, Latent Dirichlet Allocation (LDA) topic modeling, custom NER and BERT-based sentence clustering) in discovering the change of the public's concerns/behaviors over the course of COVID-19 pandemic in NC using Reddit data. Moreover, the results show that social media data can be utilized to surveil the epidemic situation in a specific community.

Analysis of Particular Parameters of a Social Situation of the Development of a Pre-School Aged Child in Conditions of the Family and Pre-School Education Institutions

The changing conditions of social development give considerable relevance to a study of parameters of a social situation of the development of the individual at the stage of childhood. The aim of the study: research of characteristics of a social situation of the development of a pre-school aged child at the present stage of society. The study was conducted in the cities of Minsk and Zhlobin (The Republic of Belarus); children of pre-school education institutions (n=100) and their parents (n=28) participated in the study. Diagnostic tools: “Parents composition”, “Choice in action”, “Stair”, “Finish a story”, a conversation on three clusters of unfinished sentences. The results of the empirical study revealed a predominance of non-constructive child-parent interaction in the respondents’ families. The analysis of a child’s position in a group of an education institution generally revealed the existence of a favourable climate in the given subsystem of relations. Preferences in one or another type of interactions are connected with the self-assessment, a position in a group and a level of awareness of ethical norms and values. Optimization of the main parameters of pre-school aged children interaction is a relevant direction of the support of positive socialization.

Research on the Natural Language Recognition Method Based on Cluster Analysis Using Neural Network

Withthe technological advent, the clustering phenomenon is recently being used in various domains and in natural language recognition. This article contributes to the clustering phenomenon of natural language and fulfills the requirements for the dynamic update of the knowledge system. This article proposes a method of dynamic knowledge extraction based on sentence clustering recognition using a neural network-based framework. The conversion process from natural language papers to object-oriented knowledge system is studied considering the related problems of sentence vectorization. This article studies the attributes of sentence vectorization using various basic definitions, judgment theorem, and postprocessing elements. The sentence clustering recognition method of the network uses the concept of prereliability as a measure of the credibility of sentence recognition results. An ART2 neural network simulation program is written using MATLAB, and the effect of the neural network on sentence recognition is utilized for the corresponding analysis. A postreliability evaluation indexing is done for the credibility of the model construction, and the implementation steps for the conjunctive rule sentence pattern are specifically introduced. A new method of structural modeling is utilized to generate the structured derivation relationship, thus completing the natural language knowledge extraction process of the object-oriented knowledge system. An application example with mechanical CAD is used in this work to demonstrate the specific implementation of the example, which confirms the effectiveness of the proposed method.