Term Frequency-inverse Document Frequency Method Research Articles

Implementation of Chatbot Artificial Intelligence in a Company Website to Improve Customer Service Automatically Using the TF-IDF Method

In today's digital age, improving customer service is one of the keys to business success. Successfully retaining and attracting new customers is a challenge that must be faced. The application of chatbot technology powered by artificial intelligence (AI) on business websites has been proven to provide more efficient and responsive customer service. This research aims to develop and implement an AI chatbot that uses the TF-IDF (Term Frequency-Inverse Document Frequency) method to automatically understand and answer customer queries. The TF-IDF method is used to extract key features from the text of customer questions and match them with the most relevant answers in the database. The results of implementing this AI chatbot showed a significant improvement in the speed and quality of customer service responses, thus helping to improve customer satisfaction and company performance. This research provides valuable insights for businesses looking to integrate AI technology into their customer service strategy.

The development of a model for monitoring the quality of online psychological and educational services by using automated analysis of user reviews

Introduction. Digitalization of education and healthcare is an inevitable response to the demands of society. However, it is associated with the emergence of many risks, and the need for regulation and evaluation of new online practices is becoming increasingly important. Questions about the quality of online services are becoming more and more significant, as well as the search for new methods to monitor large amounts of publicly available data about online interactions with helping professionals. The purpose of this study is to develop a model for monitoring psychological and educational online services through automated analysis of user reviews and feedback. Materials and methods. The sample for the study consisted of a variety of reviews about online psychological and educational services. A total of 24,351 reviews of psychologists' work and 6,812 reviews of tutors' work were uploaded. These reviews were ranked in order to identify levels of user satisfaction. We used the following methods to collect and analyze the data: 1) Automated data parsing to extract relevant feedback using the GraphQL API; 2) The TF-IDF (term frequency-inverse document frequency) method to identify the most important linguistic markers in the reviews; 3) The LDA (latent Dirichlet allocation) method for thematic modeling of the reviews. The results of the study. An empirical study of online reviews of psychological and educational services in accordance with this model revealed their thematic structure in four main areas: assessment of interaction (19.8%), assessment of results (26%), organization of professional interaction (20.4%), and assessment of professional skills relevant to clients (29.1%).The main characteristics that users perceive in psychological online services are: the importance of comfortable communication (21.9%), inclusiveness (13.9%), and specialist experience (5.9%). Additionally, the ability to structure a request is also significant (19.8%). The result of evaluating the psychologist's work can be seen in user reviews. Some of the most common responses include: understanding the problem (18.5%), seeing the issue from a different perspective (6.7%), and feeling better (11.3%). Assessments of the quality of online education services include accessibility and clarity of presentation (29.5%), as well as confidence and flexibility in communication (6.1%). Additionally, finding a common language with students is also important (6.2%). Conclusion. The analysis of user feedback on educational and psychological online services can help to understand the directions for the development of work by helping specialists, create a system for monitoring non-specialized assistance, describe criteria for the training and retraining of specialists, and create educational programs for developing digital competence among psychologists and educators.

Unraveling minnan imagery: a comprehensive analysis of traditional and modern minnan nursery rhymes through complex networks

Nursery rhymes provide insights into the traditions, beliefs, and values of a culture, thereby making it an integral part of a community’s heritage. As representative linguistic resources within the intangible cultural heritage of the Hoklo people, Minnan nursery rhymes (MNRs) play a crucial role in connecting the Chinese mainland, Taiwan Strait, and overseas Chinese communities. This study delves into features of 617 traditional and 289 modern pieces through text mining techniques, including text segmentation, the TF-IDF (term frequency-inverse document frequency) method, and the complex network analysis. We examine the frequency and emotional purity of lyrics at a larger scale than previous studies using a small set of manually annotated samples. Furthermore, we analyze the patterns of MNRs by assessing the overall, individual, core-periphery structures of the constructed MNR networks, considering key terms as nodes and co-occurrence relationships between nodes as links. Our investigation reveals the heterogeneous nature of terms in both traditional and modern MNR networks. Moreover, through the community detection method, we identify five primary imagery features presented in MNRs. Traditional MNRs place emphasis on family relationships, folk culture, and food culture, reflecting enduring aspects of Minnan cultural heritage. In contrast, modern MNRs pivot towards themes of children’s emotions and natural scenery, indicative of evolving societal values. This study represents the first large-scale complex network analysis of MNRs, providing valuable insights into the embedded Minnan culture and serving as a foundation for further research into the societal dynamics reflected in these cherished MNRs resources.

An extended TF-IDF method for improving keyword extraction in traditional corpus-based research: An example of a climate change corpus

Keyword extraction involves the application of Natural Language Processing (NLP) algorithms or models developed in the realm of text mining. Keyword extraction is a common technique used to explore linguistic patterns in the corpus linguistic field, and Dunning’s Log-Likelihood Test (LLT) has long been integrated into corpus software as a statistic-based NLP model. While prior research has confirmed the widespread applicability of keyword extraction in corpus-based research, LLT has certain limitations that may impact the accuracy of keyword extraction in such research. This paper summarized the limitations of LLT, which include benchmark corpus interference, elimination of grammatical and generic words, consideration of sub-corpus relevance, flexibility in feature selection, and adaptability to different research goals. To address these limitations, this paper proposed an extended Term Frequency-Inverse Document Frequency (TF-IDF) method. To verify the applicability of the proposed method, 20 highly cited research articles on climate change from the Web of Science (WOS) database were used as the target corpus, and a comparison was conducted with the traditional method. The experimental results indicated that the proposed method could effectively overcome the limitations of the traditional method and demonstrated the feasibility and practicality of incorporating the TF-IDF algorithm into relevant corpus-based research.

#2924 Comparison of large language models and traditional natural language processing techniques in predicting arteriovenous fistula failure

Abstract Background and Aims Large language models (LLMs) have gained significant attention in the field of natural language processing (NLP), marking a shift from traditional techniques like Term Frequency-Inverse Document Frequency (TF-IDF). We developed a traditional NLP model to predict arteriovenous fistula (AVF) failure within next 30 days using clinical notes. The goal of this analysis was to investigate whether LLMs would outperform traditional NLP techniques, specifically in the context of predicting AVF failure within the next 30 days using clinical notes. Method We defined AVF failure as the change in status from active to permanently unusable status or temporarily unusable status. We used data from a large kidney care network from January 2021 to December 2021. Two models were created using LLMs and traditional TF-IDF technique. We used “distilbert-base-uncased”, a distilled version of BERT base model [1], and compared its performance with traditional TF-IDF-based NLP techniques. The dataset was randomly divided into 60% training, 20% validation and 20% test dataset. The test data, comprising of unseen patients’ data was used to evaluate the performance of the model. Both models were evaluated using metrics such as area under the receiver operating curve (AUROC), accuracy, sensitivity, and specificity. Results The incidence of 30 days AVF failure rate was 2.3% in the population. Both LLMs and traditional showed similar overall performance as summarized in Table 1. Notably, LLMs showed marginally better performance in certain evaluation metrics. Both models had same AUROC of 0.64 on test data. The accuracy and balanced accuracy for LLMs were 72.9% and 59.7%, respectively, compared to 70.9% and 59.6% for the traditional TF-IDF approach. In terms of specificity, LLMs scored 73.2%, slightly higher than the 71.2% observed for traditional NLP methods. However, LLMs had a lower sensitivity of 46.1% compared to 48% for traditional NLP. However, it is worth noting that training on LLMs took considerably longer than TF-IDF. Moreover, it also used higher computational resources such as utilization of graphics processing units (GPU) instances in cloud-based services, leading to higher cost. Conclusion In our study, we discovered that advanced LLMs perform comparably to traditional TF-IDF modeling techniques in predicting the failure of AVF. Both models demonstrated identical AUROC. While specificity was higher in LLMs compared to traditional NLP, sensitivity was higher in traditional NLP compared to LLMs. LLM was fine-tuned with a limited dataset, which could have influenced its performance to be similar to that of traditional NLP methods. This finding suggests that while LLMs may excel in certain scenarios, such as performing in-depth sentiment analysis of patient data for complex tasks, their effectiveness is highly dependent on the specific use case. It is crucial to weigh the benefits against the resources required for LLMs, as they can be significantly more resource-intensive and costly compared to traditional TF-IDF methods. This highlights the importance of a use-case-driven approach in selecting the appropriate NLP technique for healthcare applications.

Deciphering the Systemic Impact of Herbal Medicines on Allergic Rhinitis: A Network Pharmacological Approach.

Allergic rhinitis (AR) is a systemic allergic disease that has a considerable impact on patients' quality of life. Current treatments include antihistamines and nasal steroids; however, their long-term use often causes undesirable side effects. In this context, traditional Asian medicine (TAM), with its multi-compound, multi-target herbal medicines (medicinal plants), offers a promising alternative. However, the complexity of these multi-compound traits poses challenges in understanding the overall mechanisms and efficacy of herbal medicines. Here, we demonstrate the efficacy and underlying mechanisms of these multi-compound herbal medicines specifically used for AR at a systemic level. We utilized a modified term frequency-inverse document frequency method to select AR-specific herbs and constructed an herb-compound-target network using reliable databases and computational methods, such as the Quantitative Estimate of Drug-likeness for compound filtering, STITCH database for compound-target interaction prediction (with a high confidence score threshold of 0.7), and DisGeNET and CTD databases for disease-gene association analysis. Through this network, we conducted AR-related targets and pathway analyses, as well as clustering analysis based on target-level information of the herbs. Gene ontology enrichment analysis was conducted using a protein-protein interaction network. Our research identified 14 AR-specific herbs and analyzed whether AR-specific herbs are highly related to previously known AR-related genes and pathways. AR-specific herbs were found to target several genes related to inflammation and AR pathogenesis, such as PTGS2, HRH1, and TBXA2R. Pathway analysis revealed that AR-specific herbs were associated with multiple AR-related pathways, including cytokine signaling, immune response, and allergic inflammation. Additionally, clustering analysis based on target similarity identified three distinct subgroups of AR-specific herbs, corroborated by a protein-protein interaction network. Group 1 herbs were associated with the regulation of inflammatory responses to antigenic stimuli, while Group 2 herbs were related to the detection of chemical stimuli involved in the sensory perception of bitter taste. Group 3 herbs were distinctly associated with antigen processing and presentation and NIK/NF-kappa B signaling. This study decodes the principles of TAM herbal configurations for AR using a network pharmacological approach, providing a holistic understanding of drug effects beyond specific pathways.

OPTIMIZATION NAÏVE BAYES ALGORITHM IN SENTIMENT ANALYSIS OF BUKALAPAK APP REVIEWS

Bukalapak application reviews on Google Play Store include useful information if processed correctly. The activity of analyzing application reviews is not enough to see the number of stars, it is necessary to see the entire contents of the review comments to be able to know the intent of the review. Sentiment analysis system is a system used to automatically analyze reviews. Review data is retrieved via the bukalapak application API and then classified using Naive Bayes Multinomial. A total of 1,000 reviews of bukalapak application users were collected to be used as dataset samples. The purpose of this research is to determine the accuracy level of sentiment analysis using the multinomial Naive Bayes algorithm. The stages of this research include, data collection, automatic labeling using python, pre-processing, sentiment classification, and evaluation. In the pre-processing stage there are 6 stages, namely Cleaning, Casefolding, Word Normalizer, Tokenizing, Stopword Removal and Stemming. TF-IDF (Term Frequency - Inverse Document Frequency) method is used for word weighting. The data will be grouped into two categories, namely negative and positive. The test results show an accuracy value of 90%, this result shows that the bukalapak application reviews tend to be negative. The research at this time only looks for accuracy values and provides an overview of the bulapak application to potential new users.

IMPROVEMENT OF NAIVE BAYES ALGORITHM IN SENTIMENT ANALYSIS OF SHOPEE APPLICATION REVIEWS ON GOOGLE PLAY STORE

Reviews of the shopee application on the google play store are included in useful information if processed properly. Old or new users can analyze app reviews to get information that can be used to evaluate services. The activity of analyzing application reviews is not enough just to see the number of stars, it is necessary to see the entire contents of the review comments to be able to know the intent of the review. A sentiment analysis system is a system used to automatically analyze a review to obtain information including sentiment information that is part of an online review. The data is classified using Naive Bayes. A total of 1,000 shopee app user reviews were collected to form the sample dataset. The purpose of this study is to determine the sentiment analysis of shopee application reviews in the Google Play Store using the Naive Bayes algorithm. The stages of this research include, data collection, labeling, pre-processing, sentiment classification, and evaluation. In the pre-processing stage there are 6 stages, namely Cleaning, Case folding, Word Normalizer, Tokenizing, Stopword Removal and Stemming. TF-IDF (Term Frequency - Inverse Document Frequency) method is used for word weighting. The data will be grouped into two categories, namely negative and positive. The data will then be evaluated using accuracy parameter testing. The test results show an accuracy value of 81%, this result shows that shopee application reviews tend to be negative.

Machine Learning-Based E-Archive for Archives Management of South Sumatra Province

Archives play a crucial role in institutional operations, yet efficiently retrieving specific information from them can be challenging. This research addresses this issue by developing an information retrieval system that incorporates advanced methods to enhance search efficiency. The system employs the TF-IDF (Term Frequency-Inverse Document Frequency) formula, which assesses the significance of a word within a document set, and the BM25 method, a sophisticated algorithm for ranking documents based on their relevance to the input query. Both methods undergo a preprocessing stage, enabling the system to calculate the relevance of each document to the given query accurately. The effectiveness of this system is evaluated using key performance metrics: precision (accuracy), recall (completeness), and the F1 Score (the harmonic means of precision and recall, representing the best value). Testing with various keywords revealed that the BM25 method yielded impressive results, achieving an average precision of 0.75, recall of 0.6, and an F1 Score of 0.6665. In contrast, the TF-IDF method scored lower, with a precision of 0.33, recall of 0.2, and an F1 Score of 0.2500. The system was tested using a dataset of 350 documents.

SISTEM ANALISIS PENYAKIT MATA BERBASIS PHP DAN MYSQL MENGGUNAKAN METODE TF-IDF

The development of information technology has had a positive impact on the healthcare sector, particularly in disease analysis. Eye diseases are among the health issues that require early detection for effective prevention and treatment. In this context, this research proposes the development of a PHP and MySQL-based Eye Disease Analysis System utilizing the Term Frequency-Inverse Document Frequency (TF-IDF) method. The system is designed to provide a web platform facilitating the analysis of textual data related to eye diseases. Involving PHP as the programming language and MySQL as the database, the system allows efficient storage, management, and retrieval of medical information. The TF-IDF method is employed to analyze unique characteristics within textual data, enabling the identification of symptoms, diagnosis, and treatment of eye diseases. Context Diagram, Data Flow Diagram (DFD), and Entity-Relationship Diagram (ERD) serve as visual guides in designing this system. The Context Diagram provides a general overview of the system's interaction with its external environment, while DFD illustrates the flow of data within the system. ERD is used to represent the data structure within the database.
 The research results are expected to significantly contribute to the efficiency of eye disease analysis, providing faster information to medical professionals and enhancing the general public's understanding of eye diseases. This system is anticipated to serve as a foundation for further developments in the application of information technology in the healthcare sector, particularly in disease analysis.

Innovative Framework for Context-based Recommendation using TF-IDF and Cosine Similarity

The study investigates a novel framework for context-based recommendation systems that makes use of cosine similarity and TF-IDF (Term Frequency-Inverse Document Frequency) methods. Through the recommendation of pertinent things based on context and user preferences, recommendation systems significantly improve user experience. Recommendations from traditional systems are frequently not as good because of their inability to properly include contextual information. In order to close this gap, this work suggests a unique method that represents item and user profiles using TF-IDF and calculates cosine similarity to capture semantic similarities. The framework's ability to adjust dynamically to shifting user situations improves the accuracy of recommendations in a variety of industries, including personalized services, streaming media, and e-commerce. From a methodological point of view, the study entails using and assessing the framework on a dataset customized for context-aware recommendation situations. Comparing the approach to baseline methods, experimental results show that it is successful in enhancing recommendation quality. By incorporating strong contextual understanding, the knowledge obtained from this work advances recommendation systems and opens the door to more customized and flexible user experiences in digital contexts. The concept, methods, and main conclusions of the study are briefly described in this abstract, which also emphasizes the novel use of TF-IDF and cosine similarity for context-based recommendations.

Multilayer Perceptron and TF-IDF in the Classification of Hate Speech on Twitter in Indonesian

Twitter nowadays is one of the popular social media which currently has over 300millions accounts, twitter is the rich source to learn about people’s opion and sentimental analysis. However, this also brings new problems where the practice of hate speech. This research classifies of hate speech on social media. Evaluation using dataset from previous research Ibrohim&Budi (2019), then using classification method Multilayer Perceptron which combined with feature extraction to be able to detect negations and weighting uses Term Frequency – Inverse Document Frequency (TF-IDF). Results show that the F1 score gives an accuracy rate of up to 74.51%. This research has a reasonably good effectiveness from combining the TF-IDF and Multilayer Perceptron methods, considering the results obtained from the F1 Score evaluation value.

SENTIMENT ANALYSIS ON REVIEWS OF WOMEN'S TOPS ON SHOPEE MARKETPLACE USING NAIVE BAYES ALGORITHM

Reviews of women's tops in the market are valuable information if processed properly. Merchants can conduct product review analysis to obtain information that can be used to evaluate products and services. Product review analysis activities are not enough just to see the number of stars, it is necessary to see the entire content of the review comments to be able to know the intent of the review. Sentiment analysis system is a system used to automatically analyze online product reviews to obtain information including sentiment information that is part of online reviews. The data is classified using Naive Bayes. The data collected were 1,000 product reviews of women's tops as samples. The purpose of this study is to determine the sentiment analysis of female top product reviews using the Naive Bayes algorithm. The stages of this research include data collection, labeling, pre-processing, sentiment classification, and evaluation. In the pre-processing stage there are 6 stages, namely Cleaning emoticons & symbols, Case folding, Word Normalizer, Tokenize, Stopword Removal and Stemming. TF-IDF (Term Frequency - Inverse Document Frequency) method is used for word weighting. The data will be classified into 3 (three) classes, namely negative, positive and neutral. The data will then be evaluated using accuracy parameter testing. The test results show an accuracy value of 89%, this result shows that the product reviews of women's tops are more positive.

Fusion Model for Tentative Diagnosis Inference Based on Clinical Narratives

In general, physicians make a preliminary diagnosis based on patients' admission narratives and admission conditions, largely depending on their experiences and professional knowledge. An automatic and accurate tentative diagnosis based on clinical narratives would be of great importance to physicians, particularly in the shortage of medical resources. Despite its great value, little work has been conducted on this diagnosis method. Thus, in this study, we propose a fusion model that integrates the semantic and symptom features contained in the clinical text. The semantic features of the input text are initially captured by an attention-based Bidirectional Long Short-Term Memory (BiLSTM) network. The symptom concepts, recognized from the input text, are then vectorized by using the term frequency-inverse document frequency method based on the relations between symptoms and diseases. Finally, two fusion strategies are utilized to recommend the most potential candidate for the international classification of diseases code. Model training and evaluation are performed on a public clinical dataset. The results show that both fusion strategies achieved a promising performance, in which the best performance obtained a top-3 accuracy of 0.7412.

Integrating Information Entropy and Latent Dirichlet Allocation Models for Analysis of Safety Accidents in the Construction Industry

Construction accident investigation reports contain critical information, but extracting useful insights from the voluminous Chinese text is challenging. Traditional methods rely on expert judgment, which leads to time-consuming and potentially inaccurate results. To overcome this problem, we propose a novel approach that combines text mining techniques and latent Dirichlet allocation (LDA) models to analyze standardized accident investigation reports in the Chinese construction industry. The proposed method integrates an information entropy term frequency-inverse document frequency (TF-IDF) weighting scheme to evaluate term importance and accounts for word and model uncertainty. The method was applied to a set of construction industry accident reports to identify the key factors leading to safety accidents. The results show that the causal factors of accidents in Chinese accident investigation reports consist of keywords and negative expressions, including “failure to timely identify safety hazards” and “inadequate site safety management”. Failure to timely identify safety hazards is the most common factor in accident investigation reports, and the negative expressions commonly used in the reports include “not timely” and “not in place”. The information entropy TF-IDF method is superior to traditional methods in terms of accuracy and efficiency, and the LDA model that considers word frequency and feature weights is better able to capture the underlying themes in the Chinese corpus. And the subject terms that make up the themes contain more information about the causes of accidents. This approach helps site managers more quickly and effectively understand the causal factors and key messages that lead to accidents from incident reports. It gives site managers insight into common patterns and themes associated with safety incidents, such as unsafe practices, hazardous work environments, and non-compliance with safety regulations. This enables them to make informed decisions to improve safety management practices.

Semantic Similarity Comparison Between Production Line Failures for Predictive Maintenance

With the introduction of Industry 4.0 into our lives and the creation of smart factories, predictive maintenance has become even more important. Predictive maintenance systems are often used in the manufacturing industry. On the other hand, text analysis and Natural Language Processing (NLP) techniques are gaining a lot of attention by both research and industry due to their ability to combine natural languages and industrial solutions. There is a great increase in the number of studies on NLP in the literature. Even though there are studies in the field of NLP in predictive maintenance systems, no studies were found on Turkish NLP for predictive maintenance. This study focuses on the similarity analysis of failure texts that can be used in the predictive maintenance system we developed for VESTEL, one of the leading consumer electronics manufacturers in Turkey. In the manufacturing industry, operators record descriptions of failure that occur on production lines as short texts. However, these descriptions are not often used in predictive maintenance work. In this study, semantic text similarities between fault definitions in the production line were compared using traditional word representations, modern word representations and Transformer models. Levenshtein, Jaccard, Pearson, and Cosine scales were used as similarity measures and the effectiveness of these measures were compared. Experimental data including failure texts were obtained from a consumer electronics manufacturer in Turkey. When the experimental results are examined, it is seen that the Jaccard similarity metric is not successful in grouping semantic similarities according to the other three similarity measures. In addition, Multilingual Universal Sentence Encoder (MUSE), Language-agnostic BERT Sentence Embedding (LAbSE), Bag of Words (BoW) and Term Frequency - Inverse Document Frequency (TF-IDF) outperform FastText and Language-Agnostic Sentence Representations (LASER) models in semantic discovery of error identification in embedding methods. Briefly to conclude, Pearson and Cosine are more effective at finding similar failure texts; MUSE, LAbSE, BoW and TF-IDF methods are more successful at representing the failure text.

A Novel Automatic Audiometric System Design Based on Machine Learning Methods Using the Brain's Electrical Activity Signals.

This study uses machine learning to perform the hearing test (audiometry) processes autonomously with EEG signals. Sounds with different amplitudes and wavelengths given to the person tested in standard hearing tests are assigned randomly with the interface designed with MATLAB GUI. The person stated that he heard the random size sounds he listened to with headphones but did not take action if he did not hear them. Simultaneously, EEG (electro-encephalography) signals were followed, and the waves created in the brain by the sounds that the person attended and did not hear were recorded. EEG data generated at the end of the test were pre-processed, and then feature extraction was performed. The heard and unheard information received from the MATLAB interface was combined with the EEG signals, and it was determined which sounds the person heard and which they did not hear. During the waiting period between the sounds given via the interface, no sound was given to the person. Therefore, these times are marked as not heard in EEG signals. In this study, brain signals were measured with Brain Products Vamp 16 EEG device, and then EEG raw data were created using the Brain Vision Recorder program and MATLAB. After the data set was created from the signal data produced by the heard and unheard sounds in the brain, machine learning processes were carried out with the PYTHON programming language. The raw data created with MATLAB was taken with the Python programming language, and after the pre-processing steps were completed, machine learning methods were applied to the classification algorithms. Each raw EEG data has been detected by the Count Vectorizer method. The importance of each EEG signal in all EEG data has been calculated using the TF-IDF (Term Frequency-Inverse Document Frequency) method. The obtained dataset has been classified according to whether people can hear the sound. Naïve Bayes, Light Gradient Strengthening Machine (LGBM), support vector machine (SVM), decision tree, k-NN, logistic regression, and random forest classifier algorithms have been applied in the analysis. The algorithms selected in our study were preferred because they showed superior performance in ML and succeeded in analyzing EEG signals. Selected classification algorithms also have features of being used online. Naïve Bayes, Light Gradient Strengthening Machine (LGBM), support vector machine (SVM), decision tree, k-NN, logistic regression, and random forest classifier algorithms were used. In the analysis of EEG signals, Light Gradient Strengthening Machine (LGBM) was obtained as the best method. It was determined that the most successful algorithm in prediction was the prediction of the LGBM classification algorithm, with a success rate of 84%. This study has revealed that hearing tests can also be performed using brain waves detected by an EEG device. Although a completely independent hearing test can be created, an audiologist or doctor may be needed to evaluate the results.

Research on measurement and disequilibrium of manufacturing digital transformation: Based on the text mining data of A-share listed companies

<abstract> <p>Quantitative analysis of digital transformation is an important part of relevant research in the digital field. Based on the annual report data of China's manufacturing listed companies from 2011 to 2019, this study applies cloud computing to the mining and analysis of text data, and uses the Term Frequency-Inverse Document Frequency method under machine learning to measure the digital transformation index value of manufacturing enterprises. The results show that: (1) On the whole, the current pace of digital transformation of manufacturing enterprises continues to accelerate, and the digital transformation of manufacturing has gradually spread from the eastern coastal areas to the central and western inland areas. (2) In horizontal comparison, among the five types of "ABCDE" digital modules constructed, artificial intelligence develops the fastest, cloud computing index value is second, and block chain value is the smallest. In vertical comparison, the leading provinces such as Beijing, Guangdong, and Shanghai have certain stability and a solid leading position, and there are occasional highlights in the central and western provinces. (3) In terms of polarization distribution, the digitalization of the manufacturing industry has obvious multi-peak patterns, showing the phenomenon of multi-polarization of digital services. The eastern region has both aggregate advantages and equilibrium disadvantages. (4) In terms of industry differences, the level of digital transformation in the high-end manufacturing industry is significantly higher than that in the mid-end and low-end industries. On the ownership attributes of enterprise digital transformation, private enterprises are the highest, followed by foreign-funded enterprises, and state-owned enterprises are the lowest. This research provides theoretical enlightenment and factual reference for manufacturing enterprises to carry out digital activities.</p> </abstract>

Comparative analysis of TF-IDF and loglikelihood method for keywords extraction of twitter data

Twitter has become the foremost standard of social media in today’s world. Over 335 million users are online monthly, and near about 80% are accessing it through their mobiles. Further, Twitter is now supporting 35+ which enhance its usage too much. It facilitates people having different languages. Near about 21% of the total users are from US and 79% of total users are outside of US. A tweet is restricted to a hundred and forty characters; hence it contains such information which is more concise and much valuable. Due to its usage, it is estimated that five hundred million tweets are sent per day by different categories of people including teacher, students, celebrities, officers, musician, etc. So, there is a huge amount of data that is increasing on a daily basis that need to be categorized. The important key feature is to find the keywords in the huge data that is helpful for identifying a twitter for classification. For this purpose, Term Frequency-Inverse Document Frequency (TF-IDF) and Loglikelihood methods are chosen for keywords extracted from the music field and perform a comparative analysis on both results. In the end, relevance is performed from 5 users so that finally we can take a decision to make assumption on the basis of experiments that which method is best. This analysis is much valuable because it gives a more accurate estimation which method’s results are more reliable.

Intelligent mining of safety hazard information from construction documents using semantic similarity and information entropy

Project construction on-site is known to be very dangerous workplace environments due to large numbers of safety hazards. Analysis of construction safety hazards is essential to formulate rational safety management plans and prevent accidents. Construction documents contain large volumes of safety hazard information available for analysis. However, such analyses are challenging because the safety hazard information in the construction documents is presented in an unstructured or semi-structured format. This study proposes a method for intelligent mining of safety hazard information, which comprises safety hazard technical term recognition and safety hazard information analysis. The safety hazard technical term recognition model is developed based on semantic similarity and information correlation to build a safety hazard technical term library. The safety hazard information based on the technical term library is mined and analyzed using the term frequency-inverse document frequency method (TF-IDF). Finally, the proposed method is applied to build the safety hazard technical term library, which contains 2697 technical terms, and develop a hydraulic project construction safety hazard analysis system, which can realize the intelligent recognition and application of technical terms. Meanwhile, this system can automatically extract safety hazard information and provide a visualization interface to intuitively show the safety hazard analysis results, which improves the extraction efficiency of safety hazard information. The study provides a new approach for recognizing technical terms and mining safety hazard information, which can lead to enhancing management efficiency and practical knowledge discovery for safety management.