Term Frequency-Inverse Document Frequency Features Research Articles

Comparative analysis of impact of classification algorithms on security and performance bug reports

Abstract Identification and classification of bugs, e.g., security and performance are a preemptive and fundamental practice which contributes to the development of secure and efficient software. Software Quality Assurance (SQA) needs to classify bugs into relevant categories, e.g., security and performance bugs since one type of bug may have a higher preference over another, thus facilitating software evolution and maintenance. In addition to classification, it would be ideal for the SQA manager to prioritize security and performance bugs based on the level of perseverance, severity, or impact to assign relevant developers whose expertise is aligned with the identification of such bugs, thus facilitating triaging. The aim of this research is to compare and analyze the prediction accuracy of machine learning algorithms, i.e., Artificial neural network (ANN), Support vector machine (SVM), Naïve Bayes (NB), Decision tree (DT), Logistic regression (LR), and K-nearest neighbor (KNN) to identify security and performance bugs from the bug repository. We first label the existing dataset from the Bugzilla repository with the help of a software security expert to train the algorithms. Our research type is explanatory, and our research method is controlled experimentation, in which the independent variable is prediction accuracy and the dependent variables are ANN, SVM, NB, DT, LR, and KNN. First, we applied preprocessing, Term Frequency-Inverse Document Frequency feature extraction methods, and then applied classification algorithms. The results were measured through accuracy, precision, recall, and F-measure and then the results were compared and validated through the ten-fold cross-validation technique. Comparative analysis reveals that two algorithms (SVM and LR) perform better in terms of precision (0.99) for performance bugs and three algorithms (SVM, ANN, and LR) perform better in terms of F1 score for security bugs as compared to other classification algorithms which are essentially due to the linear dataset and extensive number of features in the dataset.

Analysis of K-NN with the Integration of Bag of Words, TF-IDF, and N-Grams for Hate Speech Classification on Twitter

Social media has emerged as one of the primary communication channels in the modern world, but it has simultaneously become a platform where hate speech can spread easily. This study attempts to evaluate the performance of a hate speech classification model using the K-Nearest Neighbors (K-NN) algorithm along with various feature extraction techniques, specifically Bag of Words (BoW), Term Frequency-Inverse Document Frequency (TF-IDF), and N-Grams. The dataset used in this study consists of 13169 entries, which represent a diverse range of hate speech examples commonly encountered on social media platforms. In this experimental investigation, we assess the efficacy of the model using each feature extraction technique. The findings reveal that the K-NN model exhibits optimal performance when the k parameter is set to 3 (k=3). Under this configuration, the model achieves an accuracy of 86.88%, with a precision of 88.27%, a recall of 86.88%, and an F1-Score of 86.50%. These results show that the integration of TF-IDF feature extraction technique with K-NN algorithm produces superior performance in hate speech classification.

Reddit social media text analysis for depression prediction: using logistic regression with enhanced term frequency-inverse document frequency features

Reddit social media text analysis for depression prediction: using logistic regression with enhanced term frequency-inverse document frequency features

An Effective Method for Sentiment Analysis: Enhancing Review Classification through Machine Learning

Sentiment analysis, a critical topic in Natural Language Processing (NLP). Classification of reviews into categories that are neutral, positive, or negative is the primary goal. This study employed the dataset public available called as Amazon reviews dataset. Study investigated different aspects taken from a representation of the text data in the form of a bag of words. We have experimented with feature selection methods, particularly Term Frequency-Inverse Document Frequency (TFIDF), to improve the model's performance. Then, for the classification task, a Support Vector Machine (SVM) classifier was used. The outcomes showed that the model's performance was enhanced by combining TFIDF feature selection with bag of words feature extraction. This method of work improved the accuracy of classification models, which helps NLP applications forecast various review more accurately. Key Words: NLP, Bag of Words, TFIDF, SVM, Amazon Reviews Dataset, Machine Learning.

An efficient artificial intelligence approach for early detection of cross-site scripting attacks

Cross-Site Scripting (XSS) attacks continue to pose a significant threat to web applications, compromising the security and integrity of user data. XSS is a web application vulnerability where malicious scripts are injected into websites, allowing attackers to execute arbitrary code in the victim’s browser. The consequences of XSS attacks can be severe, ranging from financial losses to compromising sensitive user information. XSS attacks enable attackers to deface websites, distribute malware, or launch phishing campaigns, compromising the trust and reputation of affected organizations. This study proposes an efficient artificial intelligence approach for the early detection of XSS attacks, utilizing machine learning and deep learning approaches, including Long Short-Term Memory (LSTM). Additionally, advanced feature engineering techniques, such as the Term Frequency-Inverse Document Frequency (TFIDF), are applied and compared to evaluate results. We introduce a novel approach named LSTM-TFIDF (LSTF) for feature extraction, which combines temporal and TFIDF features from the cross-site scripting dataset, resulting in a new feature set. Extensive research experiments demonstrate that the random forest method achieved a high performance of 0.99, outperforming state-of-the-art approaches using the proposed features. A k-fold cross-validation mechanism is utilized to validate the performance of applied methods, and hyperparameter tuning further enhances the performance of XSS attack detection. We have applied Explainable Artificial Intelligence (XAI) to understand the interpretability and transparency of the proposed model in detecting XSS attacks. This study makes a valuable contribution to the growing body of knowledge on XSS attacks and provides an efficient model for developers and security practitioners to enhance the security of web applications.

Comparative Analysis of XGBoost Performance for Text Classification with CPU Parallel and Non-Parallel Processing

This paper shows the findings of a study that looks at how CPU parallel processing changes the way Extreme Gradient Boosting (XGBoost) classifies text. XGBoost models can sort news stories into set groups faster and more accurately, with or without CPU parallelism. This is the main goal of the study. The Keras dataset is used to prepare the text so that the TF-IDF (Term Frequency-Inverse Document Frequency) features can be found. These features will then be used to train the XGBoost model. This is used to check out two different kinds of the XGBoost classifier. There is parallelism between one of them and not it in the other. How well the model works can be observed by how accurate it is. This includes both how long it takes to learn and estimate and how well predictions work. The models take very different amounts of time to compute, but they are all pretty close in terms of how accurate they are. Parallel processing on the CPU has made tasks proceed more rapidly, and XGBoost is now better at making the most of that speed to do its task. The purpose of the study is to show that parallel processing can speed up XGBoost models without affecting their accuracy. This is helpful for putting text into categories.

An interpretable machine learning framework for opioid overdose surveillance from emergency medical services records.

The goal of this study is to develop and validate a lightweight, interpretable machine learning (ML) classifier to identify opioid overdoses in emergency medical services (EMS) records. We conducted a comparative assessment of three feature engineering approaches designed for use with unstructured narrative data. Opioid overdose annotations were provided by two harm reduction paramedics and two supporting annotators trained to reliably match expert annotations. Candidate feature engineering techniques included term frequency-inverse document frequency (TF-IDF), a highly performant approach to concept vectorization, and a custom approach based on the count of empirically-identified keywords. Each feature set was trained using four model architectures: generalized linear model (GLM), Naïve Bayes, neural network, and Extreme Gradient Boost (XGBoost). Ensembles of trained models were also evaluated. The custom feature models were also assessed for variable importance to aid interpretation. Models trained using TF-IDF feature engineering ranged from AUROC = 0.59 (95% CI: 0.53-0.66) for the Naïve Bayes to AUROC = 0.76 (95% CI: 0.71-0.81) for the neural network. Models trained using concept vectorization features ranged from AUROC = 0.83 (95% 0.78-0.88)for the Naïve Bayes to AUROC = 0.89 (95% CI: 0.85-0.94) for the ensemble. Models trained using custom features were the most performant, with benchmarks ranging from AUROC = 0.92 (95% CI: 0.88-0.95) with the GLM to 0.93 (95% CI: 0.90-0.96) for the ensemble. The custom features model achieved positive predictive values (PPV) ranging for 80 to 100%, which represent substantial improvements over previously published EMS encounter opioid overdose classifiers. The application of this approach to county EMS data can productively inform local and targeted harm reduction initiatives.

Sentiment Analysis to Identify Consumer Criticism of Artificial Intelligence: A ChatGPT Case Study

As artificial intelligence becomes increasingly integrated into various aspects of our lives, understanding consumer sentiment and criticism towards artificial intelligence technologies becomes pivotal for effective utilization. This study presents a case study focusing on ChatGPT, a popular AI application, as a means to identify consumer criticism of AI use in businesses. By harnessing sentiment analysis and clear analytics, administrators can enhance their understanding of consumer feedback and thereby improve AI integration for better user experiences. Increased consumer satisfaction is important to overall business to consumer relationships and streamlined AI use will facilitate company procedures. Our methodology revolves around machine learning techniques, specifically utilizing four classifiers, Keras logistic regression, Naive Bayes, Support Vector Machine, and random forest regressor, alongside two numerical feature representations, Bag-of-Words and Term Frequency-Inverse Document Frequency. The results show that the Term Frequency-Inverse Document Frequency features combined with the random forest regressor yielded the strongest performance in identifying criticism of ChatGPT-related media, with F1 scores of 100 and 99 percent for no criticism and criticism, respectively.

ANALYSIS OF THE EXISTENCE OF THE ACCOUNTING PROFESSION IN THE ERA OF SOCIETY 5.0 WITH THE NAIVE BAYES METHOD

Society 5.0 has opened a new phenomenon that allows humans to use modern-based science (Artificial Intelligence) for human needs with the goal of human life more easily. This phenomenon opens up various new possibilities and risks simultaneously. These changes have had a significant impact on the development of the accounting profession, resulting in fewer human resources needed in business, including accounting staff. This research aims to 1) Apply sentiment analysis to the existence of the accounting profession in the era of society 5.0. 2) Application of the naive Bayes method to classify sentiment analysis for the existence of the accounting profession in the era of society 5.0. 3) Testing the naive Bayes algorithm in sentiment analysis for the existence of the accounting profession in the era of society 5.0. This research design uses quantitative research, analysis of student sentiment on the existence of the accounting profession was carried out using data from 432 student opinions from several campuses in the East Java region. Test data using the Naïve Bayes algorithm with Term-Frequency Inverse Document Frequency feature selection and the Lexicon method. The results of this research are divided into five parts consisting of the first part 424 students' opinions on the level of accuracy (78%), the second part 427 students' opinions on the level of accuracy (62%), the third part 429 students' opinions on the level of accuracy (47%), the fourth part 425 students' opinions level of accuracy (56%), fifth part 429 student opinions level of accuracy (52%). It was found that the number of positive sentiments was more dominant than negative or neutral sentiments

A hybrid machine learning and natural language processing model for early detection of acute coronary syndrome

Acute coronary syndrome (ACS) is a leading cause of mortality and morbidity. Predicting the associated risks of patients with chest pain using electronic health record data can help identify those needing more tailored care. This study proposes the development of a reliable prediction framework to serve as a diagnostic support tool for preventing misdiagnoses among patients with clinical concerns for ACS. Data were collected from an urban, demographically diverse hospital in Detroit, Michigan, for patients presenting to the emergency department (ED) with primary chief complaints of chest pain from January 2017 to August 2020. This study incorporated term frequency-inverse document frequency features from free-text summaries, which contain anecdotal symptom descriptions and are among the first data points provided upon entering the ED. The analysis included 16,096 patients with clinical concerns for ACS and trained three machine learning models, logistic regression, AdaBoost, and linear discriminant analysis, across different data processing stages to predict patients with ACS from non-ACS etiology. The AdaBoost model outperformed the other two models with an accuracy of 94% and an F1-score of 0.943 in predicting ACS on the testing data. This study identified key independent factors from patient demographics, comorbidities, and clinical narrative data that predicted ACS in patients. The prediction framework can serve as a decision-support tool to classify ACS and inform physicians about better ACS risk factors.

Machine learning-based guilt detection in text

We introduce a novel Natural Language Processing (NLP) task called guilt detection, which focuses on detecting guilt in text. We identify guilt as a complex and vital emotion that has not been previously studied in NLP, and we aim to provide a more fine-grained analysis of it. To address the lack of publicly available corpora for guilt detection, we created VIC, a dataset containing 4622 texts from three existing emotion detection datasets that we binarized into guilt and no-guilt classes. We experimented with traditional machine learning methods using bag-of-words and term frequency-inverse document frequency features, achieving a 72% f1 score with the highest-performing model. Our study provides a first step towards understanding guilt in text and opens the door for future research in this area.

Virtually screening adults for depression, anxiety, and suicide risk using machine learning and language from an open-ended interview.

Current depression, anxiety, and suicide screening techniques rely on retrospective patient reported symptoms to standardized scales. A qualitative approach to screening combined with the innovation of natural language processing (NLP) and machine learning (ML) methods have shown promise to enhance person-centeredness while detecting depression, anxiety, and suicide risk from in-the-moment patient language derived from an open-ended brief interview. To evaluate the performance of NLP/ML models to identify depression, anxiety, and suicide risk from a single 5-10-min semi-structured interview with a large, national sample. Two thousand four hundred sixteen interviews were conducted with 1,433 participants over a teleconference platform, with 861 (35.6%), 863 (35.7%), and 838 (34.7%) sessions screening positive for depression, anxiety, and suicide risk, respectively. Participants completed an interview over a teleconference platform to collect language about the participants' feelings and emotional state. Logistic regression (LR), support vector machine (SVM), and extreme gradient boosting (XGB) models were trained for each condition using term frequency-inverse document frequency features from the participants' language. Models were primarily evaluated with the area under the receiver operating characteristic curve (AUC). The best discriminative ability was found when identifying depression with an SVM model (AUC = 0.77; 95% CI = 0.75-0.79), followed by anxiety with an LR model (AUC = 0.74; 95% CI = 0.72-0.76), and an SVM for suicide risk (AUC = 0.70; 95% CI = 0.68-0.72). Model performance was generally best with more severe depression, anxiety, or suicide risk. Performance improved when individuals with lifetime but no suicide risk in the past 3 months were considered controls. It is feasible to use a virtual platform to simultaneously screen for depression, anxiety, and suicide risk using a 5-to-10-min interview. The NLP/ML models performed with good discrimination in the identification of depression, anxiety, and suicide risk. Although the utility of suicide risk classification in clinical settings is still undetermined and suicide risk classification had the lowest performance, the result taken together with the qualitative responses from the interview can better inform clinical decision-making by providing additional drivers associated with suicide risk.

Development of a patients’ satisfaction analysis system using machine learning and lexicon-based methods

BackgroundPatients’ rights are integral to medical ethics. This study aimed to perform sentiment analysis and opinion mining on patients’ messages by a combination of lexicon-based and machine learning methods to identify positive or negative comments and to determine the different ward and staff names mentioned in patients’ messages.MethodsThe level of satisfaction and observance of the rights of 250 service recipients of the hospital was evaluated through the related checklists by the evaluator. In total, 822 Persian messages, composed of 540 negative and 282 positive comments, were collected and labeled by the evaluator. Pre-processing was performed on the messages and followed by 2 feature vectors which were extracted from the messages, including the term frequency–inverse document frequency (TFIDF) vector and a combination of the multifeature (MF) (a lexicon-based method) and TFIDF (MF + TFIDF) vectors. Six feature selectors and 5 classifiers were used in this study. For the evaluations, 5-fold cross-validation with different metrics including area under the receiver operating characteristic curve (AUC), accuracy (ACC), F1 score, sensitivity (SEN), specificity (SPE) and Precision-Recall Curves (PRC) were reported. Message tag detection, which featured different hospital wards and identified staff names mentioned in the study patients’ messages, was implemented by the lexicon-based method.ResultsThe best classifier was Multinomial Naïve Bayes in combination with MF + TFIDF feature vector and SelectFromModel (SFM) feature selection (ACC = 0.89 ± 0.03, AUC = 0.87 ± 0.03, F1 = 0.92 ± 0.03, SEN = 0.93 ± 0.04, and SPE = 0.82 ± 0.02, PRC-AUC = 0.97). Two methods of assessment by the evaluator and artificial intelligence as well as survey systems were compared.ConclusionOur results demonstrated that the lexicon-based method, in combination with machine learning classifiers, could extract sentiments in patients’ comments and classify them into positive and negative categories. We also developed an online survey system to analyze patients’ satisfaction in different wards and to remove conventional assessments by the evaluator.

Identification of offensive language in Urdu using semantic and embedding models.

Automatic identification of offensive/abusive language is very necessary to get rid of unwanted behavior. However, it is more challenging to generalize the solution due to the different grammatical structures and vocabulary of each language. Most of the prior work targeted western languages, however, one study targeted a low-resource language (Urdu). The prior study used basic linguistic features and a small dataset. This study designed a new dataset (collected from popular Pakistani Facebook pages) containing 7,500 posts for offensive language detection in Urdu. The proposed methodology used four types of feature engineering models: three are frequency-based and the fourth one is the embedding model. Frequency-based are either determined by the term frequency-inverse document frequency (TF-IDF) or bag-of-words or word n-gram feature vectors. The fourth is generated by the word2vec model, trained on the Urdu embeddings using a corpus of 196,226 Facebook posts. The experiments demonstrate that the stacking-based ensemble model with word2vec shows the best performance as a standalone model by achieving 88.27% accuracy. In addition, the wrapper-based feature selection method further improves performance. The hybrid combination of TF-IDF, bag-of-words, and word2vec feature models achieved 90% accuracy and 97% AUC. In addition, it outperformed the baseline with an improvement of 3.55% in accuracy, 3.68% in the recall, 3.60% in f1-measure, 3.67% in precision, and 2.71% in AUC. The findings of this research provide practical implications for commercial applications and future research.

Structuring of Unstructured Data from Heterogeneous Sources

Objectives: To develop a new data gathering processing under Big Data Perspectives. To convert unstructured text data into structured format by not missing out any text data available. Methods: The unstructured data is preprocessed using modified stemming and tokenization. From the stemming output, the proposed Term Frequency-Inverse Document Frequency (TF-IDF) and N-gram features are derived. Unstructured data is considered from multiple sources like twitter, consumer complaints and news blog. Findings: The proposed model with extant TF-IDF features has exposed relatively high Mean Average Error (MAE) value which is 1.4325 when compared to the proposed model without optimization to be 0.5197. Novelty: The novelty of the research work is of the stemming process where dictionary checking process is added and the improved feature extraction, interclass dispersion coefficient is computed in TF-IDF features. Keywords: Natural language processing; Structured data; Unstructured data; Big data; Feature extraction

Sentiment Analysis Against IndiHome and First Media Internet Providers Using Ensemble Stacking Method

Customer satisfaction is one of the factors that can be used to measure the success of service in a company. In the era of the 2000s until now, internet service providers have continued to grow throughout the world, including in Indonesia. IndiHome and First Media are companies that provide internet services that make it easy for the public to communicate and obtain information. With many uses of IndiHome and First Media internet services, there are often several obstacles that cause various responses from users. Users usually channel these responses to IndiHome or First Media customer care on Twitter. The dataset for this study was obtained from Twitter using the Twitter API and the Tweepy library. The dataset that has been collected is 6.962 tweets for the IndiHome dataset and 8,089 tweets for the First Media dataset. This study conducts sentiment analysis using the Ensemble Stacking with three base classifiers and a meta classifier. The base classifier used is Naïve Bayes, K-Nearest Neighbor, and Decision Tree, while the meta classifier used is Logistic Regression. This study uses the term frequency-inverse document frequency (TF-IDF) to determine the frequency value of a word in a document. This study uses two test scenarios: testing without oversampling and testing with oversampling on the dataset. The results show that Ensemble Stacking with term frequency-inverse document frequency feature extraction produces the highest accuracy, with an accuracy value of 88.27% on the IndiHome dataset and 92.56% on the First Media dataset by oversampling on both datasets.

Evaluation of clustering techniques on Urdu News head-lines: a case of short length text

ABSTRACT Document clustering and topic modelling methods are useful in determining groups of semantically similar documents. In literature, various studies have been conducted to evaluate the performance of lexical and semantic features on a clustering task. However, the focus of such studies was on the English language which is considered a high resource language. This paper analyses the effect of various feature extraction techniques combined with clustering and topic modelling algorithms on grouping news headlines written in Urdu, a low-resource language. Various features were first extracted using different approaches such as term-frequency inverse document frequency (TF-IDF), word embeddings learned using Word2vec and fastText models, deep contextualised word embeddings from BERT, and document embeddings extracted from the Doc2Vec model. Three different clustering algorithms: K-Means, Affinity Propagation and Density-based spatial clustering of applications with noise (DBSCAN) were then used to produce clusters for a given set of news headlines. The Latent Dirichlet Allocation topic model was also assessed for comparison with clustering algorithms. The evaluation based on extrinsic measures revealed that higher-quality clusters were produced when the K-Means algorithm was applied on a frequent unigrams-based TF-IDF feature matrix. In addition, several word embedding models have been trained on the domain of Urdu News headlines using state-of-the-art techniques such as Word2Vec, Doc2Vec, and fastText.

Classification of movie reviews using term frequency-inverse document frequency and optimized machine learning algorithms.

The Internet Movie Database (IMDb), being one of the popular online databases for movies and personalities, provides a wide range of movie reviews from millions of users. This provides a diverse and large dataset to analyze users’ sentiments about various personalities and movies. Despite being helpful to provide the critique of movies, the reviews on IMDb cannot be read as a whole and requires automated tools to provide insights on the sentiments in such reviews. This study provides the implementation of various machine learning models to measure the polarity of the sentiments presented in user reviews on the IMDb website. For this purpose, the reviews are first preprocessed to remove redundant information and noise, and then various classification models like support vector machines (SVM), Naïve Bayes classifier, random forest, and gradient boosting classifiers are used to predict the sentiment of these reviews. The objective is to find the optimal process and approach to attain the highest accuracy with the best generalization. Various feature engineering approaches such as term frequency-inverse document frequency (TF-IDF), bag of words, global vectors for word representations, and Word2Vec are applied along with the hyperparameter tuning of the classification models to enhance the classification accuracy. Experimental results indicate that the SVM obtains the highest accuracy when used with TF-IDF features and achieves an accuracy of 89.55%. The sentiment classification accuracy of the models is affected due to the contradictions in the user sentiments in the reviews and assigned labels. For tackling this issue, TextBlob is used to assign a sentiment to the dataset containing reviews before it can be used for training. Experimental results on TextBlob assigned sentiments indicate that an accuracy of 92% can be obtained using the proposed model.

ANALISIS SENTIMEN TANGGAPAN MASYARAKAT TERHADAPBANTUAN SOSIALPEMERINTAH DI MASA PANDEMI COVID-19 PADA PLATFORM TWITTER

The study was conducted to analyze public opinion about the government's efforts in overcoming the Covid- 19 pandemic by providing social assistance (bansos) in the form of goods, money and or services. Through social media Twitter with the topic of social assistance, the classification of positive, neutral or negative sentiments will be carried out. The results of this classification will reveal what social assistance topics are often discussed on Twitter. This classification process uses the Naive Bayes method and uses the RapidMiner application. The data used in this analysis process is 747 Twitter comment text data with a data collection time span from October to November. The classification process is supported by the Term Frequency-Inverse Document Frequency feature as the word weighting stage. This classification produces 2,382 word attributes or word vectors from 747 data, with 370 sample data for model testing which produces an accuration value of 24.32%, a true neutral recall value of 100%, and a true neutral precision value of 24.32%. The word that most often appears from the results of this sentiment analysis is the word "bantuan".

Sentiment analysis for covid-19 in Indonesia on Twitter with TF-IDF featured extraction and stochastic gradient descent

Twitter is an information platform that can be used by any internet user. The opinions of the Twitter Netizens are still random or unclassified. The technique for classifying sentiment analysis requires an algorithm. One of the classification algorithms is Stochastic Gradient Descent (SGD). The more training data provided to the machine, the accuracy of the classification function model formed by the machine is also higher. But in making representations into numerical vectors, the dimensions of data become large due to the many features. Feature optimization needs to be done to the training data by reducing the dimensions of the training data while maintaining high model accuracy. The optimization feature used is the TF-IDF (term frequency-inverse document frequency) feature extraction. sentiment analysis using TF-IDF feature extraction and stochastic gradient descent algorithm can classify Indonesian text appropriately according to positive and negative sentiment. Classification Performance using TF-IDF feature extraction and stochastic gradient descent algorithm obtained an accuracy is 85.141%.