Inverse Document Frequency Algorithm Research Articles

The mathematics approaches in the analysis of mathematics syllabuses similarities among Japanese and Bosnia and Herzegovina universities

The goal of our research is improvement of mathematics curriculum and popularization of mathematics among students of economics in developing countries. We analyze and compare curricula of pure mathematics courses that are taught to university students of faculties of economics in Japan and in Bosnia and Herzegovina. Data set contains math syllabuses in 2021/22 school year from six public universities in Bosnia and Herzegovina and seven from Japan. The text corpus was pre-processed and then the Term Frequency – Inverse Document Frequency algorithm, and Sentence Transformed Multi QA model were applied to build word vectors, find the similarity among Japanese and Bosnia and Herzegovina mathematics syllabuses using cosine similarity approach, and to find the key competences of these two countries mathematics syllabuses using the word cloud. Our results show the following similarity between the curricula: 60.7 percent using TF-IDF and 80.3 percent using Multi QA model. The key competences in the Japanese mathematics course are narrow and focused, in contrast to Bosnia and Herzegovina’s.

Sentiment Analysis and Twitter Social Media Visualization Regarding the Omnibus Law Draft

This study will classify Twitter users' positive and negative opinions about the omnibus method using a frequency-inverse document frequency algorithm and a multi-layer perceptron method. The sentiment analysis process involves several stages, including B. Collecting and preprocessing data, calculating term weights using inverse term frequencies and document frequencies, and classifying data using multi-layer perceptrons. Additionally, the study visually represents Twitter's sentiment analysis results on the omnibus method. These visualizations include word cloud, top accounts, tweet frequency, hashtags, and sentiment. Three scenarios were considered to perform the classification experiments. Scenario 1 used 700 training data, scenario 2 used 800, and Scenario 3 used 900 training data. The findings show that the Term Frequency Inverse Document Frequency algorithm and the multi-layer perceptron method are adequate for sentiment analysis, with Scenario 3 yielding the highest accuracy rate of 88%.

Implementation of K-Means Clustering Method for Trend Analysis of Thesis Topics (Case Study: Faculty of Computer Science, University of Jember)

The development of information technology causes a large number of digital documents, especially thesis documents, so that it can create opportunities for students to take the same and not varied topics. Thesis documents can be grouped by topic by identifying the abstract section. The results of the grouping can be seen with the trend with data visualization so that it can be analyzed to find out the trend of each topic. Retrieval of data in the repository of the University of Jember through a web scraping process as many as 490 thesis documents for students of the Faculty of Computer Science, University of Jember. The preprocessing stage is carried out by text mining methods which include cleaning, filtering, stemming, and tokenizing. Then calculate the weight of each word with the Term Frequency - Inverse Document Frequency algorithm, followed by the dimension reduction process using the Principal Component Analysis algorithm, which is normalized by Z-Score first. The outliers removal process is carried out before classifying documents. Furthermore, document grouping uses the K-Means Clustering method with Cosine Similarity as the distance calculation and the Silhouette Coefficient algorithm as a test. The test results were carried out with various k values and the optimal value was obtained at k = 2 with a Silhouette value of 0.80. Then the topic detection uses the Latent Dirichlet Allocation algorithm for each cluster that has been formed. Each cluster is visualized with a line chart and Trend Linear algorithm and analyzed to find out the trend. From the results of the analysis, it can be concluded that the topic of Decision Support System Development is trending down, and the topic of IT Performance Measurement and Forecasting is trending up. It can be concluded that the topic of Decision Support System Development needs to be reduced so that other topics can emerge.

Comparative Evaluation of NLP-Based Approaches for Linking CAPEC Attack Patterns from CVE Vulnerability Information

Vulnerability and attack information must be collected to assess the severity of vulnerabilities and prioritize countermeasures against cyberattacks quickly and accurately. Common Vulnerabilities and Exposures is a dictionary that lists vulnerabilities and incidents, while Common Attack Pattern Enumeration and Classification is a dictionary of attack patterns. Direct identification of common attack pattern enumeration and classification from common vulnerabilities and exposures is difficult, as they are not always directly linked. Here, an approach to directly find common links between these dictionaries is proposed. Then, several patterns, which are combinations of similarity measures and popular algorithms such as term frequency–inverse document frequency, universal sentence encoder, and sentence BERT, are evaluated experimentally using the proposed approach. Specifically, two metrics, recall and mean reciprocal rank, are used to assess the traceability of the common attack pattern enumeration and classification identifiers associated with 61 identifiers for common vulnerabilities and exposures. The experiment confirms that the term frequency–inverse document frequency algorithm provides the best overall performance.

Design and Implementation of a Big Data Evaluator Recommendation System Using Deep Learning Methodology

As the number of researchers in South Korea has grown, there is increasing dissatisfaction with the selection process for national research and development (R&D) projects among unsuccessful applicants. In this study, we designed a system that can recommend the best possible R&D evaluators using big data that are collected from related systems, refined, and analyzed. Our big data recommendation system compares keywords extracted from applications and from the full-text of the achievements of the evaluator candidates. Weights for different keywords are scored using the term frequency–inverse document frequency algorithm. Comparing the keywords extracted from the achievement of the evaluator candidates’, a project comparison module searches, scores, and ranks these achievements similarly to the project applications. The similarity scoring module calculates the overall similarity scores for different candidates based on the project comparison module scores. To assess the performance of the evaluator candidate recommendation system, 61 applications in three Review Board (RB) research fields (system fusion, organic biochemistry, and Korean literature) were recommended as the evaluator candidates by the recommendation system in the same manner as the RB’s recommendation. Our tests reveal that the evaluator candidates recommended by the Korean Review Board and those recommended by our system for 61 applications in different areas, were the same. However, our system performed the recommendation in less time with no bias and fewer personnel. The system requiresrevisions to reflect qualitative indicators, such as journal reputation, before it can entirely replace the current evaluator recommendation process.

Automatic extraction of Uyghur domain concepts based on multi‐feature for ontology extension

In the internet age, as a conceptual model of knowledge organisation, ontology has become a research hotspot. Ontology extension achieves the purpose of expanding ontology by adding new concepts and discovering the relationships between concepts in the existing ontology. In order to improve the automation and accuracy of ontology concept extraction in the Uyghur language, here the authors propose a new method to automatically extract concepts from text collection. For the characteristics of the Uyghur domain ontology concept, the text preprocessing is performed first, then the inter-word correlation of multi-feature fusion is calculated, such as Mi, Cd and Ea, and finally, the domain terminology and concept are automatically extracted, according to the term frequency–inverse document frequency algorithm. Experiment results show that, in terms of precision and recall rate, the multi-feature method proposed here represents an improvement over other methods. It also proves the feasibility and effectiveness of the authors’ method to extract the domain concept.

FastText-Based Local Feature Visualization Algorithm for Merged Image-Based Malware Classification Framework for Cyber Security and Cyber Defense

The importance of cybersecurity has recently been increasing. A malware coder writes malware into normal executable files. A computer is more likely to be infected by malware when users have easy access to various executables. Malware is considered as the starting point for cyber-attacks; thus, the timely detection, classification and blocking of malware are important. Malware visualization is a method for detecting or classifying malware. A global image is visualized through binaries extracted from malware. The overall structure and behavior of malware are considered when global images are utilized. However, the visualization of obfuscated malware is tough, owing to the difficulties encountered when extracting local features. This paper proposes a merged image-based malware classification framework that includes local feature visualization, global image-based local feature visualization, and global and local image merging methods. This study introduces a fastText-based local feature visualization method: First, local features such as opcodes and API function names are extracted from the malware; second, important local features in each malware family are selected via the term frequency inverse document frequency algorithm; third, the fastText model embeds the selected local features; finally, the embedded local features are visualized through a normalization process. Malware classification based on the proposed method using the Microsoft Malware Classification Challenge dataset was experimentally verified. The accuracy of the proposed method was approximately 99.65%, which is 2.18% higher than that of another contemporary global image-based approach.

Text visualization for construction document information management

In this study, text mining and visualization technology is applied to extract valuable information otherwise buried in the dense and abstract form of construction report text and process it into simple and intuitive graphics. This framework allows managers to quickly understand and make more informed decisions based upon key information. To extract such key information from a text automatically, the Term Frequency–Inverse Document Frequency algorithm was optimized to the characteristics of engineering texts. The key messages extracted from a case study construction report text in the form of keywords/terms were then visualized using a tag cloud algorithm. Questionnaires completed by construction managers then demonstrated that the proposed information visualization framework can facilitate rapid and informative access to key project information. This framework can thus reduce the workload and time required for construction managers to ascertain and act upon the status of their projects.

Effect of demographic structure on resource utilisation using term frequency–inverse document frequency algorithm – evidence from China

China has stepped into middle-income status, which is characterised by demographic imbalance and huge discrepancy of resource utilisation, resulting in an imbalance of regional development. Using term frequency–inverse document frequency algorithm, the authors extract the resource keywords and construct the resource utilisation index. The authors then explore the impact of demographic structure on resource utilisation. The main conclusions are as follows: (i) according to the Moran Index, there are strong spatial autocorrelations in resource utilisation at the provincial level in China; (ii) the empirical results demonstrate that the regression coefficients of spatial lags are significant. This provides compelling evidences that the neighbouring regions have obvious spatial spillover effects on local resource utilisation; (iii) the authors’ findings also reveal that the total dependency ratio which reflects the demographic structure has a negative and significant effect on resource utilisation. With other explanatory variables held constant, the child-age dependency ratio and old-age dependency ratio perform differently in the robustness testing. Ultimately, targeted and systematic policy suggestions are proposed to improve resource utilisation and adjust the demographic structure.