Web Mining Research Articles

To Study the Use of Entropy and the Preprocessing Step in Analyzing Web Users' Behavior and Improving Web Mining

The study of entropy's application to online use mining is the main goal of the work. By using the Reference Length approach, entropy provides an additional option for figuring out the ratio of auxiliary pages in the session identification. Two distinct online portals were used for the experiment. The first log file was downloaded via the web portal of a virtual learning environment course. The online gateway with anonymous access provided the second log file. When entropy and sitemap estimations of the ratio of auxiliary pages were compared, it was discovered that entropy could replace the sitemap estimate of the ratio of auxiliary pages with a full-valued substitute in the situation of sitemap abundance.

Data mining for the Analysis of Content Interaction in web-based Learning and Training Systems

The process of data mining is exploring databases for useful information and extracting it. Analysis how people interact with websites is the main goal of web usage mining, a subfield of data mining. The main, unobtrusive, and objective way to evaluate Web-based training and learning systems, namely how users engage with course materials, is web use mining. Many mining methods were developed with the classroom in mind, and we will showcase and explain them all. In order to prove effectiveness and improve instructional design as required, it is crucial to analyse and evaluate how learners behave in learning and training technology systems. This is particularly true when there are several interactive learning and training components accessible. We take a look at methods for figuring out what we want to learn, how we're going to learn it, and how our habits change as we go.

Toward Robust Graph Semi-Supervised Learning Against Extreme Data Scarcity.

The success of graph neural networks (GNNs) in graph-based web mining highly relies on abundant human-annotated data, which is laborious to obtain in practice. When only a few labeled nodes are available, how to improve their robustness is key to achieving replicable and sustainable graph semi-supervised learning. Though self-training is powerful for semi-supervised learning, its application on graph-structured data may fail because 1) larger receptive fields are not leveraged to capture long-range node interactions, which exacerbates the difficulty of propagating feature-label patterns from labeled nodes to unlabeled nodes and 2) limited labeled data makes it challenging to learn well-separated decision boundaries for different node classes without explicitly capturing the underlying semantic structure. To address the challenges of capturing informative structural and semantic knowledge, we propose a new graph data augmentation framework, augmented graph self-training (AGST), which is built with two new (i.e., structural and semantic) augmentation modules on top of a decoupled GST backbone. In this work, we investigate whether this novel framework can learn a robust graph predictive model under the low-data context. We conduct comprehensive evaluations on semi-supervised node classification under different scenarios of limited labeled-node data. The experimental results demonstrate the unique contributions of the novel data augmentation framework for node classification with few labeled data.

Penyebaran Mahasiswa ITB Ahmad Dahlan Jakarta Menggunakan Algoritma Clustering

The development of information technology has made it possible to collect and analyze large and complex data. In the context of higher education, information about student distribution patterns can be of added value for institutions to plan and manage their resources effectively. This research aims to design a web mining system that uses a clustering algorithm to grouping the distribution of students at ITB Ahmad Dahlan. In this study, we collected data from open web sources that are relevant to students, such as student profiles, academic preferences, and extracurricular activities. The data is then analyzed using a clustering algorithm to identify patterns and trends in student distribution. By using this approach, we hope to provide useful insights for educational institutions in planning infrastructure, academic programs, and student services.

Illegal Online Gambling Site Detection using Multiple Resource-Oriented Machine Learning.

The COVID-19 pandemic has led to faster digitalization and illegal online gambling has become popular. As illegal online gambling brings not only financial threats but also breaches in overall cyber security, this study defines the concept of absolute illegal online gambling (AIOG) using a machine-learning-driven approach with information gathered from public webpages. By analysing 11,172 sites to detect illegal online gambling, the proposed model classifies key features such as URLs (Uniform Resource Locator), WHOIS, INDEX, and landing page information. With a combination of text and image analyses with machine learning-driven approach, the proposed model offers the ensemble combination of attributes for high detection performance with the verification of common attributes from metadata in online gambling. This study suggests a strategy for dynamic resource utilization to increase the classification accuracy of the current environment. As a result, this research expands the scope of hybrid web mining through constant updating of data to achieve content-based filtering.

The analysis of firm web data for predicting company innovativeness: a comparison across different types of innovation

ABSTRACT This paper investigates which of the core types of innovation can be best predicted based on ’firms’ website data. In particular, we focus on four distinct key standard types of innovations in firms: product, process, organisational, and marketing innovation. Web mining of textual data on the websites of firms from Estonia, combined with the application of artificial intelligence (AI) methods, turned out to be a suitable approach to predict firm-level innovation indicators. The key novel addition to the existing literature is the finding that web mining is more applicable to predicting marketing innovation than the other three core types of innovation. As AI-based models are often black-box in nature, for transparency, we use an explainable AI approach (SHAP - SHapley Additive exPlanations), where we look at the most important words predicting a particular type of innovation. Our models confirm that the marketing innovation indicator from survey data was clearly related to marketing-related terms on the ’firms’ websites. In contrast, the results on the relevant words on websites for other innovation indicators were much less clear. Our analysis concludes that the effectiveness of web-scraping and web-text-based AI approaches in predicting cost-effective, granular and timely firm-level innovation indicators varies according to the type of innovation considered.

Exploring the Synergy of Web Usage Data and Content Mining for Personalized Effectiveness

In light of the exponential growth of web data and user volume, individuals are increasingly overwhelmed by information overload on the internet. Addressing this challenge, our study focuses on enhancing web information retrieval and presentation by leveraging web data mining techniques to uncover intrinsic relationships within textual, linkage, and usability data. Specifically, we aim to improve the performance of web information retrieval and presentation by analysing web data features. Our approach centres on web usage mining to identify usage patterns and integrate this knowledge with user profiles for personalized content delivery. Personalization, tailored to user’s characteristics and behaviours, serves to enhance engagement, conversion, and long-term commitment to websites. The objective of our research is to develop a web personalization system that enables users to access relevant website content without the need for explicit queries. This paper presents an extensive survey of various approaches proposed by researchers in the field of web personalization. It highlights the diverse methodologies and techniques employed to enhance user experience and engagement on the web. The paper identifies key challenges that require urgent attention to advance the field of web personalization.

Predicting the Students Performance Using WUM and Machine Learning Technique

Abstract: This project aims to develop a predictive model for assessing students’ academic performance by integrating web usage mining (WUM) and machine learning (ML) techniques. The model leverages user-provided data such as gender, parental profession, reading score, and writing score, along with behavioural insights from online educational platform interactions. The methodology includes data preprocessing, feature extraction, and adaptive algorithm selection to enhance prediction accuracy and adaptability across various student profiles. The integration of WUM provides a robust framework for educational institutions to tailor interventions and support systems effectively.

Enhancing E-Learning Adaptability with Automated Learning Style Identification and Sentiment Analysis: A Hybrid Deep Learning Approach for Smart Education

In smart education, adaptive e-learning systems personalize the educational process by tailoring it to individual learning styles. Traditionally, identifying these styles relies on learners completing surveys and questionnaires, which can be tedious and may not reflect their true preferences. Additionally, this approach assumes that learning styles are fixed, leading to a cold-start problem when automatically identifying styles based on e-learning platform behaviors. To address these challenges, we propose a novel approach that annotates unlabeled student feedback using multi-layer topic modeling and implements the Felder–Silverman Learning Style Model (FSLSM) to identify learning styles automatically. Our method involves learners answering four FSLSM-based questions upon logging into the e-learning platform and providing personal information like age, gender, and cognitive characteristics, which are weighted using fuzzy logic. We then analyze learners’ behaviors and activities using web usage mining techniques, classifying their learning sequences into specific styles with an advanced deep learning model. Additionally, we analyze textual feedback using latent Dirichlet allocation (LDA) for sentiment analysis to enhance the learning experience further. The experimental results demonstrate that our approach outperforms existing models in accurately detecting learning styles and improves the overall quality of personalized content delivery.

Web mining and sentiment analysis of COVID-19 discourse in online forum communities

Recently, various discussions, solutions, data, and methods related to coronavirus disease 2019 (COVID-19) have been posted in online forum communities. Although a vast amount of posting on COVID-19 analytical projects are available in the online forum communities, much of them remain untapped due to limited overview and profiling that focuses on COVID-19 analytic techniques. Thus, it is quite challenging for information diggers and researchers to distinguish the recent trends and challenges of COVID-19 analytic for initiating different and critical studies to fight against the coronavirus. This paper presents the findings of a study that executed a web mining process on COVID-19 data analytical projects from the Stack Overflow and GitHub online community platforms for data scientists. This study provides an insight on what activities can be conducted by novice researchers and others who are interested in data analysis, especially in sentiment analysis. The classification results via Naïve Bayes (NB), support vector machine (SVM) and logistic regression (LR) have returned high accuracy, indicating that the constructed model is efficient in classifying the sentiment data of COVID-19. The findings reported in this paper not only enhance the understanding of COVID-19 related content and analysis but also provides promising framework that can be applied in diverse contexts and domains.

WEB MINING TO DETECT ONLINE SPREAD OF TERRORISM

To fight online terrorism, integrating web mining techniques with data mining algorithms is crucial. Various algorithms like Logistic Regression, K-Nearest Neighbors (KNN), Support Vector Machines (SVM), Naive Bayes (NB), Decision Trees (DT), Random Forests (RF), and Gradient Boosting (GB) are studied for detecting terrorist activities online, extracting data, identifying patterns, and relevant information. Logistic Regression provides a probabilistic framework, KNN uses similarity metrics, SVM constructs hyper-planes, NB assumes feature independence, DT builds decision trees, RF applies ensemble learning, and GB boosts weak learners' performance. These algorithms aid in proactive monitoring and prevention of online terrorism through efficient analysis of structured and unstructured web data. By combining web mining and data mining strengths, this study emphasizes a comprehensive approach to combatting online terrorism dissemination, helping security agencies anticipate evolving threats and prevent terrorist propagation effectively. Key Words: Terrorism, Naive-bayes, random forest, web mining, Gradient boosting

Practical network topology in the study of online radicalisation of youth: opportunities and limitations

The paper presents key approaches to understanding and researching radicalisation, as well as the opportunities and limitations of applying some research methods to model network topology and assess content similarity of online communities. Today, Web Mining and AI methods and technologies are often applied in research on social networks and youth participation in them. However, the question how these approaches can be effectively used to study online radicalisation remains open. The answer to this question should increase the explanatory and predictive power of computational models for detecting and predicting radicalisation in the online space. In much of the Russian research on online radicalisation, a common approach has been to reduce the task of identifying the interconnectedness of individual online communities or clusters of them to assessing the degree of similarity in terms of subscribers or linguistic markers. This approach is limited in predicting new connections between communities and justifying radicalisation pathways, but is relevant in modelling information diffusion. In this paper, the authors aim to demonstrate the possibilities and limitations of applying the tf-idf, doc2vec methods to assess the content similarity of online communities without signs of radicalisation and online communities with signs of radicalisation. This approach allowed the authors to identify communities with a significant tendency to unite (to establish direct links). The paper presents the results of the comparative study in the form of social graphs formed according to the principles of subscriber commonality, similarity of significant words, and contextual similarity based on the doc2vec model. The social graph based on doc2vec method performed better in terms of clustering of online communities as well as interpretability of the results. This is crucial for detecting and predicting radicalisation online, as it opens the prospect of exploring the nature of assortativity in the observed network.

E-commerce Webpage Recommendation Scheme Base on Semantic Mining and Neural Networks

In e-commerce websites, web mining web page recommendation technology has been widely used. However, recommendation solutions often cannot meet the actual application needs of online shopping users. To address this problem, this paper proposes an e-commerce web page recommendation solution that combines semantic web mining and BP neural networks. First, the web logs of user searches are processed and 5 features are extracted: content priority, time consumption priority, online shopping users' explicit/implicit feedback on the website, recommendation semantics and input deviation amount. Then, these features are used as input features of the BP neural network to classify and identify the priority of the final output web page. Finally, the web pages are sorted according to priority and recommended to users. This project uses book sales webpages as samples for experiments. The results show that this solution can quickly and accurately identify the webpages required by users.

Machine learning and deep learning-based sentiment analysis of IMDB user reviews

Movie reviews have always been a popular and enduring subject of interest among researchers. Sentiment analysis plays a significant role in this domain. The utilization of machine learning and natural language processing techniques can provide valuable insights into the emotional responses of audiences towards movies, as well as facilitate the appraisal of their reputation and market potential. This is achieved through the analysis of sentiment expressed in movie reviews. Furthermore, this approach is highly valuable in various application domains such as data mining, web mining, and social media analysis. This paper aims to conduct a comparative analysis by utilizing typical models based on machine learning and neural networks, along with the integration of natural language processing techniques. The IMDB database, which contains 50,000 reviews, will be used, and data preprocessing will be performed before applying these models. By comparing the accuracy of each model, insights regarding movie reviews can be derived.

Multi-Instance Learning with One Side Label Noise

Multi-instance Learning (MIL) is a popular learning paradigm arising from many real applications. It assigns a label to a set of instances, which is called a bag, and the bag’s label is determined by the instances within it. A bag is positive if and only if it has at least one positive instance. Since labeling bags is more complicated than labeling each instance, we will often face the mislabeling problem in MIL. Furthermore, it is more common that a negative bag has been mislabeled to a positive one, since one mislabeled instance will lead to the change of the whole bag label. This is an important problem that originated from real applications, e.g., web mining and image classification, but little research has concentrated on it as far as we know. In this article, we focus on this MIL problem with one side label noise that the negative bags are mislabeled as positive ones. To address this challenging problem, we propose, to the best our our knowledge, a novel multi-instance learning method with one side label noise. We design a new double weighting approach under traditional framework to characterize the “faithfulness” of each instance and each bag in learning the classifier. Briefly, on the instance level, we employ a sparse weighting method to select the key instances, and the MIL problem with one size label noise is converted to a mislabeled supervised learning scenario. On the bag level, the weights of bags, together with the selected key instances, will be utilized to identify the real positive bags. In addition, we have solved our proposed model by an alternative iteration method with proved convergence behavior. Empirical studies on various datasets have validated the effectiveness of our method.

Exploring the Relationship Between Academic Science and Economics Through Bio-fuel Research: A Scientometric Analysis

Scientometrics is increasingly wielded as a powerful tool in shaping scientific policies, impacting the allocation of funding for projects and institutions by assessing priorities, viewpoints, and capabilities. The investigation of the relationship between academic science and economics is one of the most recent developments in the field of scientific metrics study. It assumes the role of an analysis method of knowledge production and dissemination in innovation systems. These results suggest that biofuel positively affected research collaboration and, as a result, scientific performance. HistCite is one of the study’s tools for analyzing data in a clear and concise manner. Manual searches were conducted on their websites to gather a condensed overview of their data for data mining purposes. Additionally, the Web of Science (WOS) database was utilized for research visualization. Vos Viewer and MS Excel were employed to create graph-like visualizations of data, particularly focusing on the key term 'biofuel' to showcase the field’s research focus and productivity rankings in this area. Statistical techniques and web mining were employed to refine and extract pertinent information. The study period was 33 years (1989-2022), and the results are presented in this paper. This may help to show that the continual growth of plants on our planet greatly outweighs men’s fundamental energy consumption when considering ecological, technical, and economic considerations; only a portion of the biomass that grows can be used to generate energy.

Neural natural language processing for long texts: A survey on classification and summarization

The adoption of Deep Neural Networks (DNNs) has greatly benefited Natural Language Processing (NLP) during the past decade. However, the demands of long document analysis are quite different from those of shorter texts, while the ever increasing size of documents uploaded online renders automated understanding of lengthy texts a critical issue. Relevant applications include automated Web mining, legal document review, medical records analysis, financial reports analysis, contract management, environmental impact assessment, news aggregation, etc. Despite the relatively recent development of efficient algorithms for analyzing long documents, practical tools in this field are currently flourishing. This article serves as an entry point into this dynamic domain and aims to achieve two objectives. First of all, it provides an introductory overview of the relevant neural building blocks, serving as a concise tutorial for the field. Secondly, it offers a brief examination of the current state-of-the-art in two key long document analysis tasks: document classification and document summarization. Sentiment analysis for long texts is also covered, since it is typically treated as a particular case of document classification. Consequently, this article presents an introductory exploration of document-level analysis, addressing the primary challenges, concerns, and existing solutions. Finally, it offers a concise definition of “long text/document”, presents an original overarching taxonomy of common deep neural methods for long document analysis and lists publicly available annotated datasets that can facilitate further research in this area.

Evaluating Datalog Tools for Meta-reasoning over OWL 2 QL

Abstract Metamodeling is a general approach to expressing knowledge about classes and properties in an ontology. It is a desirable modeling feature in multiple applications that simplifies the extension and reuse of ontologies. Nevertheless, allowing metamodeling without restrictions is problematic for several reasons, mainly due to undecidability issues. Practical languages, therefore, forbid classes to occur as instances of other classes or treat such occurrences as semantically different objects. Specifically, meta-querying in SPARQL under the Direct Semantic Entailment Regime uses the latter approach, thereby effectively not supporting meta-queries. However, several extensions enabling different metamodeling features have been proposed over the last decade. This paper deals with the Metamodeling Semantics (MS) over OWL 2 QL and the Metamodeling Semantic Entailment Regime (MSER), as proposed in Lenzerini et al. (2015, Description Logics) and Lenzerini et al. (2020, Information Systems 88, 101294), Cima et al. (2017, Proceedings of the 7th International Conference on Web Intelligence, Mining and Semantics, 1–6). A reduction from OWL 2 QL to Datalog for meta-querying was proposed in Cima et al. (2017, Proceedings of the 7th International Conference on Web Intelligence, Mining and Semantics, 1–6). In this paper, we experiment with various logic programming tools that support Datalog querying to determine their suitability as back-ends to MSER query answering. These tools stem from different logic programming paradigms (Prolog, pure Datalog, Answer Set Programming, Hybrid Knowledge Bases). Our work shows that the Datalog approach to MSER querying is practical also for sizeable ontologies with limited resources (time and memory). This paper significantly extends Qureshi and Faber (2021, International Joint Conference on Rules and Reasoning, Springer, 218–233.) by a more detailed experimental analysis and more background.

Web Mining of Online Resources for German Labor Market Research and Education: Finding the Ground Truth?

The labor market is highly dependent on vocational and academic education, training, retraining, and further education in order to master challenges such as advancing digitalization and sustainability. Further training is a key factor in ensuring a qualified workforce, the employability of all employees, and, thus, national competitiveness and innovation. In the contribution at hand, we explore an innovative way to derive knowledge about learning pathways by connecting the dots from different data sources of the German labor market. In particular, we focus on the web mining of online resources for German labor market research and education, such as online advertisements, information portals, and official government websites. A key question for working with different data sources is how to find the ground truth and common data structures that can be used to make the data interoperable. We discuss how to classify and summarize web data from different platforms and which methods can be used for extracting data, entities and relationships from online resources on the German labor market to build a network of educational pathways. Our proposed solution is based on the classification of occupations (KldB) and related document codes (DKZ), and combines natural language processing and knowledge graph technologies. Our research provides the foundation for further investigation into educational pathways and linked data for labor market research. While our work focuses on German data, it is also useful for other German-speaking countries and could easily be extended to other languages such as English.

A Survey on Machine Learning Based Classification Algorithms for Web Usage Mining

Abstract: Machine learning as well as data mining is research areas of computer technology whose quick expansion is due to the advances in data analysis research, growth in the database industry as well as the resulting market requirements for methods that are accomplished of extracting valuable knowledge from large data stores. It is a powerful platform that stores as well as retrieves mass information besides it becomes a time consuming uncomfortable task to search the information due to unstructured as well as heterogeneous nature of data on the web development. Web usage mining is one of the popular techniques of data mining that is utilized to discover besides quotation useful information from web document besides its service. This survey paper, conduct a comprehensive overview of the state of the art methods, clustering based machine learning based algorithms and its advantages and disadvantages of web usage mining.