Web Mining Techniques Research Articles

WEB MINING TO DETECT ONLINE SPREAD OF TERRORISM

To fight online terrorism, integrating web mining techniques with data mining algorithms is crucial. Various algorithms like Logistic Regression, K-Nearest Neighbors (KNN), Support Vector Machines (SVM), Naive Bayes (NB), Decision Trees (DT), Random Forests (RF), and Gradient Boosting (GB) are studied for detecting terrorist activities online, extracting data, identifying patterns, and relevant information. Logistic Regression provides a probabilistic framework, KNN uses similarity metrics, SVM constructs hyper-planes, NB assumes feature independence, DT builds decision trees, RF applies ensemble learning, and GB boosts weak learners' performance. These algorithms aid in proactive monitoring and prevention of online terrorism through efficient analysis of structured and unstructured web data. By combining web mining and data mining strengths, this study emphasizes a comprehensive approach to combatting online terrorism dissemination, helping security agencies anticipate evolving threats and prevent terrorist propagation effectively. Key Words: Terrorism, Naive-bayes, random forest, web mining, Gradient boosting

Exploring the impact of ChatGPT on education: A web mining and machine learning approach

ChatGPT, an artificial intelligence model, has garnered significant interest within education. This study examined public sentiment regarding ChatGPT's influence on education by utilizing web mining and natural language processing (NLP) techniques. By adopting an empirical approach and leveraging machine learning models to process 2003 web articles, the study extracts valuable insights. The results indicate that ChatGPT has emerged as a crucial educational tool, offering advantages for both students and educators. Notably, the study emphasized ChatGPT's role in enhancing students' writing abilities and fostering dynamic, interactive learning environments. ChatGPT's capacity to address a broad spectrum of questions demonstrates its versatility and adaptability, contributing to more inclusive and personalized educational experiences. However, the study also uncovered challenges tied to academic integrity, such as plagiarism and cheating, which stem from incorporating AI-driven tools like ChatGPT into education. This raises concerns regarding ethical aspects, including responsible AI usage and data privacy, and highlights the need for institutions to develop guidelines and policies for AI tool implementation in education. This study's findings hold theoretical and practical implications for integrating ChatGPT into educational settings. It is the first to employ web mining and NLP techniques to analyze public opinions on ChatGPT's impact on education comprehensively.

Semantic Annotation Based Mechanism for Web Service Discovery and Recommendation

Web Mining is regarded as one among the data mining techniques that aids in fetching and extraction of necessary data from the web. Conversely, Web usage mining discovers and extracts essential patterns usage over the webs which are being further utilized by various web applications. In order to discover and explore web services that are registered with documents of Web Services-Inspection, Discovery and Integration registry, Universal Description wants specific search circumstance similar to URL, category and service name. The document of Web Service Description Language (WSDL) offers a condition of the web services customers to take out operations, communications and the service format of right message. Therefore, WSDL is being utilized together with semantic explanation dependent substantiation for the extraction of different web services for related purpose, other supporting operations and attributes. The reason is that there subsist different web services having corresponding functionalities however altered or changeable attributes that are non–functional. Resultant, recognize the preeminent web service become tiresome for the user. A method is projected which caters the analysis of service resemblance with the aid of semantic annotation and machine learning (ML) algorithms depending on the analysis intended for enhancing the classification through capturing useful web services semantics related with real world. The emphasizes on the research technique of choosing preeminent web service for the user based on the semantic annotation. The research work in turn recommends a web mining technique that determines the best web service automatically thus ranking concepts in service textual documentation and classifies services on behalf of particular domains. Parallel computation is made easy with web services. The different management stages in the system of recommendation entail collection of dataset through WSDL on the semantic annotation basis, thereby recognizing the best service with the DOBT-Dynamic operation dependent discovering method, ranking through mechanisms MDBR - Multi-Dimensional based ranking, recommendation and classification. In this work, it has been employed a combination of fundamental ML estimators, namely Multinomial Naive Bayes (MNB) and Support Vector Machines (SVM), as well as ensemble techniques such as Bagging, Random Forests, and AdaBoost, to perform classification of Web services. It was observed from the investigate work that the adapted system of best web services recommendation defers high performance in contradiction of the existing recommendation technique regarding accuracy, efficiency in addition to processing time.

Predicting prices of case furniture products using web mining techniques

Predicting prices of case furniture products using web mining techniques

Analyzing Persian Wikipedia's citations to discover the effectiveness of Persian scientific papers: applied web mining techniques

PurposeThe main objective of this study is to evaluate the impact of Persian Scientific Papers (PSPs) on Persian Wikipedia by studying Wikipedia's citations to these papers.Design/methodology/approachThe present study is applied research, which has been performed by the web-mining method, such as downloading web pages, extracting information (references), identifying papers, detecting peer-review journals and calculating the frequency rates. The statistical population included 10,000 Persian Wikipedia Pages (PWPs) that were analyzed in two rounds with a six-month interval.FindingsThe number of pages containing the Persian references section was 3,994 and 4,063 out of the 10,000 pages extracted in the first and second rounds. The ratio of pages that cited scientific sources (58 and 67 pages) to the pages extracted from the PWP was equal to 0.58 and 0.67%. The ratio of pages that cited scientific sources to pages with Persian references in each round was equal to 1.45 and 1.64%. The number of references extracted from the PWP in each round equaled 30,441 and 35,891. Eight titles from reputable Persian journals had received at least three citations from Wikipedia.Originality/valueThe present study has determined the extent of interaction between science and society (knowledge flow) in the form of citations from Wikipedia articles to articles in peer-reviewed journals. The study of this issue in Persian Wikipedia in more than 2000 Persian peer-reviewed journals shows the originality of the present paper. Studying citation reliability in a collaborative and openly editable platform is another originality of the work.

Enhancing Learning Object Analysis through Fuzzy C-Means Clustering and Web Mining Methods

The development of learning objects (LO) and e-pedagogical practices has significantly influenced and changed the performance of e-learning systems. This development promotes a genuine sharing of resources and creates new opportunities for learners to explore them easily. Therefore, the need for a system of categorization for these objects becomes mandatory. In this vein, classification theories combined with web mining techniques can highlight the performance of these LOs and make them very useful for learners. This study consists of two main phases. First, we extract metadata from learning objects, using the algorithm of Web exploration techniques such as feature selection techniques, which are mainly implemented to find the best set of features that allow us to build useful models. The key role of feature selection in learning object classification is to identify pertinent features and eliminate redundant features from an excessively dimensional dataset. Second, we identify learning objects according to a particular form of similarity using Multi-Label Classification (MLC) based on Fuzzy C-Means (FCM) algorithms. As a clustering algorithm, Fuzzy C-Means is used to perform classification accuracy according to Euclidean distance metrics as similarity measurement. Finally, to assess the effectiveness of LOs with FCM, a series of experimental studies using a real-world dataset were conducted. The findings of this study indicate that the proposed approach exceeds the traditional approach and leads to viable results. Doi: 10.28991/ESJ-2023-07-03-010 Full Text: PDF

Computer Vision-Based Approach for Automatic Detection of Dairy Cow Breed

Purpose: Identification of individual cow breeds may offer various farming opportunities for disease detection, disease prevention and treatment, fertility and feeding, and welfare monitoring. However, due to the large population of cows with hundreds of breeds and almost identical visible appearance, their exact identification and detection become a tedious task. Therefore, the automatic detection of cow breeds would benefit the dairy industry. This study presents a computer-vision-based approach for identifying the breed of individual cattle. Methods: In this study, eight breeds of cows are considered to verify the classification process: Afrikaner, Brown Swiss, Gyr, Holstein Friesian, Limousin, Marchigiana, White Park, and Simmental cattle. A custom dataset is developed using web-mining techniques, comprising 1835 images grouped into 238, 223, 220, 212, 253, 185, 257, and 247 images for individual breeds. YOLOv4, a deep learning approach, is employed for breed classification and localization. The performance of the YOLOv4 algorithm is evaluated by training the model on different sets of training parameters. Results: Comprehensive analysis of the experimental results reveal that the proposed approach achieves an accuracy of 81.07%, with maximum kappa of 0.78 obtained at an image size of 608 × 608 and an intersection over union (IoU) threshold of 0.75 on the test dataset. Conclusions: The model performed better with YOLOv4 relative to other compared models. This places the proposed model among the top-ranked cow breed detection models. For future recommendations, it would be beneficial to incorporate simple tracking techniques between video frames to check the efficiency of this work.

A Semantically Enhanced Deep Neural Network Framework for Reputation System in Web Mining for Covid-19 Twitter Dataset

With the web containing a huge amount of information, the extraction of application oriented understandable data has become easier with web mining. Web Mining is the area that is derived from data mining. Unlike data mining, web mining is used to extract interesting patterns from the information available on the web. When used with deep learning, the pattern recognition becomes much easier. Deep learning works in the same way, how a human brain works in terms of predicting the outcomes when a bulk of information is provided. It deals with mathematical models that recognize the patterns efficiently. The different types of web mining techniques, namely: web content mining (WCM), web structure mining (WSM) and web usage mining (WUM) persists. Researchers and economists around the globe are keen in knowing the impact of the pandemic on the society’s economic status; this work helps find the same using reputation system. As twitter is a hub of different opinions of public, we work with covid- 19 data set from twitter. A reputation system helps finding the socio economic status of the tweets regarding covid-19 dataset. This paper has proposed a framework in which the web mining is implemented using a semantic enhanced deep neural network technique for the reputation system.

Pillar 3: Does banking regulation support stakeholders' interest in banks financial and risk profile?

The paper examines the interest of the commercial banks’ stakeholders in Pillar 3 disclosures and their behaviour during the timing of serious market turbulence. The aim is to discover to which extent current banking regulation supports stakeholders’ interest in the information required by regulators to be disclosed. The examined data consists of log files that were pre-processed using web mining techniques and from which were extracted frequent item sets by quarters and evaluated in terms of quantity. The authors have proposed a methodology to evaluate frequent item sets of web parts over a dedicated time. Based on the verification of applied methodology on two commercial banks, the results show that stakeholders’ interest in disclosures is highest in the first quarter at each year and after turbulent times in 2009 their interests decreased. Moreover, the results suggest that stakeholders expressed higher interest than in regulatory required Pillar 3 information in the following group of information: Pillar3 related information, Annual reports, Information on Group. Following our results, the paper contributes to cover the gap in the research by analysing Pillar 3 disclosures and their compliance with regulatory requirements, which also increase the interest of the relevant stakeholders to conduce them as an effective market discipline tool.

Hyphe, a Curation-Oriented Approach to Web Crawling for the Social Sciences

The web is a field of investigation for social sciences, and platform-based studies have long proven their relevance. However the generic web is rarely studied in itself though it contains crucial aspects of the embodiment of social actors: personal blogs, institutional websites, hobby-specific media… We realized that some sociologists see existing web crawlers as “black boxes” unsuitable for research though they are willing to study the broad web. In this paper we present Hyphe, a crawler developed with and for social scientists, with an innovative “curation-oriented” approach. We expose the problems of using web-mining techniques in social science research and how to overcome those by specific features such as step-by-step corpus building and a memory structure allowing researchers to redefine dynamically the granularity of their “web entities”.

Web usage analysis of Pillar 3 disclosed information by deposit customers in turbulent times

Market discipline has been a scrutinized area since the last financial crisis in 2008. Regulators strengthened their role particularly through Pillar 3 in Basel III. However, there are still some aspects of market discipline that deserve special attention to avoid future failures. This study focuses on the analysis of the interest and behaviour of deposit stakeholders based on website data dedicated to disclosures of commercial bank in Slovakia during and after turbulent times (period 2009–2012). The data consists of log files, and web mining techniques were applied (the modelling of web user behaviour in dependence on time - based on the proposals of the authors). The results show that also in turbulent times, stakeholders’ interest in Pillar 3 disclosures is low (in line with (Munk, Pilkova, Benko, & Blažeková, 2017)) and the highest interest was identified for the Pricing List category. After turbulent times, Pillar 3 categories (Pillar 3 related information and Pillar 3 disclosures) have weak interest, with peaks at the beginning of the year, and the highest increase was in the Business Conditions category. The results suggest that the enhancement of interest of key stakeholders in disclosures inevitably requires changes to deliver sufficient disclosure data structures and to design a disclosure policy that fulfils regulatory expectations.

Recommendation of a Webpage by using Web Mining Technique

In today's world of the Internet, different varieties of content are created in enormous amounts, so to give pertinent outcomes to clients, web suggestions become a significant piece of web applications. Various web proposals are made accessible to clients consistently, including pictures, recordings, Audio, question ideas, and site pages. In our research, we are targeting giving a system to website page proposals. 1) describing the fundamental technique of web mining 2) Web mining technique explanation.3) We propose the engineering for the customized website page suggestion

Web Mining Techniques and Technologies: A Landscape View

Web mining is the process of discovering important information and patterns in a weblog file. And contains loads of information. It consists of pages of various formats (HTML documents, images, etc.). And the data for this record is still increasing at a very high rate. So the process of retrieving information from it is a very difficult task. In order to obtain the required information from the network. This requires mining the weblog file. In this paper, the researcher conducted a review of previous studies and what researchers have dealt with in web prospecting. Explain web mining concepts. And how to obtain information and the visitors’ patterns through the techniques and algorithms used in Web usage mining.

A Review on Design and Development Of Sequential Patterns Algorithms In Web Usage Mining

In the recent years with the advancement in technology, a lot of information is available in different formats and extracting the knowledge from that data has become a very difficult task. Due to the vast amount of information available on the web, users are finding it difficult to extract relevant information or create new knowledge using information available on the web. To solve this problem Web mining techniques are used to discover the interesting patterns from the hidden data .Web Usage Mining (WUM), which is one of the subset of Web Mining helps in extracting the hidden knowledge present in the Web log files , in recognizing various interests of web users and also in discovering customer behaviours. Web Usage mining includes different phases of data mining techniques called Data Pre-processing, Pattern Discovery & Pattern Analysis. This paper presents an updated focused survey on various sequential pattern mining algorithms like apriori-based algorithm , Breadth First Search-based strategy, Depth First Search strategy, sequential closed-pattern algorithm and Incremental pattern mining algorithm which are used in Pattern Discovery Phase of WUM. At last , a comparison is done based on the important key features present in these algorithms. This study gives us better understanding of the approaches of sequential pattern mining.

Persian sentiment analysis of an online store independent of pre-processing using convolutional neural network with fastText embeddings.

Sentiment analysis plays a key role in companies, especially stores, and increasing the accuracy in determining customers’ opinions about products assists to maintain their competitive conditions. We intend to analyze the users’ opinions on the website of the most immense online store in Iran; Digikala. However, the Persian language is unstructured which makes the pre-processing stage very difficult and it is the main problem of sentiment analysis in Persian. What exacerbates this problem is the lack of available libraries for Persian pre-processing, while most libraries focus on English. To tackle this, approximately 3 million reviews were gathered in Persian from the Digikala website using web-mining techniques, and the fastText method was used to create a word embedding. It was assumed that this would dramatically cut down on the need for text pre-processing through the skip-gram method considering the position of the words in the sentence and the words’ relations to each other. Another word embedding has been created using the TF-IDF in parallel with fastText to compare their performance. In addition, the results of the Convolutional Neural Network (CNN), BiLSTM, Logistic Regression, and Naïve Bayes models have been compared. As a significant result, we obtained 0.996 AUC and 0.956 F-score using fastText and CNN. In this article, not only has it been demonstrated to what extent it is possible to be independent of pre-processing but also the accuracy obtained is better than other researches done in Persian. Avoiding complex text preprocessing is also important for other languages since most text preprocessing algorithms have been developed for English and cannot be used for other languages. The created word embedding due to its high accuracy and independence of pre-processing has other applications in Persian besides sentiment analysis.

A Study on Different Aspects of Web Mining and Research Issues

Web is huge and dynamically increasing day by day with exponential rate. If query is little bit complex then often or sometimes it becomes difficult to get relevant information in response. Several search engines try to optimize the users query and/or search results to provide the relevant information. Also many websites are ecommerce sites which are also competing to attract their customers and some are recommendation based. Now almost all good sites are using web mining techniques to improve their performance. In this paper, an attempt has been made to cover the key aspects used in web mining research. Initially problems faced by users and/or owners have been highlighted. Further it discusses types of data and techniques used by different categories of web mining are discussed. Finally research issues and application areas have been discussed.

HPM: A Hybrid Model for User’s Behavior Prediction Based on N-Gram Parsing and Access Logs

The continuous growth of the World Wide Web has led to the problem of long access delays. To reduce this delay, prefetching techniques have been used to predict the users’ browsing behavior to fetch the web pages before the user explicitly demands that web page. To make near accurate predictions for users’ search behavior is a complex task faced by researchers for many years. For this, various web mining techniques have been used. However, it is observed that either of the methods has its own set of drawbacks. In this paper, a novel approach has been proposed to make a hybrid prediction model that integrates usage mining and content mining techniques to tackle the individual challenges of both these approaches. The proposed method uses N-gram parsing along with the click count of the queries to capture more contextual information as an effort to improve the prediction of web pages. Evaluation of the proposed hybrid approach has been done by using AOL search logs, which shows a 26% increase in precision of prediction and a 10% increase in hit ratio on average as compared to other mining techniques.

Exploring the Potentialities of Automatic Extraction of University Webometric Information

Abstract Purpose The main objective of this work is to show the potentialities of recently developed approaches for automatic knowledge extraction directly from the universities’ websites. The information automatically extracted can be potentially updated with a frequency higher than once per year, and be safe from manipulations or misinterpretations. Moreover, this approach allows us flexibility in collecting indicators about the efficiency of universities’ websites and their effectiveness in disseminating key contents. These new indicators can complement traditional indicators of scientific research (e.g. number of articles and number of citations) and teaching (e.g. number of students and graduates) by introducing further dimensions to allow new insights for “profiling” the analyzed universities. Design/methodology/approach Webometrics relies on web mining methods and techniques to perform quantitative analyses of the web. This study implements an advanced application of the webometric approach, exploiting all the three categories of web mining: web content mining; web structure mining; web usage mining. The information to compute our indicators has been extracted from the universities’ websites by using web scraping and text mining techniques. The scraped information has been stored in a NoSQL DB according to a semi-structured form to allow for retrieving information efficiently by text mining techniques. This provides increased flexibility in the design of new indicators, opening the door to new types of analyses. Some data have also been collected by means of batch interrogations of search engines (Bing, www.bing.com) or from a leading provider of Web analytics (SimilarWeb, http://www.similarweb.com). The information extracted from the Web has been combined with the University structural information taken from the European Tertiary Education Register (https://eter.joanneum.at/#/home), a database collecting information on Higher Education Institutions (HEIs) at European level. All the above was used to perform a clusterization of 79 Italian universities based on structural and digital indicators. Findings The main findings of this study concern the evaluation of the potential in digitalization of universities, in particular by presenting techniques for the automatic extraction of information from the web to build indicators of quality and impact of universities’ websites. These indicators can complement traditional indicators and can be used to identify groups of universities with common features using clustering techniques working with the above indicators. Research limitations The results reported in this study refers to Italian universities only, but the approach could be extended to other university systems abroad. Practical implications The approach proposed in this study and its illustration on Italian universities show the usefulness of recently introduced automatic data extraction and web scraping approaches and its practical relevance for characterizing and profiling the activities of universities on the basis of their websites. The approach could be applied to other university systems. Originality/value This work applies for the first time to university websites some recently introduced techniques for automatic knowledge extraction based on web scraping, optical character recognition and nontrivial text mining operations (Bruni & Bianchi, 2020).

Spam Audit Detection Through Content Analysis

Nowadays e-commerce has gained recognition in day to day life; hence network became tremendous source for gathering customer reviews/opinions by marketplace analyzers. The count of user reviews that merchandise receives is increasing at high velocity. Opinions being posted on social media differ significantly in superiority. The client needs to essentially go through all reviews regardless of their superiority and decide whether to purchase or not purchase the manufactured goods. The main difficulty in obtainable study on opinion assessment is that all opinions are considered regardless of the implication of each of them. Hence categorization of opinions depending on implication is an essential job. In this article, attempt is made for opinion assessment depending on its superiority, and help buyer make a proper buying judgment. A web mining technique that is narrative and effective is used to assess the consumer opinion for manufactured goods depending on marked allocation are anticipated. The superiority consideration of consumer reviews are classified as genuine, near duplicate, and duplicate opinion. It is carried out in three steps: (1) Recognize opinion regions to take out opinions. (2) Take out and separate features of reviews by quartile compute and assign weights to the features that belong to each group. (3) Consider the feature weights and group belongingness to assess the reviews. Investigational output demonstrates the usefulness of the proposed method which measures the quality of review and assesses it in view of that. The efficiency of client opinion summarization task is probably improved by recognizing and discarding irrelevant opinions

Performance of Web Traffic Activities using Web Mining and Machine Learning Techniques

In recent days Web mining gathers all tools, approaches, and algorithms that had to retrieve information and knowledge through data-based data. The portion of this methodology is aimed at analyzing users ' behaviors, to continue improving the framework and content of websites visited consistently. A relevant question then arises: how much more the attempt to enhance the services provided via a website breaches the privacy of visitors? The use of important retrieval resources including web mining can threaten the privacy of users. This paper would concentrate on developing approaches to speed up the weblog mining process and also to show data visualization as a consequence of the log mining process and evaluate algorithms for data mining. The right metrics to equate algorithms will be used for the analysis of the classification methods, accuracy RMSE and MAE. The fundamental goal of the case study is to evaluate the usefulness of the expert-driven system and data-driven method for the classification of authenticated network traffic, in particular, SSH traffic from traffic log files.