Blog Dataset Research Articles

Deep auto‐encoder based clustering algorithm for graph‐based web page recommendation system

SummaryWeb‐page recommendation system (RS) plays a major role in smart web systems. The core technology is the RS, which helps users find their interests or discover item preferences. With the rapid proliferation of information technology, the present era witnessed a remarkable growth in the collection and generation of web data. Every day, it is difficult to project the appropriate information to the user and adds severe complexity to the decision making procedure. Moreover, the existing RS techniques have limitations like huge computation time, high computational cost, over specialization, and sparsity problems. This work presents an effective graph‐based web page recommendation model to overcome these difficulties. Initially, the input data is preprocessed utilizing Natural Language Tool Kit executed in Python. The snapshot‐based dynamic graph model obtains the user‐web page relationship. Weight calculation of user‐item and the user‐user edge is also processed with this graph model. The user‐user edge similarity is used to obtain the user preference vector. Then, unsupervised‐deep auto encoder based density‐based clustering algorithm is proposed to cluster the user preference vector, which groups the visited web page details into k clusters. The web pages are recommended based on the highest score calculated by the page ranking method. Finally, the topmost visited web pages are recommended to web users based on the ranking score. “All the news”, MSNBC, and Weblog datasets are taken for evaluation. The proposed method is evaluated with respect to standard measures like F1 measure, accuracy, recall, precision, and root mean square error. Compared with other existing methods, the proposed technique achieved the highest accuracy, 97.99% for All the news datasets, 93.03% for the MSNBC database, and 90.07% for the Weblog database.

Machine invasion: Automation in information acquisition and the cross-section of stock returns

We estimate the number of machines “covering” a firm by separating machine Internet protocols (IPs) from human IPs based on the intensity of information retrieval using the EDGAR web log dataset. We investigate the relationship of machine coverage and the cross-section of stock returns and find that stocks in the lowest quintile of machine coverage outperform those in the highest quintile by 6% annually after adjusting for risk. Our results indicate that automation in information processing has a significant impact on the cross-section of stock returns.

Future Link Prediction in the Blogosphere for Recommendation

The phenomenal growth in both scale and importance of social media such as blogs, micro-blogs and user-generated content, has created a need for tools that monitor information diffusion and make recommendations within these platforms. An essential element of social media, particularly blogs, is the hyperlink graph that connects various pieces of content. There are two types of links within the blogosphere; one from blog post to blog post, and another from blog post to blog channel (an event stream of blog posts). These links can be viewed as a proxy for the flow of information between blog channels and to reflect influence. Given this assumption about links, the ability to predict future links can facilitate the monitoring of information diffusion, making recommendations, and word-of-mouth (WOM) marketing. We propose different methods for link predictions and we evaluate these methods on an extensive blog dataset.

Visual Analysis System for Search Trend Data

<p indent=0mm>A visual analysis system called STVAS is presented to help users explore and analyze data collected from search engines, which includes data collection and preprocessing, calculation of streamgraph, generation of streamlines and text placement, along with interactive analysis. It presents a visualization method that combines streamgraph and text to reveal search trends and hotspots. To guide the text placement within streamgraph, this system uses a novel layout algorithm with streamlines generated from the vector field inside the streamgraph. In addition, a set of interactions is offered to help user explore and analyze data on different levels. Quantitative evaluation of the visualization method is made on five blog datasets, and case studies on two real search datasets. The results demonstrate that this system can help users understand the evolving pattern of search engine data, discover the implicit search trends, and quickly grasp public opinions from the Internet.

Mutual Information Pre-processing Based Broken-stick Linear Regression Technique for Web User Behaviour Pattern Mining

Web usage behaviour mining is a substantial research problem to be resolved as it identifies different user’s behaviour pattern by analysing web log files. But, accuracy of finding the usage behaviour of users frequently accessed web patterns was limited and also it requires more time. Mutual Information Pre-processing based Broken-Stick Linear Regression (MIP-BSLR) technique is proposed for refining the performance of web user behaviour pattern mining with higher accuracy. Initially, web log files from Apache web log dataset and NASA dataset are considered as input. Then, Mutual Information based Pre-processing (MI-P) method is applied to compute mutual dependence between the two web patterns. Based on the computed value, web access patterns which relevant are taken for further processing and irrelevant patterns are removed. After that, Broken-Stick Linear Regression analysis (BLRA) is performed in MIPBSLR for Web User Behaviour analysis. By applying the BLRA, the frequently visited web patterns are identified. With the identification of frequently visited web patterns, MIP-BSLR technique exactly predicts the usage behaviour of web users, and also increases the performance of web usage behaviour mining. Experimental evaluation of MIPBSLR method is conducted on factors such as pattern mining accuracy, false positives, time requirements and space requirements with respect to number of web patterns. Outcomes show that the proposed technique improves the pattern mining accuracy by 14%, and reduces the false positive rate by 52%, time requirement by 19% and space complexity by 21% using Apache web log dataset as compared to conventional methods. Similarly, the pattern mining accuracy of NASA dataset is increased by 16% with the reduction of false positive rate by 47%, time requirement by 20% and space complexity by 22% as compared to conventional methods.

Efficient Approach of Analyzing and Generating Intrinsic Information from Weblog

The weblog data have unstructured data, and due to this extraction, the desired information from it is a very challenging task. This kind of data is rapidly growing with huge volume. In this paper, the weblog analysis through Pyspark (WAP) algorithm is proposed to analyze the complete weblog dataset and extract useful information from it. In the proposed approach is used Resilient Distributed Dataset (RDD) for running and operating on multiple nodes to do parallel processing on a cluster; for achieving these concepts is used apache pyspark and implemented on jupyter notebook. WAP extracts different results such as host count, path count, status count, etc. This approach also generates patterns and track user behavior. The recommendation is also possible by the proposed algorithm based on user behaviors.

A network model that combines latent factors and sparse graphs

AbstractWe propose a combined model, which integrates the latent factor model and a sparse graphical model, for network data. It is noticed that neither a latent factor model nor a sparse graphical model alone may be sufficient to capture the structure of the data. The proposed model has a latent (i.e., factor analysis) model to represent the main trends (a.k.a., factors), and a sparse graphical component that captures the remaining ad‐hoc dependence. Model selection and parameter estimation are carried out simultaneously via a penalized likelihood approach. The convexity of the objective function allows us to develop an efficient algorithm, while the penalty terms push towards low‐dimensional latent components and a sparse graphical structure. The effectiveness of our model is demonstrated via simulation studies, and the model is also applied to four real datasets: Zachary's Karate club data, Kreb's U.S. political book dataset ( http://www.orgnet.com), U.S. political blog dataset , and citation network of statisticians; showing meaningful performances in practical situations.

소셜 빅데이터를 이용한 한국내의 대만음식 트렌드 분석 : 블로그 텍스트 분석을 중심으로

The purpose of this study is to analyze the trend of Taiwanese food in Korea using text mining analysis among social big data for blogs of representative portal site in Korea, and analyzed annual trends from 2015 to 2019. The frequency was analyzed by the collected data set of blogs searched for 'Taiwanese food', and 'Taiwan', 'Travel', and 'Food' were maintained high frequency steadily in the first ranking group. The main attribute of Taiwanese food is 'street food in the night market' as well. Since then, food-related words such as 'famous place for visit', 'local restaurants', 'tasty' and 'recommendation' have been frequently found in the second ranking group. Specific Taiwanese food appeared, especially 'Beef noodle', 'Mango bingsu(snow flakes with syrup)', 'Bubble tea', 'Taiwanese pancake', and 'Zippie(flat fried chicken) ' were recognized as the representative Taiwanese foods in Korea. These findings are expected to be valuable information for entering the Korean food market, which emphasizes the value for money and shows preferences for snack foods or sweet desserts.

Model Uncertainty Based Annotation Error Fixing for Web Attack Detection

Deep learning (DL) techniques have been widely used in web attack detection domain. With the stronger ability to fit data, DL models are also more sensitive to the training data, annotation errors can mislead the model training very easily. In our work, we propose model uncertainty to evaluate the prediction made by the DL based web attack model. Model uncertainty helps to find the annotation errors and also the misclassification caused by these errors. Experiments on two real web log datasets and a public benchmark dataset prove that our method can find more annotation errors than the traditional DL method. We also prove the efficiency of our detection model: with the aid of GPU acceleration, it is capable of tackling attacks on the fly.

User-centered probabilistic models for content diffusion in the blogosphere

Predicting the diffusion of information in social networks is a key problem for applications like Opinion Leader Detection, Buzz Detection or Viral Marketing. Many diffusion models are direct extensions of the Cascade and Thresholdmodels, initially proposed for epidemiology and social studies. In such models, the diffusion process is based on the dynamics of interactions between neighbor nodes in the network (the social pressure), and largely ignores important dimensions as the content diffused and the active/passive role users tend to have in social networks. We propose here a new family of models that aims at predicting how a content diffuses in a network by making use of additional dimensions: the content diffused, user’s profile and willingness to diffuse. In particular, we show how to integrate these dimensions into simple feature functions, and propose a probabilistic modeling to account for the diffusion process. These models are then illustrated and compared with other approaches on two blog datasets. The experimental results obtained on these datasets show that taking into account the content diffused is important to accurately model the diffusion process. Lastly, we study the influence maximization problem with these models and prove that it is NP-hard, prior to propose an adaptation of the greedy algorithm to approximate the optimal solution.

Cans and cants: Computational potentials for multimodality with a case study in head position

As the study of embodiment and multimodality in interaction grows in importance, there is a need for novel methodological approaches to understand how multimodal variables pattern together along social and contextual lines, and how they systematically coalesce in communicative meanings. In this work, we propose to adopt computational tools to generate replicable annotations of bodily variables: these can be examined statistically to understand their patterning with other variables across diverse speakers and interactional contexts, and can help organize qualitative analyses of large datasets. We demonstrate the possibilities thereby with a case study in head cant (side‐to‐side tilt of the head) in a dataset of video blogs and laboratory‐collected interactions, computationally extracting cant and prosody from video and audio and analyzing their interactions, looking at gender in particular. We find that head cant indexes an orientation towards the interlocutor and a sense of shared understanding, can serve a ‘bracketing’ function in interaction (for speakers to create parentheticals or asides), and has gendered associations with prosodic markers and interactional discourse particles.

Generated rules for AIDS and e-learning classifier using rough set approach

The emergence and growth of internet usage has accumulated an extensive amount of data. These data contain a wealth of undiscovered valuable information and problems of incomplete data set may lead to observation error. This research explored a technique to analyze data that transforms meaningless data to meaningful information. The work focused on Rough Set (RS) to deal with incomplete data and rules derivation. Rules with high and low left-hand-side (LHS) support value generated by RS were used as query statements to form a cluster of data. The model was tested on AIDS blog data set consisting of 146 bloggers and E-Learning@UTM (EL) log data set comprising 23105 URLs. 5-fold and 10-fold cross validation were used to split the data. Naïve algorithm and Boolean algorithm as discretization techniques and Johnson’s algorithm (Johnson) and Genetic algorithm (GA) as reduction techniques were employed to compare the results. 5-fold cross validation tended to suit AIDS data well while 10-fold cross validation was the best for EL data set. Johnson and GA yielded the same number of rules for both data sets. These findings are significant as evidence in terms of accuracy that was achieved using the proposed model

Where to go and what to play: Towards summarizing popular information from massive tourism blogs

In this work, we propose a novel method to summarize popular information from massive tourism blog data. First, we crawl blog contents and segment them into semantic word vectors separately. Then, we select the geographical terms in each word vector into a corresponding geographical term vector and present a new method to explore hot tourism locations and, in particular, their frequent sequential relations from a set of geographical term vectors. Third, we propose a novel word vector subdividing method to collect local features for each hot location, and introduce the metric of max-confidence to identify the Things of Interest (ToI) associated with the location from the collected data. We illustrate the benefits of this approach by applying it to a Chinese online tourism blog dataset. The experimental results show that the proposed method can be used to explore hot locations, as well as their sequential relations and corresponding ToI, efficiently.

Clustering of Navigation Patterns using Bolzwano_Weierstrass Theorem

Objectives: The primary objective of this research paper is to design a new and efficient clustering technique to group user navigation patterns which are useful for classification system to classify a new user with the previous users group. Methodology: Three real time web log data sets are collected from e-commerce web server, academic institution web server and a research journal web server. All three sets were collected from IIS web servers. After navigation patterns are derived from preprocessing step it is clustered into groups by using traditional Fuzzy C-Means technique. The clusters are validated and re-clustered using Bolzano_Weierstrass Theorem. Findings: Web log data is preprocessed and ICA is applied in the user session matrix to select relevant and important features. To measure the clustering accuracy of proposed and the existing methods, the parameters such as Rand Index, F measure are calculated and compared. It shows proposed BWFCM have higher rand index rate than FCM and lesser error rate. To understand the impact of the feature selection method, the data sets were implemented with the existing and proposed methods of feature selection. The parameters taken for comparison were Rand Index, Sum of Squared Errors, F-measure. The method was implemented in all the three data sets after data cleaning, session construction step. Clustering was carried out twice with the proposed clustering algorithm in all the three data sets, without selecting features and after selecting features. It was observed that the clustering results are poor when applied in full data set with irrelevant features, and the performance was increased after relevant features were selected. Conclusion: The result of the optimized clustering proves its significance and there is an increase in similarity of intra clustering and dissimilarity in inter clustering than the existing methods.

Heavy path based super-sequence frequent pattern mining on web log dataset

Mining web log datasets has been extensively studied using Frequent Pattern Mining (FPM) and its various other forms. Identifyingfrequent patterns in different sequences can help in analyzing the most common sub-sequences (e.g., the pages visitedtogether). However, this approach would not be able to identify general structures spanning over multiple sequences. In responseto understanding general structures, we introduce a new form of sequential pattern mining called super-sequence frequent patternmining (SS-FPM). In contrast to sub-sequences determined by FPM, SS-FPM determines the super-sequences that can containthe common parts from different sequences. This can be useful in understanding the general behavior/flow of users in web usagemining, classifying web pages and users, making predictions etc. In essence, finding frequent super-sequence patterns turnsout to be related to the well-known heaviest (longest) path problem in graphs, which is known to be NP-hard. Accordingly,we transform a given sequential dataset into a sequence graph and formulate the problem as k-hop heaviest path problem. Wethen propose an efficient heuristic called sequence matrix method using dynamic programming techniques. We compared ourmethod to the existing Heavypath method. The results show that our method is more efficient especially on large datasets.

Guru Dreams and Competition: An Anatomy of the Economics of Blogs

The rise of social media has encouraged guru dreams because of the low entry barrier and highly skewed distribution of public attention that characterize social media. The pursuit of guru status, however, may be achieved through information provision or cheap talk, and competition inherent to social media may incentivize participants to either process better information or express more extreme options. Using a unique dataset of blogs covering S&P 1500 stocks over the 2006-2011 period, we find evidence that social media can be informative about future stock returns but that competition distorts opinions rather than encouraging participants to process better information. In particular, competition induces exaggerated negative tones in blogs, which is unrelated to information. Our results suggest that social media may provide mixed incentives for its participants in terms of information efficiency.

NEURO FUZZY LINK BASED CLASSIFIER FOR THE ANALYSIS OF BEHAVIOR MODELS IN SOCIAL NETWORKS

In this study, a new link based classifier using ne uro fuzzy logic has been proposed for analyzing the social behavior based on Weblog dataset. In this system, d ata are processed using a multistage structure. Thi s system provides a diagnosis using a neuro fuzzy lin k based classifier that analyses the user’s behavio r to specific diagnostic categories based on their clust er category in social networks. It uses random walk s method to organize the labels. Since the links pres ent in the social network graph frequently represen t relationships among the users with respect to socia l contacts and behaviours, this work observes the l inks of the graph in order to identify the relationships re presented in the graph between the users of the soc ial network based on some new social network metrics and the past behaviour of the users. This work is use ful to provide connection between consolidated features of users based on network data and also using the traditional metrics used in the analysis of social network users. From the experiments conducted in this research work, it is observed that the proposed wor k provides better classification accuracy due to th e application of neuro fuzzy classification method in link analysis.

Hedonic Analysis for the Estimation of Condominium Rent Utilizing Web Information

Information is taken from the Web for the hedonic analysis of real-estate rent. Twenty-three wards in Tokyo are chosen as the data-collection areas and the rents of condominiums are estimated. To utilize the website information, variables signifying areal reputation are generated from weblog datasets. The results show that the contribution of such information varies from area to area, and that in areas with a good reputation the contribution tends to become larger. This suggests that such information can be useful for the hedonic analysis of areas with a good reputation, and qualitative aspects such as a good townscape and a sense of high class can be incorporated into this method to improve the accuracy of the rent or price estimation of real estate.

Effet de la précontrainte des transplants dans la reconstruction du ligament croisé antérieur, une étude en microscopie électronique à balayage

Recently there has been much interest in electronic commerce applications that use data mining techniques to explore datasets in the social media context. However, most of the applications have already been developed in an ad hoc manner, mainly, due to the lack of adequate tools, yielding difficulties in customizing applications and requesting high time consuming for constructing and maintaining these applications. This work addresses these problems and proposes a software framework for building Web mining applications in the blog world. The architecture of the proposed framework combines the use of blog crawling and data mining algorithms, in order to provide a complete and flexible solution for building general-purpose Web mining applications. The framework flexibility allows some important customizations, such as the construction of adapters for reading text from different blogs, and the use of different pre-processing techniques and data mining algorithms. In order to improve the efficacy of information extraction from blogs, ontology is used in the blog’s description. For this, there are software agents responsible for tracking and indexing blogs related to a specific tag and for mining blog datasets. Moreover, web services are used for encapsulating existing tools and maximize reuse.This framework has been instantiated in order to be applied for helping the blog users to effectively find out relevant information in the blog world. The focus of this paper is on describing the novel software architecture of the general framework (blog crawling and data mining) providing detailed information about the data mining sub-framework, which uses the semantic web services technology for automating service composition and consists on the main research contribution. A case study of an e-commerce application for analyzing the user’s sentiment regarding specific products is reported and its results considers the effort reduction when creating a web mining application by using the proposed integrated frameworks and existing data mining tools, as well as a qualitative analysis related to quality aspects of the developed application, such as the evolution impact.

FacetCube: a general framework for non-negative tensor factorization

Non-negative tensor factorization (NTF) has been successfully used to extract significant characteristics from polyadic data, such as data in social networks. Because these polyadic data have multiple dimensions (e.g., the author, content, and timestamp of a blog post), NTF fits in naturally and extracts data characteristics jointly from different data dimensions. In the traditional NTF, all information comes from the observed data, and therefore, the end users have no control over the outcomes. However, in many applications very often, the end users have certain prior knowledge, such as the demographic information about individuals in a social network or a pre-constructed ontology on the contents and therefore prefer the data characteristics extracting by NTF being consistent with such prior knowledge. To allow users’ prior knowledge to be naturally incorporated into NTF, in this paper, we present a general framework—FacetCube—that extends the standard NTF. The new framework allows the end users to control the factorization outputs at three different levels for each of the data dimensions. The proposed framework is intuitively appealing in that it has a close connection to the probabilistic generative models. In addition to introducing the framework, we provide an iterative algorithm for computing the optimal solution to the framework. We also develop an efficient implementation of the algorithm that consists of several techniques to make our framework scalable to large data sets. Extensive experimental studies on a paper citation data set and a blog data set demonstrate that our new framework is able to effectively incorporate users’ prior knowledge, improves performance over the traditional NTF on the task of personalized recommendation, and is scalable to large data sets from real-life applications.