Information Filtering Techniques Research Articles

Using Query Expansion Techniques and Content-Based Filtering for Personalizing Analysis in Big Data

The recent debates on personalizing analyses in a Big Data context are one of the most solicited challenges for business intelligence (BI) administrators. The high-volume, the high-variety, and the high-velocity of Big Data have produced difficulty in storing, processing, and analyzing data in traditional systems. These 3Vs (volume, velocity, and variety) created many new challenges and make them difficult to extract the specific needs of the users. In addition, the user may be faced with the problem of disorientation; he does not know what information really corresponds to his needs. The information personalization systems aim to overcome these problems of disorientation by using a user profile. The effectiveness of the personalization system in a Big Data context is to demonstrate by the relevance and accuracy of the content of the results obtained, according to the needs of the user and the context of the research. Nevertheless, most of the recent research focused on the relational data warehouse personalizing and ignored the integration of the user context into the analysis of OLAP cubes, which is the first concerned to execute the user's multidimensional queries. To deal with this, the authors propose in this article a dynamic personalizing approach in Big Data context using OLAP cubes, based on the Content-Based Filtering, and the Query Expansion techniques. The first step in the proposal consists of processing the user queries by an enrichment technique in order to integrate the user profile and his searching context to reduce the searching space in the OLAP cube, and use the expansion technique to extend the scope of the analysis in the OLAP cube. The retrieved results are: “as relevant as possible” compared to the user's initial request. Afterward, they use information filtering techniques such as content-based filtering to personalize the analysis in the reduced data cube according to the term frequency and cosine similarity. Finally, they present a case study and experiences results to evaluate and validate their approach.

Supporting connectivism in knowledge based engineering with graph theory, filtering techniques and model quality assurance

Mass-customization has forced manufacturing companies to put significant efforts to digitize and automate their engineering and production processes. When new products are to be developed and introduced the production is not alone to be automated. The application of knowledge regarding how the product should be designed and produced based on customer requirements also must be automated. One big academic challenge is helping industry to make sure that the background knowledge of the automated engineering processes still can be understood by its stakeholders throughout the product life cycle.The research presented in this paper aims to build an infrastructure to support a connectivistic view on knowledge in knowledge based engineering. Fundamental concepts in connectivism include network formation and contextualization, which are here addressed by using graph theory together with information filtering techniques and quality assurance of CAD-models. The paper shows how engineering knowledge contained in spreadsheets, knowledge-bases and CAD-models can be penetrated and represented as filtered graphs to support a connectivistic working approach. Three software demonstrators developed to extract filtered graphs are presented and discussed in the paper.

PowKMeans

This article describes how recommender systems are software applications or web portals that generate personalized preferences using information filtering techniques, with a goal to support decision-making of the users. Collaborative-based techniques are often used to predict the unknown preferences of the user based upon his past preferences or the preferences of the similar users that have already been identified. A user which has a high correlation with any group of users is known as white user whereas the users which have less correlation with any group of users are known as gray-sheep users. The presence of gray-sheep users affects the accuracy of the model, and generates inaccurate predictions. To improve the prediction accuracy, it is important to differentiate graysheep users from white users. Experimental results show that PowKMeans is effective in improving the prediction accuracy by 4.62%. It has also shown reduction in Mean Absolute Error by 0.7757.

A Real-Time Detection Method of System Failure Based on Kalman Filtering

The quality of information communication is the key guarantee for the normal operation of a system. Information filtering techniques and methods are important for system failure detection. Kalman filtering is a widely used method of optimal estimation as it is easy to computer programming and data can be updated and processed in real time. In this paper, the financial information system is taken for application of Kalman filtering in detecting the failure. Dynamic detecting models are established as the integration of a process model and a discriminant model. The process model is used to describe a dynamic process of financial failure and the discriminant model is used to describe discriminant rules. A real-time detection method is developed based on Kalman filtering and a general n-step-ahead forecasting algorithm is improved in order for the prospective forecast. The dynamic forecasting and detecting process is implemented through computer programming using the data of China’s communications industries. The results have proved the accuracy and advance of detecting financial failure in such case.

River segmentation using satellite image contextual information and Bayesian classifier

Satellite-based remote sensing imaging can provide continuous snapshots of the Earth’s surface over long periods. River extraction from remote sensing images is useful for the comprehensive study of dynamic changes of rivers over large areas. This paper presents a new method of extracting rivers by using training samples based on the mathematical morphology, Bayesian classifier and a dynamic alteration filter. The use of a training map from erosion morphology helps to extract the non-predictive river’s curves in the image. The algorithm has two phases: creating the profile to separate river area via evaluated morphological erosion and dilation, namely, a training map; and improving the river’s image segmentation using the Bayesian rule algorithm in which two consecutive filters swipe false positive (non-water area) along the image. The proposed algorithm was tested on the Kuala Terengganu district, Malaysia, an area that includes a river, a bridge, dam and a fair amount of vegetation. The results were compared with two standard methods based on visual perception and on peak signal-to-noise ratio, respectively. The novelty of this approach is the definition of the contextual information filtering technique, which provides an accurate extraction of river segmentation from satellite images.

A user-centered and group-based approach for social data filtering and sharing

Social networking sites (SNSs) like Facebook, Google+, Twitter, LinkedIn have become a very important part of our daily life. People are connected to multiple SNSs for networking, communicating, collaborating, sharing and seeking for information. Although, the diversity of current SNSs increases and enriches our online experience, they cause some problems. One of the major issues is that users are often overwhelmed by the huge number of social data. It is even worse as these social data are scattered across disconnected SNSs. To address such problems, we propose a user-centered and group-based approach for social data filtering and sharing. First, it allows users to aggregate their social data from different SNSs and to extract relevant contents. Users explicitly define their interests via specific queries, using information filtering techniques, the system will retrieve new corresponding contents. Second, it is expected to extend its first user-centered purpose by allowing group-based information sharing and management. Users can share some part of their own social data with and collectively define the information organization within their respective groups. To describe further and illustrate our proposed approach, a system architecture and a prototype are also presented in this paper. A primary test was carried out and showed encouraging results confirming the added values of our approach.

TrustRank: a Cold-Start tolerant recommender system

The explosive growth of the World Wide Web leads to the fast advancing development of e-commerce techniques. Recommender systems, which use personalised information filtering techniques to generate a set of items suitable to a given user, have received considerable attention. User- and item-based algorithms are two popular techniques for the design of recommender systems. These two algorithms are known to have Cold-Start problems, i.e., they are unable to effectively handle Cold-Start users who have an extremely limited number of purchase records. In this paper, we develop TrustRank, a novel recommender system which handles the Cold-Start problem by leveraging the user-trust networks which are commonly available for e-commerce applications. A user-trust network is formed by friendships or trust relationships that users specify among them. While it is straightforward to conjecture that a user-trust network is helpful for improving the accuracy of recommendations, a key challenge for using user-trust network to facilitate Cold-Start users is that these users also tend to have a very limited number of trust relationships. To address this challenge, we propose a pre-processing propagation of the Cold-Start users’ trust network. In particular, by applying the personalised PageRank algorithm, we expand the friends of a given user to include others with similar purchase records to his/her original friends. To make this propagation algorithm scalable to a large amount of users, as required by real-world recommender systems, we devise an iterative computation algorithm of the original personalised TrustRank which can incrementally compute trust vectors for Cold-Start users. We conduct extensive experiments to demonstrate the consistently improvement provided by our proposed algorithm over the existing recommender algorithms on the accuracy of recommendations for Cold-Start users.

Statistical analysis of k-nearest neighbor collaborative recommendation

Collaborative recommendation is an information-filtering technique that attempts to present information items that are likely of interest to an Internet user. Traditionally, collaborative systems deal with situations with two types of variables, users and items. In its most common form, the problem is framed as trying to estimate ratings for items that have not yet been consumed by a user. Despite wide-ranging literature, little is known about the statistical properties of recommendation systems. In fact, no clear probabilistic model even exists which would allow us to precisely describe the mathematical forces driving collaborative filtering. To provide an initial contribution to this, we propose to set out a general sequential stochastic model for collaborative recommendation. We offer an in-depth analysis of the so-called cosine-type nearest neighbor collaborative method, which is one of the most widely used algorithms in collaborative filtering, and analyze its asymptotic performance as the number of users grows. We establish consistency of the procedure under mild assumptions on the model. Rates of convergence and examples are also provided.

Knowledge reuse for software reuse

Software reuse can provide significant improvements in software productivity and quality whilst reducing development costs. Expressing software reuse intentions can be difficult though. A developer may aspire to reuse a software component but experience difficulty expressing their reuse intentions in a manner that is compatible with, or understood by, the component retrieval system. Various intelligent retrieval techniques have been developed that assist a developer in locating or discovering components in an efficient manner. These solutions share a common shortcoming: the developer must be capable of anticipating all reuse opportunities and initiating the retrieval process. There is a need for a comprehensive technique that not only assists with retrievals but that can also identify reuse opportunities. This paper advocates that component-based reuse can be supported through knowledge collaboration. Often programming tasks and solutions are replicated; this characteristic of software can be exploited for the benefit of future developments. Through the mining of existing source code solutions, knowledge, relating to how components are used by developers, can be extracted. Based on a developer's current programming task, this knowledge can subsequently be filtered and used to recommend a candidate set of reusable components. This novel recommendation approach applies and extends commonly used Information Retrieval and Information Filtering techniques such as Collaborative Filtering, Content-Based Filtering, and Bayesian Clustering Models, to the software reuse domain. This recommendation technology is applied to several thousand open-source Java classes. The most effective recommendation algorithm produces recommendations of a high quality at a low cost.

Warning message generation by information filtering techniques

This paper proposes a two-stage model for generating warning messages by using information filtering techniques. In this model, information is represented by its attributes and processed through two stages. At the first stage, exceptions are separated from normal information by the cognitive filtering technique. At the second stage, a warning message is generated from critical exceptions by the collaborative filtering approach. An example is discussed to illustrate the proposed model.

A generalised regression algorithm for Web page categorisation

This paper proposes an information system that classifies Web pages according a taxonomy, which is mainly used from seven search engines/directories. The proposed classifier is a four-layer generalised regression neural network (GRNN) that aims to perform the information segmentation according to information filtering techniques using content descriptor vectors. Eight categories of Web pages were used in order to evaluate the robustness of the method, while no restrictions were imposed except for the language of the content, which is English. The system can be used as an assistant and consultative tool for classification purposes as well as for estimating the population of Web pages at any given point in time.

Incorporating filtering techniques in a fuzzy linguistic multi-agent model for information gathering on the web

In (Computing with Words, Wiley, New York, 2001, p. 251; Soft Comput. 6 (2002) 320; Fuzzy Logic and The Internet, Physica-Verlag, Springer, Wurzburg, Berlin, 2003) we presented different fuzzy linguistic multi-agent models for helping users in their information gathering processes on the Web. In this paper we describe a new fuzzy linguistic multi-agent model that incorporates two information filtering techniques in its structure: a content-based filtering agent and a collaborative filtering agent. Both elements are introduced to increase the information filtering possibilities of multi-agent system on the Web and, in such a way, to improve its retrieval issues.

Ganging up on information overload

When information is abundant, the knowledge of which information is useful and valuable matters most. We all use our network of family, friends, and colleagues to recommend movies, books, cars, and news articles. Collaborative filtering technology automates the process of sharing opinions on the relevance and duality of information. Collaborative filtering is one technique among many information filtering techniques that range from unfiltered to personalized and from effortless to laborious. Libraries or the Web are good examples of unfiltered information sources. E-mail directed to one recipient is a good example of a filtered information source. A best-seller list requires little effort fur the user, but provides the same recommendations to all users. Filters based on demographics, such as age, sex, or marital status, require some effort from the user in providing the demographics, and provide some level of personal filtering, so they are near the middle of the chart. Collaborative filtering requires relatively little effort from the user, and provides individually targeted recommendations, so it is in the upper right of the chart. Effort, of course, can be reduced via automation. While collaborative filtering is not necessarily effortless, it requires a relatively small amount of effort on the part of the user and provides very individualized recommendations. The collaborative filtering systems that we discuss here each offer a high degree of personalization, but each system takes a different approach to automation, attempting to find the best trade-off between the amount of work the users must put into the system and the perceived value and benefits they receive in return.