Online Clustering Method Research Articles

An Efficient Online Loop Closure Detection System With Local Spatial Co-Occurrence Information

As a vital component in simultaneous localization and mapping techniques, appearance-based loop closure detection (LCD) plays important roles in bounding the long-term drift errors. In this paper, an online LCD system based on the mutual co-occurrence information among visual features is proposed. First, a feature tracker module is designed to generate distinctive visual words, exploiting tracked words tool to improve efficiency. Then an incrementally built vocabulary is organized by a hierarchical navigable small world graph, where the visual words are indexed. To merge a homologous word into the existing one, the vocabulary applies an improved high dimensional online clustering method, which regards individual cluster as a normal distribution form. At the query phase, a list of candidate frames is located due to the co-occurrence constraint. Ultimately, the loop closure is specified by passing the temporal and similarity check, which avoids the memory consumption of historic image data. Validation tests based on public datasets and experimental sequence demonstrate the merit of low running time and memory cost, while the high-precision performance is retained in the system.

MAPNEWS: A Framework for Aggregating and Organizing Online News Articles

In recent years, digital news has become increasingly prevalent, with many people getting their news and information from online sources rather than traditional print or broadcast media. This shift has been driven, in part, by the convenience and accessibility of digital platforms, as well as the ability to personalize and customize news feeds. Digital news also allows for greater interactivity and engagement with readers and can reach a global audience almost instantly. News articles contain a plethora of hidden spatial information that, when shared with readers, increases comprehension of current events. Only a few news aggregation systems make this information available to users. Many stories, on the other hand, are not clearly geotagged with their spatial information. In this work, we propose the MapNews framework, a novel system that gathers, analyzes, and presents news articles on a map interface, allowing users to take advantage of their underlying spatial information. MapNews pulls content from several different internet news sources and, using a custom-built geotagger, it extracts geographic content from articles. A rapid online clustering method is used to organize articles into story clusters. Panning and zooming MapNews' map interface allows readers to receive news based on geographic location and category importance, and they will view distinct articles depending on their location. MapNews achieved an ARI score of 0.89 for clustering and an accuracy of 95% in usability testing

Non-intrusive load monitoring techniques for the disaggregation of ON/OFF appliances

Nowadays, Non-Intrusive Load Monitoring techniques are sufficiently accurate to provide valuable insights to the end-users and improve their electricity behaviours. Indeed, previous works show that commonly used appliances (fridge, dishwasher, washing machine) can be easily disaggregated thanks to their abundance of electrical features. Nevertheless, there are still many ON/OFF devices (e.g. heaters, kettles, air conditioners, hair dryers) that present very poor power signatures, preventing their disaggregation with traditional algorithms. In this work, we propose a new online clustering method exploiting both operational features (peak power, duration) and external features (time of use, day of week, weekday/weekend) in order to recognize ON/OFF devices. The proposed algorithm is intended to support an existing disaggregation algorithm that is already able to classify at least 80% of the total energy consumption of the house. Thanks to our approach, we improved the performance of our existing disaggreation algorithm from 80% to 87% of the total energy consumption in the monitored houses. In particular, we found that 85% of the clusters were identified by only using operational features, while external features allowed us to identify the remaining 15% of the clusters. The algorithm needs to collect on average less than 40 operations to find a cluster, which demonstrates its applicability in the real world.

Efficient large-scale face clustering using an online Mixture of Gaussians

In recent years, the number of applications demanding real-time face clustering algorithms has increased, especially for security and surveillance purposes. However, state-of-the-art face clustering methods are offline, they need to repeat the whole clustering process every time new data arrives, and thus, they are not suitable for real-time applications. On the other hand, online clustering methods are highly dependent on the order and the size of the data, and they are less accurate than offline methods. To overcome these limitations, we present an online gaussian mixture-based clustering method (OGMC). The key idea of this method is the proposal that an identity can be represented by more than just one distribution or cluster. Using feature vectors extracted from the incoming faces, OGMC generates clusters that may be connected to others depending on their proximity and their robustness, and updates their connections every time their parameters are updated. With this approach, we reduce the dependency of the clustering process on the order and the size of the data and we are able to deal with complex data distributions. Experimental results show that OGMC outperforms state-of-the-art clustering methods on large-scale face clustering benchmarks not only in accuracy, but also in efficiency and scalability.

Contrastive Clustering

In this paper, we propose an online clustering method called Contrastive Clustering (CC) which explicitly performs the instance- and cluster-level contrastive learning. To be specific, for a given dataset, the positive and negative instance pairs are constructed through data augmentations and then projected into a feature space. Therein, the instance- and cluster-level contrastive learning are respectively conducted in the row and column space by maximizing the similarities of positive pairs while minimizing those of negative ones. Our key observation is that the rows of the feature matrix could be regarded as soft labels of instances, and accordingly the columns could be further regarded as cluster representations. By simultaneously optimizing the instance- and cluster-level contrastive loss, the model jointly learns representations and cluster assignments in an end-to-end manner. Besides, the proposed method could timely compute the cluster assignment for each individual, even when the data is presented in streams. Extensive experimental results show that CC remarkably outperforms 17 competitive clustering methods on six challenging image benchmarks. In particular, CC achieves an NMI of 0.705 (0.431) on the CIFAR-10 (CIFAR-100) dataset, which is an up to 19% (39%) performance improvement compared with the best baseline. The code is available at https://github.com/XLearning-SCU/2021-AAAI-CC.

An online self-organizing modular neural network for nonlinear system modeling

Modular neural network (MNN) has distinct advantage in many fields such as pattern recognition and pattern recognition. However it is still a challenge to dynamically adjust the MNN structure for dynamic nonlinear system modeling. This paper proposes a novel online self-organizing MNN (OSOMNN) for nonlinear system modeling. In OSOMNN, an online task decomposition algorithm and a self-organizing algorithm for subnetwork are introduced. Firstly, the task decomposition algorithm is implemented by the online clustering method based on distance and local density, which can online divide the original task into several simpler subtasks. Then subnetworks with single-layer feedforward neural network are built to learn the divided subtasks. Moreover, this paper develops a self-organizing algorithm for subnetwork, which can dynamically adjust its structure and is trained by the improved online gradient method with fixed memory mechanism (FMOGM). To demonstrate the effectiveness of OSOMNN for nonlinear system modeling, experimental investigations using four benchmark nonlinear systems and the monthly sunspots time series show that OSOMNN can automatically add or merge the subnetwork modules and optimize the structure of subnetworks for nonlinear system modeling with a better generalization performance than the established alternatives.

Model Learning for Multistep Backward Prediction in Dyna-${Q}$ Learning

A model-based reinforcement learning (RL) method which interplays direct and indirect learning to update ${Q}$ functions is proposed. The environment is approximated by a virtual model that can predict the transition to the next state and the reward of the domain. This virtual model is used to train ${Q}$ functions to accelerate policy learning. Lookup table methods are usually used to establish such environmental models, but these methods need to collect tremendous amounts of experiences to enumerate responses of the environment. In this paper, a stochastic model learning method based on tree structures is presented. To model the transition probability, an online clustering method is applied to equip the model learning method with the abilities to evaluate the transition probability. By the virtual model, the RL method produces simulated experience in the stage of indirect learning. Since simulated transitions and backups are more usefully focused by working backward from the state-action, the pair estimated ${Q}$ value of which changes significantly, the useful one-step backups are actions that lead directly into the one state whose value has already obviously been changed. This, however, may induce a false positive; that is, a backup state may be an invalid state, such as an absorbing or terminal state, especially in cases where the changes of ${Q}$ values at the planning stage are still needed to put back for ranking even though they are based on a simulated experience and are possibly erroneous. It is obvious that when the agent is attracted to generate simulated experience around the area of these absorbing states, the learning efficiency is deteriorated. This paper proposes three detecting methods to solve this problem. Moreover, the policy learning can speed up. The effectiveness and generality of our method is further demonstrated in three numerical simulations. The simulation results demonstrate that the training rate of our method is obviously improved.

Online Clustering of Contextual Cascading Bandits

We consider a new setting of online clustering of contextual cascading bandits, an online learning problem where the underlying cluster structure over users is unknown and needs to be learned from a random prefix feedback. More precisely, a learning agent recommends an ordered list of items to a user, who checks the list and stops at the first satisfactory item, if any. We propose an algorithm of CLUB-cascade for this setting and prove an n-step regret bound of order O(√n). Previous work corresponds to the degenerate case of only one cluster, and our general regret bound in this special case also significantly improves theirs. We conduct experiments on both synthetic and real data, and demonstrate the effectiveness of our algorithm and the advantage of incorporating online clustering method.

Adaptive online event detection in news streams

Event detection aims to discover news documents that report on the same event and arrange them under the same group. With the explosive growth of online news, there is a need for event detection to facilitate better navigation for users in news spaces. Existing works usually represent documents based on TF-IDF scheme and use a clustering algorithm for event detection. However, traditional TF-IDF vector representation suffers problems of high dimension and sparse semantics. In addition, with more news documents coming, IDF need to be incrementally updated. In this paper, we present a novel document representation method based on word embeddings, which reduces the dimension and alleviates the sparse semantics compared to TF-IDF, and thus improves the efficiency and accuracy. Based on the document representation, we propose an adaptive online clustering method for online news event detection, which improves both the precision and recall by using time slicing and event merging respectively. The resulted events are further improved by an adaptive post-processing step which can automatically detect noisy events and further process them. Experiments on standard and real-world datasets show that our proposed adaptive online event detection method significantly improves the performance of event detection in terms of both efficiency and accuracy compared to state-of-the-art methods.

A time-varying quadratic programming for online clustering of streaming data

An online clustering method based on a time-varying quadratic programming is proposed which can precisely detect streaming data clustering structure when no assumption is desired on the shape and density of data classes. In the proposed method, online clustering is achieved through simulating some dynamical equations which yield optimum solution of the time-varying quadratic programming over time. A new framework is also proposed which efficiently permits streaming data clustering based on a relatively small and renewable dataset. This framework reduces the need for incoming data storage memory to a small and independent of original data size. The performance of the proposed method is evaluated through the experiments using synthetic data as well as the KDD cup 99 dataset. The results illustrate higher performance of the proposed method in comparison with a range of benchmark methods.

Pursuit Reinforcement Competitive Learning: PRCL based Online Clustering with Tracking Algorithm and its Application to Image Retrieval

Pursuit Reinforcement guided Competitive Learning: PRCL based on relatively fast online clustering that allows grouping the data in concern into several clusters when the number of data and distribution of data are varied of reinforcement guided competitive learning is proposed. One of applications of the proposed method is image portion retrievals from the relatively large scale of the images such as Earth observation satellite images. It is found that the proposed method shows relatively fast on the retrievals in comparison to the other existing conventional online clustering such as Vector Quatization: VQ. Moreover, the proposed method shows much faster than the others for the multi-stage retrievals of image portion as well as scale estimation. A new approach for online clustering based on reinforcement learning, called Pursuit Reinforcement Guided Competitive Learning. PRCL which is derived from pursuit method in reinforcement learning that maintain both action- value and action preferences, with the preferences continually pursuing the action that is greedy according to the current action-value estimates together with learning automata is proposed. PRCL can be used as online clustering method. One of the applications is, then introduced for evacuation simulation. The following section describes the proposed PRCL with learning automata together with the existing conventional online clustering methods of RGCL, SRGCL and VQ. Then preliminary experiments are described followed by its application of image retrievals. After all, conclusion is described with some discussions.

Pursuit Reinforcement Competitive Learning: PRCL based Online Clustering with Learning Automata

A new online clustering method based on not only reinforcement and competitive learning but also pursuit algorithm (Pursuit Reinforcement Competitive Learning: PRCL) as well as learning automata is proposed for reaching a relatively stable clustering solution in comparatively short time duration. UCI repository data which are widely used for evaluation of clustering performance in usual is used for a comparative study among the existing conventional online clustering methods of Reinforcement Guided Competitive Learning: RGCL, Sustained RGCL: SRGCL, Vector Quantization, and the proposed PRCL. The results show that the clustering accuracy of the proposed method is superior to the conventional methods. More importantly, it is found that the proposed PRCL is much faster than the conventional methods. The proposed method is then applied to the evacuation simulation study. It is found that the proposed method is much faster than the conventional method of vector quatization to find the most appropriate evacuation route. Due to the fact that the proposed PRCL method allows finding the most appropriate evacuation route, collisions among peoples who have to evacuate for the proposed method is much less than that of vector quatization.

Online clustering via energy scoring based on low‐rank and sparse representation

Subspace clustering is very useful in many fields, such as computer vision and machine learning. However, most of the clustering methods cannot deal with out-of-sample data directly. For each new sample, these methods need to relearn the representations of all (new and original) data for clustering. This is unrealistic in many practical applications. A new online clustering method to cluster out-of-sample data in terms of the meaningful energy scores of data is proposed. By interpreting low-rank representation (LRR) as a dynamical system, a computation method for energy scores of data has been developed. The scores can be calculated by integration, independent of the LRR learning procedure. Then, a linear classifier is used to cluster out-of-sample data using their energy scores. Experimental results demonstrate the effectiveness and efficiency of the method.

MLJ

In this demonstration, we introduce MLJ (MultiLingual Journalism, http://mljournalism.com), a first Web-based system that enables users to search any topic of latest tweets posted by media outlets and journalists beyond languages. Handling multilingual tweets in real time involves many technical challenges: language barrier, sparsity of words, and real-time data stream. To overcome the language barrier and the sparsity of words, MLJ harnesses CL-ESA, a Wikipedia-based language-independent method to generate a vector of Wikipedia pages (entities) from an input text. To continuously deal with tweet stream, we propose one-pass DP-means, an online clustering method based on DP-means. Given a new tweet as an input, MLJ generates a vector using CL-ESA and classifies it into one of clusters using one-pass DP-means. By interpreting a search query as a vector, users can instantly search clusters containing latest related tweets from the query without being aware of language differences. MLJ as of March 2014 supports nine languages including English, Japanese, Korean, Spanish, Portuguese, German, French, Italian, and Arabic covering 24 countries.

On-line modeling via fuzzy support vector machines and neural networks

Unlike the other fuzzy neural networks, in this paper the fuzzy system and the neural network are separated, and they are corresponding to structure identification and parameter identification. The fuzzy model is generated automatically by on-line clustering method and fuzzy support vector machines. This fuzzy model is not updated, but its modeling error is compensated by a neural network. The neural network acts as the parameter identification model. The benefits of this new method are the structure is simple, and the physical meanings of each part of the model are clear. For the neural compensator, a new stable and fast training algorithm is proposed. Finally, this modeling method is successfully applied to model a magnetic tube recovery ratio in a metal company in China.

Video abstraction based on the visual attention model and online clustering

With the fast evolution of digital video, research and development of new technologies are greatly needed to lower the cost of video archiving, cataloging and indexing, as well as improve the efficiency and accessibility of stored video sequences. A number of methods to respectively meet these requirements have been researched and proposed. As one of the most important research topics, video abstraction helps to enable us to quickly browse a large video database and to achieve efficient content access and representation. In this paper, a video abstraction algorithm based on the visual attention model and online clustering is proposed. First, shot boundaries are detected and key frames in each shot are extracted so that consecutive key frames in a shot have the same distance. Second, the spatial saliency map indicating the saliency value of each region of the image is generated from each key frame and regions of interest (ROI) is extracted according to the saliency map. Third, key frames, as well as their corresponding saliency map, are passed to a specific filter, and several thresholds are used so that the key frames containing less information are discarded. Finally, key frames are clustered using an online clustering method based on the features in ROIs. Experimental results demonstrate the performance and effectiveness of the proposed video abstraction algorithm.

Mining spatio-temporal information on microblogging streams using a density-based online clustering method

Social networks have been regarded as a timely and cost-effective source of spatio-temporal information for many fields of application. However, while some research groups have successfully developed topic detection methods from the text streams for a while, and even some popular microblogging services such as Twitter did provide information of top trending topics for selection, it is still unable to fully support users for picking up all of the real-time event topics with a comprehensive spatio-temporal viewpoint to satisfy their information needs. This paper aims to investigate how microblogging social networks (i.e. Twitter) can be used as a reliable information source of emerging events by extracting their spatio-temporal features from the messages to enhance event awareness. In this work, we applied a density-based online clustering method for mining microblogging text streams, in order to obtain temporal and geospatial features of real-world events. By analyzing the events detected by our system, the temporal and spatial impacts of the emerging events can be estimated, for achieving the goals of situational awareness and risk management.

Evidential evolving Gustafson–Kessel algorithm for online data streams partitioning using belief function theory

A new online clustering method called E2GK (Evidential Evolving Gustafson–Kessel) is introduced. This partitional clustering algorithm is based on the concept of credal partition defined in the theoretical framework of belief functions. A credal partition is derived online by applying an algorithm resulting from the adaptation of the Evolving Gustafson–Kessel (EGK) algorithm. Online partitioning of data streams is then possible with a meaningful interpretation of the data structure. A comparative study with the original online procedure shows that E2GK outperforms EGK on different entry data sets. To show the performance of E2GK, several experiments have been conducted on synthetic data sets as well as on data collected from a real application problem. A study of parameters’ sensitivity is also carried out and solutions are proposed to limit complexity issues.

Indirect adaptive controller based on a self-structuring fuzzy system for nonlinear modeling and control

Indirect adaptive controller based on a self-structuring fuzzy system for nonlinear modeling and controlIn this paper, a unified nonlinear modeling and control scheme is presented. A self-structuring Takagi-Sugeno (T-S) fuzzy model is used to approximate the unknown nonlinear plant based on I/O data collected on-line. Both the structure and the parameters of the T-S fuzzy model are updated by an on-line clustering method and a recursive least squares estimation (RLSE) algorithm. The rules of the fuzzy model can be added, replaced or deleted on-line to allow a more flexible and compact model structure. The overall controller consists of an indirect adaptive controller and a supervisory controller. The former is the dominant controller, which maintains the closed-loop stability when the fuzzy system is a good approximation of the nonlinear plant. The latter is an auxiliary controller, which is activated when the tracking error reaches the boundary of a predefined constraint set. It is proven that global stability of the closed-loop system is guaranteed in the sense that all the closed-loop signals are bounded and simulation examples demonstrate the effectiveness of the proposed control scheme.

A Self-Evolving Interval Type-2 Fuzzy Neural Network With Online Structure and Parameter Learning

This paper proposes a self-evolving interval type-2 fuzzy neural network (SEIT2FNN) with online structure and parameter learning. The antecedent parts in each fuzzy rule of the SEIT2FNN are interval type-2 fuzzy sets and the fuzzy rules are of the Takagi-Sugeno-Kang (TSK) type. The initial rule base in the SEIT2FNN is empty, and the online clustering method is proposed to generate fuzzy rules that flexibly partition the input space. To avoid generating highly overlapping fuzzy sets in each input variable, an efficient fuzzy set reduction method is also proposed. This method independently determines whether a corresponding fuzzy set should be generated in each input variable when a new fuzzy rule is generated. For parameter learning, the consequent part parameters are tuned by the rule-ordered Kalman filter algorithm for high-accuracy learning performance. Detailed learning equations on applying the rule-ordered Kalman filter algorithm to the SEIT2FNN consequent part learning, with rules being generated online, are derived. The antecedent part parameters are learned by gradient descent algorithms. The SEIT2FNN is applied to simulations on nonlinear plant modeling, adaptive noise cancellation, and chaotic signal prediction. Comparisons with other type-1 and type-2 fuzzy systems in these examples verify the performance of the SEIT2FNN.