Distributed Clustering Techniques Research Articles

Distributed Clustering Approach by Apache Pyspark Based on SEER for Clinical Data

Data clustering is a thoroughly studied data mining issue. As the amount of information being analyzed grows exponentially, there are several problems with clustering diagnostic large datasets like the monitoring, microbiology, and end results (SEER) carcinoma feature sets. These traditional clustering methods are severely constrained in terms of speed, productivity, and adaptability. This paper summarizes the most modern distributed clustering algorithms, organized according to the computing platforms used to process vast volumes of data. The purpose of this work was to offer an optimized distributed clustering strategy for reducing the algorithm’s total execution time. We obtained, preprocessed, and analyzed clinical SEER data on liver cancer, respiratory cancer, human immunodeficiency virus (HIV)-related lymphoma, and lung cancer for large-scale data clustering analysis. Three major contributions and their effects were covered in this paper: To begin, three current Pyspark distributed clustering algorithms were evaluated on SEER clinical data using a simulated New York cancer dataset. Second, systemic inflammatory response syndrome (SIRS) model inference was done and described using three SEER cancer datasets. Third, employing lung cancer data, we suggested an optimized distributed bisecting [Formula: see text]-means method. We have shown the outcomes of our suggested optimized distributed clustering technique, demonstrating the performance enhancement.

Distributed robust data clustering in wireless sensor networks using diffusion moth flame optimization

Traditional K-Means based distributed clustering used in wireless sensor networks has limitation of getting stuck into local minima, thus many times results in giving inaccurate cluster partitions. To alleviate this drawback, evolutionary based robust distributed clustering techniques are proposed in this paper. These techniques have the capability to determine the global optima thus results in effective cluster partitioning. A moth flame optimization based method is proposed here which minimizes the intra-cluster distance to determine the optimal partition at every sensor node. A diffusion method of cooperation is employed by sharing the best moth position and corresponding fitness value (intra cluster distance) to the neighboring nodes. To introduce robustness, a weight based method for detection and removal of outliers is employed. In this method a weight based on volume and density is given to each data point for outlier detection, a larger weight is considered as an outlier. The simulation study is carried out on one synthetic and two real datasets. The performance of proposed approach is compared with diffusion particle swarm optimization (DPSO), diffusion whale optimization algorithm (DWOA), diffusion elephant herding optimization (DEHO) and distributed K-Means (DK-Means) in terms of Dunn’s index, Silhouette index and time complexity. The minimum average Euclidean deviation of proposed diffusion moth flame optimization is 7.16%, 3.25%, 5.24% and 21.70% lower compared to DPSO, DWOA, DEHO and DK-Means respectively for cook agronomy farm dataset.

A Survey on Distributed Clustering Techniques for Wireless Sensor Networks

International Journal of Computer Sciences and Engineering (A UGC Approved and indexed with DOI, ICI and Approved, DPI Digital Library) is one of the leading and growing open access, peer-reviewed, monthly, and scientific research journal for scientists, engineers, research scholars, and academicians, which gains a foothold in Asia and opens to the world, aims to publish original, theoretical and practical advances in Computer Science,Information Technology, Engineering (Software, Mechanical, Civil, Electronics & Electrical), and all interdisciplinary streams of Computing Sciences. It intends to disseminate original, scientific, theoretical or applied research in the field of Computer Sciences and allied fields. It provides a platform for publishing results and research with a strong empirical component. It aims to bridge the significant gap between research and practice by promoting the publication of original, novel, industry-relevant research.

Parallel and distributed clustering framework for big spatial data mining

ABSTRACTClustering techniques are very attractive for identifying and extracting patterns of interests from datasets. However, their application to very large spatial datasets presents numerous challenges such as high-dimensionality, heterogeneity, and high complexity of some algorithms. Distributed clustering techniques constitute a very good alternative to the Big Data challenges (e.g., Volume, Variety, Veracity, and Velocity). In this paper, we developed and implemented a Dynamic Parallel and Distributed clustering (DPDC) approach that can analyse Big Data within a reasonable response time and produce accurate results, by using existing and current computing and storage infrastructure, such as cloud computing. The DPDC approach consists of two phases. The first phase is fully parallel and it generates local clusters and the second phase aggregates the local results to obtain global clusters. The aggregation phase is designed in such a way that the final clusters are compact and accurate while the overall process is efficient in time and memory allocation. DPDC was thoroughly tested and compared to well-known clustering algorithms BIRCH and CURE. The results show that the approach not only produces high-quality results but also scales up very well by taking advantage of the Hadoop MapReduce paradigm or any distributed system.

A Cluster-Based Data Fusion Technique to Analyze Big Data in Wireless Multi-Sensor System

With the development of the latest technologies and changes in market demand, the wireless multi-sensor system is widely used. These multi-sensors are integrated in a way that produces an overwhelming amount of data, termed as big data. The multi-sensor system creates several challenges, which include getting actual information from big data with high accuracy, increasing processing efficiency, reducing power consumption, providing a reliable route toward destination using minimum bandwidth, and so on. Such shortcomings can be overcome by exploiting some novel techniques, such as clustering, data fusion, and coding schemes. Moreover, data fusion and clustering techniques are proven architectures that are used for efficient data processing; resultant data have less uncertainty, providing energy-aware routing protocols. Because of the limited resources of the multi-sensor system, it is a challenging task to reduce the energy consumption to survive a network for a longer period. Keeping challenges above in view, this paper presents a novel technique by using a hybrid algorithm for clustering and cluster member selection in the wireless multi-sensor system. After the selection of cluster heads and member nodes, the proposed data fusion technique is used for partitioning and processing the data. The proposed scheme efficiently reduces the blind broadcast messages but also decreases the signal overhead as the result of cluster formation. Afterward, the routing technique is provided based on the layered architecture. The proposed layered architecture efficiently minimizes the routing paths toward the base station. Comprehensive analysis is performed on the proposed scheme with state-of-the-art centralized clustering and distributed clustering techniques. From the results, it is shown that the proposed scheme outperforms competitive algorithms in terms of energy consumption, packet loss, and cluster formation.

A Wireless Intelligent Network for Industrial Control

In this paper, we describe a distributed clustering technique for wireless sensor node tracking in an industrial environment. The research builds on extant work on wireless sensor node clustering by reporting on: (1) the development of a novel distributed management approach for tracking mobile nodes in an industrial wireless sensor network; and (2) an objective assessment of the cluster management approach both in terms of its tracking performance and its use of network resources. To support this work, we introduce two classes of metrics: a set of three tracking performance metrics, and a set of three network efficiency metrics. The results of our experiments show that the proposed distributed system adapts readily to changes in the sensing environment, but this higher level of adaptability is at the cost of overall efficiency.

A Comparison of Alternative Distributed Dynamic Cluster Formation Techniques for Industrial Wireless Sensor Networks.

In this paper, we investigate alternative distributed clustering techniques for wireless sensor node tracking in an industrial environment. The research builds on extant work on wireless sensor node clustering by reporting on: (1) the development of a novel distributed management approach for tracking mobile nodes in an industrial wireless sensor network; and (2) an objective comparison of alternative cluster management approaches for wireless sensor networks. To perform this comparison, we focus on two main clustering approaches proposed in the literature: pre-defined clusters and ad hoc clusters. These approaches are compared in the context of their reconfigurability: more specifically, we investigate the trade-off between the cost and the effectiveness of competing strategies aimed at adapting to changes in the sensing environment. To support this work, we introduce three new metrics: a cost/efficiency measure, a performance measure, and a resource consumption measure. The results of our experiments show that ad hoc clusters adapt more readily to changes in the sensing environment, but this higher level of adaptability is at the cost of overall efficiency.

Lifetime Enhancement of WSN using Energy-Balanced Distributed Clustering Algorithm with Honey Bee Optimization

Wireless sensor network consists of sensor nodes with sensing and communication capabilities. Typical communication model of wireless sensor network consists of several sensor nodes as group members, sensor head and sinks. Past few years have witnessed increased in the potential use of Wireless Sensor Network (WSN) such as military surveillance, tracking and monitoring, disaster management and combat field reconnaissance. Many issues in WSN are formulated as multidimensional optimization problem and solved through energy based distributed clustering techniques [1]. The main problem in WSN is the lifetime of network. To support scalability, nodes are often grouped in clusters having a leader, often referred as Sensor Head (SH). A SH is responsible for not only sending data to base station but also assists the general nodes to send sensed data to target nodes. The energy consumption of SH is greater than general nodes. Therefore SH selection will affect the lifetime of WSN and also support packet loss. This proposed approach is based on Energy-Balanced Distributed Clustering (EBDC) [2–3] algorithm. Honey bee optimization with some parameters is employed over EBDC algorithm for effective dynamic cluster head selection. This approach helps in increasing the energy consumption. This proposed technique works in three stages: Cluster nodes send data to SH, SH sends data directly to leader and leader sends data to Base Station (BS). Sensor Head (SH) has responsibility to select neighbor node as a Sub Sensor Head (SSH) to reduce overload. Simulations results demonstrate that EBDC-HBO algorithm improves the life time of network.

Distributed Information Theoretic Clustering

Distributed data collection and analysis over networks are ubiquitous, especially over the wireless sensor networks (WSNs). Distributed clustering is one of the most important topics in distributed data analysis. It is desired to explore the hidden structure of the data collected/stored in geographically distributed nodes. In recent years, several distributed data clustering techniques have been developed based on the K-means algorithm or the Gaussian mixture model. In these methods, data structures are captured by measures only based on the first and the second order statistics. When the structure of cluster data is complicated, these statistics are insufficient and may lead to unsatisfactory clustering results. In such a case, using information theoretic measures can achieve better clustering performance since they take the whole distribution of cluster data into account. In this work, we incorporate an information theoretic measure into the cost function of the distributed clustering, to present a linear and a kernel distributed clustering algorithms. In the algorithms, each node solves a local clustering problem through diffusion cooperation with its neighboring nodes. In order to preserve privacy and save communication costs, in the cooperation, nodes merely exchange a few parameters instead of original data with their one-hop neighbors. Simulation results show that the proposed distributed algorithms can achieve almost as good clustering results as the corresponding centralized information theoretic clustering algorithms on both synthetic and real data.

An Efficient Blocking Technique for Reference Matching using MapReduce

Document Clustering has become an increasingly important task in the area of data mining and information retrieval. With growing data volumes, CPU—and memory-efficient techniques for clustering algorithms are receiving considerable attention in the research community. To deal with huge amounts of data (e.g., documents from Wikipedia or CiteSeerX which are several GB in size), distributed clustering techniques have been designed to provide scalable and flexible approaches. We study the problem of document clustering in the area of Entity Matching, where documents from various data sources are matched together. More specifically, we focus on a common optimization technique called blocking which reduces the enormous search space by clustering the data sources into smaller groups and processes comparisons only within a group. In this article, we describe our experiences and findings in applying the MapReduce framework to deal with huge bibliographic data sets and to provide a flexible, scalable and easy-to-use blocking technique to reduce the search space for Entity Matching.

A Database Grid System for Multi-Domain Dynamic Data Integration

With the richness of common database resources, distributed users in wide areas hope to transparently access and use these data resources on demand. DS_Grid (database grid) is an SOA (service-oriented architecture) based database grid system for data sharing in multiple application domains. DS_Grid adopts a P2P (peer-to-peer) Multi-Chord (MultiChord) architecture to realize the distributed storage, query processing and dynamic data integration of data resources. According to the text similarity, the data resources are registered to the corresponding domains to realize rapidly discovering data resources. The domain ontology knowledge and reasoning rules are used to support the semantics based intelligent query. A multi-root and multi-peer maintenance based data resources replica management mechanism is applied to improve the reliability of the system. A keyword filter based distributed data integration strategy is adopted to reduce the communication cost. A distributed clustering technique is used to summarize the huge data information. The experiments demonstrate the feasibility and effectiveness of the key techniques of DS_Grid.

Distributed Clustering Using Collective Principal Component Analysis

This paper considers distributed clustering of high-dimensional heterogeneous data using a distributed principal component analysis (PCA) technique called the collective PCA. It presents the collective PCA technique, which can be used independent of the clustering application. It shows a way to integrate the Collective PCA with a given off-the-shelf clustering algorithm in order to develop a distributed clustering technique. It also presents experimental results using different test data sets including an application for web mining.