Kernel Clustering Research Articles

Multiple Kernel Learning Based on Discriminative Kernel Clustering for Hyperspectral Band Selection

In hyperspectral images, band selection plays a crucial role for land-cover classification. Multiple kernel learning (MKL) is a popular feature selection method by selecting the relevant features and classifying the images simultaneously. Unfortunately, a large number of spectral bands in hyperspectral images result in excessive kernels, which limit the application of MKL. To address this problem, a novel MKL method based on discriminative kernel clustering (DKC) is proposed. In the proposed method, a discriminative kernel alignment (KA) (DKA) is defined. Traditional KA measures kernel similarity independently of the current classification task. Compared with KA, DKA measures the similarity of discriminative information by introducing the comparison of intraclass and interclass similarities. It can evaluate both kernel redundancy and kernel synergy for classification. Then, DKA-based affinity-propagation clustering is devised to reduce the kernel scale and retain the kernels having high discrimination and low redundancy for classification. Additionally, an analysis of necessity for DKC in hyperspectral band selection is provided by empirical Rademacher complexity. Experimental results on several hyperspectral images demonstrate the effectiveness of the proposed band selection method in terms of classification performance and computation efficiency.

Embedded non-parametric kernel learning for kernel clustering

Non-parametric kernel learning (NPKL) methods have been attracted much more attention and achieved outstanding classification performance in the past few years. Their models generally utilize pairwise constraints and are built based on the manifold assumption. But, such an assumption might be invalid for some high-dimensional and sparse data due to the curse of dimensionality, which has a negative influence on the kernel learning performance. In this paper, we try to address this problem using joint dimensionality reduction and kernel learning. Different from traditional approaches which conduct dimensionality reduction and learning tasks in sequence, we propose a novel framework which can seamlessly combine semi-supervised NPKL with dimensionality reduction. Several semi-supervised NPKL algorithms can be derived from this framework, which not only effectively utilize pairwise constraints, but can address the issue of the manifold assumption invalidation. In addition, we apply the proposed method to improve the performance of kernel clustering. Experimental results demonstrate that the proposed method outperforms state-of-the-art semi-supervised NPKL methods and can significantly enhance the performance of kernel clustering.

Two-stage SAR image segmentation framework with an efficient union filter and multi-objective kernel clustering

This paper presents a novel two-stage image segmentation framework by artificial immune system (AIS) thereof to partition synthetic aperture radar (SAR) images. The following three crucial tough problems have not been completely solved in current SAR image processing community thus far: (1) the automatic ability of discovering the true number of categories in different types of land covers; (2) the skill of smoothing speckle noise in SAR imagery, which is different from classical Gaussian and Salt & Pepper noise; (3) the better clustering performance in segmenting thousands of highly contaminating pixels in SAR image. With above three problems as goals, an effective two-stage SAR image segmentation framework (TSIS) is discussed here. Firstly, a union filter, combing maximum likelihood estimator and partial nonlocal means filter, is designed. Afterwards, a searching algorithm with variable length of chromosomes is designed to automatically discover the clustering numbers in SAR images. Finally, an efficient multi-objective clustering paradigm in AIS and kernel mapping thereof to implement final image partition is proposed. To test its performance, a systematic comparison of TSIS versus three famous variations of fuzzy c-means (FCM) and two graph partitioning methods is given. Experiments results show that TSIS can provide an effective option in segmenting the SAR imagery.

Adaptive Morphological Filtering Method for Structural Fusion Restoration of Hyperspectral Images

Recovering hyperspectral image (HSI) from mixed noise degradation is a challenging and promising theme in remote sensing, particularly when stripes and deadlines exist in several contiguous bands. This paper proposes a HSI's restoration method making use of adaptive morphological filtering (AMF) and fusing structure information of an auxiliary color image. An adaptive structuring element (ASE) indicating morphological features of each pixel is generated through information fusion, to simultaneously remove the mixed noise and preserve fine spatial structures. This key technology contains three main steps. First, edges are extracted from the auxiliary image exploiting its color information; then, an edge-constraint growing algorithm is used to generate the clustering kernel; finally, the ASE is obtained via goal-guided k-means clustering. The ASE has extensive application value, for it can be an enhancing module for most filters-based restoration methods, to mitigate the structural damage due to the fixed mask. Among these methods, Gaussian filter for preprocessing and majority voting for postprocessing are introduced in this paper as representatives. In addition, the auxiliary image can be both visible image of multisensor and false RGB component of the undamaged bands of the HSI, so it is relatively available. Experiments on simulated and real data sets show obvious effects on denoising and destriping both subjectively and objectively. The advantage of ASE on structure details preserving, compared to conventional approaches, is clearly demonstrated. The application value of the proposed restoration frame and ASE is further proved through the decision-level postprocessing experiments.

Research on New Multi-Feature Large-Scale Image Retrieval Algorithm based on Semantic Parsing and Modified Kernel Clustering Method

Because of the feature points can describe the local characteristics of the image in a reasonable manner, effective use of feature point of content based image retrieval become the current hot issues in the field of computer vision. Aiming at this problem, we put forward a kind of combination clustering based on feature points, a new method of image retrieval. The method includes the combination of feature point clustering algorithm and based on the algorithm of local color histogram construction strategy. With the existing and local color histogram retrieval method based on feature points, compared to the method can effectively solve the current method of feature point location information and feature point center relying too much on the problem. Subjectivity and as a result of the manual annotation image accuracy, the traditional image retrieval methods cannot meet the needs of the user. Multidimensional indexing technology is only from the perspective of how to improve the indexing algorithm to adapt to the large-scale database to consider a problem, in content-based image retrieval. Our research combines the advantages of the semantic analysis and kernel clustering which will enhance the performance of the traditional image retrieval methods and strengthen the feasibility of the algorithm.

An Improved Kernel Clustering Algorithm for Mixed-Type Data in Network Forensic

Clustering algorithm is a common analysis technology for network forensics, which, lacking of any prior knowledge, can effectively find out the invasions by analyzing the collected real-time communication data flowing through the network. This paper proposed an improved dynamic kernel clustering algorithm for mixed numeric and categorical network communication data. First, centroid prototype based on the mean and distribution centroid was put forward to represent the cluster center. Then by using Gaussian kernel function, the paper introduced a new dissimilarity measure between the data object and the centroid prototype in combination with the significance of different categorical values. On this basis, the objective function was defined, which took into account both the compact degree in a cluster and the discrete degree among the clusters. After that an improved kernel clustering algorithm was designed. In the process of clustering, centroid prototype and the value of the clustering parameter dynamically updated for a better description of the characteristics of clusters’ change. Finally, in order to verify the feasibility and effectiveness of the algorithm, the paper further applied it to network forensics, and the experimental results showed that the method could mine the intrusion behavior more accurately.

Combination of Ant Colony Optimization with Local Triangular Kernel Clustering for Vehicle Routing Problem with Time Windows

VRP is a common problem that occurred in logistics, including determination a route of products from the source to the destination. VRPTW is variation of VRP that use routing concepts in the serving process at the certain time interval. Recently, many methods are used to solve this optimization problem, for example ACO. LTKC-ACO was developed to improve the ACO solutions that apply LTKC to obtain a number of classes that are considered as the candidate list in ACO. Local Search is also used to avoid ACO getting stuck in the local optimum. In this study, two types of LTKC-ACO are developed that’s related to time windows parameter usage in clustering. The experimental result of 56 Solomon’s datasets showed that LTKC-ACO can improve the ACO solutions on 73,21% of datasets and can out performed then the other methods, especially on the datasets that have longer scheduling of service time.

Combination of Ant Colony Optimization with Local Triangular Kernel Clustering for Vehicle Routing Problem with Time Windows

VRP is a common problem that occurred in logistics, including determination a route of products from the source to the destination. VRPTW is variation of VRP that use routing concepts in the serving process at the certain time interval. Recently, many methods are used to solve this optimization problem, for example ACO. LTKC-ACO was developed to improve the ACO solutions that apply LTKC to obtain a number of classes that are considered as the candidate list in ACO. Local Search is also used to avoid ACO getting stuck in the local optimum. In this study, two types of LTKC-ACO are developed that’s related to time windows parameter usage in clustering. The experimental result of 56 Solomon’s datasets showed that LTKC-ACO can improve the ACO solutions on 73,21% of datasets and can out performed then the other methods, especially on the datasets that have longer scheduling of service time. Full Text: PDF DOI: http://dx.doi.org/10.11591/telkomnika.v16i2.8800

A Hybrid Kernel-Based Change Detection Method for Remotely Sensed Data in a Similarity Space

Detection of damages caused by natural disasters is a delicate and difficult task due to the time constraints imposed by emergency situations. Therefore, an automatic Change Detection (CD) algorithm, with less user interaction, is always very interesting and helpful. So far, there is no existing CD approach that is optimal and applicable in the case of (a) labeled samples not existing in the study area; (b) multi-temporal images being corrupted by either noise or non-normalized radiometric differences; (c) difference images having overlapped change and no-change classes that are non-linearly separable from each other. Also, a low degree of automation is not optimal for real-time CD applications and also one-dimensional representations of classical CD methods hide the useful information in multi-temporal images. In order to resolve these problems, two automatic kernel-based CD algorithms (KCD) were proposed based on kernel clustering and support vector data description (SVDD) algorithms in high dimensional Hilbert space. In this paper (a( a new similarity space was proposed in order to increase the separation between change and no-change classes, and also to decrease the processing time, (b) three kernel-based approaches were proposed for transferring the multi-temporal images from spectral space into high dimensional Hilbert space, (c) automatic approach was proposed to extract the precise labeled samples; (d) kernel parameter was selected automatically by optimizing an improved cost function and (e) initial value of the kernel parameter was estimated by a statistical method based on the L2-norm distance. Two different datasets including Quickbird and Landsat TM/ETM+ imageries were used for the accuracy of analysis of proposed methods. The comparative analysis showed the accuracy improvements of kernel clustering based CD and SVDD based CD methods with respect to the conventional CD techniques such as Minimum Noise Fraction, Independent Component Analysis, Spectral Angle Mapper, Simple Image differencing and Image Rationing, and also the computational cost analysis showed that implementation of the proposed CD method in similarity space decreases the processing runtime.

An effective fuzzy kernel clustering analysis approach for gene expression data.

Fuzzy clustering is an important tool for analyzing microarray data. A major problem in applying fuzzy clustering method to microarray gene expression data is the choice of parameters with cluster number and centers. This paper proposes a new approach to fuzzy kernel clustering analysis (FKCA) that identifies desired cluster number and obtains more steady results for gene expression data. First of all, to optimize characteristic differences and estimate optimal cluster number, Gaussian kernel function is introduced to improve spectrum analysis method (SAM). By combining subtractive clustering with max-min distance mean, maximum distance method (MDM) is proposed to determine cluster centers. Then, the corresponding steps of improved SAM (ISAM) and MDM are given respectively, whose superiority and stability are illustrated through performing experimental comparisons on gene expression data. Finally, by introducing ISAM and MDM into FKCA, an effective improved FKCA algorithm is proposed. Experimental results from public gene expression data and UCI database show that the proposed algorithms are feasible for cluster analysis, and the clustering accuracy is higher than the other related clustering algorithms.

A cluster-based method for marine sensitive object extraction and representation

Within the context of global change, marine sensitive factors or Marine Essential Climate Variables have been defined by many projects, and their sensitive spatial regions and time phases play significant roles in regional sea-air interactions and better understanding of their dynamic process. In this paper, we propose a cluster-based method for marine sensitive region extraction and representation. This method includes a kernel expansion algorithm for extracting marine sensitive regions, and a field-object triple form, integration of object-oriented and field-based model, for representing marine sensitive objects. Firstly, this method recognizes ENSO-related spatial patterns using empirical orthogonal decomposition of long term marine sensitive factors and correlation analysis with multiple ENSO index. The cluster kernel, defined by statistics of spatial patterns, is initialized to carry out spatial expansion and cluster mergence with spatial neighborhoods recursively, then all the related lattices with similar behavior are merged into marine sensitive regions. After this, the Field-object triple form of is used to represent the marine sensitive objects, both with the discrete object with a precise extend and boundary, and the continuous field with variations dependent on spatial locations. Finally, the marine sensitive objects about sea surface temperature are extracted, represented and analyzed as a case of study, which proves the effectiveness and the efficiency of the proposed method.

Smart Multitask Bregman Clustering and Multitask Kernel Clustering

Traditional clustering algorithms deal with a single clustering task on a single dataset. However, there are many related tasks in the real world, which motivates multitask clustering. Recently some multitask clustering algorithms have been proposed, and among them multitask Bregman clustering (MBC) is a very applicable method. MBC alternatively updates clusters and learns relationships between clusters of different tasks, and the two phases boost each other. However, the boosting does not always have positive effects on improving the clustering performance, it may also cause negative effects. Another issue of MBC is that it cannot deal with nonlinear separable data. In this article, we show that in MBC, the process of using cluster relationship to boost the cluster updating phase may cause negative effects, that is, cluster centroids may be skewed under some conditions. We propose a smart multitask Bregman clustering (S-MBC) algorithm which can identify the negative effects of the boosting and avoid the negative effects if they occur. We then propose a multitask kernel clustering (MKC) framework for nonlinear separable data by using a similar framework like MBC in the kernel space. We also propose a specific optimization method, which is quite different from that of MBC, to implement the MKC framework. Since MKC can also cause negative effects like MBC, we further extend the framework of MKC to a smart multitask kernel clustering (S-MKC) framework in a similar way that S-MBC is extended from MBC. We conduct experiments on 10 real world multitask clustering datasets to evaluate the performance of S-MBC and S-MKC. The results on clustering accuracy show that: (1) compared with the original MBC algorithm MBC, S-MBC and S-MKC perform much better; (2) compared with the convex discriminative multitask relationship clustering (DMTRC) algorithms DMTRC-L and DMTRC-R which also avoid negative transfer, S-MBC and S-MKC perform worse in the (ideal) case in which different tasks have the same cluster number and the empirical label marginal distribution in each task distributes evenly, but better or comparable in other (more general) cases. Moreover, S-MBC and S-MKC can work on the datasets in which different tasks have different number of clusters, violating the assumptions of DMTRC-L and DMTRC-R. The results on efficiency show that S-MBC and S-MKC consume more computational time than MBC and less computational time than DMTRC-L and DMTRC-R. Overall S-MBC and S-MKC are competitive compared with the state-of-the-art multitask clustering algorithms in synthetical terms of accuracy, efficiency and applicability.

A Comparative Study of MRI Image Segmentation based on Fast Kernel Clustering Analysis

Kernel-based clustering provides a better analysis tool for pattern classification, which implicitly maps input samples to a highdimensional space for improving pattern separability. For this implicit space map, the kernel trick is believed to elegantly tackle the problem of “curse of dimensionality”, which has actually been more challenging for kernel-based clustering in terms of computational complexity and classification accuracy, which traditional kernelized algorithms cannot effectively deal with. In this paper, we have analyzed the merits and deficiencies of KFCM-I/KFCM-II, and KFMC-III and pointed out the connections of these three algorithms.

Reservoir Geological Uncertainty Reduction: an Optimization-Based Method Using Multiple Static Measures

Uncertainty in reservoir geological properties has a major impact in reservoir design and operations decision-making. To quantify the production uncertainty and to make optimal decisions in reservoir development, flow simulation is widely used. However, reservoir flow simulation is a computationally intensive task due to complex geological heterogeneities and numerical thermal modeling. Normally only a small number of realizations are chosen from a large superset for flow simulation. In this paper, a mixed-integer linear optimization-based geological realization reduction method is proposed to select geological realizations. The method minimizes the probability distance between the discrete distribution represented by the superset of realizations and the reduced discrete distribution represented by the selected realizations. The proposed method was compared with the traditional ranking technique and the distance-based kernel clustering method. Results show that the proposed method can effectively select realizations and assign probabilities such that the extreme and expected reservoir performances are recovered better than any of the single static measure-based ranking methods or the kernel clustering method.

An optimal method for data clustering

An algorithm for optimizing data clustering in feature space is studied in this work. Using graph Laplacian and extreme learning machine (ELM) mapping technique, we develop an optimal weight matrix W for feature mapping. This work explicitly performs a mapping of the original data for clustering into an optimal feature space, which can further increase the separability of original data in the feature space, and the patterns points in same cluster are still closely clustered. Our method, which can be easily implemented, gets better clustering results than some popular clustering algorithms, like k-means on the original data, kernel clustering method, spectral clustering method, and ELM k-means on data include three UCI real data benchmarks (IRIS data, Wisconsin breast cancer database, and Wine database).

A novel granular support vector machine based on fuzzy kernel clustering

For the traditional granular support vector machine (GSVM), the training samples are granulated in the original space and then are mapped into the kernel space. However, this method will lead to the inconsistent distribution of the data between original space and kernel space, thereby reducing the generalisation of GSVM. To solve this problem, a method called granular support vector machine based on fuzzy kernel cluster is proposed. This method directly granulates the training data and selects support vector particles in kernel space, and then trains the support vector particles in the same kernel space by GSVM. Finally, the experiments on benchmark datasets demonstrate the effectiveness of the proposed approach.

Computer Vision-Based Color Image Segmentation with Improved Kernel Clustering

Abstract Color image segmentation has been widely applied to diverse fields in the past decades for containing more information than gray ones, whose essence is a process of clustering according to the color of pixels. However, traditional clustering methods do not scale well with the number of data, which limits the ability of handling massive data effectively. We developed an improved kernel clustering algorithm for computing the different clusters of given color images in kernel-induced space for image segmentation. Compared to other popular algorithms, it has the competitive performances both on training time and accuracy. The experiments performed on both synthetic and real-world data sets demonstrate the validity of the proposed algorithm.

Intelligent Kernel K-Means for Clustering Gene Expression

Abstract Intelligent Kernel K-Means is a fully unsupervised clustering algorithm based on kernel. It is able to cluster kernel matrix without any information regarding to the number of required clusters. Our experiment using gene expression of human colorectal carcinoma had shown that the genes were grouped into three clusters. Global silhouette value and davies-bouldin index of the resulted clusters indicated that they are trustworthy and compact. To analyze the relationship between the clustered genes and phenotypes of clinical data, we performed correlation (CR) between each of three phenotypes (distant metastasis, cancer and normal tissues, and lymph node) with genes in each cluster of original dataset and permuted dataset. The result of the correlation had shown that Cluster 1 and Cluster 2 of original dataset had significantly higher CR than that of the permuted dataset. Among the three clusters, Cluster 3 contained smallest number of genes, but 16 out of 21 genes in that cluster were genes listed in Tumor Classifier List (TCL).

Data preprocessing and re kernel clustering for letter

Many classifiers and methods are proposed to deal with letter recognition problem. Among them, clustering is a widely used method. But only one time for clustering is not adequately. Here, we adopt data preprocessing and a re kernel clustering method to tackle the letter recognition problem. In order to validate effectiveness and efficiency of proposed method, we introduce re kernel clustering into Kernel Nearest Neighbor classification (KNN), Radial Basis Function Neural Network (RBFNN), and Support Vector Machine (SVM). Furthermore, we compare the difference between re kernel clustering and one time kernel clustering which is denoted as kernel clustering for short. Experimental results validate that re kernel clustering forms fewer and more feasible kernels and attain higher classification accuracy.

Technology Focus: Data Management and Communication (October 2014)

Technology Focus The Petroleum Domain Practitioner—A Multiscience Talented Professional The pursuit of excellence in the upstream petroleum area in the last 3 decades has been supported by rapidly changing digital-information technologies as well as by classical sciences such as math, thermodynamics, and transport phenomena; however, interestingly, it has been reinforced by data-driven sciences such as descriptive statistics, automatic-process-control theory, data mining, and predictive-analytics-modeling techniques. The latter have been referred to as artificial-intelligence techniques as they are incorporated into the oil field. Data-driven knowledge-search case histories proliferate throughout the petroleum (and other) literature. Many of those have clear value, self-learning tutorials, and simple and clear implementation paths; others are far from practical. The development and production of oil and gas resources supported by more-available continuous real-time data sources entail incessant learning and knowledge search over a relatively long period of time. As we drill more wells, as we produce for various decades and increasingly acquire more data and information, uncertainty about the success of new wells is progressively reduced and production forecasts and economic value of oil and gas fields are continuously improved. Fundamental-Physics vs. Data-Driven Models Because fundamental-physics models intend to prompt us about the thermodynamic principles that hold data together, engineers identify and characterize the fundamental modeling elements affecting petroleum production forecasts in a multidisciplinary fashion and, at the same time, analyze existing and new data on the basis of cross-validation approaches and dimensionality-reduction modeling techniques to uncover patterns that would be useful for making predictions on production and recovery. A hybrid of these approaches would provide the most promising approach to oilfield-knowledge management. The fundamental-physics approach would propose the structure of the models, and data analysis would refine possible model structures and determine values or related parameters so robust predictions can be made. For example, time and spatial dependency are important factors in the resolution of uncertainty associated with oil and gas occurrence and performance. Therefore, hybrid approaches considering both data-driven modeling and physics modeling that considers fundamental analytic relations of time and space seem to be reasonable approaches. The multiple time and space scales we manage in the oil field require further advances in how data are validated and interpreted and, most important, how results are communicated fast enough to make the right decisions at the right time. The following selected case studies demonstrate how these hybrid approaches can be used to improve our geological knowledge as more wells are drilled (SPE 164816), how to uncover patterns to increase recovery as new key events in data are interpreted (SPE 167836), and how to optimize drilling as live data are analyzed (OTC 25164). JPT Recommended additional reading at OnePetro: www.onepetro.org. SPE 164465 Improved Permeability Prediction From Seismic and Log Data Using Artificial-Intelligence Techniques by Fatai A. Anifowose, King Fahd University of Petroleum and Minerals, et al. SPE 167398 Automated Work Flows To Monitor, Diagnose, Optimize, and Perform Multiscenario Forecasts of Waterflooding in Low-Permeability Carbonate Reservoirs by P. Ranjan, Halliburton, et al. SPE 167399 Multivariate Analysis of Job-Pause-Time Data Using Classification and Regression Tree and Kernel Clustering by Marko Maucec, Halliburton, et al. SPE 167839 Advanced Machine- Learning Methods for Production- Data-Pattern Recognition by Niranjan Subrahmanya, ExxonMobil Research and Engineering Company, et al.