Multi-objective Evolutionary Clustering Research Articles

Multitask Learning for Joint Diagnosis of Multiple Mental Disorders in Resting-State fMRI.

Facing the increasing worldwide prevalence of mental disorders, the symptom-based diagnostic criteria struggle to address the urgent public health concern due to the global shortfall in well-qualified professionals. Thanks to the recent advances in neuroimaging techniques, functional magnetic resonance imaging (fMRI) has surfaced as a new solution to characterize neuropathological biomarkers for detecting functional connectivity (FC) anomalies in mental disorders. However, the existing computer-aided diagnosis models for fMRI analysis suffer from unstable performance on large datasets. To address this issue, we propose an efficient multitask learning (MTL) framework for joint diagnosis of multiple mental disorders using resting-state fMRI data. A novel multiobjective evolutionary clustering algorithm is presented to group regions of interests (ROIs) into different clusters for FC pattern analysis. On the optimal clustering solution, the multicluster multigate mixture-of-expert model is used for the final classification by capturing the highly consistent feature patterns among related diagnostic tasks. Extensive simulation experiments demonstrate that the performance of the proposed framework is superior to that of the other state-of-the-art methods. Moreover, the potential for practical application of the framework is also validated in terms of limited computational resources, real-time analysis, and insufficient training data. The proposed model can identify the remarkable interpretative biomarkers associated with specific mental disorders for clinical interpretation analysis.

An adaptive dynamic community detection algorithm based on multi-objective evolutionary clustering

PurposeCommunity detection of dynamic networks provides more effective information than static network community detection in the real world. The mainstream method for community detection in dynamic networks is evolutionary clustering, which uses temporal smoothness of community structures to connect snapshots of networks in adjacent time intervals. However, the error accumulation issues limit the effectiveness of evolutionary clustering. While the multi-objective evolutionary approach can solve the issue of fixed settings of the two objective function weight parameters in the evolutionary clustering framework, the traditional multi-objective evolutionary approach lacks self-adaptability.Design/methodology/approachThis paper proposes a community detection algorithm that integrates evolutionary clustering and decomposition-based multi-objective optimization methods. In this approach, a benchmark correction procedure is added to the evolutionary clustering framework to prevent the division results from drifting.FindingsExperimental results demonstrate the superior accuracy of this method compared to similar algorithms in both real and synthetic dynamic datasets.Originality/valueTo enhance the clustering results, adaptive variances and crossover probabilities are designed based on the relative change amounts of the subproblems decomposed by MOEA/D (A Multiobjective Optimization Evolutionary Algorithm based on Decomposition) to dynamically adjust the focus of different evolutionary stages.

Multiple population-based multi-objective evolutionary semi-supervised multi-kernel region fuzzy clustering image segmentation

Multiple population-based multi-objective evolutionary semi-supervised multi-kernel region fuzzy clustering image segmentation

Improving temporal smoothness and snapshot quality in dynamic network community discovery using NOME algorithm.

The goal of dynamic community discovery is to quickly and accurately mine the network structure for individuals with similar attributes for classification. Correct classification can effectively help us screen out more desired results, and it also reveals the laws of dynamic network changes. We propose a dynamic community discovery algorithm, NOME, based on node occupancy assignment and multi-objective evolutionary clustering. NOME adopts the multi-objective evolutionary algorithm MOEA/D framework based on decomposition, which can simultaneously decompose the two objective functions of modularization and normalized mutual information into multiple single-objective problems. In this algorithm, we use a Physarum-based network model to initialize populations, and each population represents a group of community-divided solutions. The evolution of the population uses the crossover and mutation operations of the genome matrix. To make the population in the evolution process closer to a better community division result, we develop a new strategy for node occupancy assignment and cooperate with mutation operators, aiming at the boundary nodes in the connection between the community and the connection between communities, by calculating the comparison node. The occupancy rate of the community with the neighbor node, the node is assigned to the community with the highest occupancy rate, and the authenticity of the community division is improved. In addition, to select high-quality final solutions from candidate solutions, we use a rationalized selection strategy from the external population size to obtain better time costs through smaller snapshot quality loss. Finally, comparative experiments with other representative dynamic community detection algorithms on synthetic and real datasets show that our proposed method has a better balance between snapshot quality and time cost.

A knee point driven Kriging-assisted multi-objective robust fuzzy clustering algorithm for image segmentation

To enhance the segmentation performance and optimization efficiency of multi-objective evolutionary clustering algorithms for noisy images, this study presents a knee point driven Kriging-assisted multi-objective robust fuzzy clustering algorithm for image segmentation (K2 MORFC). First, pixel and region-level fitness functions are developed to comprehensively consider the spatial constraints and region consistency in an image. In the pixel-level fitness function, a Kullback–Leibler (KL) divergence-based spatial constraint term is employed and combined with intra-class compactness to preserve the membership similarity between an arbitrary image pixel and its non-local neighbors. In the region-level fitness function, image edge information is utilized to evaluate the regional consistency of the partition. Moreover, an adaptive determination mechanism of the weight factor for the spatial constraint term is designed in the pixel-level fitness function to control the influence of the spatial constraint for each pixel. Furthermore, considering that the preference toward knee points can effectively improve the convergence performance, a knee point driven Kriging-assisted evolutionary algorithm (K2EA) is proposed to efficiently optimize these two fitness functions, including a dynamic subspace knee point searching strategy, a knee point guided environmental selection strategy, and a Kriging model management mechanism. Finally, we construct a fuzzy clustering validity index with a KL divergence-based non-local spatial constraint term to select the optimal solution from the nondominated solution set. Competitive experimental results on the DTLZ benchmark verify the effectiveness of K2EA. Segmentation results on color images from the Berkeley dataset and magnetic resonance images from the BrainWeb and IBSR datasets confirm the performance of K2MORFC.

CLUSTMOSA: Clustering for GPS trajectory data based on multi-objective simulated annealing to develop mobility application

Mobility analysis is the core idea of many applications such as vehicle navigation, trajectory analysis, POI recommendation, and traffic flow analysis. These applications collect huge spatio-temporal information represented as trajectories of a moving object such as a vehicle or people using Global Positioning System enabled devices. Various techniques are evolved to process, manage and extract useful information from trajectories. Among these techniques, clustering plays an important and integral role in developing various mobility applications. Popular traditional clustering techniques such as DBSCAN, K-means, OPTICS, hierarchical clustering, and DJ-clustering are used for this purpose. However, these techniques suffer from major issues such as entrapping in local optima and being less effective in varying densities. Further, these methods have low search capability in search space, work upon single criteria optimization, and are less scalable for the big dataset. To overcome these issues, a new multi-objective criterion-based evolutionary clustering termed CLUSTMOSA is proposed. It exploits the search capability of archived multi-objective simulated annealing (AMOSA) to cluster the dataset. It stabilizes the exploratory and exploitative behavior of the solution. In this paper, three clustering evaluation metrics are simultaneously exploited as objective functions of CLUSTMOSA. Also, a new segmentation method is presented using bearing measurement for trajectory data. It helps to eliminate multiple waypoints localized over the straight roads and prevents multiple cluster formations for the same segment. To investigate the performance, the proposed CLUSTMOSA, along with a new segmentation method using bearing measurement is compared with the state-of-art methods of trajectory data mining. The extensive experiments and analysis prove the superiority of our clustering model over state-of-art approaches.

Broad learning approach to Surrogate-Assisted Multi-Objective evolutionary fuzzy clustering algorithm based on reference points for color image segmentation

For some real-world problems, the objective function evaluation is time-consuming and computationally expensive in multi-objective evolutionary algorithms. Surrogate-assistance approaches can be used to estimate the objective function and solve the computationally expensive problem. Taking advantages of the flatted structure and incremental learning, broad learning system (BLS) has shown a great improvement of time efficiency on the premise of ensuring the performance on the classification and regression problems. In this paper, BLS is adopted to design a BLS surrogate-assisted evolutionary framework to boost the time efficiency of clustering-based image segmentation. A BLS-assisted multi-objective evolutionary fuzzy clustering algorithm with reference points (BLS-MOEFC) is proposed. Firstly, two fuzzy clustering objective functions integrating image region information are constructed as the optimized fitness functions for obtaining satisfactory segmentation quality. Secondly, in order to improve the optimization efficiency and reduce the computational time cost, a BLS surrogate model-assisted multi-objective evolutionary framework with reference points is designed to optimize the constructed objective functions. Thirdly, an adaptively updating strategy of parameters and a model update mechanism based on the non-dominated sorting are adopted to improve the prediction accuracy of the BLS surrogate model. Finally, a novel fuzzy clustering validity index integrating region information is designed to select an optimal solution from the final set of non-dominated solutions. Experimental results on Berkeley and Weizmann images show that the proposed algorithm behaves well in the segmentation performance and time cost.

A multi-objective model for optimizing the redundancy allocation, component supplier selection, and reliable activities for multi-state systems

This paper presents a multi-objective availability-redundancy allocation optimization model for a hyper-system. The hyper-system consists of B systems with shared resources. The structure of the systems is series-parallel subsystems consisting of multi-failure and multi-state components. The components may be purchased from different suppliers based on their price and discounts. It is assumed that technical and organizational activities continuously affect the components' failure rates and the subsystems' working conditions before starting the system's mission horizon. The model aims to find the optimal number and the type of the subsystems' components for all systems from each supplier and the level of the technical and organizational activities. The problem is classified as an NP-hard class of problems; thus, four multi-objective meta-heuristics are adapted to solve the proposed model. These four meta-heuristics are non-dominated sorting genetic algorithm type II (NSGA-II) and type III (NSGA-III), non-dominated ranking genetic algorithm (NRGA), and multi-objective evolutionary algorithm based on decomposition (MOEA/D). The parameters of all algorithms are calibrated using response surface methodology. Moreover, a method based on a sequential combination of multi-objective evolutionary algorithms and data clustering is used on the Pareto solutions to yield a smaller and more manageable set of prospective solutions. The results showed the superiority of the NSGA-III and the MOEA/D algorithms to solve the presented model.

An Empirical Study of Cluster-Based MOEA/D Bare Bones PSO for Data Clustering †

Previously, cluster-based multi or many objective function techniques were proposed to reduce the Pareto set. Recently, researchers proposed such techniques to find better solutions in the objective space to solve engineering problems. In this work, we applied a cluster-based approach for solution selection in a multiobjective evolutionary algorithm based on decomposition with bare bones particle swarm optimization for data clustering and investigated its clustering performance. In our previous work, we found that MOEA/D with BBPSO performed the best on 10 datasets. Here, we extend this work applying a cluster-based approach tested on 13 UCI datasets. We compared with six multiobjective evolutionary clustering algorithms from the existing literature and ten from our previous work. The proposed technique was found to perform well on datasets highly overlapping clusters, such as CMC and Sonar. So far, we found only one work that used cluster-based MOEA for clustering data, the hierarchical topology multiobjective clustering algorithm. All other cluster-based MOEA found were used to solve other problems that are not data clustering problems. By clustering Pareto solutions and evaluating new candidates against the found cluster representatives, local search is introduced in the solution selection process within the objective space, which can be effective on datasets with highly overlapping clusters. This is an added layer of search control in the objective space. The results are found to be promising, prompting different areas of future research which are discussed, including the study of its effects with an increasing number of clusters as well as with other objective functions.

Coarse–fine surrogate model driven multiobjective evolutionary fuzzy clustering algorithm with dual memberships for noisy image segmentation

To improve the segmentation performance and boost evolutionary efficiency of multiobjective evolutionary clustering algorithms on noisy images, this paper proposes a coarse–fine surrogate model driven multiobjective evolutionary fuzzy clustering algorithm with dual membership functions (CFS-MOEFC). The existing fuzzy clustering validity-based fitness functions generally consider one single fuzzy membership function, which cannot fully process the uncertainty in an image. Moreover, most existing fitness functions used in image segmentation only utilize one type of spatial information derived from the image, which cannot behave robust on images with uncertain types of noise. To deal with the above problems, dual fuzzy membership functions are first designed by utilizing the local and non-local image spatial information. Then, the designed dual membership functions and the two kinds of spatial information are both used to construct spatial information-motivated fitness functions for image segmentation. To promote the time efficiency, the coarse–fine surrogate model is employed to assist the evolutionary optimization process, such as approximating the fitness functions instead of real function evaluations and evolving satisfactory cluster centers for CFS-MOEFC. Besides, a dual memberships-driven cluster validity index combining the local and non-local spatial information is designed for selecting an optimal solution from the final non-dominated solution set of CFS-MOEFC. Experiments on Berkeley and Magnetic Resonance (MR) images indicate that CFS-MOEFC greatly improves the segmentation accuracy on images with multiple types of noise, preserves more significant detailed image information, and behaves well on time cost comparing with state-of-the-art methods.

Multi-objective evolutionary clustering for large-scale dynamic community detection

The research of dynamic community detection is becoming increasingly popular since it can disclose how the community structures change over time in dynamic networks. Evolutionary clustering is often utilized for the goal and has achieved some success, however, still has some major drawbacks: (1) The absence of error correction may lead to the result-drifting problem and the error accumulation problem; (2) The NP-hardness of modularity based community detection makes it low efficiency to get an exact solution. In this paper, an efficient and effective multi-objective method, namely DYN-MODPSO, is proposed, and where the traditional evolutionary clustering framework and the particle swarm algorithm are modified and enhanced, respectively. The main contributions include that: (1) A novel strategy, namely the recent future reference, is devised for the initial clustering result correction to make the dynamic community detection more effective; (2) The traditional particle swarm algorithm is improved and integrated with the evolutionary clustering framework by profitably exploiting the proposed strategy; (3) The de-redundant random walk based population initialization is proposed to diversify the individuals in a quality-guaranteed way. Furthermore, the multi-individual crossover operator and the improved interference operator are carefully designed to keep the solution from local optimization. Extensive experiments conducted on the real and the synthetic dynamic networks manifest that the proposed DYN-MODPSO outperforms the competitors in terms of both effectiveness and efficiency.

Multiobjective multiple features fusion: A case study in image segmentation

Most of existing image segmentation algorithms are only based on the color feature. However, the spatial distribution of an image can not be well described by using the color feature alone. Thus, it is necessary to add additional features to design efficient segmentation algorithms. Although many researchers also try to use multiple features for image segmentation, it is extremely difficult to combine multiple features automatically. This paper proposes a multiojective multiple features fusion strategy for image segmentation. The basic idea is to convert the segmentation problem into a multiobjective optimization problem, in which each objective considers one feature. It contains three steps. First, the original image is split into a set of over-segmented regions by using Meanshift to preserve the spatial details and to simplify the segmentation problem. Second, both the color and texture features are extracted to describe the regions. And two similarity matrices are designed by computing the similarity between each pair of regions in two features respectively. Third, a multiobjective evolutionary clustering algorithm is applied to merge these over-segmented regions. In this stage, two objective functions are designed based on the color and texture features respectively. A region index encoding scheme is introduced to design the individual, which contains some cluster representative regions. Some evolutionary operators are proposed to generate the new population. In the final generation, the best solution is selected from nondominated solutions for subsequent segmentation. Experiment results show that the proposed method provides promising segmentation results in combining the color and texture features.

Multi-objective evolutionary clustering with complex networks

Evolutionary clustering (EC) refers to the applications of evolutionary optimization algorithms such as genetic algorithm to data clustering. Although multi-objective evolutionary clustering algorithms were proposed to simultaneously consider different cluster properties such as compactness and separation, these techniques usually suffer from a reasonable initial population and a pre-defined number of clusters. Besides, the effectiveness of evolutionary operators is decreased in dealing with the clustering problem. On the other side, complex networks play an essential role in different fields of machine learning. In a complex network, points are considered as nodes, and the dataset is shown as a connected weighted graph. Also, complex networks tend to present a modular structure. This paper applies two concepts of complex networks including node centrality and community modularity to introduce a novel multi-objective evolutionary clustering. The proposed centrality modularity-based multi-objective evolutionary clustering (CMMOEC) takes the advantage of nodes similarity to find the best initial population of clustering solutions and provide new structural-based modularity to determine the optimal number of clusters automatically. Moreover, the proposed modularity is used to design a new recombination and mutation operator so that it generates offspring solutions that satisfy more diversity. Experiments carried out on several artificial and real-world datasets with different structures. The performance of the proposed algorithm is evaluated by the Adjusted Rand Index (ARI). Simulation results indicate that the proposed algorithm satisfies better performance in comparison to traditional methods.

Unsupervised EA-Based Fuzzy Clustering for Image Segmentation

This paper presents an unsupervised fuzzy clustering based on evolutionary algorithm for image segmentation. It needs no prior information about exact numbers of segments. Local and nonlocal spatial information derived from observed images are incorporated into fuzzy clustering process. It consists of three major procedures. First, a multi-objective evolutionary sampling is proposed to locate image pixels with a variety of image information. Secondly, optimizing fuzzy compactness and fuzzy separation, a multi-objective evolutionary fuzzy clustering with spatial information is performed on sampling pixels. The particular numbers of segments and balances of spatial information can be obtained. Then fuzzy clustering segmentation on whole image is carried out by two fuzzy clustering approaches, which are depended on fuzzy c-means and evolutionary algorithm respectively. To enhance qualities of final segmentation results, a label correction based on entropy and local spatial information is introduced. Experiments on different types of images demonstrate the effectiveness of our approach for image segmentation.

Detecting the evolving community structure in dynamic social networks

Identifying the evolving community structure of social networks has recently drawn increasing attention. Evolutionary clustering, previously proposed to detect the evolution of clusters over time, presents a temporal smoothness framework to simultaneously maximize clustering accuracy and minimize the clustering drift between two successive time steps. Under this framework, evolving patterns of communities in dynamic networks were detected by finding the best trade-off between the clustering accuracy and temporal smoothness. However, two main drawbacks in previous methods limit the effectiveness of dynamic community detection. One is that the classic operators implemented by existing methods cannot avoid that a node is often inter-connected to most of its neighbors. The other is that those methods take it for granted that an inter-connection cannot exist between nodes clustered into the same community, which results in a limited search space. In this paper, we propose a novel multi-objective evolutionary clustering algorithm called DECS, to detect the evolving community structure in dynamic social networks. Specifically, we develop a migration operator cooperating with efficient operators to ensure that nodes and their most neighbors are grouped together, and use a genome matrix encoding the structure information of networks to expand the search space. DECS calculates the modularity based on the genome matrix as one of objectives to optimize. Experimental results on synthetic networks and real-world social networks demonstrate that DECS outperforms in both clustering accuracy and smoothness, contrasted with other state-of-the-art methods.

Transfer learning-assisted multi-objective evolutionary clustering framework with decomposition for high-dimensional data

Although multi-objective evolutionary subspace clustering approaches have shown promise in handling high-dimensional datasets, their performance is restricted by two main drawbacks. First, their local search strategies have not been well investigated. Second, while exploring the search space, they neglect the useful knowledge from previously solved problems. To tackle these issues, this paper proposes a transfer learning-assisted multi-objective evolutionary clustering framework with decomposition. Firstly, we provide a decomposition-based local search strategy. To capture a comprehensive data structure, this strategy updates the weights of features by considering both the within-class compactness and between-class separation, and spontaneously balances the two properties. Secondly, we develop a knowledge transfer strategy. By transferring search experience from a previously solved clustering problem, the strategy improves the search efficiency, consequently enhances the clustering accuracy of the current problem. It has a closed-form solution and can transfer knowledge across both homogeneous and heterogeneous problems from either different or the same domains. Finally, we conduct an extensive experimental study on the framework by comparing with six representative subspace clustering approaches on a wide range of benchmarks and real-world applications. Results demonstrate the superiority of our framework.

Single-Cell RNA Sequencing Data Interpretation by Evolutionary Multiobjective Clustering.

In recent years, single-cell RNA sequencing reveals diverse cell genetics at unprecedented resolutions. Such technological advances enable researchers to uncover the functionally distinct cell subtypes such as hematopoietic stem cell subpopulation identification. However, most of the related algorithms have been hindered by the high-dimensionality and sparse nature of single-cell RNA sequencing (RNA-seq) data. To address those problems, we propose a multiobjective evolutionary clustering based on adaptive non-negative matrix factorization (MCANMF) for multiobjective single-cell RNA-seq data clustering. First, adaptive non-negative matrix factorization is proposed to decompose data for feature extraction. After that, a multiobjective clustering algorithm based on learning vector quantization is proposed to analyze single-cell RNA-seq data. To validate the effectiveness of MCANMF, we benchmark MCANMF against 15 state-of-the-art methods including seven feature extraction methods, seven clustering methods, and the kernel-based similarity learning method on six published single-cell RNA sequencing datasets comprehensively. When compared with those 15 state-of-the-art methods, MCANMF performs better than the others on those single-cell RNA sequencing datasets according to multiple evaluation metrics. Moreover, the MCANMF component analysis, time complexity analysis, and parameter analysis are conducted to demonstrate various properties of our proposed algorithm.

Noise Robust Multiobjective Evolutionary Clustering Image Segmentation Motivated by the Intuitionistic Fuzzy Information

Images are always contaminated by noise, increasing uncertainty. Fuzzy set (FS) theory is a useful tool for dealing with uncertainty in images. When comparing with the FS, an intuitionistic fuzzy set (IFS) can better describe the blurred characteristic in images due to the membership, nonmembership, and hesitation degrees. However, when applied to an image segmentation, the IFS cannot completely overcome the influence of noise. With the aim of performing noisy image segmentation under several criteria, this paper defines a noise robust IFS (NR-IFS) for an image and then presents a novel noise robust multiobjective evolutionary intuitionistic fuzzy clustering algorithm (NR-MOEIFC). A majority dominated suppressed similarity measure using the neighborhood statistics and the competitive learning is proposed to obtain the NR-IFS representation for the image corrupted by noise. Then, the NR-IFS is fully used to motivate the whole process of multiobjective evolutionary clustering: first, computing a three-parameter intuitionistic fuzzy distance measure; second, constructing intuitionistic fuzzy fitness functions; third, designing a nonuniform intuitionistic fuzzy mutation operator; and forth, defining an intuitionistic fuzzy cluster validity index to select the optimal solution from the final nondominated solution set. The histogram statistics of NR-IFS are adopted in the NR-MOEIFC to greatly reduce the computational complexity. Experimental results on Berkeley and real magnetic resonance images reveal that the NR-MOEIFC behaves well in noise robustness and segmentation performance while requiring a low time cost.

A Fast Multiobjective Fuzzy Clustering with Multimeasures Combination

Most of the existing clustering algorithms are often based on Euclidean distance measure. However, only using Euclidean distance measure may not be sufficient enough to partition a dataset with different structures. Thus, it is necessary to combine multiple distance measures into clustering. However, the weights for different distance measures are hard to set. Accordingly, it appears natural to keep multiple distance measures separately and to optimize them simultaneously by applying a multiobjective optimization technique. Recently a new clustering algorithm called ‘multiobjective evolutionary clustering based on combining multiple distance measures’ (MOECDM) was proposed to integrate Euclidean and Path distance measures together for partitioning the dataset with different structures. However, it is time-consuming due to the large-sized genes. This paper proposes a fast multiobjective fuzzy clustering algorithm for partitioning the dataset with different structures. In this algorithm, a real encoding scheme is adopted to represent the individual. Two fuzzy clustering objective functions are designed based on Euclidean and Path distance measures, respectively, to evaluate the goodness of each individual. An improved evolutionary operator is also introduced accordingly to increase the convergence speed and the diversity of the population. In the final generation, a set of nondominated solutions can be obtained. The best solution and the best distance measure are selected by using a semisupervised method. Afterwards, an updated algorithm is also designed to detect the optimal cluster number automatically. The proposed algorithms are applied to many datasets with different structures, and the results of eight artificial and six real-life datasets are shown in experiments. Experimental results have shown that the proposed algorithms can not only successfully partition the dataset with different structures, but also reduce the computational cost.

A Kriging-Assisted Reference Vector Guided Multi-Objective Evolutionary Fuzzy Clustering Algorithm for Image Segmentation

In order to reduce the computational complexity of multi-objective evolutionary optimization-based clustering algorithms, a Kriging-assisted reference vector guided multi-objective robust spatial fuzzy clustering algorithm (KRV-MRSFC) is proposed and then successfully applied to image segmentation. We first construct objective functions with noise robust local spatial information derived from the image to improve the robustness to noise and then use the Kriging model to approximate each objective function to decrease the computational cost. Meanwhile, in order to improve the approximation accuracy of the Kriging model, an angle-penalized distance-based expected improvement sampling criterion is presented in the KRV-MRSFC, which can select individuals with better exploitation and exploration to update the Kriging model. In addition, KRV-MRSFC adopts a clustering validity index with noise robust local image spatial information to select the optimal solution from the final non-dominated solution set to perform image segmentation. The experiments performed on Berkeley and real magnetic resonance images indicate that the proposed method not only achieves satisfactory segmentation performance on noisy images but also requires a low time cost.