Computational Intelligence Community Research Articles

A novel multi-hybrid differential evolution algorithm for optimization of frame structures

Differential evolution (DE) is a robust optimizer designed for solving complex domain research problems in the computational intelligence community. In the present work, a multi-hybrid DE (MHDE) is proposed for improving the overall working capability of the algorithm without compromising the solution quality. Adaptive parameters, enhanced mutation, enhanced crossover, reducing population, iterative division and Gaussian random sampling are some of the major characteristics of the proposed MHDE algorithm. Firstly, an iterative division for improved exploration and exploitation is used, then an adaptive proportional population size reduction mechanism is followed for reducing the computational complexity. It also incorporated Weibull distribution and Gaussian random sampling to mitigate premature convergence. The proposed framework is validated by using IEEE CEC benchmark suites (CEC 2005, CEC 2014 and CEC 2017). The algorithm is applied to four engineering design problems and for the weight minimization of three frame design problems. Experimental results are analysed and compared with recent hybrid algorithms such as laplacian biogeography based optimization, adaptive differential evolution with archive (JADE), success history based DE, self adaptive DE, LSHADE, MVMO, fractional-order calculus-based flower pollination algorithm, sine cosine crow search algorithm and others. Statistically, the Friedman and Wilcoxon rank sum tests prove that the proposed algorithm fares better than others.

METHOD OF BUILDING ENSEMBLES OF MODELS FOR DATA CLASSIFICATION BASED ON DECISION CORRELATIONS

The scientific work highlights the problem of increasing the accuracy of binary classification predictions using machine learning algorithms. Over the past few decades, systems that consist of many machine learning algorithms, also called ensemble models, have received increasing attention in the computational intelligence and machine learning community. This attention is well deserved, as ensemble systems have proven to be very effective and extremely versatile in a wide range of problem domains and real-world applications. One algorithm may not make a perfect prediction for a particular data set. Machine learning algorithms have their limitations, so creating a model with high accuracy is a difficult task. If you create and combine several models by combining and aggregating the results of each model, there is a chance to improve the overall accuracy, this problem is dealt with by ensembling. The basis of the information system of binary classification is the ensemble model. This model, in turn, contains a set of unique combinations of basic classifiers – a kind of algorithmic primitives. An ensemble model can be considered as some kind of meta-algorithm, which consists of unique sets of machine learning (ML) classification algorithms. The task of the ensemble model is to find such a combination of basic classification algorithms that would give the highest performance. The performance is evaluated according to the main ML metrics in classification tasks. Another aspect of scientific work is the creation of an aggregation mechanism for combining the results of basic classification algorithms. That is, each unique combination within the ensemble consists of a set of basic models (harbingers), the results of which must be aggregated. In this work, a non-hierarchical clustering method is used to aggregate (average) the predictions of the base models. A feature of this study is to find the correlation coefficients of the base models in each combination. With the help of the magnitude of correlations, the relationship between the prediction of the classifier (base model) and the true value is established, as a result of which space is opened for further research on improving the ensemble model (meta-algorithm)

A practical tutorial on solving optimization problems via PlatEMO

PlatEMO is an open-source platform for solving complex optimization problems, which provides a variety of metaheuristics including evolutionary algorithms, swarm intelligence algorithms, multi-objective optimization algorithms, surrogate-assisted optimization algorithms, and many others. Due to the problem-independent nature of most metaheuristics, they are versatile for solving problems with various difficulties such as multimodal landscapes, discrete search spaces, multiple objectives, strict constraints, and expensive evaluations, regardless of the fields the problems belong to. Since PlatEMO was published in 2017, it has been used by many researchers from both academia and industry in the computational intelligence community. However, the basic terms and concepts about optimization may confuse practitioners and junior researchers new to metaheuristics. Hence, this paper presents a practical introduction to the use of PlatEMO 4.0, focusing on the procedures of defining problems, selecting suitable metaheuristics, and collecting results. Note, however, that a description of the technical details of metaheuristics is beyond the scope of this paper and interested readers may refer to the cited references.

Personalized Image Aesthetics Assessment via Multi-Attribute Interactive Reasoning

Due to the subjective nature of people’s aesthetic experiences with respect to images, personalized image aesthetics assessment (PIAA), which can simulate the aesthetic experiences of individual users to estimate images, has received extensive attention from researchers in the computational intelligence and computer vision communities. Existing PIAA models are usually built on prior knowledge that directly learns the generic aesthetic results of images from most people or the personalized aesthetic results of images from a large number of individuals. However, the learned prior knowledge ignores the mutual influence of the multiple attributes of images and users in their personalized aesthetic experiences. To this end, this paper proposes a personalized image aesthetics assessment method via multi-attribute interactive reasoning. Different from existing PIAA models, the multi-attribute interaction constructed from both images and users is used as more effective prior knowledge. First, we designed a generic aesthetics extraction module from the perspective of images to obtain the aesthetic score distribution and multiple objective attributes of images rated by most users. Then, we propose a multi-attribute interactive reasoning network from the perspective of users. By interacting multiple subjective attributes of users with multiple objective attributes of images, we fused the obtained multi-attribute interactive features and aesthetic score distribution to predict personalized aesthetic scores. Experimental results on multiple PIAA datasets demonstrated our method outperformed state-of-the-art PIAA methods.

A Penalized h-Likelihood Variable Selection Algorithm for Generalized Linear Regression Models with Random Effects

Reinforcement learning is one of the paradigms and methodologies of machine learning developed in the computational intelligence community. Reinforcement learning algorithms present a major challenge in complex dynamics recently. In the perspective of variable selection, we often come across situations where too many variables are included in the full model at the initial stage of modeling. Due to a high-dimensional and intractable integral of longitudinal data, likelihood inference is computationally challenging. It can be computationally difficult such as very slow convergence or even nonconvergence, for the computationally intensive methods. Recently, hierarchical likelihood (h-likelihood) plays an important role in inferences for models having unobservable or unobserved random variables. This paper focuses linear models with random effects in the mean structure and proposes a penalized h-likelihood algorithm which incorporates variable selection procedures in the setting of mean modeling via h-likelihood. The penalized h-likelihood method avoids the messy integration for the random effects and is computationally efficient. Furthermore, it demonstrates good performance in relevant-variable selection. Throughout theoretical analysis and simulations, it is confirmed that the penalized h-likelihood algorithm produces good fixed effect estimation results and can identify zero regression coefficients in modeling the mean structure.

Comparison of Circular Symmetric Low-Pass Digital IIR Filter Design Using Evolutionary Computation Techniques

The design of two-dimensional Infinite Impulse Response (2D-IIR) filters has recently attracted attention in several areas of engineering because of their wide range of applications. Synthesizing a user-defined filter in a 2D-IIR structure can be interpreted as an optimization problem. However, since 2D-IIR filters can easily produce unstable transfer functions, they tend to compose multimodal error surfaces, which are computationally difficult to optimize. On the other hand, Evolutionary Computation (EC) algorithms are well-known global optimization methods with the capacity to explore complex search spaces for a suitable solution. Every EC technique holds distinctive attributes to properly satisfy particular requirements of specific problems. Hence, a particular EC algorithm is not able to solve all problems adequately. To determine the advantages and flaws of EC techniques, their correct evaluation is a critical task in the computational intelligence community. Furthermore, EC algorithms are stochastic processes with random operations. Under such conditions, for obtaining significant conclusions, appropriate statistical methods must be considered. Although several comparisons among EC methods have been reported in the literature, their conclusions are based on a set of synthetic functions, without considering the context of the problem or appropriate statistical treatment. This paper presents a comparative study of various EC techniques currently in use employed for designing 2D-IIR digital filters. The results of several experiments are presented and statistically analyzed.

Guest Editorial: Deep Fuzzy Models

The papers in this special section focus on recent developments and emerging topics in the area of deep fuzzy models that address some of the problems and limitations above. These models have been known under different names, such as hierarchical fuzzy systems and fuzzy networks. They are usually well suited for performing multiple functional compositions at either crisp or linguistic level. Deep learning has gained significant attention within the computational intelligence community in recent years. Its success has been mainly due to the increased power of modern computational platforms in terms of their ability to collect, store, and process large volumes of data. This has led to a substantial increase in the effectiveness and efficiency of data management. As a result, it has become possible to achieve high accuracy for some benchmark learning tasks, such as object classification and image recognition within a short time frame. The most common implementation of deep learning has been through neural networks due to the ability of their layers of neurons to perform multiple functional compositions as part of a multistage learning process.

To All Members of the Computational Intelligence Community [President's Message

To All Members of the Computational Intelligence Community [President's Message

On the Relationship Between Similarity Measures and Thresholds of Statistical Significance in the Context of Comparing Fuzzy Sets

Comparing fuzzy sets by computing their similarity is common, with a large set of measures of similarity available. However, while commonplace in the computational intelligence community, the application and results of similarity measures are less common in the wider scientific context, where statistical approaches are the standard for comparing distributions. This is challenging, as it means that developments around similarity measures arising from the fuzzy community are inaccessible to the wider scientific community and that the fuzzy community fails to take advantage of a strong statistical understanding, which may be applicable to comparing (fuzzy membership) functions. In this paper, we commence a body of work on systematically relating the outputs of similarity measures to the notion of statistically significant difference; that is, how (dis)similar do two fuzzy sets need to be for them to be statistically different? We explain that in this context, it is useful to initially focus on dis-similarity, rather than similarity, as the former aligns directly with the widely used concept of statistical difference. We propose two methods of applying statistical tests to the outputs of fuzzy dissimilarity measures to determine significant difference. We show how the proposed work provides deeper insight into the behavior and possible interpretation of degrees of dis-similarity and, consequently, similarity, and how the interpretation differs with respect to context (e.g., the complexity of the fuzzy sets).

An Adaptive Archive-Based Evolutionary Framework for Many-Task Optimization

Multi-task optimization is an emerging research topic in computational intelligence community. In this paper, we propose a novel evolutionary framework, many-task evolutionary algorithm (MaTEA), for many-task optimization. In the proposed MaTEA, an adaptive selection mechanism is proposed to select suitable “assisted” task for a given task by considering the similarity between tasks and the accumulated rewards of knowledge transfer during the evolution. Besides, a knowledge transfer schema via crossover is adopted to exchange information among tasks to improve the search efficiency. In addition, to facilitate measuring similarity between tasks and transferring knowledge among tasks that arrive at different time instances, multiple archives are integrated with the proposed MaTEA. Experiments on both single-objective and multi-objective optimization problems have demonstrated that the proposed MaTEA can outperform the state-of-the-art multi-task evolutionary algorithms, in terms of search efficiency and solution accuracy. Besides, the proposed MaTEA is also capable of solving dynamic many-task optimization where tasks arrive at different time instances.

A Fast Feature Selection Method Based on Coefficient of Variation for Diabetics Prediction Using Machine Learning

Diabetes has become a prevalent metabolic disease nowadays, affecting patients of all age groups and large populations around the world. Early detection would facilitate early treatment that helps the prognosis. In the literature of computational intelligence and medical care communities, different techniques have been proposed in predicting diabetes based on the historical records of related symptoms. The researchers share a common goal of improving the accuracy of a diabetes prediction model. In addition to the model induction algorithms, feature selection is a significant approach in retaining only the relevant attributes for the sake of building a quality prediction model later. In this article, a novel and simple feature selection criterion called Coefficient of Variation (CV) is proposed as a filter-based feature selection scheme. By following the CV method, attributes that have a data dispersion too low are disqualified from the model construction process. Thereby the attributes which are factors leading to poor model accuracy are discarded. The computation of CV is simple, hence enabling an efficient feature selection process. Computer simulation experiments by using the Prima Indian diabetes dataset is used to compare the performance of CV with other traditional feature selection methods. Superior results by CV are observed.

Computational Intelligence Techniques in Bioinformatics and Bioengineering [Guest Editorial

The computational intelligence community in bioinformatics and bioengineering is fragmented and large. The aim of the special issue is to capture some of the ongoing interdisciplinary research that draws upon joint expertise in the domains of optimization and learning via computational intelligence techniques and bioinformatics and bioengineering. After a rigorous review process, two papers were selected for publication in the special issue. The papers are briefly summarized.

Learning a No-Reference Quality Assessment Model of Enhanced Images With Big Data.

In this paper, we investigate into the problem of image quality assessment (IQA) and enhancement via machine learning. This issue has long attracted a wide range of attention in computational intelligence and image processing communities, since, for many practical applications, e.g., object detection and recognition, raw images are usually needed to be appropriately enhanced to raise the visual quality (e.g., visibility and contrast). In fact, proper enhancement can noticeably improve the quality of input images, even better than originally captured images, which are generally thought to be of the best quality. In this paper, we present two most important contributions. The first contribution is to develop a new no-reference (NR) IQA model. Given an image, our quality measure first extracts 17 features through analysis of contrast, sharpness, brightness and more, and then yields a measure of visual quality using a regression module, which is learned with big-data training samples that are much bigger than the size of relevant image data sets. The results of experiments on nine data sets validate the superiority and efficiency of our blind metric compared with typical state-of-the-art full-reference, reduced-reference and NA IQA methods. The second contribution is that a robust image enhancement framework is established based on quality optimization. For an input image, by the guidance of the proposed NR-IQA measure, we conduct histogram modification to successively rectify image brightness and contrast to a proper level. Thorough tests demonstrate that our framework can well enhance natural images, low-contrast images, low-light images, and dehazed images. The source code will be released at https://sites.google.com/site/guke198701/publications.

Extreme Learning Machine for Multi-Label Classification

Extreme learning machine (ELM) techniques have received considerable attention in the computational intelligence and machine learning communities because of the significantly low computational time required for training new classifiers. ELM provides solutions for regression, clustering, binary classification, multiclass classifications and so on, but not for multi-label learning. Multi-label learning deals with objects having multiple labels simultaneously, which widely exist in real-world applications. Therefore, a thresholding method-based ELM is proposed in this paper to adapt ELM to multi-label classification, called extreme learning machine for multi-label classification (ELM-ML). ELM-ML outperforms other multi-label classification methods in several standard data sets in most cases, especially for applications which only have a small labeled data set.

Universal Approximation with Convex Optimization: Gimmick or Reality? [Discussion Forum

This paper surveys in a tutorial fashion the recent history of universal learning machines starting with the multilayer perceptron. The big push in recent years has been on the design of universal learning machines using optimization methods linear in the parameters, such as the Echo State Network, the Extreme Learning Machine and the Kernel Adaptive filter. We call this class of learning machines convex universal learning machines or CULMs. The purpose of the paper is to compare the methods behind these CULMs, highlighting their features using concepts of vector spaces (i.e. basis functions and projections), which are easy to understand by the computational intelligence community. We illustrate how two of the CULMs behave in a simple example, and we conclude that indeed it is practical to create universal mappers with convex adaptation, which is an improvement over backpropagation.

How to Deal with Sports Activity Datasets for Data Mining and Analysis

The aim of this paper is to briefly present the fundamentals of sports training to computational intelligence community and present a short description on recently released sport activity datasets for data mining and data analysis. Training plan for professional mountainbiker is presented in details. Moreover, tools for parsing sport activities files are discussed. Finally, tips and open challenges are exposed that refer to the data analysis and data mining of the proposed sports activity datasets.

A Multi-Learning Immune Algorithm for Numerical Optimization

The emergence of nature-inspired algorithms (NIA) is a great milestone in the field of computational intelligence community. As one of the NIAs, the artificial immune algorithm (AIS) mimics the principles of the biological immune system, and has exhibited its effectiveness, implicit parallelism, flexibility and applicability when solving various engineering problems. Nevertheless, AIS still suffers from the issues of evolution premature, local minima trapping and slow convergence due to its inherent stochastic search dynamics. Much effort has been made to improve the search performance of AIS from different aspects, such as population diversity maintenance, adaptive parameter control, etc. In this paper, we propose a novel multi-learning operator into the AIS to further enrich the search dynamics of the algorithm. A framework of embedding multiple commonly used mutation operators into the antibody evolution procedure is also established. Four distinct learning operators including baldwinian learning, cauchy mutation, gaussian mutation and lateral mutation are selected to merge together as a multi-learning operator. It can be expected that the multi-learning operator can effectively balance the exploration and exploitation of the search by enriched dynamics. To verify its performance, the proposed algorithm, which is called multi-learning immune algorithm (MLIA), is applied on a number of benchmark functions. Experimental results demonstrate the superiority of the proposed algorithm in terms of convergence speed and solution quality.

The Emerging "Big Dimensionality"

The world continues to generate quintillion bytes of data daily, leading to the pressing needs for new efforts in dealing with the grand challenges brought by Data. Today, there is a growing consensus among the computational intelligence communities that data volume presents an immediate challenge pertaining to the scalability issue. However, when addressing volume in Data analytics, researchers in the data analytics community have largely taken a one-sided study of volume, which is the Big Instance Size factor of the data. The flip side of volume which is the dimensionality factor of Data, on the other hand, has received much lesser attention. This article thus represents an attempt to fill in this gap and places special focus on this relatively under-explored topic of Big Dimensionality, wherein the explosion of features (variables) brings about new challenges to computational intelligence. We begin with an analysis on the origins of Dimensionality. The evolution of feature dimensionality in the last two decades is then studied using popular data repositories considered in the data analytics and computational intelligence research communities. Subsequently, the state-of-the-art feature selection schemes reported in the field of computational intelligence are reviewed to reveal the inadequacies of existing approaches in keeping pace with the emerging phenomenon of Dimensionality. Last but not least, the curse and blessing of Dimensionality are delineated and deliberated.

Optimal Adaptive Control and Differential Games by Reinforcement Leanring Principles [Book review

This book introduces and leverages concepts from reinforcement learning (RL). RL methods have been formalized by the computational intelligence community based on the conditioned reflex concept and serve as the bridge between adaptive and optimal control methods. In this book, the authors focus on RL methods to design adaptive controllers that learn approximate optimal solution methods. These optimal adaptive control approaches make use of neural networks (NNs) that serve as the actor-critic for discrete and continuous optimal control and differential game problems. The book contains 11 chapters, with most chapters representing the results from recent journal publications. For readers with an understanding of concepts such as Lyapunov-based analysis and adaptive and optimal control methods, this book presents a foundation for online optimal control solutions for continuous systems. The text uses RL to merge concepts from optimal and adaptive control, and the results are solutions to a class of challenging and highly relevant optimal control problems. The text is the first of its kind specialized for the control systems community. The mixed use of typical state feedback and sampled past data to adjust the adaptive update laws presents an exciting new domain rich for new contributions. As control practitioners seek to apply the approaches in this text to more complex, higher-dimensional systems, new challenges will likely arise such as finding appropriate basis functions for the learning laws; developing alternatives to or ensuring persistence of excitation, extending to interconnected (potentially distributed) networks; and applying to systems where the uncertainty enters the dynamics in unique ways. In summary, this book presents an exciting blend of ideas and methods developed in machine learning and adaptive control to provide new inroads into online approximate optimal control for complex and potentially uncertain systems.

Novel Concepts for Realizing Neural Networks as Services in the Sky

A Virtual Organization (VO) is logical orchestration of globally dispersed resources to achieve common goals fostering new computing paradigms as Utility Computing, Grid Computing, Autonomic Computing, Clusters Computing and Cloud computing. The Computational Intelligence community is striving hard to build an online community to share resources such as data, paradigms, human expertise, models and methods. We build up on the N2Sky system, a Virtual Organization architecture for the Computational Intelligence community, which provides sharing and exchange of neural network resources, neural network specific knowledge, neural network objects and paradigms. In this paper we present novel extensions to N2Sky: automatic exploitation of parallelization capabilities of modern hardware (Multi-core servers, GPGPU) and softwares (OpenMP, CUDA), SLA management for Quality-of-Service guarantees of Cloud resources, and NN2Query, a novel query interface facilitating the user to find solutions to problems by smart searching of neural network resources using semantic web technology.