Competitive Learning Neural Network Research Articles

Optimization of number of wireless temperature sensors using clustering algorithm for deep learning algorithm-based Kimchi quality prediction

The production and storage of kimchi in a factory require monitoring temperature and humidity to maintain optimal quality. Recently, smart packaging devices such as wireless sensors have been used, but their installation and maintenance are costly and require additional resources. In this study, kimchi is packaged in polyethylene packages and placed in plastic baskets. Thirty baskets are stacked in the middle of the storage room. Wireless sensors placed inside the baskets collect temperature data alongside the kimchi for 60 days. The data is sent to a server through communication unit using 4G network. Two clustering algorithms, Competitive Learning Neural Network (CLNN) and k-means, are used to determine the optimal number and location of sensors. Deep learning algorithms, specifically Long Short-Term Memory (LSTM), predict the acidity and ripeness grade of the kimchi. The study found that the optimal number of temperature sensor groups for CLNN was 6 for acidity and 4 for ripeness grade classification. Increasing the number of sensors improved the kimchi acidity prediction, and the accuracy of ripeness classification increased with the number of sensors. The location of the wireless sensors was determined based on the heights of the stacked plastic baskets using the CLNN + LSTM model. This work presents a process for optimizing sensor placement and evaluating kimchi quality during storage and fermentation using clustering and deep learning techniques to maintain optimal quality.

Wind turbine contaminant classification using machine learning techniques

It has been well established in the literature that surface contamination can adversely affect the aerodynamic performance of aerofoils and hence the efficiency with which turbines can convert wind to electrical power. Hence it is critical to ensure that turbine blades are kept as free as possible of contaminants. In this manuscript, we discuss LIBS and machine learning techniques on contaminated wind turbine blades with a view to the possibility of integrating these methods into a standoff laser ablation setup in the future. A LIBS signal will be a key step in the decision process, i.e., to determine if a wind turbine blade has been fouled in the first place. Relatedly one must also determine the point at which the laser has adequately removed contaminants from the current Area of Irradiation (AoI) before moving to the adjacent AoI, i.e., interim end-point-detection. For both steps (presence / absence of fouling) we are investigating laser-induced breakdown spectroscopy (LIBS) to discriminate clean from fouled wind turbine blade samples. In particular we have performed LIBS in both the Vacuum Ultraviolet (VUV) and Ultraviolet Visible (UV–Vis) spectral ranges. Analysis of the spectra showed only slight variations in the constituent materials between clean and contaminated blade samples. In order to address this challenge, the efficacy of a number of machine learning and statistical methods for clean versus contaminated blade classification was investigated. Four methods (Partial Least Squares Discriminant Analysis (PLS-DA), Support Vector Machines (SVM), Competitive Learning (CL), and Convolutional Neural Networks (CNN)) were evaluated. The spectral regions where machine learning algorithms were applied was determined via a volumetric ellipsoid overlap test based on Principal Component Analysis (PCA). It was found that SVMs provided the most accurate methodology for binary classification of clean vs contaminated blades whilst also yielding the shortest run time.

Energy‐Efficient Memristive Euclidean Distance Engine for Brain‐Inspired Competitive Learning

Inspired by competitive rules of the nature, competitive learning contributes to the specialization of the human brain and the general creativity of mankind. However, the construction of hardware competitive learning neural network still faces great challenges due to the lack of an accurate distance computation method and a self‐adaptive in situ training scheme. Herein, a fully memristive Euclidean distance (ED) engine based on analog multiply‐accumulate operation in a 32 × 32 TiN/TaO x /HfO x /TiN 1T1R array is demonstrated. The dual‐layer devices perform multilevel modulation under the target‐aware programming method with excellent read linearity in a dynamic range of 10–100 μS. The ED calculation is verified experimentally on a test board with an O(1) temporal complexity. Furthermore, in situ training and offline inference schemes for competitive learning, based on the ED engine, are developed and the simulated results show comparable success rates with those obtained by the CPU‐based software. Compared with a state‐of‐the‐art RTX6000 GPU (0.5 TOPS W−1), the energy efficiency of competitive learning models on ED engines can yield 100× improvements by utilizing optimized memristive devices.

A Multi-Swarm ABC Algorithm for Parameters Optimization of SOFM Neural Network in Dynamic Environment

Self-organizing feature map (SOFM) neural network is a kind of competitive unsupervised learning neural network, which has strong self-organizing and self-learning capabilities. It has been widely used in the fields of data classification and data clustering. A crucial step for SOFM neural network is to set its weight parameters correctly because the output accuracy and efficiency of the network depend much on these parameters. Most of current methods for parameter setting are based on static data. However, in a dynamic environment, the statistical characteristics of the generated data will change unpredictably over time. If the SOFM network cannot react to the changes of the environment, its performance will degrade. To deal with this problem, a more powerful multi-swarm artificial bee colony algorithm (MABC) is proposed. In the algorithm, the classic ABC algorithm is improved with multi-swarm and exclusive operation strategies to make it suitable for tracking optimal parameter settings of the SOFM network, so that the SOFM network can be applied to a dynamic environment. Two real data streams, which are regarded as coming from dynamic environments, are used to evaluate the effectiveness of the algorithm. Results show that the proposed algorithm is superior to the classic SOFM algorithm in terms of clustering purity and effectiveness. It is a promising method for the classification of data streams from dynamic environments.

Comments on “Quantum Classification Algorithm Based on Competitive Learning Neural Network and Entanglement Measure”

In a paper on competitive learning neural network, Mohammed Zidan et al. (Applied Sciences 9.7 (2019): 1277) proposed a new algorithm for the entanglement classification of two qubit states using the competitive learning and quantum computation. In this letter, we claim that the result after statement “the state of the two-qubit system…” in step 3 of the section “Classification an Input Using the Proposed Algorithm” in this manuscript is wrong.

Adaptive Landmark-Based Spectral Clustering for Big Datasets

Clustering has emerged as an effective tool for the processing and assessment of the vast data generated by modern applications; its primary aim is to classify data into clusters in which the items are grouped into a given category. However, various challenges, such as volume, velocity, and variety, occur during the clustering of big data. Different algorithms have been proposed to enhance the performance of clustering. The landmark-based spectral clustering (LSC) technique has been proven to be efficient in clustering big datasets. In this study, an algorithm called adaptive landmark-based spectral clustering (ALSC) is proposed for clustering big datasets. The proposed algorithm comprises the adaptive competitive learning neural network (ACLNN) algorithm, which can be efficiently used to determine the number of clusters and the LSC technique. The ACLNN algorithm can also be used with small datasets. Thus, in our implementation, the original big dataset is split into N small sub datasets, which run in parallel by N copies of the ACLNN algorithm. To evaluate the performance of the proposed algorithm, two distinctive datasets, namely, Fashion-MNIST and United States Postal Service are used. The experiments show that the proposed ALSC algorithm produces high clustering accuracy with the identification of the number of clusters. Results reveal that the normalized mutual information and adjusted Rand index of the proposed algorithm outperform state-of-the-art models.

Enhancement of classification accuracy of multi-spectral satellites’ images using Laplacian pyramids

Processed satellites’ images are not widely used in remote sensing classification due to the changes in spectral properties that may confuse the classifiers. In pixel-based classification there is a certain debate concerning with the boundary pixels. Most of miss-classified pixels are boundary pixels due to the sudden change in the spectral properties of the contacted objects. This research work is an investigation to study the proper enhancement in classification accuracy that may occur if the Laplacian pyramids are used in classification. The reference map is prepared to study the performance of the proposed system. The Laplacian image is constructed for each band of the satellite image. Then the classification is carried out for both the Laplacian image pyramid and the original satellite image using competitive learning neural networks (CLNN) method. The evaluation is carried out by comparing the classified Laplacian image with the classified original image. A statistical test is carried out to study the significance of using the classified Laplacian image in classification.

POTENTIALITY OF USING DIGITAL WAVELET/QMF PYRAMIDS IN REMOTELY SENSED SATELLITES’ IMAGES CLASSIFICATION

Wavelet or quadrature mirror filter (QMF) satellites’ images are not commonly used in classification because of the modification in spectral responses that may confuse any classifier. Boundary pixels are hardly classified correctly in pixel-based classification especially in medium and coarse resolution. In such case, the sudden change in landcover is not measurable by the classifiers because the pixel may contain mor than one class. This research work is a trial to investigate the proper enhancement in accuracy that may occur by using wavelet/QMF bands’ pyramids are in classification instead of the original image bands. The reference map is prepared traditionally to measure the performance of the new system. The Wavelet/QMF image is constructed for each band of the satellite image. Then the classification is carried out for both the Wavelet/QMF image pyramid and the original satellite image using competitive learning neural networks (CLNN) method. The evaluation is carried out by comparing the classified Wavelet/QMF image with the classified original image. A statistical test is carried out to study the significance of using the classified Wavelet/ QMF image in classification.

Studying the Potentiality of Using Digital Gaussian Pyramids in Multi-spectral Satellites Images Classification

Sized satellite images (pyramids) are not widely used in classification due to their dissimilarity with respect to the original image. But varied size images can reduce the cost, the time and mass storage of the classification system and can give us an initial impression of the result. This research work is an investigation to study the dependency upon varied size images in classification instead of using the original satellite image. The reference map is prepared to study the performance of the proposed system. Then, the varied size image is constructed for each band of the satellite image. Then, the classification is carried out using competitive learning neural networks (CLNN) method for all digital image pyramids, either the original satellite image or its sized images. The last step is the evaluation of the studied elements such as accuracy, classification time and storage volume. Some resized images are useful in such application, so the study suggests which size suits such applications.

Quantum Classification Algorithm Based on Competitive Learning Neural Network and Entanglement Measure

In this paper, we develop a novel classification algorithm that is based on the integration between competitive learning and the computational power of quantum computing. The proposed algorithm classifies an input into one of two binary classes even if the input pattern is incomplete. We use the entanglement measure after applying unitary operators to conduct the competition between neurons in order to find the winning class based on wining-take-all. The novelty of the proposed algorithm is shown in its application to the quantum computer. Our idea is validated via classifying the state of Reactor Coolant Pump of a Risky Nuclear Power Plant and compared with other quantum-based competitive neural networks model.

Expert systems: Special issue on “Machine Learning Methods Neural Networks applied to Vision and Robotics (MLMVR)”

The International Joint Conference on Neural Networks (IJCNN) was held in Anchorage (Alaska) in May 2017. This top conference in the field of neural networks included many tracks and special sessions. In particular, a special session on Machine Learning Methods Neural Networks applied to Vision and Robotics (MLMVR) was organized by the authors receiving a large volume of excellent contributions. Only a small set of outstanding papers presented at this special session were invited to submit extended versions of their work. After a rigorous revision process, four of these papers were accepted. Over the last decades, there has been an increasing interest in using machine learning methods combined with computer vision techniques to create autonomous systems that solve vision problems in different fields. This special issue was designed to serve researchers and developers to publish original, innovative, and state-of-the art machine learning algorithms and architectures for applications in the areas of computer vision, image processing, biometrics, virtual and augmented reality, robot vision, intelligent interfaces, and biomimetic object-vision recognition. This issue is composed of four papers selected from the original 15 submissions received in the special session MLMVR. The papers are divided in two groups: a set of two papers that applied deep learning methods to solve computer vision challenging problems. A second set of two papers that deal with image enhancement by using a cellular automata and competitive neural networks. In the first group, Yao et al. revisit the trajectory clustering problem by developing a method that can detect space- and time-invariant trajectory clusters. The method uses recurrent neural networks (RNNs) to convert trajectories into a fixed-length representation that well encodes the object's moving behaviours. Oprea et al. present a Schaeffer language recognition system with the purpose of teaching children with autism disorder the correct way to communicate using gestures in combination with speech reproduction. The purpose is to accelerate the learning process and increase children interest using a technological approach. A Long Short-Term Memory (LSTM) model has been implemented for this purpose reporting a 93.13% classification success rate over a subset of 25 Schaeffer gestures. In the second group, Priego et al. describe a novel spatio-temporal cellular automata-based filtering algorithm (st-CAF) intended for performing image sequence denoising processes. The approach presents several advantages over more traditional single-frame denoising techniques or even over their adaptation to sequences. Especially the fact that the cellular automaton used is able to contemplate information concerning the type of noise through the use of specific sequences to tune the algorithm, as well as temporal information by means of a spatio-temporal neighbourhood when processing each pixel of the sequence. These two elements lead to significant improvements in the results with respect to simple spatial or temporal sets of neighbours. Finally, Thurnhofer-Hemsi et al. present a novel approach for constructing a panoramic background model based on competitive learning neural networks and a subsequent piecewise linear interpolation by Delaunay triangulation. The approach can handle arbitrary camera directions and zooms for a Pan-Tilt-Zoom (PTZ) camera-based surveillance system. In summary, the selected papers present recent developments of machine learning and deep learning methods applied to the very active research fields like mobile robotics and computer vision.

An Efficient Hardware Circuit for Spike Sorting Based on Competitive Learning Networks.

This study aims to present an effective VLSI circuit for multi-channel spike sorting. The circuit supports the spike detection, feature extraction and classification operations. The detection circuit is implemented in accordance with the nonlinear energy operator algorithm. Both the peak detection and area computation operations are adopted for the realization of the hardware architecture for feature extraction. The resulting feature vectors are classified by a circuit for competitive learning (CL) neural networks. The CL circuit supports both online training and classification. In the proposed architecture, all the channels share the same detection, feature extraction, learning and classification circuits for a low area cost hardware implementation. The clock-gating technique is also employed for reducing the power dissipation. To evaluate the performance of the architecture, an application-specific integrated circuit (ASIC) implementation is presented. Experimental results demonstrate that the proposed circuit exhibits the advantages of a low chip area, a low power dissipation and a high classification success rate for spike sorting.

Ameliorate Competitive Learning Neural Networks for System Intrusion detection

Ameliorate Competitive Learning Neural Networks for System Intrusion detection

Tone mapping with contrast preservation and lightness correction in high dynamic range imaging

In real-world environments, the human visual system perceives a wide range of luminance in a scene. However, the full range of tones in a high dynamic range (HDR) scene cannot be displayed on standard display devices due to their low dynamic range (LDR). Therefore, tone mapping is necessary to faithfully display a HDR scene on an LDR display device. Existing tone mapping methods have problems because details and contrast in a scene are not preserved faithfully, and they also distort the colors in a scene. Thus, we propose a tone mapping method for preserving the detail in an HDR scene using a weighted least squares filter, which preserves the global contrast in a scene by using a competitive learning neural network, before applying a tone reproduction operator to preserve the color without shifting the lightness. According to the Helmholtz–Kohlrausch effect, the perception of brightness depends on the lightness, chroma, and hue of a color. For example, the luminance of pixels with specific colors such as red and blue is low in an HDR scene. The proposed method corrects the lightness of pixels according to the color (i.e., lightness, chroma, and hue) of a tone-mapped image. Experimental results with several test sets of images demonstrated that the proposed tone mapping method with contrast preservation and lightness correction is more suitable for dynamic range compression than existing tone mapping methods, while it also preserves the color of a scene in an effective way.

Topology-based analysis of self-organizing maps for time series prediction

A self-organizing map (SOM) is a competitive learning neural network architecture that uses a set of classificatory neurons, which self-organize spatially based on input patterns. The application of standard SOM, and its variants, to time series prediction has been studied in the literature under different conditions and techniques. In this paper, we have focused on the standard Self-Organizing Maps, and its application for time series forecasting. We provide extensive numerical analysis considering regular and complex topologies, and real time series. Concretely, we study the impact of the number of neurons, the effect of the best-matching unit over its neighborhood, the use of nonlinear learning rate functions, and the importance of a proportional training together with a sampled input space as uniformly as possible. We have found that probabilistic updating in neighborhoods may be used as a second learning rate parameter within the SOM. We have also found that small-world and scale-free topologies are able to improve the error of regular lattices, depending on their own set of parameters, the number of neurons, and the length of the training set. Finally, we have found coherence between results of real time series and analytic benchmarks for forecasting.

Using self-organizing maps with complex network topologies and coalitions for time series prediction

A Self-organizing Map (SOM) is a competitive learning neural network architecture that make available a certain amount of classificatory neurons, which self-organize spatially based on input patterns. In this paper we explore the use of complex network topologies, like small-world, scale-free or random networks; for connecting the neurons within a SOM, and apply them for Time Series Prediction. We follow the VQTAM model for function prediction, and consider several benchmarks to evaluate the quality of the predictions. Afterwards, we introduce the CASOM algorithm (Coalitions and SOM) that uses coalitions to temporarily extend the neighborhood of a neuron, and to provide more accuracy in prediction problems than classical SOM. The results presented in this work suggest that the most regular the network topology is, the better results it provides in prediction. Besides, we have found that not updating all the neurons at the same time provides much better results. Finally, we describe how the use of coalitions can enhance the capability of SOM for Time Series Prediction.

Adaptive competitive learning neural networks

In this paper, the adaptive competitive learning (ACL) neural network algorithm is proposed. This neural network not only groups similar input feature vectors together but also determines the appropriate number of groups of these vectors. This algorithm uses a new proposed criterion referred to as the ACL criterion. This criterion evaluates different clustering structures produced by the ACL neural network for an input data set. Then, it selects the best clustering structure and the corresponding network architecture for this data set. The selected structure is composed of the minimum number of clusters that are compact and balanced in their sizes. The selected network architecture is efficient, in terms of its complexity, as it contains the minimum number of neurons. Synaptic weight vectors of these neurons represent well-separated, compact and balanced clusters in the input data set. The performance of the ACL algorithm is evaluated and compared with the performance of a recently proposed algorithm in the literature in clustering an input data set and determining its number of clusters. Results show that the ACL algorithm is more accurate and robust in both determining the number of clusters and allocating input feature vectors into these clusters than the other algorithm especially with data sets that are sparsely distributed.

Computer-aided diagnosis of breast cancer based on fine needle biopsy microscopic images

Prompt and widely available diagnostics of breast cancer is crucial for the prognosis of patients. One of the diagnostic methods is the analysis of cytological material from the breast. This examination requires extensive knowledge and experience of the cytologist. Computer-aided diagnosis can speed up the diagnostic process and allow for large-scale screening. One of the largest challenges in the automatic analysis of cytological images is the segmentation of nuclei. In this study, four different clustering algorithms are tested and compared in the task of fast nuclei segmentation. K-means, fuzzy C-means, competitive learning neural networks and Gaussian mixture models were incorporated for clustering in the color space along with adaptive thresholding in grayscale. These methods were applied in a medical decision support system for breast cancer diagnosis, where the cases were classified as either benign or malignant. In the segmented nuclei, 42 morphological, topological and texture features were extracted. Then, these features were used in a classification procedure with three different classifiers. The system was tested for classification accuracy by means of microscopic images of fine needle breast biopsies. In cooperation with the Regional Hospital in Zielona Góra, 500 real case medical images from 50 patients were collected. The acquired classification accuracy was approximately 96–100%, which is very promising and shows that the presented method ensures accurate and objective data acquisition that could be used to facilitate breast cancer diagnosis.

Self-organizing map for symbolic data

Kohonen's self-organizing map (SOM) is a competitive learning neural network that uses a neighborhood lateral interaction function to discover the topological structure hidden in the data set. It is an unsupervised learning which has both visualization and clustering properties. In general, the SOM neural network is constructed as a learning algorithm for numeric (vector) data. Although there are different SOM clustering methods for numeric data with real applications in the literature, there is less consideration in a SOM clustering for symbolic data. In this paper, we modify the SOM so that it can treat symbolic data and a so-called symbolic SOM (S-SOM) is then proposed. We first use novel structures to represent symbolic neurons. We then use a suppression concept to create a learning rule for neurons. Therefore, the S-SOM is created for treating symbolic data by embedding the novel structure and the suppression learning rule. Some real data sets are applied with the S-SOM. The experimental results show the feasibility and effectiveness of the proposed S-SOM in these real applications.

Improved competitive learning neural networks for network intrusion and fraud detection

In this research, we propose two new clustering algorithms, the improved competitive learning network (ICLN) and the supervised improved competitive learning network (SICLN), for fraud detection and network intrusion detection. The ICLN is an unsupervised clustering algorithm, which applies new rules to the standard competitive learning neural network (SCLN). The network neurons in the ICLN are trained to represent the center of the data by a new reward-punishment update rule. This new update rule overcomes the instability of the SCLN. The SICLN is a supervised version of the ICLN. In the SICLN, the new supervised update rule uses the data labels to guide the training process to achieve a better clustering result. The SICLN can be applied to both labeled and unlabeled data and is highly tolerant to missing or delay labels. Furthermore, the SICLN is capable to reconstruct itself, thus is completely independent from the initial number of clusters. To assess the proposed algorithms, we have performed experimental comparisons on both research data and real-world data in fraud detection and network intrusion detection. The results demonstrate that both the ICLN and the SICLN achieve high performance, and the SICLN outperforms traditional unsupervised clustering algorithms.