Pattern Classification Problems Research Articles

Eigenspectrum regularisation reverse neighbourhood discriminative learning

AbstractLinear discriminant analysis is a classical method for solving problems of dimensional reduction and pattern classification. Although it has been extensively developed, however, it still suffers from various common problems, such as the Small Sample Size (SSS) and the multimodal problem. Neighbourhood linear discriminant analysis (nLDA) was recently proposed to solve the problem of multimodal class caused by the contravention of independently and identically distributed samples. However, due to the existence of many small‐scale practical applications, nLDA still has to face the SSS problem, which leads to instability and poor generalisation caused by the singularity of the within‐neighbourhood scatter matrix. The authors exploit the eigenspectrum regularisation techniques to circumvent the singularity of the within‐neighbourhood scatter matrix of nLDA, which is called Eigenspectrum Regularisation Reverse Neighbourhood Discriminative Learning (ERRNDL). The algorithm of nLDA is reformulated as a framework by searching two projection matrices. Three eigenspectrum regularisation models are introduced to our framework to evaluate the performance. Experiments are conducted on the University of California, Irvine machine learning repository and six image classification datasets. The proposed ERRNDL‐based methods achieve considerable performance.

Sparse and robust support vector machine with capped squared loss for large-scale pattern classification

Support vector machine (SVM), being considered one of the most efficient tools for classification, has received widespread attention in various fields. However, its performance is hindered when dealing with large-scale pattern classification tasks due to high memory requirements and running very slow. To address this challenge, we construct a novel sparse and robust SVM based on our newly proposed capped squared loss (named as Lcsl-SVM). To solve Lcsl-SVM, we first focus on establishing optimality theory of Lcsl-SVM via our defined proximal stationary point, which is convenient for us to efficiently characterize the Lcsl support vectors of Lcsl-SVM. We subsequently demonstrate that the Lcsl support vectors comprise merely a minor fraction of entire training data. This observation leads us to introduce the concept of the working set. Furthermore, we design a novel subspace fast algorithm with working set (named as Lcsl-ADMM) for solving Lcsl-SVM, which is proven that Lcsl-ADMM has both global convergence and relatively low computational complexity. Finally, numerical experiments show that Lcsl-ADMM has excellent performances in terms of getting the best classification accuracy, using the shortest time and presenting the smallest numbers of support vectors when solving large-scale pattern classification problems.

Backpropagation through time learning for recurrence-aware long-term cognitive networks

Fuzzy Cognitive Mapping (FCM) and the extensive family of models derived from it have firmly established their strong position in the landscape of machine learning algorithms. Specifically designed for pattern classification and multi-output regression, the recently introduced Recurrence-aware Long-term Cognitive Network (r-LTCN) model is one of these FCM-inspired extensions. On the one hand, this recurrent neural network connects all temporal states generated during the reasoning process with the decision-making layer. On the other hand, it uses a quasi-nonlinear reasoning rule devoted to avoiding convergence issues caused by unique fixed points, which typically emerge in other FCM models. In the original paper, the authors employed a combination of unsupervised and supervised learning to compute the r-LTCNs’ learnable parameters. Despite r-LTCNs’ astounding performance for a wide variety of pattern classification problems, the literature reports no attempt to train these recurrent neural systems in a fully supervised manner nor provide insights into their performance in other machine learning settings. This paper brings forward a modified Backpropagation Through Time learning (BPTT) algorithm devoted to training r-LTCN models used for multi-output regressions tasks rather than pattern classification. The proposed BPPT includes a simple yet effective mechanism to deal with the vanishing gradient within the recurrent layer that operates as a closed system while being tailored to the quasi-nonlinear reasoning mechanism. Empirical evaluation of the proposed BPTT algorithm using 20 multi-output regression problems reveals that it produces lower prediction errors compared with other state-of-the-art learning approaches.

A novel quantum Dempster's rule of combination for pattern classification

Dempster's rule of combination (DRC) is widely used for uncertainty reasoning in intelligent information system, which is generalized to complex domain recently. However, as the increase of identification framework elements, the computational complexity of Dempster's rule of combination increases exponentially. To address this issue, we propose a novel quantum Dempster's rule of combination (QDRC) by means of Toffoli gate. The QDRC combination process is completely implemented using quantum circuits. To validate the efficiency of the proposed QDRC, we conduct simulation experiments on IBM and IonQ quantum cloud platforms. Besides, we implement the QDRC algorithm to the real quantum computer of IonQ Harmony. The experiment results show that the result of QDRC is getting more and more accurate as the number of measurements increases. In addition, we propose a quantum multisource information fusion method on the basis of QDRC. Finally, we demonstrate its effectiveness to solve pattern classification problem under several diverse real UCI datasets.

Song’s & Non-Song’s Pattern Classification Using Neural Network

The problem of pattern classification between musical or non-musical song audio snippets and non-songs is the main subject of this paper. The machine learning approach accomplishes the goal of automatically interpreting recordings to see if they are songs or not. There are two types of songs: musical and non-musical. This is due to the possibility that it could be difficult to discern between non-musical songs and non-song patterns. Thus, it is necessary to have such systems that differentiate between non-song patterns and songs, whether or not they are melodic. The extraction and selection of characteristics have made advantage of the preprocessing phases. The features that were obtained were pitch, intensity, length (duration), tempo, and sample rate. The Back-propagation Multi-layer Perceptron Neural Network model is used for both dataset testing and model training. This technique will distinguish between a musical or non-musical song and a non-musical piece (such conversation and speech). Training data for the dataset included recordings of speeches and dialogues in Hindi and Urdu for non-song audio files, as well as audio of various male and female singers from Pakistan and India for song audio files in Hindi and Urdu. When the classifier was evaluated using a range of audio samples, 90% of its classifications for songs and non-songs were accurate.

Precision forecasting of spray-dry desulfurization using Gaussian noise data augmentation and k-fold cross-validation optimized neural computing

Perceptron models have become integral tools for pattern recognition and classification problems in engineering fields. This study envisioned implementing artificial neural networks to forecast the performance of a mini-spray dryer for desulfurization activities. This work adopted k-fold cross-validation, a rigorous technique that evaluates model performance across multiple data segments. Several ANN models (21) were trained on data obtained from sulfation conditions, including sulfation temperature (120 °C–200 °C), slurry pH (4–12), stoichiometric ratio (0.5–2.5), slurry solid concentration (6%–14%) as the feed input and sulfur capture as the response. Three hundred synthetic datasets generated using the Gaussian noise data augmentation underwent a 10-fold cross-validation process before simulation on neural networks triggered by the logsig and tansig activation functions. The computation accuracy was further evaluated by altering the number of hidden cells from 2 to 10. The ANN architectures were assessed using statistical metrics such as mean square error (MSE), root mean square error (RMSE), mean absolute percentage error (MAPE), and the coefficient of determination (R 2) techniques. Overall, error estimation suggests cross-validation and data augmentation are critical in efficient neural network generalization. The logsig function trained with 10 hidden cells presented closer data articulation when mapped onto actual values.

Recurrence-Aware Long-Term Cognitive Network for Explainable Pattern Classification.

Machine-learning solutions for pattern classification problems are nowadays widely deployed in society and industry. However, the lack of transparency and accountability of most accurate models often hinders their safe use. Thus, there is a clear need for developing explainable artificial intelligence mechanisms. There exist model-agnostic methods that summarize feature contributions, but their interpretability is limited to predictions made by black-box models. An open challenge is to develop models that have intrinsic interpretability and produce their own explanations, even for classes of models that are traditionally considered black boxes like (recurrent) neural networks. In this article, we propose a long-term cognitive network (LTCN) for interpretable pattern classification of structured data. Our method brings its own mechanism for providing explanations by quantifying the relevance of each feature in the decision process. For supporting the interpretability without affecting the performance, the model incorporates more flexibility through a quasi-nonlinear reasoning rule that allows controlling nonlinearity. Besides, we propose a recurrence-aware decision model that evades the issues posed by the unique fixed point while introducing a deterministic learning algorithm to compute the tunable parameters. The simulations show that our interpretable model obtains competitive results when compared to state-of-the-art white and black-box models.

Fast generalized ramp loss support vector machine for pattern classification

Support vector machine (SVM) is widely recognized as an effective classification tool and has demonstrated superior performance in diverse applications. However, for large-scale pattern classification problems, it may require much memory and incur prohibitive computational costs. Motivated by this, we propose a new SVM model with novel generalized ramp loss (LR-SVM). The first-order optimality conditions for the non-convex and non-smooth LR-SVM are developed by the newly developed P-stationary point, based on which, the LR support vectors and working set of LR-SVM are defined, interestingly, which shows that all of the LR support vectors are on the two support hyperplanes under mild conditions. A fast proximal alternating direction method of multipliers with working set (LR-ADMM) is developed to handle LR-SVM and LR-ADMM has been demonstrated to achieve global convergence while maintaining a significantly low computational complexity. Numerical comparisons with nine leading solvers show that LR-ADMM demonstrates outstanding performance, particularly when applied to large-scale pattern classification problems with fewer support vectors, higher prediction accuracy and shorter computational time.

Fractal belief Jensen–Shannon divergence-based multi-source information fusion for pattern classification

Multi-source information fusion is an effective method to handle pattern classification problems. Dempster–Shafer evidence theory (DSET) plays an important role in handling uncertainty problems in multi-source information fusion. However, highly conflicting evidence in DSET may cause counter-intuitive fusion results. Belief divergence theory is one of the solutions to conflict management, which is also beneficial for the improvement of accuracies of pattern classification. In this paper, a novel belief divergence measurement method, fractal belief Jensen–Shannon (FBJS) divergence is proposed to better measure the discrepancy between Basic probability assignments (BPAs) and address the problem of highly conflicting evidence in DSET. The proposed FBJS divergence is the first belief divergence that incorporates the belief divergence theory and the concept of fractal. In addition, it has the properties of non-negativeness and symmetry. Then, based on FBJS divergence, a novel multi-source information fusion algorithm is proposed. Ultimately, the proposed algorithm is effectively applied to solve a pattern classification problem with a higher classification accuracy.

(Invited) Design and Modeling of Resistive Switching Devices for Neuromorphic Applications

Resistive switching devices are a critical component for the realization of computational systems that mimic biological neural systems. In biological systems, connection weights between the neurons are dependent on the time lapse between the neurons’ action potentials. Resistive switching devices (memristors) with multiple conductance states can mimic this biological spike-time-dependent plasticity (STDP) behaviour, with multiple conductance states being analogous to different synaptic weights. Additionally, memristors are one of the contenders as storage elements in non-volatile memories. The change in conductance in memristors can be broadly attributed to vacancy migration inside the switching layer (valence change memory) or the formation of metallic filaments inside the switching layer (electrochemical metallization).Electrochemical metallization-based memristors with vertical transport and small inter-electrode distances have been reported recently. Their device characteristics exhibit multiple conductance states with relatively low switching voltages, which makes them well-suited for low-power neuromorphic applications. Our work models the transport in these memristors, focusing on explaining and capturing their current-voltage characteristics. Our model also captures the switching dynamics (with an emphasis on estimating switching energies and delays) and explains the experimentally observed STDP behaviour in these devices. We have proposed models for filament growth and dissolution along one dimension (axial) and two dimensions (axial and radial). The simulation results obtained using our model (and implemented in Verilog-A) have been validated with experimental data from multiple sources. Our work demonstrates the flexibility of including different transport mechanisms in a unified framework.We have characterized valence change memory (VCM) memristors with HfO2 and HfZrO2 as switching layers. They exhibit bipolar switching, negative differential resistance, multiple conductance states, and STDP behaviour, which have been analyzed. We have also modeled the transport in these VCM-based memristors, and simulated the I-V and dynamic switching characteristics, and STDP behaviour in these devices. We have simulated arrays of, both, ECM and VCM devices using the proposed models, and tried to demonstrate a typical pattern classification problem using in-memory computation. We believe that our models provide useful insight into the design of memristors for neuromorphic and memory applications.

A study on predicting crisis information dissemination in epidemic-level public health events

Crisis information dissemination plays a key role in the development of emergency responses to epidemic-level public health events. Therefore, clarifying the causes of crisis information dissemination and making accurate predictions to effectively control such situations have attracted extensive attention. Based on media richness theory and persuasion theory, this study constructs an index system of crisis information dissemination impact factors from two aspects: the crisis information publisher and the published crisis information content. A multi-layer perceptron is used to analyze the weight of the index system, and the prediction is transformed into a pattern classification problem to test crisis information dissemination. In this study, COVID-19 is considered a representative event. An experiment is conducted to predict the crisis information dissemination of COVID-19 in two megacities. Data related to COVID-19 from these two megacities are acquired from the well-known Chinese social media platform Weibo. The experimental results show that not only the identity but also the social influence of the information publisher has a significant impact on crisis information dissemination in epidemic-level public health events. Furthermore, the proposed model achieves more than 95% test accuracy, precision rate, recall value and f1-score in the prediction task. The study provides decision-making support for government departments and a guide for correctly disseminating crisis information and public opinion during future epidemic-level public health events.

Quantum Pattern Classification in a Three-Qubit System

The problem of pattern classification in quantum data has been of great importance over the past few years. This study investigates the effect of deploying Grover’s, the partial diffusion, and the fixed-phase algorithms separately to amplify the amplitudes of a desired pattern in an unstructured dataset. These quantum search operators were applied to symmetric and antisymmetric input superpositions on a three-qubit system for 20 iterations each. After each iteration, different probabilities of classification were calculated in order to determine the accuracy of classification for each of the three quantum search operators. The results indicated that, in the case of applying the three quantum search operators to incomplete superposition input states, the partial diffusion operator outperformed the other operators with a probability of correct classification that reached 100% in certain iterations. It also showed that the classification accuracy of the fixed-phase operator exceeded the accuracy of the other two operators by 40% in most cases when the input state was a uniform superposition, and some of the basis states were phase-inverted.

One-Pass Online Learning Based on Gradient Descent for Multilayer Spiking Neural Networks

Online supervised learning algorithms update synaptic weights in real-time during the running process of spiking neural networks (SNNs), which are important for modeling the behavior and cognitive process of the brain. This article proposes an online supervised learning algorithm based on gradient descent for multilayer feedforward SNNs, where precisely timed spike trains are used to represent neural information. The online learning rule is derived from the real-time error function and backpropagation mechanism. The synaptic weights are adjusted when an output neuron fires a spike. Results of spike train learning demonstrate that the proposed online learning algorithm can achieve higher learning accuracy and requires fewer learning epochs than the corresponding offline learning method and other typical supervised learning algorithms. Furthermore, the proposed algorithm is used for solving pattern classification problems, where the one-pass learning approach is employed for training SNNs. Results show that the proposed algorithm can obtain comparable classification accuracy compared with other state-of-the-art algorithms even in the case of only one iteration. It indicates that the proposed algorithm is effective for solving spatio-temporal pattern recognition problems.

A new method for writer identification based on historical documents

Abstract Identifying the writer of a handwritten document has remained an interesting pattern classification problem for document examiners, forensic experts, and paleographers. While mature identification systems have been developed for handwriting in contemporary documents, the problem remains challenging from the viewpoint of historical manuscripts. Design and development of expert systems that can identify the writer of a questioned manuscript or retrieve samples belonging to a given writer can greatly help the paleographers in their practices. In this context, the current study exploits the textural information in handwriting to characterize writer from historical documents. More specifically, we employ oBIF(oriented Basic Image Features) and hinge features and introduce a novel moment-based matching method to compare the feature vectors extracted from writing samples. Classification is based on minimization of a similarity criterion using the proposed moment distance. A comprehensive series of experiments using the International Conference on Document Analysis and Recognition 2017 historical writer identification dataset reported promising results and validated the ideas put forward in this study.

Formulation of pdi-Bagging and its Evaluation

For pattern classification problems, there is ensemble learning method that identifies multiple weak classifiers by the learning data and combines them together to improve the discriminant rate of testing data. We have already proposed pdi-Bagging (possibilistic data interpolation-Bagging) which improves the discriminant rate of testing data by adding virtually generated data to learning data. However, the accuracy of the correct virtual data type is not stable because the virtual data generate in the wide area of the data space. In addition, the discriminant accuracy is not high because the evaluation index for changing the generation class of virtual data is defined in each dimension. In this paper, we propose a new method to specify the generation area of virtual data and change the generation class of virtual data. As a result, the discriminant accuracy is improved since five new bagging methods which generate virtual data around correct discriminant data and error discriminant data are formulated, and the class of virtual data is determined with the proposed new evaluation index in multidimensional space. We formulate a new pdi-Bagging algorithm, and discuss the usefulness of the proposed method using numerical examples.

Deep second generation wavelet autoencoders based on curvelet pooling for brain pathology classification

Neurological disorders represent anomalies relevant to the human nervous system. They also contain biochemical, anatomical and electrical modifications in the central nervous system, the spinal cord and the brain. These disorders provoke different symptoms. Early diagnosis of such changes is necessary for treatment, to limit disease progression. An accurate and CAD system is introduced in this paper to classify brain magnetic resonance images, which overcomes crucial problems in pattern classification, such as extracting certain features in the training phase. Our contribution is to merge a Second-Generation Wavelet Transform Network (SGWTN) and deep learning architectures, hence suggesting novel supervised feature extraction approaches for pattern classification. Our novel architecture allows the classification of all classes of our dataset by the reconstruction of a deep stacked SGWTN-Autoencoder (SGWTN-AE). Combining Curvelet Pooling (CP) with the Adam gradient calculation method can improve their accuracy. We implement CP with Adam in this work using the Haar curvelet (CurvPool-AH) and the Shannon Curvelet (CurvPool-AS). This Network is obtained after a series of SGWTN-AEs followed by a Softmax classifier at the last layer. We find that CurvPool performs fairly well on all datasets. However, overfitting is an issue with all the approaches considered. CP has the potential to outperform current approaches, particularly when merged with an adaptive gradient and curvelets chosen specifically for the data. Our architecture is tested with various image datasets. These latter are as follows: DS-66, DS-90, DS-160, and DS-255. Based on 5×5 cross-validation, our suggested approach can outperform the state-of-the-art methods in terms of classification accuracy.

Метрика схожості категоріальних розподілів, що враховує спорідненість різних категорій

Estimating a level of similarity of two objects is a common problem in pattern recognition, clustering and classification. Among these problems can be reviewer recommendation, similar text documents analysis, human pose detection in video, species distribution clustering, recommendation in internet-shops etc. In case of categorical attributes an object is described as a distribution of membership degrees over categories. Similarity metrics of such distributions are usually defined as a superposition of objects’ similarities for each category. Most often it is a sum of similarities in separate categories. In addition to that each category is considered independently and in isolation from the others. Some practical problems have categories that are kinship. Therefore, it is expedient to consider objects’ similarity not only directly, as a similarity between equivalent categories, but it is also necessary to consider an indirect similarity, cross-similarity through kinship categories. It is such similarity metric of two categorical distributions that accounts for the kinship of different categories is proposed in this paper. The metric has two components. The first component is defined as Czekanowski metric. It defines a direct similarity of categorical distributions as a sum of intersection of distributions’ membership degrees of two objects. After the intersection the residuals are accounted for in the second component of the metric. The second metric’s component is defined as element-wise product of two matrices: matrix of residuals composition from membership degrees of two categorical distributions and matrix of categories’ paired kinship. It is assumed that kinship indices for each pair of categories are known. As a result, with a large number of categories the overall noisy contribution from weakly kinship categories is prominent. Therefore, it is proposed to filter the noise and account only for contribution from strongly kinship categories.

Chaotic Metaheuristics with Multi-Spiking Neural Network Based Cloud Intrusion Detection

Cloud Computing (CC) provides data storage options as well as computing services to its users through the Internet. On the other hand, cloud users are concerned about security and privacy issues due to the increased number of cyberattacks. Data protection has become an important issue since the users’ information gets exposed to third parties. Computer networks are exposed to different types of attacks which have extensively grown in addition to the novel intrusion methods and hacking tools. Intrusion Detection Systems (IDSs) can be used in a network to manage suspicious activities. These IDSs monitor the activities of the CC environment and decide whether an activity is legitimate (normal) or malicious (intrusive) based on the established system’s confidentiality, availability and integrity of the data sources. In the current study, a Chaotic Metaheuristics with Optimal Multi-Spiking Neural Network-based Intrusion Detection (CMOMSNN-ID) model is proposed to secure the cloud environment. The presented CMOMSNN-ID model involves the Chaotic Artificial Bee Colony Optimization-based Feature Selection (CABC-FS) technique to reduce the curse of dimensionality. In addition, the Multi-Spiking Neural Network (MSNN) classifier is also used based on the simulation of brain functioning. It is applied to resolve pattern classification problems. In order to fine-tune the parameters relevant to the MSNN model, the Whale Optimization Algorithm (WOA) is employed to boost the classification results. To demonstrate the superiority of the proposed CMOMSNN-ID model, a useful set of simulations was performed. The simulation outcomes inferred that the proposed CMOMSNN-ID model accomplished a superior performance over other models with a maximum accuracy of 99.20%.

Prolog-based agnostic explanation module for structured pattern classification

This paper presents a Prolog-based reasoning module to generate counterfactual explanations given the predictions computed by a black-box classifier. Our approach comprises four well-defined stages that can be applied to any structured pattern classification problem. Firstly, we pre-process the given dataset by imputing missing values and normalizing the numerical features. Secondly, we transform numerical features into symbolic ones using fuzzy clustering such that extracted fuzzy clusters are mapped to an ordered set of predefined symbols. Thirdly, we encode instances as a Prolog rule using the nominal values, the predefined symbols, the decision classes, and the confidence values. Fourthly, we compute the overall confidence of each Prolog rule using fuzzy-rough set theory to handle the uncertainty caused by transforming numerical quantities into symbols. This step comes with an additional theoretical contribution to a new similarity function to compare the previously defined Prolog rules involving confidence values. Finally, we implement a chatbot as a proxy between humans and the Prolog-based reasoning module to resolve natural language queries and generate counterfactual explanations. During the numerical simulations using synthetic datasets, we study the performance of our system when using different fuzzy operators and similarity functions.

EFNC-Exp: An evolving fuzzy neural classifier integrating expert rules and uncertainty

Data stream classification processes with neuro-fuzzy approaches may involve situations where uncertainties arise, which may directly interfere with the quality of the results of the evolving models. Another factor that can help improve the performance of neuro-fuzzy evolving models is using a priori knowledge about a topic and incorporating it into the model's training procedure. The definition of fuzzy rules with a high degree of representativeness for certain classes can help models increase the significance of the representation of these labels and thus boost their predictive performance for these classes. This article proposes the integration of uncertainty in experts' feedback on the class labels and the integration of expert rules into the classifier architecture and the evolving, adaptive learning engine. This uncertainty integration occurs by combining it in defining neurons' weights in the first layer of the model and incorporating these weight values in the Gaussian neurons in the model's first layer; furthermore, uncertainty is integrated into an incremental feature weighting concept (inducing a weighted version of it) for the curse of dimensionality reduction. The proof of the new concepts will be carried out through tests on binary pattern classification problems in real-world data streams and a comparison between our approach and several related state-of-the-art works in evolving (neuro-) fuzzy modeling. The results obtained by the model showed that, by explicitly respecting the uncertainty of the class labels in the process of updating the evolving neuro-fuzzy classifier, the accuracy trend lines showed a robust behavior as the degree of distortion existing in the class labels of the samples due to uncertainty could be partially compensated.