Performance Of Neural Network Classifier Research Articles

A Spatio-Temporal Capsule Neural Network with Self-Correlation Routing for EEG Decoding of Semantic Concepts of Imagination and Perception Tasks.

Decoding semantic concepts for imagination and perception tasks (SCIP) is important for rehabilitation medicine as well as cognitive neuroscience. Electroencephalogram (EEG) is commonly used in the relevant fields, because it is a low-cost noninvasive technique with high temporal resolution. However, as EEG signals contain a high noise level resulting in a low signal-to-noise ratio, it makes decoding EEG-based semantic concepts for imagination and perception tasks (SCIP-EEG) challenging. Currently, neural network algorithms such as CNN, RNN, and LSTM have almost reached their limits in EEG signal decoding due to their own short-comings. The emergence of transformer methods has improved the classification performance of neural networks for EEG signals. However, the transformer model has a large parameter set and high complexity, which is not conducive to the application of BCI. EEG signals have high spatial correlation. The relationship between signals from different electrodes is more complex. Capsule neural networks can effectively model the spatial relationship between electrodes through vector representation and a dynamic routing mechanism. Therefore, it achieves more accurate feature extraction and classification. This paper proposes a spatio-temporal capsule network with a self-correlation routing mechaninsm for the classification of semantic conceptual EEG signals. By improving the feature extraction and routing mechanism, the model is able to more effectively capture the highly variable spatio-temporal features from EEG signals and establish connections between capsules, thereby enhancing classification accuracy and model efficiency. The performance of the proposed model was validated using the publicly accessible semantic concept dataset for imagined and perceived tasks from Bath University. Our model achieved average accuracies of 94.9%, 93.3%, and 78.4% in the three sensory modalities (pictorial, orthographic, and audio), respectively. The overall average accuracy across the three sensory modalities is 88.9%. Compared to existing advanced algorithms, the proposed model achieved state-of-the-art performance, significantly improving classification accuracy. Additionally, the proposed model is more stable and efficient, making it a better decoding solution for SCIP-EEG decoding.

Melanoma image synthesis: a review using generative adversarial networks

Melanoma is a highly malignant skin cancer that may be fatal if not promptly detected and treated. The limited availability of high-quality melanoma images, which are needed for training machine learning models, is one of the obstacles to detecting melanoma. Generative adversarial networks (GANs) have grown in popularity as a strong technique for image synthesis. This research is also targeted at the sustainable development goal (SDG) for health care. In this study, we survey existing GAN-based melanoma image synthesis methods. In this work, we briefly introduce GANs and how they may be used for generating synthetic images. Ensuring healthy lifestyles and promoting well-being for everyone, regardless of age, is the main aim. A comparative study is carried out on how GANs are used in current research to generate melanoma images and how they improve the classification performance of neural networks. Various public and proprietary datasets for training GANs in melanoma image synthesis are also discussed. Lastly, we assess the examined studies' performance using measures like the Frechet Inception distance (FID), Inception score, structural similarity ındex (SSIM), and various classification performance metrics. We compare the evaluated findings and suggest further GAN-based melanoma image-creation research.

Assessing the Impact of Image Quality on Deep Learning Classification of Infectious Keratitis

To investigate the impact of corneal photograph quality on convolutional neural network (CNN) predictions. A CNN trained to classify bacterial and fungal keratitis was evaluated using photographs of ulcers labeled according to 5 corneal image quality parameters: eccentric gaze direction, abnormal eyelid position, over/under-exposure, inadequate focus, and malpositioned light reflection. All eligible subjects with culture and stain-proven bacterial and/or fungal ulcers presenting to Aravind Eye Hospital in Madurai, India, between January 1, 2021 and December 31, 2021. Convolutional neural network classification performance was compared for each quality parameter, and gradient class activation heatmaps were generated to visualize regions of highest influence on CNN predictions. Area under the receiver operating characteristic and precision recall curves were calculated to quantify model performance. Bootstrapped confidence intervals were used for statistical comparisons. Logistic loss was calculated to measure individual prediction accuracy. Individual presence of either light reflection or eyelids obscuring the corneal surface was associated with significantly higher CNN performance. No other quality parameter significantly influenced CNN performance. Qualitative review of gradient class activation heatmaps generally revealed the infiltrate as having the highest diagnostic relevance. The CNN demonstrated expert-level performance regardless of image quality. Future studies may investigate use of smartphone cameras and image sets with greater variance in image quality to further explore the influence of these parameters on model performance. Proprietary or commercial disclosure may be found after the references.

Crop Classification Based on GDSSM-CNN Using Multi-Temporal RADARSAT-2 SAR with Limited Labeled Data

Crop classification is an important part of crop management and yield estimation. In recent years, neural networks have made great progress in synthetic aperture radar (SAR) crop classification. However, the insufficient number of labeled samples limits the classification performance of neural networks. In order to solve this problem, a new crop classification method combining geodesic distance spectral similarity measurement and a one-dimensional convolutional neural network (GDSSM-CNN) is proposed in this study. The method consisted of: (1) the geodesic distance spectral similarity method (GDSSM) for obtaining similarity and (2) the one-dimensional convolutional neural network model for crop classification. Thereinto, a large number of training data are extracted by GDSSM and the generalized volume scattering model which is based on radar vegetation index (GRVI), and then classified by 1D-CNN. In order to prove the effectiveness of the GDSSM-CNN method, the GDSSM method and 1D-CNN method are compared in the case of a limited sample. In terms of evaluation and verification of methods, the GDSSM-CNN method has the highest accuracy, with an accuracy rate of 91.2%, which is 19.94% and 23.91% higher than the GDSSM method and the 1D-CNN method, respectively. In general, the GDSSM-CNN method uses a small number of ground measurement samples, and it uses the rich polarity information in multi-temporal fully polarized SAR data to obtain a large number of training samples, which can quickly improve the accuracy of classification in a short time, which has more new inspiration for crop classification.

TOWARDS BETTER COASTAL MAPPING USING FUSION OF HIGH TEMPORAL SENTINEL-2 AND PLANETSCOPE-2 IMAGERIES: 12 BANDS AT 3 M THROUGH NEURAL NETWORK MODELLING

Abstract. Coastal interfaces are subject to an unprecedented rate of risks, gathering waves and rainfalls’ hazards, human assets’ densification, sea-level rise and precipitation intensification. Their sound management requires iterative observation at the highest possible spatial resolution. Sentinel-2 (S-2), provided with 13 spectral bands, datasets leverage high temporal resolution (one week) but spatial resolution (from 60 to 10 m) often remains too coarse to finely classify and monitor the coastal patches. PlanetScope-2 (PS-2) imagery benefits from very high temporal resolution (< one week) and high spatial resolution (3 m) for its blue-green-red-near-infrared dataset.This research paper proposes to, first, downscale 12 S-2 bands (cirrus S10 being evicted) by using neural network (NN) regressions built on the 4 PS-2 bands following two methods, and second, evaluate the NN classification performance of the 12-band datasets at 3 m for mapping 8 common coastal classes on a representative site (Brittany, France). Straightforward and stepwise downscaling procedures, respectively based on 12 and 22 NN regressions, generated very good performances (R2test=0.92 ± 0.02 and 0.95 ± 0.01, respectively). The 3-m NN classifications were considerably improved by the number of spectral bands (overall accuracy, OA, of the 4 bands: 48.12%) but also the precision of the downscaling (OA of the straightforward and stepwise downscaling: 75.25% and 93.57%, respectively). For the best classification, examination of the contribution of the individual bands revealed that S5, S7, S1, S9, S6 and S8A were meaningful (62.42, 55.02, 50.82, 46.4, 45.1, 31.02%, respectively), contrary to S12, S11 and S12 (12.47, 0 and 0%, respectively).

A new density core graph-cut class decomposition to improve neural network classification performance

<span>This research presents a new pre-processed class decomposition technique called density core graph-cut (DCGC). The method uses supervised clustering instead of a traditional unsupervised one to decompose the class. Supervised clustering requires additional label information to function and with that it gains a better understanding of the distribution. DCGC employs nearest neighbors to form a density core graph for each class. Then, the edges of each graph to be removed or cut is identified utilizing class information. Lastly, it yields final clusters by grouping all connected cores and assigning the remaining samples to a cluster where the nearest core belongs. Training neural network classifiers on complex label data will yield a better accuracy with the modified class representation. Intuitively, the decision boundaries separating classes based on the modified labels are less complex, and classifiers’ chance to reach these hyperplanes is higher. The results present that training neural networks using label representations from DCGC significantly helps improve the classification accuracy of neural networks on syntactic datasets as high as 30%. For real-world problems, the experiment presents a mixed result in which some datasets moderately benefit from DCGC.</span>

Revisiting Dropout Regularization for Cross-Modality Person Re-Identification

This paper investigated targeted dropout regularization for improving convolutional neural network classifier performance on cross-modality person re-identification problems. The dropout regularization is carefully applied on specific layers (targeted dropout regularization) or regions of the features map (spatially targeted dropout regularization). The intuition behind the spatially targeted dropout regularization is that the importantness of feature map regions is not equally the same, and the specific object is more likely placed at the center. We experimented heavily with PKU-Sketch-ReID and multi-modality person re-identification datasets with SwinTransformer deep neural network architecture. Three different targeted dropout regularizations are used for the experiments, including block-wise dropout, horizontal block-wise dropout, and vertical-horizontal block-wise dropout. Experiments on three different sketch re-identification datasets show that the proposed spatially targeted dropout regularization can improve the performance of the deep neural network classifiers with the best rank-1 of 73.20% on the PKU-Sketch-ReID, 52.73% on the SYSU-MM01 dataset, and 72.74% on the RegDB dataset.

Discriminative feature generation for classification of imbalanced data

The data imbalance problem is a frequent bottleneck in the classification performance of neural networks. In this paper, we propose a novel supervised discriminative feature generation (DFG) method for a minority class dataset. DFG is based on the modified structure of a generative adversarial network consisting of four independent networks: generator, discriminator, feature extractor, and classifier. To augment the selected discriminative features of the minority class data by adopting an attention mechanism, the generator for the class-imbalanced target task is trained, and the feature extractor and classifier are regularized using the pre-trained features from a large source data. The experimental results show that the DFG generator enhances the augmentation of the label-preserved and diverse features, and the classification results are significantly improved on the target task. The feature generation model can contribute greatly to the development of data augmentation methods through discriminative feature generation and supervised attention methods.

Experimental stability analysis of neural networks in classification problems with confidence sets for persistence diagrams

We investigate classification performance of neural networks (NNs) based on topological insight in an attempt to guarantee stability of their inference. NNs which can accurately classify a dataset map it into a hidden space while disentangling intertwined data. NNs sometimes acquire forcible mapping to disentangle the data, and this forcible mapping generates outliers. The mapping around the outliers is unstable because the outputs change drastically. Hence, we define stable NNs to mean that they do not generate outliers. To investigate the possibility of the existence of outliers, we use persistent homology and a method to estimate the confidence set for persistence diagrams. The combined use enables us to test whether the focused geometry is topologically simple, that is, no outliers. In this work, we use the MNIST and CIFAR-10 datasets and investigate the relationship between the classification performance and the topological characteristics with several NNs. Investigation results with the MNIST dataset show that the test accuracy of all the networks is superior, exceeding 98%, even though the transformed dataset is not topologically simple. Results with the CIFAR-10 dataset also show that the possibility of the existence of outliers is shown in the mapping by the accurate convolutional NNs. Therefore, we conclude that the presented investigation is necessary to guarantee that the NNs, in particular deep NNs, do not acquire unstable mapping for forcible classification.

A Novel Quantum-Inspired Fuzzy Based Neural Network for Data Classification

The performance of the neural network (NN) depends on the various parameters such as structure, initial weight, number of hidden layer neurons, and learning rate. The improvement in classification performance of NN without changing its structure is a challenging issue. This paper proposes a novel learning model called Quantum-inspired Fuzzy Based Neural Network (Q-FNN) to solve two-class classification problems. In the proposed model, NN architecture is formed constructively by adding neurons in the hidden layer and learning is performed using the concept of Fuzzy c-Means (FCM) clustering, where the fuzziness parameter ( <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">m</i> ) is evolved using the quantum computing concept. The quantum computing concept provides a large search space for a selection of <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">m</i> , which helps in finding the optimal weights and also optimizes the network architecture. This paper also proposes a modified step activation function for the formation of hidden layer neurons, which handles the overlapping samples belong to different class regions. The performance of the proposed Q-FNN model is superior and competitive with the state-of-the-art methods in terms of accuracy, sensitivity, and specificity on 15 real-world benchmark datasets.

Classification of ground-truth fire debris samples using artificial neural networks

Neural networks are a class of biologically inspired machine learning models that are used for classification and regression problems. This work assesses the classification performance of neural networks on ground-truth fire debris samples using ions that are representative of several compound classes that are typically present in ignitable liquids. An optimal neural network model was selected from a subset of candidate models that were trained on in-silico mixed fire debris samples from the National Center for Forensic Science Substrate and Ignitable Liquid Reference Collection databases. An optimal decision threshold was determined using a defined ratio of misclassification costs. A cost ratio corresponding to a false positive classification having a cost that is 10 times greater than a false negative classification resulted in a decision threshold of log likelihood ratio of 0.966. This decision threshold resulted in a false positive rate of 0.07 and a true positive rate of 0.59 for the ground-truth validation data. This study demonstrates the selection of an optimal decision threshold using ROC analysis and exhibits the potential of neural network models for the evaluation of fire debris evidence.

Forecasting significant stock price changes using neural networks

Stock price prediction is a rich research topic that has attracted interest from various areas of science. The recent success of machine learning in speech and image recognition has prompted researchers to apply these methods to asset price prediction. The majority of literature has been devoted to predicting either the actual asset price or the direction of price movement. In this paper, we study a hitherto little explored question of predicting significant changes in stock price based on previous changes using machine learning algorithms. We are particularly interested in the performance of neural network classifiers in the given context. To this end, we construct and test three neural network models including multi-layer perceptron, convolutional net, and long short term memory net. As benchmark models we use random forest and relative strength index methods. The models are tested using 10-year daily stock price data of four major US public companies. Test results show that predicting significant changes in stock price can be accomplished with a high degree of accuracy. In particular, we obtain substantially better results than similar studies that forecast the direction of price change.

Cell Image Classification: A Comparative Overview.

Cell image classification methods are currently being used in numerous applications in cell biology and medicine. Applications include understanding the effects of genes and drugs in screening experiments, understanding the role and subcellular localization of different proteins, as well as diagnosis and prognosis of cancer from images acquired using cytological and histological techniques. The article also reviews three main approaches for cell image classification most often used: numerical feature extraction, end-to-end classification with neural networks (NNs), and transport-based morphometry (TBM). In addition, we provide comparisons on four different cell imaging datasets to highlight the relative strength of each method. The results computed using four publicly available datasets show that numerical features tend to carry the best discriminative information for most of the classification tasks. Results also show that NN-based methods produce state-of-the-art results in the dataset that contains a relatively large number of training samples. Data augmentation or the choice of a more recently reported architecture does not necessarily improve the classification performance of NNs in the datasets with limited number of training samples. If understanding and visualization are desired aspects, TBM methods can offer the ability to invert classification functions, and thus can aid in the interpretation of results. These and other comparison outcomes are discussed with the aim of clarifying the advantages and disadvantages of each method. © 2020 International Society for Advancement of Cytometry.

A Machine Learning Approach for Lamb Meat Quality Assessment Using FTIR Spectra

The food industry requires automatic methods to establish authenticity of food products. In this work, we address the problem of the certification of suckling lamb meat with respect to the rearing system. We evaluate the performance of neural network classifiers as well as different dimensionality reduction techniques, with the aim of categorizing lamb fat by means of spectroscopy and analyzing the features with more discrimination power. Assessing the stability of feature ranking algorithms also becomes particularly important. We assess six feature selection techniques (χ 2 , Information Gain, Gain Ratio, Relief and two embedded techniques based on the decision rule 1R and SVM (Support Vector Machine). Additionally, we compare them with common approaches in the chemometrics field like the Partial Least Square (PLS) model and Principal Component Analysis (PCA) regression. Experimental results with a fat sample dataset collected from carcasses of suckling lambs show that performing feature selection contributes to classification performance increasing accuracy from 89.70% with the full feature set to 91.80% and 93.89% with the SVM approach and PCA, respectively. Moreover, the neural classifiers yield a significant increase in the accuracy with respect to the PLS model (85.60% accuracy). It is noteworthy that unlike PCA or PLS, the feature selection techniques that select relevant wavelengths allow the user to identify the regions in the spectrum with the most discriminant power, which makes the understanding of this process easier for veterinary experts. The robustness of the feature selection methods is assessed via a visual approach.

A novel ensembling method to boost performance of neural networks

ABSTRACTClassification is one of the important tasks in data mining. Over the last few decades, neural networks have proved to be very effective in solving this task. This paper presents a novel algorithm, called Neural Network Boosting (NNBOOST) to improve the classification performance of neural networks using ensembling technique. The ensembling technique proposed here is based on the boosting concept to ensemble classifiers. A boosting algorithm sequentially learns a new model by reassigning pattern weights. NNBOOST algorithm adapts a new technique to assign weights to patterns based on the Euclidean distances of patterns from the centroids of respective classes. It assigns more weight to patterns closest to the centroid. The pattern weights are updated subsequently after building or learning a neural network model, based on the updated centroids to build the next neural network model. Weighted Majority voting scheme is used to combine all the learned neural network models. Weight or confidence of a learned neural network model is calculated as the ratio of correctly classified to misclassified training patterns. The algorithm is validated with six real life data sets taken from UCI repository. Results show the effectiveness of the algorithm in ensembling neural networks.

Attacking Black-Box Image Classifiers With Particle Swarm Optimization

In order to better solve the shortcomings of Deep Neural Networks (DNNs) susceptible to adversarial examples, evaluating existing neural network classification performance and increasing training sets to improve the robustness of classification models require more effective methods of the adversarial examples generation. Under the black-box condition, less information about parameters of the classification model, limited query times, and less feedback information available, it is difficult to generate adversarial examples against the black-box model. In order to further improve the efficiency of the adversarial images generation, we propose two different variants of Partial Swarm Optimization algorithm (vPSO) base on the traditional Partial Swarm Optimization for the targeted and non-targeted attack under conditions of the completely black-box. Compared with the existing of the state-of-the-art generation algorithm, the vPSO effectively reduce the number of queries to the black-box classifier and the dependence on the feedback information. The success rate of the targeted attack is up to 96.0% and the average number of queries for the black-box model is greatly reduced. Furthermore, we propose an efficient target image screening method in targeted attacks, as well as the concept of easy-to-attack and hard-to-attack images in non-targeted attacks, and give corresponding distinctions.

Neurons With Paraboloid Decision Boundaries for Improved Neural Network Classification Performance.

In mathematical terms, an artificial neuron computes the inner product of a d -dimensional input vector x with its weight vector w , compares it with a bias value w0 and fires based on the result of this comparison. Therefore, its decision boundary is given by the equation wTx+w0=0 . In this paper, we propose replacing the linear hyperplane decision boundary of a neuron with a curved, paraboloid decision boundary. Thus, the decision boundary of the proposed paraboloid neuron is given by the equation (hTx+h0)2-||x-p||22=0 , where h and h0 denote the parameters of the directrix and p denotes the coordinates of the focus. Such paraboloid neural networks are proven to have superior recognition accuracy in a number of applications.

Training neural network classifiers through Bayes risk minimization applying unidimensional Parzen windows

A new training algorithm for neural networks in binary classification problems is presented. It is based on the minimization of an estimate of the Bayes risk by using Parzen windows applied to the final one-dimensional nonlinear transformation of the samples to estimate the probability of classification error. This leads to a very general approach to error minimization and training, where the risk that is to be minimized is defined in terms of integrated one-dimensional Parzen windows, and the gradient descent algorithm used to minimize this risk is a function of the window that is used. By relaxing the constraints that are typically applied to Parzen windows when used for probability density function estimation, for example by allowing them to be non-symmetric or possibly infinite in duration, an entirely new set of training algorithms emerge. In particular, different Parzen windows lead to different cost functions, and some interesting relationships with classical training methods are discovered. Experiments with synthetic and real benchmark datasets show that with the appropriate choice of window, fitted to the specific problem, it is possible to improve the performance of neural network classifiers over those that are trained using classical methods.

A Novel Rotation Forest Modality Based on Hybrid NNs: RF (ScPSO-NN)

Neural Network (NN), hybrid NN methods and Rotation Forest (RF) ensemble classifier are preferred in pattern analysis owing to their ability for finding efficient solutions on different problems. NN architecture usually includes backpropagation type algorithms in which error is exposed to fluctuations. Hybrid NN methods are generally designed to improve the classification performance of NN. Scout Particle Swarm Optimization (ScPSO) is one of these optimization algorithms including the effective parts of Particle Swarm Optimization (PSO) and Artificial Bee Colony Optimization (ABC). Moreover, RF algorithm usually indicates the same performance as in hybrid NN methods, although it is comprised of Decision Tree (DT) classifiers. At this point, our paper investigates whether RF using the hybrid NNs can outperform other ensemble classifiers in binary-medical pattern classification, or not. With this intention, PSO, ABC and ScPSO are placed in NN algorithms instead of back propagation, and hybrid methods (PSO-NN, ABC-NN and ScPSO-NN) are realized. As a result, RF (PSO-NN), RF (ABC-NN) and RF (ScPSO-NN) architectures are obtained. Classification Accuracy (CA), Area Under Curve (AUC), Sensitivity, Specificity, F-measure, Gmean and Precision metrics are used for a statistical performance comparison, and a test based on 2-fold cross validation method was realized on five medical datasets.

An effective image denoising using PPCA and classification of CT images using artificial neural networks

The main aim of denoising is to remove the noise while recollecting as much possible important signal features. This appears to be very simple when considered under practical situations, where the type of images and noises are all variable parameters. This paper deals with removal of combination of noises from image and classification of normal and abnormal images. At first phase, median filter is used to remove the noises present in the images. To improve the denoised output, we are using PSM and PPCA with morphological operations, filter and region props. In the second phase, to analyse the denoised output, neural network-based classification is proposed. The use of artificial intelligent techniques for classification shows a great potential in this field. Hence the performance of neural network classifier is estimated in terms of training performance and classification accuracy and is compared with the existing method to show the system is effective.