Complex Classification Problems Research Articles

Deep Learning with Unsupervised Data Labeling for Weed Detection in Line Crops in UAV Images

In recent years, weeds have been responsible for most agricultural yield losses. To deal with this threat, farmers resort to spraying the fields uniformly with herbicides. This method not only requires huge quantities of herbicides but impacts the environment and human health. One way to reduce the cost and environmental impact is to allocate the right doses of herbicide to the right place and at the right time (precision agriculture). Nowadays, unmanned aerial vehicles (UAVs) are becoming an interesting acquisition system for weed localization and management due to their ability to obtain images of the entire agricultural field with a very high spatial resolution and at a low cost. However, despite significant advances in UAV acquisition systems, the automatic detection of weeds remains a challenging problem because of their strong similarity to the crops. Recently, a deep learning approach has shown impressive results in different complex classification problems. However, this approach needs a certain amount of training data, and creating large agricultural datasets with pixel-level annotations by an expert is an extremely time-consuming task. In this paper, we propose a novel fully automatic learning method using convolutional neuronal networks (CNNs) with an unsupervised training dataset collection for weed detection from UAV images. The proposed method comprises three main phases. First, we automatically detect the crop rows and use them to identify the inter-row weeds. In the second phase, inter-row weeds are used to constitute the training dataset. Finally, we perform CNNs on this dataset to build a model able to detect the crop and the weeds in the images. The results obtained are comparable to those of traditional supervised training data labeling, with differences in accuracy of 1.5% in the spinach field and 6% in the bean field.

Method for Recognizing Mechanical Status of Container Crane Motor Based on SOM Neural Network

The recognition of mechanical state of container crane motor is a complex nonlinear mode classification problem. In this paper, a deep learning method based on self-organizing feature map (SOM) neural network was developed for the cluster analysis of motor vibration signals to characterize the mechanical state of the lifting motor. Through Python software simulation, the vibration intensity signal of the lifting motor can be divided into five different categories, and the interval range of each type of vibration intensity signal is statistically obtained, which corresponds to the five mechanical states of the motor. The SOM neural network can realize the effective and rapid self-adaptive classification of the motor vibration signal, realize the recognition of the mechanical status of the crane motor, and provide a certain basis for the maintenance of the motor.

An All-Memristor Deep Spiking Neural Computing System: A Step Toward Realizing the Low-Power Stochastic Brain

Deep analog artificial neural networks (ANNs) perform complex classification problems with remarkably high accuracy. However, they rely on humongous amount of power to perform the calculations, veiling the accuracy benefits. The biological brain, on the other hand, is significantly more powerful than such networks and consumes orders of magnitude less power, indicating us about some conceptual mismatch. Given that the biological neurons communicate using energy efficient trains of spikes, and the behavior is nondeterministic, incorporating these effects in deep artificial neural networks may drive us few steps toward a more realistic neuron. In this paper, we propose how the inherent stochasticity of nanoscale resistive devices can be harnessed to emulate the functionality of a spiking neuron that can be incorporated in deep stochastic spiking neural networks (SNN). At the algorithmic level, we propose how the training can be modified to convert an ANN to an SNN while supporting the stochastic activation function offered by these devices. We devise circuit architectures to incorporate stochastic memristive neurons along with memristive crossbars, which perform the functionality of the synaptic weights. We tested the proposed all-memristor deep stochastic SNN for image classification and observed only about 1% degradation in accuracy with the ANN baseline after incorporating the circuit and device related nonidealities. We witnessed that the network is robust to certain variations and consumes ~6.4 × less energy than its complementary metal oxide semiconductor (CMOS) counterpart.

Метод формалізації процесу прийняття рішення на базі теорії порогових елементів

The conceptual principles and the theoretical substantiation of the peculiarities of the threshold elements mathematical apparatus application for the making decision of complex classification problems are formulated. This makes it possible to significantly simplify the procedure for formalizing the DSS for objects with quantitative evaluation parameters. The method of formalizing the making decision process based on the theory of threshold elements for objects with quantitative evaluation parameters is suggested, which, unlike existing approaches, in particular the method of linear weighted sums, allows us to substantiate weighting of estimating parame-ters. This allows for making decision of a clear and one-valued solution provided that the combinations of valuation parameters are incompletely checked. The process of constructing the logical function of choice is formalized and the algorithm of the transition from the logical selection function to the threshold function is proposed. Construction of the logical choice function requires the definition of its minimum disjunctive normal form. This led to the need to find the number of simple implicants that should have this form of function. Such a process was identified by proving the authors of the corresponding theorem. A corresponding approach to the making classification decision, when functionally meaningless elements that are theoretically possible to be ignored can’t be taken into account, and threshold selection functions will not ensure the attribution of an element to any class is formulated. The algorithm of making decision in the process of application of the method described above is produced. The peculiarities of such an algorithm are that it is clearly and cardinally described and quite simply implemented with the use of modern computer technology.

A new fuzzy learning vector quantization method for classification problems based on a granular approach

In this paper, a new Fuzzy Learning Vector Quantization (FuzzLVQ) method for classification is presented. FuzzLVQ is a hybrid method based on LVQ neural networks and fuzzy systems. FuzzLVQ was implemented using modular architectures based on a granular approach, to further improve its performance in complex classification problems. The contribution of this research work is the development of the new fuzzy learning quantization method (FuzzLVQ) for classification problems, which is a hybrid method based on LVQ neural networks and fuzzy inference systems. In this work, a set of 15 experiments were performed for each presented architecture and a comparison between the classical LVQ algorithm and the new FuzzLVQ is presented. The obtained results are favorable, and the classification accuracy is slightly higher with the FuzzLVQ method. The hybridization proved to be beneficial also in other aspects of the method, such as in the training times of the neural network, and the number of cluster centers, which are reduced in respect to the classical LVQ performance. A proper optimization method could help improve the classification accuracy even more.

Improved Classification by Non Iterative and Ensemble Classifiers in Motor Fault Diagnosis

Data driven approach for multi-class fault diagnosis of induction motor using MCSA at steady state condition is a complex pattern classification problem. This investigation has exploit ...

Initiative For Thyroid Cancer Diagnosis: Decision Support System For Anaplast Thyroid Cancer

Due to the high level exposure of biomedical image analysis, Medical image mining has become one of the well-established research area(s) of machine learning. AI (Artificial Intelligence) techniques have been vastly used to solve the complex classification problems of thyroid cancer. Since the persistence of copycat chromatin properties and unavailability of nuclei measurement techniques, it is really problem for doctors to determine the initial phases of nuclei enlargement and to assess the early changes of chromatin distribution. For example involvement of multiple transparent overlapping of nuclei may become the cause of confusion to infer the growth pattern of nuclei variations. Un-decidable nuclei eccentric properties may become one of the leading causes for misdiagnosis in Anaplast cancers. In-order to mitigate all above stated problems this paper proposes a novel methodology so called “Decision Support System for Anaplast Thyroid Cancer” and it proposes a medical data preparation algorithm AD (Analpast_Cancers) which helps to select the appropriate features of Anaplast cancers such as (1) enlargement of nuclei, (2) persistence of irregularity in nuclei and existence of hyper chromatin. Proposed methodology comprises over four major layers, first layer deals with the noise reduction, detection of nuclei edges and object clusters. Second layer selects the features of object of interest such as nuclei enlargement, irregularity and hyper chromatin. Third layer constructs the decision model to extract the hidden patterns of disease associated variables and final layer evaluates the performance evaluation by using confusion matrix, precision and recall measures. The overall classification accuracy is measured about 97.2% with 10-k fold cross validation.

DANNP: an efficient artificial neural network pruning tool

Background Artificial neural networks (ANNs) are a robust class of machine learning models and are a frequent choice for solving classification problems. However, determining the structure of the ANNs is not trivial as a large number of weights (connection links) may lead to overfitting the training data. Although several ANN pruning algorithms have been proposed for the simplification of ANNs, these algorithms are not able to efficiently cope with intricate ANN structures required for complex classification problems. Methods We developed DANNP, a web-based tool, that implements parallelized versions of several ANN pruning algorithms. The DANNP tool uses a modified version of the Fast Compressed Neural Network software implemented in C++ to considerably enhance the running time of the ANN pruning algorithms we implemented. In addition to the performance evaluation of the pruned ANNs, we systematically compared the set of features that remained in the pruned ANN with those obtained by different state-of-the-art feature selection (FS) methods. Results Although the ANN pruning algorithms are not entirely parallelizable, DANNP was able to speed up the ANN pruning up to eight times on a 32-core machine, compared to the serial implementations. To assess the impact of the ANN pruning by DANNP tool, we used 16 datasets from different domains. In eight out of the 16 datasets, DANNP significantly reduced the number of weights by 70%–99%, while maintaining a competitive or better model performance compared to the unpruned ANN. Finally, we used a naïve Bayes classifier derived with the features selected as a byproduct of the ANN pruning and demonstrated that its accuracy is comparable to those obtained by the classifiers trained with the features selected by several state-of-the-art FS methods. The FS ranking methodology proposed in this study allows the users to identify the most discriminant features of the problem at hand. To the best of our knowledge, DANNP (publicly available at www.cbrc.kaust.edu.sa/dannp) is the only available and on-line accessible tool that provides multiple parallelized ANN pruning options. Datasets and DANNP code can be obtained at www.cbrc.kaust.edu.sa/dannp/data.php and https://doi.org/10.5281/zenodo.1001086.

An Improvised k-NN Respecting Diversity of Data for Network Intrusion Detection

Network Intrusion Detection is a complex classification problem aimed at discriminating the legitimate from illegitimate and potentially harmful network connections over the communication network. What adds to the complexity of the problem is the near real - time response to a threat, imbalanced datasets to deal with and finally the data being mixed in nature with some features being numeric some discrete an d some nominal. In this work we have applied Synthetic Minority Oversampling Technique ( SMOTE ) to balance the dataset and eliminate the skewness of the class distribution. The success of k - Nearest Neighbour ( k - NN ) depends upon the set of neighbours deemed to be very close or similar to a data point which is in turn determined by the similarity /distance metric empl oyed, where most of the metrics employed in literature deal with numeric data only, and either need con version of categorical features to numeric features or simply eliminated the categorical features, which often leads to reduction in the results. As for this work is considered, we take into consideration both the categories of features simultaneously by replac ing the conventional Euclidean metric with Gower metric, which is better suited for mixed data . Gower metric provides a mechanism to deal with heterogeneous features differently and ultimately yields a quantifiable value that determines the similarit y of the two instances. Experimental results show that improvised version of k - NN outperforms its conventional counterpart in terms of the Accuracy, Detection Rate, Precision, Recall, f - Measure, and Receiver Operating Characteristic ( ROC ) curve .

Data equilibrium based automatic image annotation by fusing deep model and semantic propagation

Automatic image annotation is a challenging research problem that includes a large number of tags and various features. Traditional shallow machine learning algorithms lack generalization performance when dealing with complex classification problems. Automatic image annotation based on a stacked auto-encoder (SAE) is proposed to enhance the annotation generalization performance. In this paper, two kinds of strategies, the annotation model and the annotation process, are proposed to solve the main problem of unbalanced data in image annotation. 1) For the annotation model itself, to improve the annotation effect of low frequency tags, we propose a balanced and stacked auto-encoder (BSAE) that can enhance training for low frequency tags. On the basis of this model, a robust BSAE (RBSAE) algorithm which enhances training for sub BSAE model by group is proposed to enhance the annotation stability. This strategy ensures that the model itself has a strong ability to deal with the problem of unbalanced data. 2) For the annotation process, we propose a framework of attribute discrimination annotation (ADA). We first take an unknown image. Then we construct a local equilibrium dataset based on the unknown image and discriminate the high- and low-frequency attribute of the image to determine the corresponding annotation process. One process called the local semantic propagation (LDE-SP) algorithm annotates the low frequency image and the RBSAE algorithm annotates the high frequency image. This strategy improves the overall image annotation effect and ensures that the annotation process has a strong ability to deal with the problem of unbalanced data. For each SAE (including BSAE and RBSAE) annotation model, we propose two kinds of optimization methods, namely, one that is based on non-linear optimization and one on linear optimization. Experimental results on three benchmark datasets show that the proposed model outperforms the previous models in many performance indices.

A Two Phase Hybrid Classifier based on Structure Similarities and Textural Features for Accurate Meningioma Classification

Meningioma subtype classification is a complex pattern classification problem of digital pathology due to het-erogeneity issues of tumor texture, low inter-class and high intra-class texture variations of tumor samples, and architec-tural variations of cellular components. The basic aim is the achievement of significantly high classification results for all the subtypes of meningioma while dealing with inherent complexity and texture variations. The ultimate goal is to mimic the prognosis decision of expert pathologists and assist newer pathologists in making right and quick decisions. In this paper, a novel hybrid classification framework based on nuclei shape matching and texture analysis is proposed for classification of four subtypes of grade-I benign meningioma. Meningothelial and fibroblastic subtypes are classified on basis of nuclei shape matching through skeletons and shock graphs while an optimized texture-based evolutionary framework is designed for the classification of transi-tional and psammomatous subtypes. Classifier-based evolutionary feature selection is performed using Genetic Algorithm (GA) in combination with Support Vector Machine (SVM) to select the optimal combination of higher-order statistical features extracted from morphologically processed RGB color channel images. The proposed hybrid classifier employed leave-one-patient-out 5-fold cross validation and achieved an overall 95.63% mean classification accuracy.

An artificial immune system for offline isolated handwritten arabic character recognition

Character recognition plays an important role in the modern world. In recent years, character recognition systems for different languages has gain importance. The recognition of Arabic writing is still an important challenge due to its cursive nature and great topological variability. The Artificial Immune System is a supervised learning technique that embodies the concepts of natural immunity to cope with complex classification problems. The objective of this research is to investigate the applicability of an Artificial Immune System in Offline Isolated Handwritten Arabic Characters. The developed system is composed of three main modules: preprocessing, feature extraction and recognition. The system was trained and tested with ten-fold cross-validation technique on an original realistic database that we built from the well-known IFN/ENIT benchmark. Parameter tuning was performed with a grid-search algorithm with leave-one-out cross-validation. The obtained results of the proposed system are promising with a classification rate of 93.25% and often outperform most well-known classifiers from Scikit Learn Library.

A regularized root–quartic mixture of experts for complex classification problems

Mixture of experts is a neural network based ensemble learning approach consisting of several experts and a gating network. In this paper, we introduce regularized root–quartic mixture of experts (R-RTQRT-ME) by incorporating a regularization term into the error function to control the complexity of model and to increase robustness in confronting with over-fitting and noise. The average of the results of R-RTQRT-ME on 20 classification benchmark datasets, shows that this algorithm performs 1.75%, 2.50%, 2.29% better than multi objective regularized negative correlation learning, multi objective negative correlation learning and multi objective neural network, respectively. Also, the average of improvements of R-RTQRT-ME is 1.16%, 2.31%, 3.40%, 3.39% in comparison with root-quartic mixture of experts, mixture of negatively correlated experts, mixture of experts and negative correlation learning, respectively. Furthermore, the effect of the regularization penalty term in R-RTQRT-ME on noisy data is analyzed which shows the robustness of R-RTQRT-ME in these situations.

Letter to the Editor: Suicide as a complex classification problem: machine learning and related techniques can advance suicide prediction - a reply to Roaldset (2016).

Letter to the Editor: Suicide as a complex classification problem: machine learning and related techniques can advance suicide prediction - a reply to Roaldset (2016).

Root-quatric mixture of experts for complex classification problems

Mixture of experts (ME) as an ensemble method consists of several experts and a gating network to decompose the input space into some subspaces regarding to the experts specialties. To increase the diversity between experts in ME, this paper incorporates a correlation penalty function into the error function of ME. The significant of this modification is providing an occasion to encourage experts to specialize on different parts of the input space and to create decorrelated experts. The experimental results of this approach reveals that the impacts of this penalty function is extremely improved the diversity of experts and the tradeoff between the accuracy and the diversity in ME. Moreover in the implementation of this method, the experts are trained simultaneously and they can communicate by the aid of the correlation penalty function. The performance of the proposed method on ten classification benchmark datasets shows that the average of accuracy of this method improves 1.94%, 3.7%, and 3.74% compared with the mixture of negatively correlated experts, ME and the negative correlation learning, respectively. Thus the proposed method can be considered as a better classifier for healthy and medical problems and also when the great non-stationary data should be classified.

A fast training algorithm for extreme learning machine based on matrix decomposition

Extreme Learning Machine (ELM), a competitive machine learning technique for single-hidden-layer feedforward neural networks (SLFNNs), has proven to be efficient and effective algorithm for regression and classification problems. However, traditional ELM involves a large number of hidden nodes for complex real world regression and classification problems which increasing the computation burden. In this paper, a decomposition based fast ELM (DFELM) algorithm is proposed to effectively reduce the computational burden for large number of hidden nodes condition. Compared with ELM algorithm, DFELM algorithm has faster training time with a large number of hidden nodes maintaining the same accuracy performance. Experiment on three regression problems, six classification problems and a complex blast furnace modeling problem are carried out to verify the performance of DFELM algorithm. Moreover, the decomposition method can be extended to other modified ELM algorithms to further reduce the training time.

An effective multivariate time series classification approach using echo state network and adaptive differential evolution algorithm

The multivariate time series (MTS) classification is a very difficult process because of the complexity of the MTS data type. Among all the methods to resolve this problem, the attribute–value representation classification approaches are the most popular. Despite their proven effectiveness of these however, these approaches are time consuming, sensitive to noise, or prone to damage of inner data properties as well as capable of producing undesirable accuracy. In this paper, we propose a new approach (CADE) for MTS classification that utilizes recurrent neural network (RNN) and adaptive differential evolution (ADE) algorithm. The approach can effectively overcome specific shortcomings of the attribute–value representation approaches. The principle of this approach adheres to three steps. First, an RNN is used to project the training MTS samples into different state clouds (samples in the same class are projected into a state cloud). Second, classifiers from these state clouds are induced for different classes. Third, the final MTS classifiers are obtained using ADE for parameter optimization. This approach makes full use of the network state space of a given RNN to induce classifiers rather than to train the network. Experimental results performed on 18 data sets demonstrate the accuracy and robustness of the proposed approach for MTS classification. As a new and universal approach, CADE can be very effective and stable for handling a variety of complex classification problems.

Analysis of the socioecological structure and dynamics of the territory using a hybrid Bayesian network classifier

Territorial planning and management requires that the spatial structure of the socioecological sectors is adequately understood. Several classification techniques exist that have been applied to detect ecological, or socioeconomic sectors, but not simultaneously in the same model; and also, with a limited number of variables. We have developed and applied a new probabilistic methodology – based on hierarchical hybrid Bayesian network classifiers – to identify the different socioecological sectors in Andalusia, a region in southern Spain, and incorporate a scenario of change. Results show that a priori, the socioecological structure is highly heterogeneous, with an altitude gradient from the river basin to the mountain peaks. However, under a scenario of global environmental change this heterogeneity is lost, making the territory more vulnerable to any alteration or disturbance. The methodology applied allows dealing with complex problems, containing a large number of variables, by splitting them into several sub-problems that can be easily solved. In the case of territorial planning, each component of the territory is modelled independently before combining them into a general classifier model. Furthermore, it can be applied to any complex unsupervised classification problem with no modification to the methodology.

Belief rule-based classification system: Extension of FRBCS in belief functions framework

Among the computational intelligence techniques employed to solve classification problems, the fuzzy rule-based classification system (FRBCS) is a popular tool capable of building a linguistic model interpretable to users. However, it may face lack of accuracy in some complex applications, by the fact that the inflexibility of the concept of the linguistic variable imposes hard restrictions on the fuzzy rule structure. In this paper, we extend the fuzzy rule in FRBCS with a belief rule structure and develop a belief rule-based classification system (BRBCS) to address imprecise or incomplete information in complex classification problems. The two components of the proposed BRBCS, i.e., the belief rule base (BRB) and the belief reasoning method (BRM), are designed specifically by taking into account the pattern noise that existes in many real-world data sets. Four experiments based on benchmark data sets are carried out to evaluate the classification accuracy, robustness, interpretability and time complexity of the proposed method.

Function Clustering Self-Organization Maps (FCSOMs) for mining differentially expressed genes in Drosophila and its correlation with the growth medium.

The central task of this study was to mine the gene-to-medium relationship. Adequate knowledge of this relationship could potentially improve the accuracy of differentially expressed gene mining. One of the approaches to differentially expressed gene mining uses conventional clustering algorithms to identify the gene-to-medium relationship. Compared to conventional clustering algorithms, self-organization maps (SOMs) identify the nonlinear aspects of the gene-to-medium relationships by mapping the input space into another higher dimensional feature space. However, SOMs are not suitable for huge datasets consisting of millions of samples. Therefore, a new computational model, the Function Clustering Self-Organization Maps (FCSOMs), was developed. FCSOMs take advantage of the theory of granular computing as well as advanced statistical learning methodologies, and are built specifically for each information granule (a function cluster of genes), which are intelligently partitioned by the clustering algorithm provided by the DAVID_6.7 software platform. However, only the gene functions, and not their expression values, are considered in the fuzzy clustering algorithm of DAVID. Compared to the clustering algorithm of DAVID, these experimental results show a marked improvement in the accuracy of classification with the application of FCSOMs. FCSOMs can handle huge datasets and their complex classification problems, as each FCSOM (modeled for each function cluster) can be easily parallelized.