Generalization Capacity Research Articles

Semi-Supervised Classification for PolSAR Data With Multi-Scale Evolving Weighted Graph Convolutional Network

Although deep learning-based methods have been successfully applied to polarimetric synthetic aperture radar (PolSAR) image classification tasks, most of the available techniques are not suitable to deal with PolSAR data on irregular domains, e.g., superpixel graphs, because they are naturally designed as grid-based architectures in Euclidean space. To overcome this limitation and achieve robust PolSAR image classification, this article proposes the multiscale evolving weighted graph convolutional network, where weighted graphs based on superpixel technique and Wishart-derived distance are constructed to enable efficient handling of graphical PolSAR data representations. In this article, we derive a new architectural design named graph evolving module that combines pairwise latent feature similarity and kernel diffusion to refine the graph structure in each scale. Finally, we propose a graph integration module based on self-attention to perform robust hierarchical feature extraction and learn an optimal linear combination of various scales to exploit effective feature propagation on multiple graphs. We validate the superiority of proposed approach on classification performance with four real-measured datasets and demonstrate significant improvements compared to state-of-the-art methods. Additionally, the proposed method has shown strong generalization capacity across datasets with similar land covers.

Generalizability of machine learning models and empirical equations for the estimation of reference evapotranspiration from temperature in a semiarid region.

The Penman-Monteith equation is recommended for the estimation of reference evapotranspiration (ETo). However, it requires meteorological data that are commonly unavailable. Thus, this study evaluates artificial neural network (ANN), multivariate adaptive regression splines (MARS), and the original and calibrated Hargreaves-Samani (HS) and Penman-Monteith temperature (PMT) equations for the estimation of daily ETo using temperature. Two scenarios were considered: (i) local, models were calibrated/developed and evaluated using data from individual weather stations; (ii) regional, models were calibrated/developed using pooled data from several stations and evaluated independently in each one. Local models were also evaluated outside the calibration/training station. Data from 9 stations were used. The original PMT outperformed the original HS, but after local or regional calibrations, they performed similarly. The locally calibrated equations and the local machine learning models exhibited higher performances than their regional versions. However, the regional models had higher generalization capacity, with a more stable performance between stations. The machine learning models performed better than the equations evaluated. When comparing the ANN models with the HS equation, mean RMSE reduced from 0.96 to 0.87 and from 0.84 to 0.73, in regional and local scenarios, respectively. ANN and MARS performed similarly, with a slight advantage for ANN.

Новая интеллектуальная система для обнаружения сахарного диабета 2-го типа с модифицированной функцией потерь и регуляризацией

Type 2 Diabetes (T2DM) makes up about 90% of diabetes cases, as well as tough restriction on continuous monitoring and detecting become one of key aspects in T2DM. This research aims to develop an ensemble of several machine learning and deep learning models for early detection of T2DM with high accuracy. With high diversity of models, the ensemble will provide more excessive performance than single models. Methodology: The proposed system is modified enhanced ensemble of machine learning models for T2DM prediction. It is composed of Logistic Regression, Random Forest, SVM and Deep Neural Network models to generate a modified ensemble model. Results: The output of each model in the modified ensemble is used to figure out the final output of the system. The datasets being used for these models include Practice Fusion HER, Pima Indians diabetic's data, UCI AIM94 Dataset and CA Diabetes Prevalence 2014. In comparison to the previous solutions, the proposed ensemble model solution exposes the effectiveness of accuracy, sensitivity, and specificity. It provides an accuracy of 87.5% from 83.51% in average, sensitivity of 35.8% from 29.59% as well as specificity of 98.9% from 96.27%. The processing time of the proposed model solution with 96.6ms is faster than the state-of-the-art with 97.5ms. Conclusion: The proposed modified enhanced system in this work improves the overall prediction capability of T2DM using an ensemble of several machine learning and deep learning models. A majority voting scheme utilizes the output from several models to make the final accurate prediction. Regularization function in this work is modified in order to include the regularization of all the models in ensemble, that helps prevent the overfitting and encourages the generalization capacity of the proposed system.

A Survey on Curriculum Learning.

Curriculum learning (CL) is a training strategy that trains a machine learning model from easier data to harder data, which imitates the meaningful learning order in human curricula. As an easy-to-use plug-in, the CL strategy has demonstrated its power in improving the generalization capacity and convergence rate of various models in a wide range of scenarios such as computer vision and natural language processing etc. In this survey article, we comprehensively review CL from various aspects including motivations, definitions, theories, and applications. We discuss works on curriculum learning within a general CL framework, elaborating on how to design a manually predefined curriculum or an automatic curriculum. In particular, we summarize existing CL designs based on the general framework of Difficulty Measurer + Training Scheduler and further categorize the methodologies for automatic CL into four groups, i.e., Self-paced Learning, Transfer Teacher, RL Teacher, and Other Automatic CL. We also analyze principles to select different CL designs that may benefit practical applications. Finally, we present our insights on the relationships connecting CL and other machine learning concepts including transfer learning, meta-learning, continual learning and active learning, etc., then point out challenges in CL as well as potential future research directions deserving further investigations.

Concentration inequalities for two-sample rank processes with application to bipartite ranking

The ROC curve is the gold standard for measuring the performance of a test/scoring statistic regarding its capacity to discriminate between two statistical populations in a wide variety of applications, ranging from anomaly detection in signal processing to information retrieval, through medical diagnosis. Most practical performance measures used in scoring/ranking applications such as the AUC, the local AUC, the p-norm push, the DCG and others, can be viewed as summaries of the ROC curve. In this paper, the fact that most of these empirical criteria can be expressed as two-sample linear rank statistics is highlighted and concentration inequalities for collections of such random variables, referred to as two-sample rank processes here, are proved, when indexed by VC classes of scoring functions. Based on these nonasymptotic bounds, the generalization capacity of empirical maximizers of a wide class of ranking performance criteria is next investigated from a theoretical perspective. It is also supported by empirical evidence through convincing numerical experiments.

A Hybrid Approach for Neural Network in Pattern Storage

Your mind does not manufacture mind. Your mind forms neural networks. Neural networks had been effectively carried out to numerous sample garage and type troubles in phrases in their mastering ability, excessive discrimination electricity, and exceptional generalization ability. The achievement of many mastering schemes isn't always assured, however, seeing that algorithms like backpropagation have many drawbacks like stepping into the nearby minima, for that reason imparting suboptimal solutions. In the case of classifying big sets and complicated patterns, the traditional neural networks are afflicted by numerous problems inclusive of the dedication of the shape and length of the network, the computational complexity, and so on. This paper introduces the neural computing techniques especially radial foundation features network. Various upgrades and trends made in an artificial neural network for rushing up training, keeping off neighborhood minima, growing the generalization capacity, and different capabilities are reviewed.

Meta-Reinforcement Learning of Machining Parameters for Energy-Efficient Process Control of Flexible Turning Operations

Energy-efficient machining has become imperative for energy conservation, emission reduction, and cost saving of manufacturing sectors. Optimal machining parameter decision is regarded as an effective way to achieve energy efficient turning. For flexible machining, it is of utmost importance to determine the optimal parameters adaptive to various machines, workpieces, and tools. However, very little research has focused on this issue. Hence, this paper undertakes this challenge by integrated meta-reinforcement learning (MRL) of machining parameters to explore the commonalities of optimization models and use the knowledge to respond quickly to new machining tasks. Specifically, the optimization problem is first formulated as a finite Markov decision process (MDP). Then, the continuous parametric optimization is approached with actor-critic (AC) framework. On the basis of the framework, meta-policy training is performed to improve the generalization capacity of the optimizer. The significance of the proposed method is exemplified and elucidated by a case study with a comparative analysis. Note to Practitioners —Here, we consider a real-world application problem of energy-aware machining parameter optimization encountered in flexible turning operations, namely, design of a parametric optimization method that can be generalized to various machining tasks where multiple objectives and constraints varying with the machining configurations. This paper presents a novel meta-reinforcement learning (MRL)-based optimization method to improve the generalization by training optimizer with multiple machining tasks. To the best of our knowledge, this is the first MRL-based method of adaptive parameter decision for energy-efficient flexible machining. It should be highly emphasized that technologists benefit from the reduced decision-making time and the improved energy saving opportunity.

Nature inspired optimization tools for SVMs - NIOTS

Support Vector Machines (SVMs) technique for achieving classifiers and regressors. However, to obtain models with high accuracy and low complexity, it is necessary to define the kernel parameters as well as the parameters of the training model, which are called hyperparameters. The challenge of defining the more suitable value to hyperparameters is called the Parameter Selection Problem (PSP). However, minimizing the complexity and maximizing the generalization capacity of the SVMs are conflicting criteria. Therefore, we propose the Nature Inspired Optimization Tools for SVMs (NIOTS) that offers a method to automate the search process for the best possible solution for the PSP, allowing the user to quickly obtain several sets of good solutions and choose the one most appropriate for his specific problem.•The PSP has been modeled as a Multi-objective Optimization Problem (MOP) with two objectives: (1) good precision and (2) low complexity (low number of support vectors).•The user can evaluate multiple solutions included in the Pareto front, in terms of precision and low complexity of the model.•Apart from the Adaptive Parameter with Mutant Tournament Multiobjective Differential Evolution (APMT-MODE), the user can choose other metaheuristics and also among several kernel options.

Deep learning for pattern recognition of photovoltaic energy generation

With the rapid growth in computational complexities of statistical pattern recognition of photovoltaic (PV) energy measurements, the need for new data-driven models has emerged. Among machine learning frameworks, deep neural networks yield promising solutions due to their high generalization capacity, low estimation bias, and ease of implementation. In this study, first, the discriminative deep models including autoencoders, Long Short-Term Memory networks, and Rectified Linear Units are introduced as a class of mathematical models that directly estimate the future solar energy given historical measurements. In addition, Convolutional Neural Networks are explored to show the merit of spatiotemporal pattern recognition to increase PV generation prediction accuracy. Then, we explore deep sparse coding algorithms for behind-the-meter (BTM) energy disaggregation (ED) with PV generation in residential and commercial customers. We show how the combination of deep discriminative models with dictionary learning leads to a promising performance in decomposing the net electricity demand into the solar generation and demand. Finally, deep generative modeling for BTM ED with PV generation is proposed which learns powerful probability distribution functions from PV time series rather than crisp features captured by the classic discriminative deep learning. Our numerical results on real-world ED datasets of residential units show significant improvements in ED accuracy compared to the state-of-the-art sparse coding approaches.

Smooth dendrite morphological neurons

A typical feature of hyperbox-based dendrite morphological neurons (DMN) is the generation of sharp and rough decision boundaries that inaccurately track the distribution shape of classes of patterns. This feature is because the minimum and maximum activation functions force the decision boundaries to match the faces of the hyperboxes. To improve the DMN response, we introduce a dendritic model that uses smooth maximum and minimum functions to soften the decision boundaries. The classification performance assessment is conducted on nine synthetic and 28 real-world datasets. Based on the experimental results, we demonstrate that the smooth activation functions improve the generalization capacity of DMN. The proposed approach is competitive with four machine learning techniques, namely, Multilayer Perceptron, Radial Basis Function Network, Support Vector Machine, and Nearest Neighbor algorithm. Besides, the computational complexity of DMN training is lower than MLP and SVM classifiers.

Efficient prediction of transonic flutter boundaries for varying Mach number and angle of attack via LSTM network

Transonic aeroelastic analysis can be carried out accurately and efficiently by using the aerodynamic Reduced-Order Modeling (ROM) approach. However, the efficiency and generalization capability of traditional time-dependent ROM should be further enhanced, especially when dealing with the case for varying flight parameters. For such a purpose, a set of flight samples for different Mach numbers and mean angles of attack in transonic regime are selected to cover the concerned parameter space. Subsequently, a typical filtered white Gaussian noise is used as the input signal to excite the dynamical behavior of the aerodynamic system via the direct Computational Fluid Dynamic (CFD) technique, and the corresponding input and output data at all the flight samples are used as the training data set. Afterwards, based on the CFD training data set, the dynamical relationship between aerodynamic output and displacement input for varying Mach number and mean angle of attack can be approximately fitted by using the Long Short Term Memory (LSTM) network, which is a time-series prediction approach of deep learning method. Finally, the transonic flutter boundaries of a NACA 64A010 airfoil are investigated to assess the validity of the proposed approach. The comparison with CFD results shows that, the ROM can predict the unsteady aerodynamic response and aeroelastic characteristics well with low computation cost. In particular, the flutter boundaries of the concerned airfoil at different Mach numbers and mean angles of attack are obtained, due to the absence of time-delay term in surrogate model, the generalization capacity and modeling efficiency of the ROM are improved.

Artificial potential field neuro-fuzzy controller for autonomous navigation of mobile robots

Some autonomous navigation methods, when implemented alone, can lead to poor performance, whereas their combinations, when well thought out, can yield exceptional performances. We have demonstrated this by combining the artificial potential field and fuzzy logic methods in the framework of mobile robots’ autonomous navigation. In this article, we investigate a possible combination of three methods widely used in the autonomous navigation of mobile robots, and whose individual implementation still does not yield the expected performances. These are as follows: the artificial potential field, which is quick and easy to implement but faces local minima and robustness problems. Fuzzy logic is robust but computationally intensive. Finally, neural networks have an exceptional generalization capacity, but face data collection problems for the learning base and robustness. This article aims to exploit the advantages offered by each of these approaches to design a robust, intelligent, and computationally efficient controller. The combination of the artificial potential field and interval type-2 fuzzy logic resulted in an interval type-2 fuzzy logic controller whose advantage over the classical interval type-2 fuzzy logic controller was the small size of the rule base. However, it kept all the classical interval type-2 fuzzy logic controller characteristics, with the major disadvantage that type-reduction remains the main cause of high computation time. In this article, the type-reduction process is replaced with two layers of neural networks. The resulting controller is an interval type-2 fuzzy neural network controller with the artificial potential field controller’s outputs as auxiliary inputs. The results obtained by performing a series of experiments on a mobile platform demonstrate the proposed navigation system’s efficiency.

DFF-ResNet: An insect pest recognition model based on residual networks

Insect pest control is considered as a significant factor in the yield of commercial crops. Thus, to avoid economic losses, we need a valid method for insect pest recognition. In this paper, we proposed a feature fusion residual block to perform the insect pest recognition task. Based on the original residual block, we fused the feature from a previous layer between two 1×1 convolution layers in a residual signal branch to improve the capacity of the block. Furthermore, we explored the contribution of each residual group to the model performance. We found that adding the residual blocks of earlier residual groups promotes the model performance significantly, which improves the capacity of generalization of the model. By stacking the feature fusion residual block, we constructed the Deep Feature Fusion Residual Network (DFF-ResNet). To prove the validity and adaptivity of our approach, we constructed it with two common residual networks (Pre-ResNet and Wide Residual Network (WRN)) and validated these models on the Canadian Institute For Advanced Research (CIFAR) and Street View House Number (SVHN) benchmark datasets. The experimental results indicate that our models have a lower test error than those of baseline models. Then, we applied our models to recognize insect pests and obtained validity on the IP102 benchmark dataset. The experimental results show that our models outperform the original ResNet and other state-of-the-art methods.

COVIDGR Dataset and COVID-SDNet Methodology for Predicting COVID-19 Based on Chest X-Ray Images.

Currently, Coronavirus disease (COVID-19), one of the most infectious diseases in the 21st century, is diagnosed using RT-PCR testing, CT scans and/or Chest X-Ray (CXR) images. CT (Computed Tomography) scanners and RT-PCR testing are not available in most medical centers and hence in many cases CXR images become the most time/cost effective tool for assisting clinicians in making decisions. Deep learning neural networks have a great potential for building COVID-19 triage systems and detecting COVID-19 patients, especially patients with low severity. Unfortunately, current databases do not allow building such systems as they are highly heterogeneous and biased towards severe cases. This article is three-fold: (i) we demystify the high sensitivities achieved by most recent COVID-19 classification models, (ii) under a close collaboration with Hospital Universitario Clínico San Cecilio, Granada, Spain, we built COVIDGR-1.0, a homogeneous and balanced database that includes all levels of severity, from normal with Positive RT-PCR, Mild, Moderate to Severe. COVIDGR-1.0 contains 426 positive and 426 negative PA (PosteroAnterior) CXR views and (iii) we propose COVID Smart Data based Network (COVID-SDNet) methodology for improving the generalization capacity of COVID-classification models. Our approach reaches good and stable results with an accuracy of [Formula: see text], [Formula: see text], [Formula: see text] in severe, moderate and mild COVID-19 severity levels. Our approach could help in the early detection of COVID-19. COVIDGR-1.0 along with the severity level labels are available to the scientific community through this link https://dasci.es/es/transferencia/open-data/covidgr/.

PICCOLO White-Light and Narrow-Band Imaging Colonoscopic Dataset: A Performance Comparative of Models and Datasets

Colorectal cancer is one of the world leading death causes. Fortunately, an early diagnosis allows for effective treatment, increasing the survival rate. Deep learning techniques have shown their utility for increasing the adenoma detection rate at colonoscopy, but a dataset is usually required so the model can automatically learn features that characterize the polyps. In this work, we present the PICCOLO dataset, that comprises 3433 manually annotated images (2131 white-light images 1302 narrow-band images), originated from 76 lesions from 40 patients, which are distributed into training (2203), validation (897) and test (333) sets assuring patient independence between sets. Furthermore, clinical metadata are also provided for each lesion. Four different models, obtained by combining two backbones and two encoder–decoder architectures, are trained with the PICCOLO dataset and other two publicly available datasets for comparison. Results are provided for the test set of each dataset. Models trained with the PICCOLO dataset have a better generalization capacity, as they perform more uniformly along test sets of all datasets, rather than obtaining the best results for its own test set. This dataset is available at the website of the Basque Biobank, so it is expected that it will contribute to the further development of deep learning methods for polyp detection, localisation and classification, which would eventually result in a better and earlier diagnosis of colorectal cancer, hence improving patient outcomes.

Moore-Penrose pseudo-inverse and artificial neural network modeling in performance prediction of switched reluctance machine

PurposeThe purpose of this paper is to present the Moore-Penrose pseudoinverse (PI) modeling and compare with artificial neural network (ANN) modeling for switched reluctance machine (SRM) performance.Design/methodology/approachIn a design of an SRM, there are a number of parameters that are chosen empirically inside a certain interval, therefore, to find an optimal geometry it is necessary to define a good model for SRM. The proposed modeling uses the Moore-Penrose PI for the resolution of linear systems and finite element simulation data. To attest to the quality of PI modeling, a model using ANN is established and the two models are compared with the values determined by simulations of finite elements.FindingsThe proposed PI model showed better accuracy, generalization capacity and lower computational cost than the ANN model.Originality/valueThe proposed approach can be applied to any problem as long as experimental/computational results can be obtained and will deliver the best approximation model to the available data set.

Full-scale identification of ice load on ship hull by least square support vector machine method

For ensuring the structural safety of an icebreaker, the ice load is identified from the strains measured on the frames or plates of the hull along the waterline. The strain sensors are difficult to install, however, at the required locations on the hull because the watertight compartment is cramped. In this situation, the far-field load identification method can be applied to overcome such restriction on the installation location. To determine the nonlinear relationship between ice load and ice-induced strains, the least square support vector machine (LS_SVM) method is adopted. Among the advantages that this method affords for ice load identification are its capacity for small sample learning, global optimization, and strong generalization. In this study, a ship-based ice load measurement system installed on icebreaker XueLong is introduced. The experimental application is performed to verify the feasibility of the LS_SVM procedure and establish a full-scale ice load identification model. With this method, the identified ice load is reasonably verified through comparative analysis and case study. To estimate the ice load through far-field measurements, the predictive ability of the LS_SVM algorithm can further be applied.

Ensemble of CheXNet and VGG-19 Feature Extractor with Random Forest Classifier for Pediatric Pneumonia Detection.

Pneumonia, an acute respiratory infection, causes serious breathing hindrance by damaging lung/s. Recovery of pneumonia patients depends on the early diagnosis of the disease and proper treatment. This paper proposes an ensemble method-based pneumonia diagnosis from Chest X-ray images. The deep Convolutional Neural Networks (CNNs)—CheXNet and VGG-19 are trained and used to extract features from given X-ray images. These features are then ensembled for classification. To overcome data irregularity problem, Random Under Sampler (RUS), Random Over Sampler (ROS) and Synthetic Minority Oversampling Technique (SMOTE) are applied on the ensembled feature vector. The ensembled feature vector is then classified using several Machine Learning (ML) classification techniques (Random Forest, Adaptive Boosting, K-Nearest Neighbors). Among these methods, Random Forest got better performance metrics than others on the available standard dataset. Comparison with existing methods shows that the proposed method attains improved classification accuracy, AUC values and outperforms all other models providing 98.93% accurate prediction. The model also exhibits potential generalization capacity when tested on different dataset. Outcomes of this study can be great to use for pneumonia diagnosis from chest X-ray images.

Detecting Plant Invasion in Urban Parks with Aerial Image Time Series and Residual Neural Network

Invasive plants are a major agent threatening biodiversity conservation and directly affecting our living environment. This study aims to evaluate the potential of deep learning, one of the fastest-growing trends in machine learning, to detect plant invasion in urban parks using high-resolution (0.1 m) aerial image time series. Capitalizing on a state-of-the-art, popular architecture residual neural network (ResNet), we examined key challenges applying deep learning to detect plant invasion: relatively limited training sample size (invasion often confirmed in the field) and high forest contextual variation in space (from one invaded park to another) and over time (caused by varying stages of invasion and the difference in illumination condition). To do so, our evaluations focused on a widespread exotic plant, autumn olive (Elaeagnus umbellate), that has invaded 20 urban parks across Mecklenburg County (1410 km2) in North Carolina, USA. The results demonstrate a promising spatial and temporal generalization capacity of deep learning to detect urban invasive plants. In particular, the performance of ResNet was consistently over 96.2% using training samples from 8 (out of 20) or more parks. The model trained by samples from only four parks still achieved an accuracy of 77.4%. ResNet was further found tolerant of high contextual variation caused by autumn olive’s progressive invasion and the difference in illumination condition over the years. Our findings shed light on prioritized mitigation actions for effectively managing urban invasive plants.

Prediction of the reflection intensity of natural hydroxyapatite using generalized linear model and ensemble learning methods

AbstractLaboratory data acquisition and analysis of X‐ray diffraction (XRD) data involves a lot of tedious human engineering and is time‐consuming. To put in context, a summation of the material synthesis procedure leading to the analysis of the structure of the material can span several days. To curb this challenge and to enhance innovations in engineering pedagogy, this article investigates an alternative method that uses supervised learning algorithms based on ensemble techniques and a generalized linear model (GLM) for predicting reflection intensity (XRD patterns) of raw and natural hydroxyapatite under varying sintering temperature conditions given Bragg angles as input to the machine learning algorithms. For the experiment, we trained GLM and ensemble learning models (CatBoost, LightGBM, and two variants of XGBoost based on manual and genetic algorithm for tuning of the hyperparameters). The results show that most instances of the XGBoost yielded a robust performance that surpasses all other approaches, when predicting X‐ray reflection intensities ascribed to the biomaterials subjected to varying sintering temperature conditions. In addition, the results show that all the ensemble techniques significantly outperform the GLM, this indicates that the former exhibits better generalization capacity. The ensemble learning techniques and the GLM presents a reduced computational complexity.