Good Generalization Performance Research Articles

A critical examination of robustness and generalizability of machine learning prediction of materials properties

Recent advances in machine learning (ML) have led to substantial performance improvement in material database benchmarks, but an excellent benchmark score may not imply good generalization performance. Here we show that ML models trained on Materials Project 2018 can have severely degraded performance on new compounds in Materials Project 2021 due to the distribution shift. We discuss how to foresee the issue with a few simple tools. Firstly, the uniform manifold approximation and projection (UMAP) can be used to investigate the relation between the training and test data within the feature space. Secondly, the disagreement between multiple ML models on the test data can illuminate out-of-distribution samples. We demonstrate that the UMAP-guided and query by committee acquisition strategies can greatly improve prediction accuracy by adding only 1% of the test data. We believe this work provides valuable insights for building databases and models that enable better robustness and generalizability.

Strip Surface Defect Detection Algorithm Based on YOLOv5.

In order to improve the detection accuracy of the surface defect detection of industrial hot rolled strip steel, the advanced technology of deep learning is applied to the surface defect detection of strip steel. In this paper, we propose a framework for strip surface defect detection based on a convolutional neural network (CNN). In particular, we propose a novel multi-scale feature fusion module (ATPF) for integrating multi-scale features and adaptively assigning weights to each feature. This module can extract semantic information at different scales more fully. At the same time, based on this module, we build a deep learning network, CG-Net, that is suitable for strip surface defect detection. The test results showed that it achieved an average accuracy of 75.9 percent (mAP50) in 6.5 giga floating-point operation (GFLOPs) and 105 frames per second (FPS). The detection accuracy improved by 6.3% over the baseline YOLOv5s. Compared with YOLOv5s, the reference quantity and calculation amount were reduced by 67% and 59.5%, respectively. At the same time, we also verify that our model exhibits good generalization performance on the NEU-CLS dataset.

Human Activity Recognition Based on Two-Channel Residual–GRU–ECA Module with Two Types of Sensors

With the thriving development of sensor technology and pervasive computing, sensor-based human activity recognition (HAR) has become more and more widely used in healthcare, sports, health monitoring, and human interaction with smart devices. Inertial sensors were one of the most commonly used sensors in HAR. In recent years, the demand for comfort and flexibility in wearable devices has gradually increased, and with the continuous development and advancement of flexible electronics technology, attempts to incorporate stretch sensors into HAR have begun. In this paper, we propose a two-channel network model based on residual blocks, an efficient channel attention module (ECA), and a gated recurrent unit (GRU) that is capable of the long-term sequence modeling of data, efficiently extracting spatial–temporal features, and performing activity classification. A dataset named IS-Data was designed and collected from six subjects wearing stretch sensors and inertial sensors while performing six daily activities. We conducted experiments using IS-Data and a public dataset called w-HAR to validate the feasibility of using stretch sensors in human action recognition and to investigate the effectiveness of combining flexible and inertial data in human activity recognition, and our proposed method showed superior performance and good generalization performance when compared with the state-of-the-art methods.

BabyNutri

The physical and physiological development of infants and toddlers requires the proper amount of macronutrient intake, making it an essential problem to estimate the macronutrient in baby food. Nevertheless, existing solutions are either too expensive or poor performing, preventing the widespread use of automatic baby nutrient intake logging. To narrow this gap, this paper proposes a cost-effective and portable baby food macronutrient estimation system, BabyNutri. BabyNutri exploits a novel spectral reconstruction algorithm to reconstruct high-dimensional informative spectra from low-dimensional spectra, which are available from low-cost spectrometers. We propose a denoising autoencoder for the reconstruction process, by which BabyNutri can reconstruct a 160-dimensional spectrum from a 5-dimensional spectrum. Since the high-dimensional spectrum is rich in light absorption features of macronutrients, it can achieve more accurate macronutrient estimation. In addition, considering that baby food contains complex ingredients, we also design a CNN nutrition estimation model with good generalization performance over various types of baby food. Our extensive experiments over 88 types of baby food show that the spectral reconstruction error of BabyNutri is only 5.91%, reducing 33% than the state-of-the-art baseline with the same time complexity. In addition, the nutrient estimation performance of BabyNutri not only obviously outperforms state-of-the-art and cost-effective solutions but also is highly correlated with the professional spectrometer, with the correlation coefficients of 0.81, 0.88, 0.82 for protein, fat, and carbohydrate, respectively. However the price of our system is only one percent of the commercial solution. We also validate that BabyNutri is robust regarding various factors, e.g., ambient light, food volume, and even unseen baby food samples.

A Non-Intrusive speech quality evaluation algorithm for hearing aids via an auxiliary training task

With the increasing severity of hearing problems caused by an aging population, it has become more urgent to study speech quality evaluation methods for hearing aids to automatically judge the effect of parameter adjustments to solve the problem of lack of audiologists. A new evaluation algorithm, which is based on the multi-task learning strategy that combines the main task of non-intrusive speech quality evaluation with the auxiliary task of quality classification, is proposed. The main task network is composed of a bi-directional long short-term memory (BiLSTM) network combined with the self-attention mechanism, and the auxiliary task network is formed by fully connected layers based on batch normalization. The proposed algorithm firstly uses a convolutional neural network (CNN) to extract frame-level features. Then, the features are inputted into the main task network and the auxiliary task network to extract utterance-level features, respectively. Finally, the objective scores and quality ratings are obtained by the different mappers. Evaluation experiments of the Hearing Aid Speech Quality Index (HASQI) show that compared with others algorithms, the proposed algorithm effectively improves the speech quality evaluation. And it also shows strong robustness and good generalization performances under different distorted conditions.

Development of prediction models to estimate extubation time and midterm recovery time of ophthalmic patients undergoing general anesthesia: a cross-sectional study

BackgroundTo develop prediction models for extubation time and midterm recovery time estimation in ophthalmic patients who underwent general anesthesia.MethodsTotally 1824 ophthalmic patients who received general anesthesia at Joint Shantou International Eye Center were included. They were divided into a training dataset of 1276 samples, a validation dataset of 274 samples and a check dataset of 274 samples. Up to 85 to 87 related factors were collected for extubation time and midterm recovery time analysis, respectively, including patient factors, anesthetic factors, surgery factors and laboratory examination results. First, multiple linear regression was used for predictor selection. Second, different methods were used to develop predictive models for extubation time and midterm recovery time respectively. Finally, the models’ generalization abilities were evaluated using a same check dataset with MSE, RMSE, MAE, MAPE, R-Squared and CCC.ResultsThe fuzzy neural network achieved the highest R-Squared of 0.956 for extubation time prediction and 0.885 for midterm recovery time, and the RMSE value was 6.637 and 9.285, respectively.ConclusionThe fuzzy neural network developed in this study had good generalization performance in predicting both extubation time and midterm recovery time of ophthalmic patients undergoing general anesthesia.Trial registrationThis study is prospectively registered in the Chinese Clinical Trial Registry, registration number: CHiCRT2000036416, registration date: August 23, 2020.

Enhanced task parameterized dynamic movement primitives by GMM to solve manipulation tasks

PurposeUnder the development trend of intelligent manufacturing, the unstructured environment requires the robot to have a good generalization performance to adapt to the scene changes. The purpose of this paper aims to present a learning from demonstration (LfD) method (task parameterized [TP]-dynamic movement primitives [DMP]-GMR) that combines DMPs and TP-LfD to improve generalization performance and solve object manipulation tasks.Design/methodology/approachThe dynamic time warping algorithm is applied to processing demonstration data to obtain a more standard learning model in the proposed method. The DMPs are used to model the basic trajectory learning model. The Gaussian mixture model is introduced to learn the force term of DMPs and solve the problem of learning from multiple demonstration trajectories. The robot can learn more local geometric features and generalize the learned model to unknown situations by adding task parameters.FindingsAn evaluation criterion based on curve similarity calculated by the Frechet distance was constructed to evaluate the model’s interpolation and extrapolation performance. The model’s generalization performance was assessed on 2D virtual data sets, and first, the results show that the proposed method has better interpolation and extrapolation performance than other methods.Originality/valueThe proposed model was applied to the axle-hole assembly task on real robots, and the robot’s posture in grasping and placing the axle part was taken as the task parameter of the model. The experiment results show that The proposed model is competitive with other models.

A Semantic Learning-Based SQL Injection Attack Detection Technology

Over the years, injection vulnerabilities have been at the top of the Open Web Application Security Project Top 10 and are one of the most damaging and widely exploited types of vulnerabilities against web applications. Structured Query Language (SQL) injection attack detection remains a challenging problem due to the heterogeneity of attack loads, the diversity of attack methods, and the variety of attack patterns. It has been demonstrated that no single model can guarantee adequate security to protect web applications, and it is crucial to develop an efficient and accurate model for SQL injection attack detection. In this paper, we propose synBERT, a semantic learning-based detection model that explicitly embeds the sentence-level semantic information from SQL statements into an embedding vector. The model learns representations that can be mapped to SQL syntax tree structures, as evidenced by visualization work. We gathered a wide range of datasets to assess the classification performance of the synBERT, and the results show that our approach outperforms previously proposed models. Even on brand-new, untrained models, accuracy can reach 90% or higher, indicating that the model has good generalization performance.

Prediction of line heating deformation for sheet based on SSA-ELM algorithm

Line heating is highly susceptible to plastic deformation leading to structural changes, which seriously affects shipbuilding efficiency. It is extremely important to obtain the precise line heating deformation law. There is much research on plate deformation prediction, but there is still no mature, accurate and fast prediction method. Based on this, it is necessary to research the line heating deformation prediction method. ELM has the advantages of fast learning speed, good generalization performance and strong function approximation ability. To address the poor stability of the regression model due to the randomly generated weights and thresholds of ELM, a combined prediction model (SSA-ELM) based on a squirrel search algorithm (SSA) optimized limit learning machine is proposed. The results show that the relative errors of transverse shrinkage and angular deformation of SSA-ELM are improved by 4.7% and 3.9%. The SSA-ELM prediction model has good regression accuracy and generalization ability. It gives more accurate prediction results in terms of line heating deformation prediction.

Vision-and-Language Navigation Based on Cross-Modal Feature Fusion in Indoor Environment

It is challenging for an agent to simultaneously decipher visual and language information and make decisions to perform corresponding actions. Recently, the vision-and-language navigation task has been proposed to allow the agent to navigate based on a language instruction and the currently visible visual point information in a 3-D indoor real environment. The key to this task is that the agent needs to understand the information of the two models of vision and language in an unknown environment to navigate effectively. In this study, we capture the alignment relationship between visual features and language features using a cross-modal feature fusion method. Attention is used to set up the cross-modal fusion module so that visual features contain language information and language features contain visual information, thereby allowing the model to learn more feature relationships and improving the success rate (SR) of agent navigation. Considering the practical significance of the navigation of the agent, we aim to shorten the trajectory length of the agent as much as possible while ensuring that the agent reaches the target position successfully. We employ a reinforcement learning algorithm based on the advantage actor critic to constrain the action selection of the agent to shorten the trajectory length. In order to further improve the performance of the model and reduce the difference between the performance of the agent in known environments and unknown environments, we propose the data augmentation method Cro-Speaker, and the three training methods Speaker data augmentation (SD), Cro-Speaker data augmentation (CSD), and Speaker and Cro-Speaker data augmentation (SCSD) based on this method. We evaluate the proposed method based on the Room-to-Room data set. The results show that the proposed method improves the SR of the agent navigation, shortens the length of the navigation trajectory, and exhibits a good generalization performance in known and unknown environments.

DRA-net: A new deep learning framwork for non-intrusive load disaggregation

The non-intrusive load decomposition method helps users understand the current situation of electricity consumption and reduce energy consumption. Traditional methods based on deep learning are difficult to identify low usage appliances, and are prone to model degradation leading to insufficient classification capacity. To solve this problem, this paper proposes a dilated residual aggregation network to achieve non-intrusive load decomposition. First, the original power data is processed by difference to enhance the data expression ability. Secondly, the residual structure and dilated convolution are combined to realize the cross layer transmission of load characteristic information, and capture more long sequence content. Then, the feature enhancement module is proposed to recalibrate the local feature mapping, so as to enhance the learning ability of its own network for subtle features. Compared to traditional network models, the null-residual aggregated convolutional network model has the advantages of strong learning capability for fine load features and good generalisation performance, improving the accuracy of load decomposition. The experimental results on several datasets show that the network model has good generalization performance and improves the recognition accuracy of low usage appliances.

Solving One-Dimensional Cutting Stock Problems with the Deep Reinforcement Learning

It is well known that the one-dimensional cutting stock problem (1DCSP) is a combinatorial optimization problem with nondeterministic polynomial (NP-hard) characteristics. Heuristic and genetic algorithms are the two main algorithms used to solve the cutting stock problem (CSP), which has problems of small scale and low-efficiency solutions. To better improve the stability and versatility of the solution, a mathematical model is established, with the optimization objective of the minimum raw material consumption and the maximum remaining material length. Meanwhile, a novel algorithm based on deep reinforcement learning (DRL) is proposed in this paper. The algorithm consists of two modules, each designed for different functions. Firstly, the pointer network with encoder and decoder structure is used as the policy network to utilize the underlying mode shared by the 1DCSP. Secondly, the model-free reinforcement learning algorithm is used to train network parameters and optimize the cutting sequence. The experimental data show that the one-dimensional cutting stock algorithm model based on deep reinforcement learning (DRL-CSP) can obtain the approximate satisfactory solution on 82 instances of 3 data sets in a very short time, and shows good generalization performance and practical application potential.

L1-Norm Robust Regularized Extreme Learning Machine with Asymmetric C-Loss for Regression

Extreme learning machines (ELMs) have recently attracted significant attention due to their fast training speeds and good prediction effect. However, ELMs ignore the inherent distribution of the original samples, and they are prone to overfitting, which fails at achieving good generalization performance. In this paper, based on expectile penalty and correntropy, an asymmetric C-loss function (called AC-loss) is proposed, which is non-convex, bounded, and relatively insensitive to noise. Further, a novel extreme learning machine called L1 norm robust regularized extreme learning machine with asymmetric C-loss (L1-ACELM) is presented to handle the overfitting problem. The proposed algorithm benefits from L1 norm and replaces the square loss function with the AC-loss function. The L1-ACELM can generate a more compact network with fewer hidden nodes and reduce the impact of noise. To evaluate the effectiveness of the proposed algorithm on noisy datasets, different levels of noise are added in numerical experiments. The results for different types of artificial and benchmark datasets demonstrate that L1-ACELM achieves better generalization performance compared to other state-of-the-art algorithms, especially when noise exists in the datasets.

Unsupervised classification based on deep adaptation network for sonar images

The research of underwater sonar image classification is beneficial to the development of marine resources. In recent years, deep neural networks have achieved great success in the fields of image classification and image recognition. However, deep neural networks require a large amount of training data. Due to the difficulty of obtaining sonar image datasets, the existing public sonar datasets are generally small, and sonar image classification can be regarded as a small sample problem. To solve this problem, we designed a deep adaptive sonar image classification network (DASCN) based on deep learning and domain adaptation. The feature extraction module in DASCN extracts multiscale features of images; the attention module learns the importance of different channel features; and the domain adaptation module reduces the difference between the source domain and the target domain. It is worth noting that our DASCN does not require a large number of training samples, and sonar images used for training do not need to be labeled. As demonstrated in comprehensive experiments, the classification accuracy of the DASCN model reached 89.4% on the sonar image dataset. Our DASCN achieves unsupervised accurate classification of small sample sonar images. In addition, our DASCN has good classification results on the Office-31 dataset and the Office Home dataset and has good generalization performance.

Machine learning based approach for modeling and forecasting of GPS–TEC during diverse solar phase periods

The ionosphere is a dispersive medium and it is the most important source of error affecting radio communications. As a result, precise improvements in ionospheric Total Electron Content (TEC) predictions and early warning alerts are critical for space-based radio communication and navigation systems services. In the present study, we examined the Global Positioning System (GPS)–derived TEC observations of Bangalore station (Geog. Lat. 13.02° N and Geog. Long. 77.57° E) during the 24th solar cycle, which extends the 11 years from 2009 to 2019. Based on Solar Radio Flux (sfu) and seasonal features, machine learning frameworks have been developed to forecast monthly/hourly GPS-TEC values. Motivated by the preliminary results, the problem of traditional approaches for predicting ionospheric GPS-TEC values and improvements using the machine learning approach is discussed. The Kernel extreme learning machine (KELM) model has characteristics of accuracy and good generalization performance compared with the traditional method of Holt-Winters (HW) and (Auto Regressive Moving Average (ARMA). The experimental results show that the statistical validations of the KELM model are superior to the other statistical models, with average Mean Absolute Error (MAE) and Root Mean Square Error values, which is of 0.94 and 1.305 TECU (KELM), 1.70 and 2.29 TECU (ARMA) and 2.91 and 3.86 TECU (HW) during different phases of the solar cycle. In addition, the proposed model is validated by comparison with global ionospheric models (International Reference Ionosphere (IRI) −2016 and Global Ionosphere Map (GIM). The results show that the proposed approach is superior to other alternatives using prediction accuracy. This paper outlines a machine learning framework for space weather predictions and recommends KELM classifier could be used for other query functions in Active Learning (AL) practice approaches and other datasets of the Global Navigation Satellite System (GNSS).

A Two-Terminal Fault Location Fusion Model of Transmission Line Based on CNN-Multi-Head-LSTM with an Attention Module

Most traditional artificial intelligence-based fault location methods are very dependent on fault signal selection and feature extraction, which is often based on prior knowledge. Further, these methods are usually very sensitive to line parameters and selected fault characteristics, so the generalization performance is poor and cannot be applied to different lines. In order to solve the above problems, this paper proposes a two-terminal fault location fusion model, which combines a convolutional neural network (CNN), an attention module (AM), and multi-head long short-term memory (multi-head-LSTM). First, the CNN is used to accomplish the self-extraction of fault data features. Second, the CBAM (convolutional block attention module) model is embedded into the convolutional neural network to selectively learn fault features autonomously. Furthermore, the LSTM is combined to learn the deep timing characteristics. Finally, a MLP output layer is used to determine the optimal weights to construct a fusion model based on the results of the two-terminal relative fault location model and then output the final location result. Simulation studies show that this method has a high location accuracy, does not require the design of complex feature extraction algorithms, and exhibits good generalization performance for lines with different parameters, which is of great importance for the development of AI-based methods of fault location.

Efficient numerical simulation on dielectric barrier discharges at atmospheric pressure integrated by deep neural network

Numerical simulation is an essential way to investigate the discharge behaviors of atmospheric low-temperature plasmas (LTPs). In this study, a deep neural network (DNN) with multiple hidden layers is constructed to surrogate the fluid model to investigate the discharge characteristics of atmospheric helium dielectric barrier discharges (DBDs) with very high computational efficiency, working as an example to show the ability and validity of DNN to explore LTPs. The DNN is trained by the well-formed training datasets obtained from a verified fluid model, and a designed loss function coupled in the DNN program is continuously optimized to achieve a better prediction performance. The predicted data show that the essential discharge characteristics of atmospheric DBDs such as the discharge current waveforms, spatial profiles of charged particles, and electric field can be yielded by the well-trained DNN program with great accuracy only in several seconds, and the predicted evolutionary discharge trends are consistent with the previous simulations and experimental observations. Additionally, the constructed DNN shows good generalization performance for multiple input attributes, which indicates a great potential promise for vastly extending the range of discharge parameters. This study provides a useful paradigm for future explorations of machine learning-based methods in the field of atmospheric LTP simulation without high-cost calculation.

Series Arc Fault Detection Using Regular Signals and Time-Series Reconstruction

This article investigates into signal regularity in ac series arc fault (SAF) detection. The regularity can solve the vital problem that SAF current characteristics change or disappear in unknown multiload circuits. A coupling method is adopted to capture high-frequency differential signals. The coupling signals show that the waveform in single-load circuits closely resembles the waveform in multiload ones. However, the coupling method confuses normal signals with arcing ones in dimmer loads. To address the issue, this article presents a time-series reconstruction method based on spectral features. First, the spectral features are analyzed between fault-like signals and fault signals. According to the spectral features and the desirable margin, the time series is self-adaptively decomposed and reconstructed. Then, the pulse-recognition algorithm is used to extract arcing features of the reconstructed signals. Finally, the detection method determined by single-load circuits is used to identify SAFs in unknown multiload ones. The results show the presented approach has good generalization performance and identification precision under arbitrary circuits.

Prediction of Incident Solar Radiation Using a Hybrid Kernel Based Extreme Learning Machine

Forecasting solar radiation for a given region is an emerging field of study. It will help to identify the places for installing large-scale photovoltaic-systems, designing energy-efficient buildings and energy estimation. The different machine learning kernel-based approaches for prediction problems uses either a local or global kernel. These models can provide either strong training capability or good generalization performance. In this paper, a new hybrid kernel is proposed using the combination of a local and global kernel. A novel algorithm Hybrid Kernel-based Extreme Learning Machine is proposed for predictive modelling of Incident Solar Radiation (ISR) time series using new hybrid kernel. The proposed algorithm uses the surface, atmospheric, cloud properties obtained from the MODIS instrument and observed ISR at time t – 1 to predict ISR at time t. This study is conducted for 41 diverse sites of Australia for the period of 2012–2015. Further, the proposed model is experimented with other time series datasets to prove its efficacy. It is shown that the proposed methodology outperforms five other benchmarking methods in terms of MAE and Willmott’s Index (WI). Therefore, the suggested approach can be used for modelling solar energy at a national scale using remotely-sensed satellite footprints.

Radiomics feature analysis and model research for predicting histopathological subtypes of non-small cell lung cancer on CT images: A multi-dataset study.

Classifying the subtypes of non-small cell lung cancer (NSCLC) is essential for clinically adopting optimal treatment strategies and improving clinical outcomes, but the histological subtypes are confirmed by invasive biopsy or post-operative examination at present. Based on multi-center data, this study aimed to analyze the importance of extracted CT radiomics features and develop the model with good generalization performance for precisely distinguishing major NSCLC subtypes: adenocarcinoma (ADC) and squamous cell carcinoma (SCC). We collected a multi-center CT dataset with 868 patients from eight international databases on the cancer imaging archive (TCIA). Among them, patients from five databases were mixed and split to training and test sets (560:140). The remaining three databases were used as independent test sets: TCGA set (n=97) and lung3 set (n=71). A total of 1409 features containing shape, intensity, and texture information were extracted from tumor volume of interest (VOI), then the ℓ2,1 -norm minimization was used for feature selection and the importance of selected features was analyzed. Next, the prediction and generalization performance of 130 radiomics models (10 common algorithms and 120 heterogeneous ensemble combinations) were compared by the average AUC value on three test sets. Finally, predictive results of the optimal model were shown. After feature selection, 401 features were obtained. Features of intensity, texture GLCM, GLRLM, and GLSZM had higher classification weight coefficients than other features (shape, texture GLDM, and NGTDM), and the filtered image features exhibited significant importance than original image features (p-value=0.0210). Moreover, five ensemble learning algorithms (Bagging, AdaBoost, RF, XGBoost, GBDT) had better generalization performance (p-value=0.00418) than other non-ensemble algorithms (MLP, LR, GNB, SVM, KNN). The Bagging-AdaBoost-SVM model had the highest AUC value (0.815±0.010) on three test sets. It obtained AUC values of 0.819, 0.823, and 0.804 on test set, TCGA set and lung3 set, respectively. Our multi-dataset study showed that intensity features, texture features (GLCM, GLRLM, and GLSZM) and filtered image features were more important for distinguishing ADCs from SCCs. The method of ensemble learning can improve the prediction and generalization performance on the complicated multi-center data. The Bagging-AdaBoost-SVM model had the strongest generalization performance, and it showed promising clinical value for non-invasively predicting the histopathological subtypes of NSCLC.