High Generalization Ability Research Articles

Privacy-Preserving Online Medical Prediagnosis Training Model Based on Soft-Margin SVM

Online medical prediagnosis systems have already shown great achievement in providing the guidance of healthcare services with lower time and cost. Achieving a high-precision medical primary diagnosis system faces many severe challenges on the privacy of individual health information, the distributed storage of medical data and the diversity of the disease. In this paper, we propose an efficient and privacy-preserving framework for obtaining a pre-clinical guide model, which allows an authorized data analysis center to train a disease classifier using a combination of medical data gathered from different entities. Our proposed scheme is based on soft-margin support vector machine (SVM) which takes Taylor polynomial of exponential-loss as penalty. Our scheme achieves the following advantages: the trained model can tolerate some abnormal samples therefore has higher generalization ability, and the training process can constraint the inefficient operations in the encrypted domain thus leads to the availability of partial homomorphic encryption system. Lately, we prove that the proposed scheme achieves the goal of medical prediagnosis system construction and data without privacy leakage to data analysis center and model parameters without exposure to data providers, as well as demonstrating its utility and efficiency using real-world medical datasets.

Application of depth learning in prediction of glutenite reservoir

Sedimentary facies of gravel reservoir has the characteristics of multiple sources and short flow. Sand body thickness changes rapidly in plane and space. Model-based multi-attribute regression seismic reservoir prediction method often results in unreasonable inter-well prediction results because of insufficient generalization ability of the model. For this reason, a deep learning reservoir prediction method based on fault blocks is proposed. Within a fault block range (regardless of wells outside the fault block), the seismic attributes extracted from the sliding time window of the isochronous sedimentary surface are used as input, and the sandstone thickness data of the target layer of the well point are used as expected output to train the depth neural network model of the multi-layer perceptor. Then, the trained model is used to predict the sandstone thickness distribution in the well distribution area. The application of actual data in A oilfield shows that the predictive coincidence rate of sandstone thickness in known wells is more than 95%, and the drilling occurrence rate of sandstone in horizontal section after completion of a posterior horizontal well is more than 80%. This method has high prediction accuracy and good generalization ability.

Wideband DOA Estimation Based on Deep Residual Learning With Lyapunov Stability Analysis

This letter puts forward a new method to estimate the direction of arrivals (DOAs) along with the source number under wideband circumstances. The method constructs a multiple-input-single-output (MISO) framework to make full use of the data from different frequency subbands. Each channel in the framework extracts angular features in parallel through stacked residual blocks. As a result, we can obtain an integrated output vector with smaller error. Because of the well-designed loss function, outputs of the trained model under the proposed framework are self-adjustable with the source number so that the regression of angle values and classification of signal numbers can be completed simultaneously. Moreover, we have proved herein that, if the equivalent matrices of residual blocks are negative definite, then the proposed framework is stable in the sense of Lyapunov. This stability is of great significance to practical applications. Numerical simulations on X-band data demonstrate high estimation accuracy and powerful generalization ability of the proposed scheme in the presence of nonuniform noise.

Ensemble Deep Learning-Based Non-Crossing Quantile Regression for Nonparametric Probabilistic Forecasting of Wind Power Generation

Probabilistic forecasting that quantifies the prediction uncertainties is crucial for decision-making in power systems. As a prevalent nonparametric probabilistic forecasting approach, traditional machine learning-based quantile regression encounters the quantile crossing problem. This paper proposes a novel ensemble deep learning based non-crossing quantile regression (EDNQR) model for probabilistic wind power forecasting, which does not need any distribution assumption and demonstrates high generalization ability. A unique non-crossing quantile regression strategy is proposed to generate monotonous quantiles with deep learning models. The exponential stacking mapping method is proposed to guarantee the monotonicity of quantiles, and Huber norm pinball loss is introduced for deep learning quantile regression model training. To improve the generalization capability, a two-stage ensemble framework is proposed to integrate homogeneous and heterogeneous deep learning models. The newly-trained base model considers the training performance of the last base model, which benefits the performance boosting of the ensemble model. To overcome the complexity of conventional weight optimization-based model integration, an overall quantile loss index is proposed to serve as an indicator to directly integrate both the homogeneous and heterogeneous deep learning base models. Comprehensive numerical studies based on actual wind farm data validate the superior performance of the proposed EDNQR model in terms of reliability, overall skill and generalization ability.

Genetic yield control in sugar beet hybrids obtained by diallel crosses

Purpose. The aim of our study was to determine the combination ability of the studied pollinators and identify the genetic control of the yield sign in interline diallel hybrids of sugar beet.
 Methods. The source material was homozygous as a result self-pollination. The combination ability and genetic control of quantitative traits were determined by the Hayman model.
 Results. Genetic analysis revealed that sugar beet pollinator line demonstrated genetic control of yield in diallel sugar beet hybrids by 14 genes and gene groups. Lines (BZ 1 and BZ 4) with high general combination ability demonstrated a significant additive effect of genes were selected. The effects of specific combination ability, which significantly affected the yield of heterosis hybrids, were revealed. Their share of influence was 36.4 and 23.8%, respectively. High-yielding hybrid combinations of parent genotypes were selected. They are transferred to reproduction and testing for ecological plasticity.
 Conclusions. Genetic control of the yield sign in diallel hybrids is found based on the Hаyman model. The influence of the combination ability of sugar beet pollinators was determined and the best parent genotype pairs were selected. According to the effects of specific gene interaction, the best combinations have been identified that can be used as sources of economically valuable traits.

Predicting army cadets’ performance: The role of character strengths, GPA and GMA

Abstract The purpose of this study was to examine how well a set of 12 character strengths (Leadership, Integrity, Open-Mindedness, Bravery, Teamwork, Persistence, Social Intelligence, Love of Learning, Fairness, Self-Regulation, Perspective and Creativity) will predict academic performance (AP) and military performance (MP), compared to high school grade point average (GPA) and general mental ability (GMA). The study sample comprised 123 army cadets of two cohorts from the three-year bachelor's degree programme at the Norwegian Military Academy (NMA). GPA predicted AP (r = 0.32, p ≤ 0.05), but not MP (r = 0.14, n.s.), while GMA correlated significantly with neither AP nor MP. All 12 character strengths correlated significantly with MP (rs ranging from 0.27 to 0.65), and all except for Fairness correlated significantly with AP (rs ranging from 0.18 to 0.58). An average score of the 12 character strengths showed incremental validity beyond GMA and GPA in predicting both AP and MP. Our results suggest that character strengths should be considered when selecting and training army cadets.

An optimized Parkinson's disorder identification through evolutionary fast learning network

PurposeParkinson's disease (PD) is a well-known complex neurodegenerative disease. Typically, its identification is based on motor disorders, while the computer estimation of its main symptoms with computational machine learning (ML) has a high exposure which is supported by researches conducted. Nevertheless, ML approaches required first to refine their parameters and then to work with the best model generated. This process often requires an expert user to oversee the performance of the algorithm. Therefore, an attention is required towards new approaches for better forecasting accuracy.Design/methodology/approachTo provide an available identification model for Parkinson disease as an auxiliary function for clinicians, the authors suggest a new evolutionary classification model. The core of the prediction model is a fast learning network (FLN) optimized by a genetic algorithm (GA). To get a better subset of features and parameters, a new coding architecture is introduced to improve GA for obtaining an optimal FLN model.FindingsThe proposed model is intensively evaluated through a series of experiments based on Speech and HandPD benchmark datasets. The very popular wrappers induction models such as support vector machine (SVM), K-nearest neighbors (KNN) have been tested in the same condition. The results support that the proposed model can achieve the best performances in terms of accuracy and g-mean.Originality/valueA novel efficient PD detection model is proposed, which is called A-W-FLN. The A-W-FLN utilizes FLN as the base classifier; in order to take its higher generalization ability, and identification capability is also embedded to discover the most suitable feature model in the detection process. Moreover, the proposed method automatically optimizes the FLN's architecture to a smaller number of hidden nodes and solid connecting weights. This helps the network to train on complex PD datasets with non-linear features and yields superior result.

The Storage Period Discrimination of Bolete Mushrooms Based on Deep Learning Methods Combined With Two-Dimensional Correlation Spectroscopy and Integrative Two-Dimensional Correlation Spectroscopy

Boletes are favored by consumers because of their delicious taste and high nutritional value. However, as the storage period increases, their fruiting bodies will grow microorganisms and produce substances harmful to the human body. Therefore, we need to identify the storage period of boletes to ensure their quality. In this article, two-dimensional correlation spectroscopy (2DCOS) images are directly used for deep learning modeling, and the complex spectral data analysis process is transformed into a simple digital image processing problem. We collected 2,018 samples of boletes. After laboratory cleaning, drying, grinding, and tablet compression, their Fourier transform mid-infrared (FT-MIR) spectroscopy data were obtained. Then, we acquired 18,162 spectral images belonging to nine datasets which are synchronous 2DCOS, asynchronous 2DCOS, and integrative 2DCOS (i2DCOS) spectra of 1,750–400, 1,450–1,000, and 1,150–1,000 cm–1 bands. For these data sets, we established nine deep residual convolutional neural network (ResNet) models to identify the storage period of boletes. The result shows that the accuracy with the train set, test set, and external validation set of the synchronous 2DCOS model on the 1,750–400-cm–1 band is 100%, and the loss value is close to zero, so this model is the best. The synchronous 2DCOS model on the 1,150–1,000-cm–1 band comes next, and these two models have high accuracy and generalization ability which can be used to identify the storage period of boletes. The results have certain practical application value and provide a scientific basis for the quality control and market management of bolete mushrooms. In conclusion, our method is novel and extends the application of deep learning in the food field. At the same time, it can be applied to other fields such as agriculture and herbal medicine.

QL-IQA: Learning distance distribution from quality levels for blind image quality assessment

Recently, blind image quality assessment (BIQA) has been intensively studied with deep learning. However, the limited quality-annotated datasets restrict its further development. Although patch-based methods have been leveraged to generate more training data, they usually assign the image quality score to all patches in an image. Consequently, much noise would be introduced. To avoid this, we propose a method learning distance distribution from quality levels to expand the training data, which is pseudo Siamese network-based no-reference image quality assessment (NR-IQA). Specifically, we firstly use the K-means clustering algorithm to classify the quality scores of distorted images into five levels. Subsequently, the pseudo Siamese network is adopted to learn the distance distribution from these quality levels. We mainly decrease distances between the quality scores of images from the same quality level and increase distances between the quality scores of images from different quality levels, in which the distorted images can be compared across different distortion types. Finally, we introduce a fusion layer to average the quality scores learnt by the two branches of the trained pseudo Siamese network, and take fine-tuning on this basis to learn image quality score from one image. To demonstrate the effectiveness of the proposed approach, we evaluate it on benchmark databases (LIVE, TID2013 and LIVE MD). The result shows superior performance to some widely used NR-IQA methods and full-reference IQA (FR-IQA) methods. Cross-database evaluation verifies high generalization ability and high effectiveness of our model.

Minimum Distribution Support Vector Clustering.

Support vector clustering (SVC) is a boundary-based algorithm, which has several advantages over other clustering methods, including identifying clusters of arbitrary shapes and numbers. Leveraged by the high generalization ability of the large margin distribution machine (LDM) and the optimal margin distribution clustering (ODMC), we propose a new clustering method: minimum distribution for support vector clustering (MDSVC), for improving the robustness of boundary point recognition, which characterizes the optimal hypersphere by the first-order and second-order statistics and tries to minimize the mean and variance simultaneously. In addition, we further prove, theoretically, that our algorithm can obtain better generalization performance. Some instructive insights for adjusting the number of support vector points are gained. For the optimization problem of MDSVC, we propose a double coordinate descent algorithm for small and medium samples. The experimental results on both artificial and real datasets indicate that our MDSVC has a significant improvement in generalization performance compared to SVC.

An improved stacking ensemble learning-based sensor fault detection method for building energy systems using fault-discrimination information

Sensor fault detection is essential to maintain operations of heating, ventilation, and air conditioning systems (HVACs) in buildings. Data-driven sensor fault detection methods are becoming increasingly popular recently. However, for the relatively limited modelling data available in the heating, ventilation, and air conditioning system, it is a major challenge to extract fault-discrimination information (FDI) from limited data to develop a data-driven model with higher fault detection performance and lower false-alarm rate. Therefore, this study proposed an improved Stacking (IStacking) sensor fault detection method using fault-discrimination information. Fault-discrimination-information is extracted from original samples by subtracting the statistics with the threshold of each single model. Four different single models (i.e., principal component analysis, one-class support vector machine, K-Means clustering, and autoencoder) are employed to develop the ensemble learning detection method with relatively high generalization ability using a Stacking training manner. Sensor bias faults from two different sources are used for validation. Area under the ROC (receiver operating characteristic) curve (AUC) and false positive rate (FPR) are used to evaluate the detection performance and false-alarm rate. Results indicated that IStacking outperforms the four single models and traditional Stacking when the added biases are ranged [-4 °C, −2 °C] and [3.7 °C, 4 °C]. Since single models are stacked and fault-discrimination information is used to replace the original input, IStacking averagely increases Area under the ROC curve by 2.85% and decreases false positive rate by 5.76%.

Feature-Based Multi-Class Classification and Novelty Detection for Fault Diagnosis of Industrial Machinery

Given the strategic role that maintenance assumes in achieving profitability and competitiveness, many industries are dedicating many efforts and resources to improve their maintenance approaches. The concept of the Smart Factory and the possibility of highly connected plants enable the collection of massive data that allow equipment to be monitored continuously and real-time feedback on their health status. The main issue met by industries is the lack of data corresponding to faulty conditions, due to environmental and safety issues that failed machinery might cause, besides the production loss and product quality issues. In this paper, a complete and easy-to-implement procedure for streaming fault diagnosis and novelty detection, using different Machine Learning techniques, is applied to an industrial machinery sub-system. The paper aims to offer useful guidelines to practitioners to choose the best solution for their systems, including a model hyperparameter optimization technique that supports the choice of the best model. Results indicate that the methodology is easy, fast, and accurate. Few training data guarantee a high accuracy and a high generalization ability of the classification models, while the integration of a classifier and an anomaly detector reduces the number of false alarms and the computational time.

Probabilistic analysis of soil-water characteristic curve based on machine learning algorithms

The soil-water characteristic curve (SWCC) is the constitutive relationship curve of soil suction and water content, which has important engineering significance for studying the soil strength, permeability coefficient and volume change of unsaturated soils. Considering the experimental measurement of the SWCC is time-consuming, many empirical methods have been suggested to estimate the SWCC. This paper proposed a machine learning algorithm to predict SWCC from limited sets of soil properties. By predicting the parameters of the Fredlund and Xing model (which is called FX model), the most probable SWCC can be obtained. Since SWCC can’t be accurately determined, the residual probability distribution functions of parameters of FX model are also derived in this paper. Finally, this paper predicted the 90% confidence interval of the SWCC. The result shows that the machine learning algorithm has high accuracy and generalization ability. The suggested method provides a practical means to estimate the SWCC and the variability of it, so that the error associated with the prediction model can be explicitly considered.

Real-time evaluation of compaction quality based on RF- ACGWO with high robustness and generalization ability

The effective evaluation of compaction quality is a key issue for the safety of earth-rock dams. However, existing prediction models of compaction quality are designed to improve prediction accuracy but generally ignore generalizability and robustness, resulting in deviations from practical evaluation results, making these models inapplicable to complex construction environments. To address these problems, a novel real-time evaluation model for construction unit compaction quality based on random forest optimized by adaptive chaos grey wolf algorithm (RF-ACGWO) is proposed in this article. In RF-ACGWO, RF predicts compaction quality, while ACGWO increases efficiency and accuracy for traditional RF parameter selection and improves the generalizability and robustness of the model. Also, meteorological factors at a project site are also considered to affect the model, thereby improving model accuracy. After embedding the proposed method in a Three-Dimensions (3D) rolling monitoring system, real-time evaluation, guidance and feedback on a project site can be obtained. Compared to the conventional evaluation methods, RF-ACGWO achieves the highest accuracy of 0.838, the best generalizability of 0.793 and the most stable robustness when applied to a large-scale, real-life hydraulic engineering project.

A simple and efficient text matching model based on deep interaction

In recent years, text matching has gained increasing research focus and shown great improvements. However, due to the long-distance dependency and polysemy, existing text matching models cannot effectively capture the contextual and implicit semantic information of texts. Additionally, existing models are lack of generalization ability when applied to different scenarios. In this study, we propose a novel Deep Interactive Text Matching (DITM) model by integrating the encoder layer, the co-attention layer, and the fusion layer as an interaction module, based on a matching-aggregation framework. In particular, the interaction process is iterated multiple times to obtain the in-depth interaction information, and the relationship between the text pair is extracted through the multi-perspective pooling. We conduct extensive experiments on four text matching tasks, i.e., opinion retrieval, answer selection, paraphrase identification and natural language inference. Compared with the state-of-the-art text matching methods, the proposed model achieved the best results on most of the tasks, which proves that our model could effectively capture the interactive information between text pairs, and has a high generalization ability among different tasks. Further multi-lingual investigations show the similarities of the performance between English and Chinese, which suggest that our model could be ported to other languages. The research contributes a simple and efficient implementation of text matching in a situation where there is limited computing capacity, and sheds light on leveraging text matching models to facilitate a range of downstream tasks.

Analysis on route information failure in IP core networks by NFV-based test environment

Stable and high-quality Internet connectivity is mandatory for 5G mobile networks. However, the pandemic of COVID-19 has forced global and large-scale staying at home and telecommuting in many countries. The increasing traffic has induced more pressure on networks, devices and cloud data centers. It becomes an essential task for network operators to enable their ability to automatically and rapidly detect network and device failures. We propose a highly practical method based on highly practical technology. Our method has a high generalization ability that can efficiently extract features from large-scale unstructured data and ensure high accuracy prediction. First, 997 useful features are extracted from 28GB-per-day network logs. Then, a differential approach is employed to preprocess the extracted features so as to highlight the differences between normal and abnormal states. Third, those features are refined based on the feature importance we calculated. According to our experiment, the proposed feature extraction and refinement method can reduce computation without degrading the performance. Among the five types of failures, we achieve a 100% recall rate in four types and the rest can also reach 71%. Overall, the total average prediction accuracy of the proposed method is 94%.

Minimum training sample size requirements for achieving high prediction accuracy with the BN model: A case study regarding seismic liquefaction

The complexity of a Bayesian network (BN) model and the number of training samples used have significant impacts on the prediction accuracy of the model regarding seismic liquefaction. The required training sample size for ensuring that a BN model has high generalization ability is a critical issue in parameter learning. To address this issue, this study analyses the relationship between the predictive performance of the BN model and the complexity of the model, training sample size, and average discrete intervals. Taking seismic liquefaction prediction as an example, 4536 statistical experiments are designed to investigate the training and testing performances of 21 different BN models under the conditions of nine different training sample size ratios (5%, 10%, 20%, 30%, 40%, 50%, 60%, 70%, and 80% of all data) and testing samples using 20% of all data. The results reveal that the learning performance of a BN is not sensitive to the training sample size but is related to the complexity of the model. The larger the sample size is, the stronger the generalization ability of the model. The minimum training sample requirements are related to the maximum in-degrees and the average discrete intervals, not the numbers of nodes and edges and the maximum out-degree of the BN structure. In addition, a modified structural entropy can characterize the complexity of a BN structure better than the existing structural entropy, but it has a worse relationship with the minimum training sample requirements than that of the maximum in-degree. To quickly determine the minimum training sample size requirements of a BN model with a predictive accuracy of 80%, a fitting function that considers the effects of the maximum in-degree and the average discrete intervals is presented, and its effectiveness is validated by two examples.

Adaptivity and stability parameters of heterotic tomato hybrids (Solanum lycopersicum L.) in soil greenhouses

Varieties and hybrids which are stable in yield and suitable for cultivation in various soil and climatic zones have a particular value within the conditions of industrial production. New varieties created by breeders should be characterized by the highest yield in favorable conditions, and, at the same time, by the stable yield in other conditions, i.e. be highly accommodative. The aim of the research was to assess environmental stability and plasticity parameters of tomato hybrids according to the main characteristics of yield and then to select the hybrids F1 with high productivity and stability in soil greenhouses. The calculation of adaptive ability and ecological stability indexes was carried out according to the method of A.V. Kilchevsky and L.V. Khotyleva using the ADIS program. As a result, some hybrids were identified as heavy yielding and with high general adaptive ability according to the following parameters: early fruit yield - NikolaçLine‑2, Line‑4×Line‑2, NikolaçLine 19/3, NikolaçLine-9, Line TX-144×Irishka, Line B-3-1-8×Line 19/0, Line C-9464×Line 19/0; on the basis of “commercial fruit yield” - NikolaçLine-2, TX-144×Line-2, Line TX-144×Irishka, TX140×Line-2, Line TX-140×Line 19/3, Line B-3-1-8×Line 19/0, Line C-9464×Line 19/0; by the weight of commercial fruit - NikolaçLine 19/3, Line TX-144çLine 19/3, Line B-3-1-8×Line 19/0. Greater part of yield parameters was stable for hybrids NikolaçLine-2, Line-4×Line-2, TX-140×Line-2, Line B-3-1-8×Line 19/0, Line TX-144×Irishka. Hybrids combinations Azart and Vityaz have successfully passed the test and were recommended for planting on the territory of the Republic of Belarus. Thus, feasibility of ecological stability and plasticity assessment at the final stage of breeding process has been proven. It reduces possibility of errors in choosing hybrids for State testing procedures. Acknowledgments. The research was carried out within the framework of the State Program “Innovative Biotechnologies” for 2010-2012 and for the period up to 2015”, as well as the Interstate Target Program of the Eurasian Economic Community “Innovative Biotechnologies” for 2011-2015.

A Machine Learning Model for Photorespiration Response to Multi-Factors

Photorespiration results in a large amount of leaf photosynthesis consumption. However, there are few studies on the response of photorespiration to multi-factors. In this study, a machine learning model for the photorespiration rate of cucumber leaves’ response to multi-factors was established. It provides a theoretical basis for studies related to photorespiration. Machine learning models of different methods were designed and compared. The photorespiration rate was expressed as the difference between the photosynthetic rate at 2% O2 and 21% O2 concentrations. The results show that the XGBoost models had the best fit performance with an explained variance score of 0.970 for both photosynthetic rate datasets measured using air and 2% O2, with mean absolute errors of 0.327 and 0.181, root mean square errors of 1.607 and 1.469, respectively, and coefficients of determination of 0.970 for both. In addition, this study indicates the importance of the features of temperature, humidity and the physiological status of the leaves for predicted results of photorespiration. The model established in this study performed well, with high accuracy and generalization ability. As a preferable exploration of the research on photorespiration rate simulation, it has theoretical significance and application prospects.

Prediction of three-phase product distribution and bio-oil heating value of biomass fast pyrolysis based on machine learning

In this work, by mining the experimental data of fast pyrolysis of lignocellulosic biomass in bubbling fluidized bed in previous literature, regression prediction models were established for three-phase product distribution and bio-oil heating value (HHV) based on gradient boosting, random forest, support vector machine, and multilayer perceptron algorithms. Comprehensive feedstock characteristics and pyrolysis conditions were considered and compared as input features. Among the several algorithms, random forest is most suitable for the prediction of three-phase product yields and bio-oil HHV with the benefits of high accuracy and good generalization ability. Visual analysis of the model shows that pyrolysis temperature is the most critical factor affecting three-phase product distribution, while bio-oil HHV is more affected by the feedstock characteristics such as the contents of C and H. The highest yield and HHV of bio-oil is obtained at about 480 °C, suggesting 480 °C as the optimum pyrolysis temperature of fast pyrolysis of biomass in a bubbling fluidized bed. As for the feedstock characteristics, high contents of C and H and low content of O are favorable to the enhancement of bio-oil HHV, indicating the crucial importance of feedstock pretreatment such as torrefaction to the quality improvement of bio-oil.