General Machine Learning Methods Research Articles

Estimating the composition ratios of network services carried in mixed traffic

Network performance management (NPM) and quality of service (QoS) guarantee are the key concerns of network operators and service providers/consumers, respectively. The basis for traffic scheduling is one of the vital elements guiding the implementation of both NPM and QoS assurance. Although traffic identification technologies can theoretically be applied to providing basis for traffic scheduling, the computational complexity caused by the flow-by-flow processing makes them unsuitable for actual large-scale high-throughput network scenarios. In this work, we introduce a new scheme to provide a basis for service-oriented NPM and QoS guarantee. The proposed scheme treats the mixed traffic as a whole and estimates the composition ratios of network services carried in mixed traffic, which makes traffic scheduling for NPM and QoS more reasonable and flexible. To estimate the composition ratios, the observed mixed traffic is expressed as a bipartite graph according to the communication relationship. Then, the bipartite graph is further projected to a multi-dimensional hyper-image based on node coding, random walking and embedding mapping. By this way, we convert the service composition ratio estimation into a problem similar to image classification and enable it to be solved by general machine learning methods like convolutional neural networks. The proposed scheme facilitates batch traffic scheduling and can reorganize the distribution of compositions of network services for each link based on given management strategies. Its computational complexity is significantly lower than that of flow-by-flow processing methods. We validate the proposed solution via real network data and evaluate its performance with some benchmark methods based on traffic identification. The experimental results show that the proposed solution can accurately infer the service composition ratios of mixed traffic with low computational complexity.

Discovery of sparse, reliable omic biomarkers with Stabl.

Adoption of high-content omic technologies in clinical studies, coupled with computational methods, has yielded an abundance of candidate biomarkers. However, translating such findings into bona fide clinical biomarkers remains challenging. To facilitate this process, we introduce Stabl, a general machine learning method that identifies a sparse, reliable set of biomarkers by integrating noise injection and a data-driven signal-to-noise threshold into multivariable predictive modeling. Evaluation of Stabl on synthetic datasets and five independent clinical studies demonstrates improved biomarker sparsity and reliability compared to commonly used sparsity-promoting regularization methods while maintaining predictive performance; it distills datasets containing 1,400-35,000 features down to 4-34 candidate biomarkers. Stabl extends to multi-omic integration tasks, enabling biological interpretation of complex predictive models, as it hones in on a shortlist of proteomic, metabolomic and cytometric events predicting labor onset, microbial biomarkers of pre-term birth and a pre-operative immune signature of post-surgical infections. Stabl is available at https://github.com/gregbellan/Stabl .

A machine learning surrogate modeling benchmark for temperature field reconstruction of heat source systems

Temperature field reconstruction of heat source systems (TFR-HSS) with limited monitoring sensors occurred in thermal management plays an important role in real time health detection system of electronic equipment in engineering. However, prior methods with common interpolations usually cannot provide accurate reconstruction performance as required. In addition, there exists no public dataset for widely research of reconstruction methods to further boost the reconstruction performance and engineering applications. To overcome this problem, this work develops a machine learning modelling benchmark for TFR-HSS task. First, the TFR-HSS task is mathematically modelled from real-world engineering problem and four types of numerically modellings have been constructed to transform the problem into discrete mapping forms. Then, this work proposes a set of machine learning modelling methods, including the general machine learning methods and the deep learning methods, to advance the state-of-the-art methods over temperature field reconstruction. More importantly, this work develops a novel benchmark dataset, namely Temperature Field Reconstruction Dataset (TFRD), to evaluate these machine learning modelling methods for the TFR-HSS task. Finally, a performance analysis of typical methods is given on TFRD, which can be served as the baseline results on this benchmark.

Multiobjective evolutionary discovery of equation-based analytical models for dynamical systems

In this article, an approach to modeling dynamical systems in case of unknown governing physical laws has been introduced. The systems of differential equations obtained by means of a data-driven algorithm are taken as the desired models. In this case, the problem of predicting the state of the process is solved by integrating the resulting differential equations. In contrast to classical data-driven approaches to dynamical systems representation, based on the general machine learning methods, the proposed approach is based on the principles, comparable to the analytical equation-based modeling. Models in forms of systems of differential equations, composed as combinations of elementary functions and operation with the structure, were determined by adapted multi-objective evolutionary optimization algorithm. Time-series describing the state of each element of the dynamic system are used as input data for the algorithm. To ensure the correct operation of the algorithm on data characterizing real-world processes, noise reduction mechanisms are introduced in the algorithm. The use of multicriteria optimization, held in the space of complexity and quality criteria for individual equations of the differential equation system, makes it possible to improve the diversity of proposed candidate solutions and, therefore, to improve the convergence of the algorithm to a model that best represents the dynamics of the process. The output of the algorithm is a set of Pareto-optimal solutions of the optimization problem where each individual of the set corresponds to one system of differential equations. In the course of the work, a library of data-driven modeling of dynamic systems based on differential equation systems was created. The behavior of the algorithm was studied on a synthetic validation dataset describing the state of the hunter-prey dynamic system given by the Lotka-Volterra equations. Finally, a toolset based on the solution of the generated equations was integrated into the algorithm for predicting future system states. The method is applicable to data-driven modeling of arbitrary dynamical systems (e.g. hydrometeorological systems) in cases where the processes can be described using differential equations. Models generated by the algorithm can be used as components of more complex composite models, or in an ensemble of methods as an interpretable component.

An interpretable analytic framework of the relationship between carsharing station development patterns and built environment for sustainable urban transportation

The in-depth understanding of the relationship between development patterns of carsharing stations and built environment are important to the comprehensive station evaluation, layout optimization and urban spatial resources planning. However, the previous researches mainly study the operation of carsharing by aggregate methods with cross-sectional data and rarely discovered patterns within carsharing operation time series. Therefore, an interpretable analytic framework is proposed for predicting development patterns of carsharing stations, which is composed of a development pattern construction method based on Time Series Clustering and an interpretable prediction method based on CatBoost and SHAP models. The temporal variations of time series data are sufficiently utilized by time series clustering to identify patterns and CatBoost-SHAP has better classification performance and interpretability than general machine learning methods. The proposed framework is applied to explore the relationship between the development pattern of one-way carsharing stations and the built environment influencing factors. The result shows that the carsharing stations of Nanjing EVCARD are divided into two types: increasing pattern and decreasing pattern. The built environment factors that have the greatest impact on model output and the impact of pairwise factors are visually analysed. Moreover, this is also effective for a specific individual station to analyze the causes of its status quo. Therefore, this study provides data-driven intuitive decision references for carsharing operators, which helps the operators effectively manage carsharing stations.

An empirical evaluation of sampling methods for the classification of imbalanced data.

In numerous classification problems, class distribution is not balanced. For example, positive examples are rare in the fields of disease diagnosis and credit card fraud detection. General machine learning methods are known to be suboptimal for such imbalanced classification. One popular solution is to balance training data by oversampling the underrepresented (or undersampling the overrepresented) classes before applying machine learning algorithms. However, despite its popularity, the effectiveness of sampling has not been rigorously and comprehensively evaluated. This study assessed combinations of seven sampling methods and eight machine learning classifiers (56 varieties in total) using 31 datasets with varying degrees of imbalance. We used the areas under the precision-recall curve (AUPRC) and receiver operating characteristics curve (AUROC) as the performance measures. The AUPRC is known to be more informative for imbalanced classification than the AUROC. We observed that sampling significantly changed the performance of the classifier (paired t-tests P < 0.05) only for few cases (12.2% in AUPRC and 10.0% in AUROC). Surprisingly, sampling was more likely to reduce rather than improve the classification performance. Moreover, the adverse effects of sampling were more pronounced in AUPRC than in AUROC. Among the sampling methods, undersampling performed worse than others. Also, sampling was more effective for improving linear classifiers. Most importantly, we did not need sampling to obtain the optimal classifier for most of the 31 datasets. In addition, we found two interesting examples in which sampling significantly reduced AUPRC while significantly improving AUROC (paired t-tests P < 0.05). In conclusion, the applicability of sampling is limited because it could be ineffective or even harmful. Furthermore, the choice of the performance measure is crucial for decision making. Our results provide valuable insights into the effect and characteristics of sampling for imbalanced classification.

Distance-based Support Vector Machine to Predict DNA N6- methyladenine Modification

Background: DNA N6-methyladenine plays an important role in the restriction-modification system to isolate invasion from adventive DNA. The shortcomings of the high time consumption and high costs of experimental methods have been exposed, and some computational methods have emerged. The support vector machine theory has received extensive attention in the bioinformatics field due to its solid theoretical foundation and many good characteristics. Objective: General machine learning methods include an important step of extracting features. The research has omitted this step and replaced with easy-to-obtain sequence distances matrix to obtain better results. Method: First sequence alignment technology was used to achieve the similarity matrix. Then, a novel transformation turned the similarity matrix into a distance matrix. Next, the similarity-distance matrix was made positive semi-definite so that it can be used in the kernel matrix. Finally, the LIBSVM software was applied to solve the support vector machine. Results: The five-fold cross-validation of this model on rice and mouse data has achieved excellent accuracy rates of 92.04% and 96.51%, respectively. This shows that the DB-SVM method has obvious advantages over traditional machine learning methods. Meanwhile, this model achieved 0.943,0.982 and 0.818 accuracy; 0.944, 0.982, and 0.838 Matthews correlation coefficient; and 0.942, 0.982 and 0.840 F1 scores for the rice, M. musculus and cross-species genome datasets, respectively. Conclusion: These outcomes show that this model outperforms the iIM-CNN and csDMA in the prediction of DNA 6mA modification, which is the latest research finding on DNA 6mA.

Chemistry-informed macromolecule graph representation for similarity computation, unsupervised and supervised learning

The near-infinite chemical diversity of natural and artificial macromolecules arises from the vast range of possible component monomers, linkages, and polymers topologies. This enormous variety contributes to the ubiquity and indispensability of macromolecules but hinders the development of general machine learning methods with macromolecules as input. To address this, we developed a chemistry-informed graph representation of macromolecules that enables quantifying structural similarity, and interpretable supervised learning for macromolecules. Our work enables quantitative chemistry-informed decision-making and iterative design in the macromolecular chemical space.

A Machine Learning Method for Material Property Prediction: Example Polymer Compatibility

Prediction of material property is a key problem because of its significance to material design and screening. We present a brand-new and general machine learning method for material property prediction. As a representative example, polymer compatibility is chosen to demonstrate the effectiveness of our method. Specifically, we mine data from related literature to build a specific database and give a prediction based on the basic molecular structures of blending polymers and, as auxiliary, the blending composition. Our model obtains at least 75% accuracy on the dataset consisting of thousands of entries. We demonstrate that the relationship between structure and properties can be learned and simulated by machine learning method.

Transfer State of Health Estimation Based on Cross-Manifold Embedding

The data-driven approach is currently a research hotspot of battery state of health (SOH) estimation. Such methods have advantages in nonlinear fitting; no human intervention is required in their implementation process. In existing research, general data-driven models are developed for specific battery objects. However, different battery objects need to be dealt with in practical applications, and the batteries may have different characteristics. To make SOH estimates for various battery types, electric vehicle maintainers usually require a model to have portability. However, the general machine learning methods are based on the data consistency assumption. The differences in the battery characteristics make the model migration difficult. To address this issue, we propose a novel cross-manifold transfer learning method. This method obtains a small amount of data from the target battery, and at the same time, brings relevant information from related tasks through cross manifold embedding, thereby achieving small sample SOH estimation. Experimental results show that traditional machine learning methods may suffer serious over-fitting problems when the training and target objects are very different. With the cross manifold embedding method, the knowledge learned by data-driven models can be well generalized to unseen battery objects. In this way, a data-driven model can perform a practical SOH estimation with a small amount of target data.

Random Sampling High Dimensional Model Representation Gaussian Process Regression (RS-HDMR-GPR) for representing multidimensional functions with machine-learned lower-dimensional terms allowing insight with a general method

We present an implementation for the RS-HDMR-GPR (Random Sampling High Dimensional Model Representation Gaussian Process Regression) method. The method builds representations of multivariate functions with lower-dimensional terms, either as an expansion over orders of coupling or using terms of only a given dimensionality. This facilitates, in particular, recovering functional dependence from sparse data. The method also allows for imputation of missing values of the variables and for a significant pruning of the useful number of HDMR terms. It can also be used for estimating relative importance of different combinations of input variables, thereby adding an element of insight to a general machine learning method, in a way that can be viewed as extending the automatic relevance determination approach. The capabilities of the method and of the associated Python software tool are demonstrated on test cases involving synthetic analytic functions, the potential energy surface of the water molecule, kinetic energy densities of materials (crystalline magnesium, aluminum, and silicon), and financial market data.

Balanced Distribution Adaptation for Metal Oxide Semiconductor Gas Sensor Array Drift Compensation.

Drift compensation is an important issue for metal oxide semiconductor (MOS) gas sensor arrays. General machine learning methods require constant calibration and a large amount of label gas data. At the same time, recalibration will cause a lot of costs, and label gas is difficult to obtain in practice. In this paper, a novel drift compensation method based on balanced distribution adaptation (BDA) is proposed. First, the BDA drift compensation method can adjust the conditional distribution and marginal distribution between the two domains through the weight balance factor, thereby more effectively reducing the mismatch between the two domains. When the BDA method performs classification tasks through machine learning, no labeled data is required in the target domain. Then, the particle swarm optimization algorithm is used to improve the accuracy of drift compensation. Individuals in the population are initialized randomly, and their fitness values are calculated. Iterative optimization of the population individuals is conducted until the optimal weight balance factor parameters are calculated. Finally, the BDA method is experimentally verified on the public gas sensor drift data set. Experimental results showed that the BDA method was significantly better than the existing joint distribution adaptation (JDA) method and other standard drift compensation methods such as K-Nearest Neighbor (KNN). In the two setting groups, the recognition accuracy was 4.54% and 1.62% ahead of the JDA method, and 12.23% and 15.83% ahead of the KNN method.

Learning Data Correction for Myoelectric Hand Based on "Survival of the Fittest".

In recent years, myoelectric hands have become multi-degree-of-freedom (DOF) devices, which are controlled via machine learning methods. However, currently, learning data for myoelectric hands are gathered manually and thus tend to be of low quality. Moreover, in the case of infants, gathering accurate learning data is nearly impossible because of the difficulty of communicating with them. Therefore, a method that automatically corrects errors in the learning data is necessary. Myoelectric hands are wearable robots and thus have volumetric and weight constraints that make it infeasible to store large amounts of data or apply complex processing methods. Compared with general machine learning methods such as image processing, those for myoelectric hands have limitations on the data storage, although the amount of data to be processed is quite large. If we can use this huge amount of processing data to correct the learning data without storing the processing data, the machine learning performance is expected to improve. We then propose a method for correcting the learning data through utilisation of the signals acquired during the use of the myoelectric hand. The proposed method is inspired by “survival of the fittest.” The effectiveness of the method was verified through offline analysis. The method reduced the amount of learning data and learning time by approximately a factor of 10 while maintaining classification rates. The classification rates improved for one participant but slightly deteriorated on average among all participants. To solve this problem, verifying the method via interactive learning will be necessary in the future.

Chaotic neural network model for SMISs reliability prediction based on interdependent network SMISs reliability prediction by chaotic neural network

AbstractWith the development of industrial Internet, smart manufacturing information systems (SMISs) are faced with more uncertainties, dynamics, and complexity. These problems bring more challenges to the reliability operation of SMISs. To solve the above problem, a prediction model based on phase space reconstruction, chaos analysis, and back propagation (BP) neural network is proposed to predict SMISs reliability. First, we decompose failure data series into some subdata series components with strong regularity by using C‐C algorithm and Cao algorithm. On this basis, we use the maximum Lyapunov index to identify chaotic characteristics of failure data series. And then, we establish BP neural network prediction model by using reconstructing failure data to predict SMISs failure behaviors. Finally, we use two groups of failure data series to verify the effectiveness of chaotic BP neural network model, and the experiment results verify that chaotic BP neural network model has more accurate prediction results compared with BP network, support vector machine, long short term memory networks (LSTM), and autoregressive model (AR).LEAD PARAGRAPHSMISs are open and complex systems, human error and external environment cause the uncertainty of reliability. The threat of the external environment mainly comes from malicious attacks and the threat of the human error mainly comes from the wrong operation of the operator. Human error and external environment often cause frequent failures of software and hardware of the system or physical failures of devices. As an open complex system, the reliability operation of SMISs is very important for manufacturing enterprises. However, in the daily use of SMISs, the most common failures are time failure caused by human error and external environment. Therefore, it is very important to study the time failure of SMISs.Main points of this paper: (1) SMISs are complex and open systems, so we establish an interdependent network based on the characteristics of SMISs, and use the cascade effect of the complex network to point out that when SMISs fail, the system will easily fall into failure. (2) The phase space reconstruction method is used to restore the real data characteristics of failure data. (3) By using the reconstructed data and the neural network, the failure behavior can be predicted accurately. Compared with other popular prediction methods, it is found that general machine learning methods cannot predict data with chaotic characteristics.The research results of this paper find that when SMISs fail, the failure behavior can easily lead SMISs into chaos through the propagation of interdependent network. Therefore, when future scholars conduct fault analysis on SMISs, they should consider the chaos of the data, otherwise the systems fault analysis and diagnosis cannot be carried out accurately.

An Effective Fault Feature Extraction Method for Gas Turbine Generator System Diagnosis

Fault diagnosis is very important to maintain the operation of a gas turbine generator system (GTGS) in power plants, where any abnormal situations will interrupt the electricity supply. The fault diagnosis of the GTGS faces the main challenge that the acquired data, vibration or sound signals, contain a great deal of redundant information which extends the fault identification time and degrades the diagnostic accuracy. To improve the diagnostic performance in the GTGS, an effective fault feature extraction framework is proposed to solve the problem of the signal disorder and redundant information in the acquired signal. The proposed framework combines feature extraction with a general machine learning method, support vector machine (SVM), to implement an intelligent fault diagnosis. The feature extraction method adopts wavelet packet transform and time-domain statistical features to extract the features of faults from the vibration signal. To further reduce the redundant information in extracted features, kernel principal component analysis is applied in this study. Experimental results indicate that the proposed feature extracted technique is an effective method to extract the useful features of faults, resulting in improvement of the performance of fault diagnosis for the GTGS.

Screening drug target proteins based on sequence information

Identifying new drug target (DT) proteins is important in pharmaceutical and biomedical research. General machine learning method (GMLM) classifiers perform fairly well at prediction if the training dataset is well prepared. However, a common problem in preparing the training dataset is the lack of a negative dataset. To address this problem, we proposed two methods that can help GMLM better select the negative training dataset from the test dataset. The prediction accuracy was improved with the training dataset from the proposed strategies. The classifier identified 1797 and 227 potential DT proteins, some of which were mentioned in previous research, which added correlative weight to the new method. Practically, these two sets of potential DT proteins or their homologues are worth considering.

Edge-over-Erosion Error Prediction Method Based on Multi-Level Machine Learning Algorithm

SUMMARY As VLSI process node continue to shrink, chemical mechanical planarization (CMP) process for copper interconnect has become an essential technique for enabling many-layer interconnection. Recently, Edge-over-Erosion error (EoE-error), which originates from overpolishing and could cause yield loss, is observed in various CMP processes, while its mechanism is still unclear. To predict these errors, we propose an EoEerror prediction method that exploits machine learning algorithms. The proposed method consists of (1) error analysis stage, (2) layout parameter extraction stage, (3) model construction stage and (4) prediction stage. In the error analysis and parameter extraction stages, we analyze test chips and identify layout parameters which have an impact on EoE phenomenon. In the model construction stage, we construct a prediction model using the proposed multi-level machine learning method, and do predictions for designed layouts in the prediction stage. Experimental results show that the proposed method attained 2.7∼19.2% accuracy improvement of EoE-error prediction and 0.8∼10.1% improvement of non-EoE-error prediction compared with general machine learning methods. The proposed method makes it possible to prevent unexpected yield loss by recognizing EoE-errors be

An approach to predict the risk of glaucoma development by integrating different attribute data

Primary open-angle glaucoma (POAG) is one of the major causes of blindness worldwide and considered to be influenced by inherited and environmental factors. Recently, we demonstrated a genome-wide association study for the susceptibility to POAG by comparing patients and controls. In addition, the serum cytokine levels, which are affected by environmental and postnatal factors, could be also obtained in patients as well as in controls, simultaneously. Here, in order to predict the effective diagnosis of POAG, we developed an “integration approach” using different attribute data which were integrated simply with several machine learning methods and random sampling. Two data sets were prepared for this study. The one is the “training data set”, which consisted of 42 POAG and 42 controls. The other is the “test data set” consisted of 73 POAG and 52 controls. We first examined for genotype and cytokine data using the training data set with general machine learning methods. After the integration approach was applied, we obtained the stable accuracy, using the support vector machine method with the radial basis function. Although our approach was based on well-known machine learning methods and a simple process, we demonstrated that the integration with two kinds of attributes, genotype and cytokines, was effective and helpful in diagnostic prediction of POAG.

Using support vector regression coupled with the genetic algorithm for predicting acute toxicity to the fathead minnow

The potential toxicity of chemicals may present adverse effects to the environment and human health. The quantitative structure–activity relationship (QSAR) provides a useful method for hazard assessment. In this study, we constructed a QSAR model based on a highly heterogeneous data set of 571 compounds from the US Environmental Protection Agency, for predicting acute toxicity to the fathead minnow (Pimephales promelas). An approach coupling support vector regression (SVR) with the genetic algorithm (GA) was developed to build the model. The generated QSAR model showed excellent data fitting and prediction abilities: the squared correlation coefficients (r 2) for the training set and the test set were 0.826 and 0.802, respectively. Only eight critical descriptors, most of which are closely related to the toxicity mechanism, were chosen by GA-SVR, making the derived model readily interpretable. In summary, the successful case reported here highlights that our GA-SVR approach can be used as a general machine learning method for toxicity prediction.

ANALYZING FRENCH JUSTICE WITH A GENETIC-BASED INDUCTIVE ALGORITHM

Abstract This paper describes a genetic learning system called SIA, which learns rules from a set of preclassified examples where many attribute values are missing or have “don't care” or undefined values. SIA was motivated by a data analysis task in the French justice domain and by the fact that the general machine learning methods used to deal with unknown values are not appropriate for this task. SIA is somewhat similar to the AQ algorithm because it takes an example as a seed and generalizes it (here, using a genetic process) to find a rule maximizing a rule evaluation criterion. Two mechanisms are used to deal efficiently with unknown values: the dropping of the unknown attributes in the seed example and a restricted matching operator that prevents matching with unknown values. SIA is comparable to AQ and other algorithms on two standard learning tasks and can help understanding of the French justice domain.