Multi-objective Machine Learning Research Articles

Predicting multiple taste sensations with a multiobjective machine learning method

Taste perception plays a pivotal role in guiding nutrient intake and aiding in the avoidance of potentially harmful substances through five basic tastes - sweet, bitter, umami, salty, and sour. Taste perception originates from molecular interactions in the oral cavity between taste receptors and chemical tastants. Hence, the recognition of taste receptors and the subsequent perception of taste heavily rely on the physicochemical properties of food ingredients. In recent years, several advances have been made towards the development of machine learning-based algorithms to classify chemical compounds’ tastes using their molecular structures. Despite the great efforts, there remains significant room for improvement in developing multi-class models to predict the entire spectrum of basic tastes. Here, we present a multi-class predictor aimed at distinguishing bitter, sweet, and umami, from other taste sensations. The development of a multi-class taste predictor paves the way for a comprehensive understanding of the chemical attributes associated with each fundamental taste. It also opens the potential for integration into the evolving realm of multi-sensory perception, which encompasses visual, tactile, and olfactory sensations to holistically characterize flavour perception. This concept holds promise for introducing innovative methodologies in the rational design of foods, including pre-determining specific tastes and engineering complementary diets to augment traditional pharmacological treatments.

Interactive Hyperparameter Optimization in Multi-Objective Problems via Preference Learning

Hyperparameter optimization (HPO) is important to leverage the full potential of machine learning (ML). In practice, users are often interested in multi-objective (MO) problems, i.e., optimizing potentially conflicting objectives, like accuracy and energy consumption. To tackle this, the vast majority of MO-ML algorithms return a Pareto front of non-dominated machine learning models to the user. Optimizing the hyperparameters of such algorithms is non-trivial as evaluating a hyperparameter configuration entails evaluating the quality of the resulting Pareto front. In literature, there are known indicators that assess the quality of a Pareto front (e.g., hypervolume, R2) by quantifying different properties (e.g., volume, proximity to a reference point). However, choosing the indicator that leads to the desired Pareto front might be a hard task for a user. In this paper, we propose a human-centered interactive HPO approach tailored towards multi-objective ML leveraging preference learning to extract desiderata from users that guide the optimization. Instead of relying on the user guessing the most suitable indicator for their needs, our approach automatically learns an appropriate indicator. Concretely, we leverage pairwise comparisons of distinct Pareto fronts to learn such an appropriate quality indicator. Then, we optimize the hyperparameters of the underlying MO-ML algorithm towards this learned indicator using a state-of-the-art HPO approach. In an experimental study targeting the environmental impact of ML, we demonstrate that our approach leads to substantially better Pareto fronts compared to optimizing based on a wrong indicator pre-selected by the user, and performs comparable in the case of an advanced user knowing which indicator to pick.

Multi-objective machine learning for designing enzyme cofactor specificity

Multi-objective machine learning for designing enzyme cofactor specificity

Multi-objective Machine Learning for control performance assessment in PID control loops

Control Performance Assessment (CPA) is a critical endeavor in industrial processes, ensuring optimal functioning of control systems. Traditionally, CPA has been addressed through solutions using some control performance indicators. Nowadays, the integration of data science and machine learning has emerged as a viable alternative, particularly in classification tasks related to CPA. That is, in a binary classification scheme, the goal is to predict whether incoming data from the control loop belongs to class 0 or 1, representing the absence or presence of an anomaly (performance degradation). In such a case, a trade-of between false positives and false negatives should be obtained, via the training phase of a given supervised machine learning structure for example. Usually, this is a conflicting trade-of, where multi-objective optimization techniques in the training phase of such learners could bring interesting results. In this paper, we explore the usability of multi-objective optimization training in machine learning, for control performance assessment classification. A database describing 30 control performance indicators (features) in a PID control loop is used. The obtained results indicate that the proposed approach could bring interesting applications to improve the performance of CPA classification systems.

Industrial carbon emission efficiency prediction and carbon emission reduction strategies based on multi-objective particle swarm optimization-backpropagation: A perspective from regional clustering

Against the backdrop of global climate change, industrial carbon emission reduction has become an important pathway to for global low-carbon development. This study constructs a framework of geographic spatial constraints regionalization and multi-objective machine learning to predict future industrial carbon emission efficiency (ICEE) and explore strategies for carbon emission reduction. Firstly, the ICEE of 285 Chinese cities were calculated by the super-efficiency slacks-based measure. Secondly, the cities were classified into four ICEE level regions through the spatially constrained multivariate clustering. Next, the multi-objective particle swarm optimization-BP (MOPSO-BP) model was constructed to predict the future trends of ICEE in the four regions. Finally, the geographical detector and multi-scale geographically weighted regression were employed for exploring driving force and carbon emission reduction strategies in different regions. The results show that most cities had low or medium ICEE, while super efficiency cities were mainly distributed in the east coastal areas. The prediction performance of the MOPSO-BP model for the four regions was better than the ordinary particle swarm optimization-BP and traditional BP model. Except for the Agricultural Production Region, there is considerable room for improving the ICEE of other regions over the next decade. Macroeconomic and microeconomic development have a global effect in promoting regional ICEE improvement, urban construction shows a promoting or inhibiting effect in different regions, and information technology has significant spatial heterogeneity in its influence within each region. The analysis framework developed in the study is a reliable solution for managing and planning ICEE and provides constructive suggestions for future regional low-carbon development.

Prediction of triaxial mechanical properties of rocks based on mesoscopic finite element numerical simulation and multi-objective machine learning

The deformation and strength characteristics of rocks are crucial for the effective development of underground resources and the construction of underground engineering projects. In this study, a novel approach is proposed to predict the triaxial mechanical properties of rocks by utilizing mesoscopic finite element numerical simulation and multi-objective machine learning. First, the mesoscopic mechanical properties of rocks are obtained through microscale finite element simulations to generate a training dataset. Then, three machine learning algorithms, namely support vector regression (SVR), artificial neural network (ANN), and random forest (RF), are employed to perform multi-objective machine learning on the triaxial elastic modulus, Poisson's ratio, and compressive strength of rocks, using uniaxial elastic modulus, Poisson's ratio, tensile strength, compressive strength, and confining pressure as feature variables. The performance evaluation, based on 10-fold cross-validation, demonstrates that all three models exhibit excellent predictive capabilities, especially the SVR and RF models, which show high accuracy and correlation, respectively. Among the five feature variables, confining pressure is the most important feature, while uniaxial tensile strength is the least important feature. The absence of uniaxial tensile strength does not significantly impact the predictive performance of the models. These findings offer novel insights into the investigation of the triaxial mechanical properties of rocks.

Fast Simulation of Urban Waterlogging Based on Multi-Objective Machine Learning Model

The frequent occurrence of urban waterlogging disasters induced by rainstorm has recently caused serious economic losses and casualties in China. Numerical simulation of waterlogging is an important tool for disaster prewarning and forecasting as well as disaster prevention and control; however, the traditional numerical physical models have the disadvantage of low computational efficiency, which makes it difficult to meet the demand for real-time simulation and real-time early warning and forecast. To this end, this study combines the respective advantages of coupled rainstorm-flood models with physical mechanisms and machine learning algorithms, and proposes a rapid prediction and simulation method for inundated depth of urban waterlogging based on multi-objective machine learning algorithms. The forecasting performances of K-Nearest Neighbors(KNN), Multi-Objective Random Forest(MORF), Extreme Gradient Boosting(XGBoost) and their integrated models are discussed, respectively. The results show that: (1)The coupled rainstorm- flood model based on SWMM and LISFLOOD-FP has good applicability in the simulation of urban waterlogging induced by rainstorm in the study area. On this basis, the database with a total of 70 scenarios of rainstorm-inundation with different characteristics were generated. (2)The KNN, MORF, XGBoost and their integrated models all have good results in predicting water depth, with Pearson correlation coefficient (PCC)values all above 0. 812, mean absolute error(MAE)below 6. 9 cm, and root-mean-square error (RMSE)less than 0. 116. The KNN-MORF-XGBoost integrated model has the best overall results, with the average values of MAE, PCC and RMSE reaching 2. 4cm, 0. 965 and 0. 043, respectively. (3)In addition to the high prediction accuracy, the prediction speed of the constructed multi-objective machine learning prediction model is extremely fast, and the water depth simulation efficiency is more than 20 times higher than that of the coupled rainstorm-flood model. This study can provide a reference for the application of machine learning in the rapid simulation of urban waterlogging induced by rainstorm, which is of great value for the early warning and forecast of urban waterlogging disaster.

High-throughput calculation and machine learning of two-dimensional halide perovskite materials: Formation energy and band gap

Both band gap and stability of halide perovskites are prerequisites for deployable photovoltaic devices; however, many machine learning researches focus on one target output and a systematic machine learning workflow for achieving multiple targets is desirable. In this manuscript, we employ machine learning (ML) coupled with high-throughput density functional theory (DFT) calculation to predict potential two-dimensional lead-free halide perovskite materials with appropriate band gap and stability for solar cell applications. This is realized by the construction of two machine learning models based on the random forest algorithm with each targeting on band gap or formation energy, followed by the candidate intersection for the materials screening. The multi-objective DFT+ML framework predicts three possible lead-free two-dimensional halide perovskite materials with suitable stability and band gap, which are further evaluated via the molecular dynamics to evaluate their thermodynamic stability. Their spectroscopic limited maximum efficiencies (SLMEs) are calculated to confirm their photovoltaic capabilities. In order to comprehensively evaluate the features, new descriptors for the halide perovskite materials with better correlation with the target output are automatically formulated via symbolic regression based on genetic algorithms, and an alternative feature analysis method based on the literature textual data and natural language processing (NLP) is proposed. Post hoc analysis is performed via DFT and molecular dynamics to provide more detailed information on the materials prediction. This study highlights the developments multi-objective machine learning workflow for inverse materials design and analysis.

Machine learning-based multi-objective optimization for efficient identification of crystal plasticity model parameters

A set of constitutive model parameters along with crystallography governs the activation of deformation mechanisms in crystal plasticity. The constitutive parameters are typically established by fitting of mechanical data, while microstructural data is used for verification. This paper develops a Pareto-based multi-objective machine learning methodology for efficient identification of crystal plasticity constitutive parameters. Specifically, the methodology relays on a Gaussian processes-based surrogate model to limit the number of calls to a given crystal plasticity model, and, consequently, to increase the computational efficiency. The constitutive parameters pertaining to an Elasto-Plastic Self-Consistent (EPSC) crystal plasticity model including a dislocation density-based hardening law, a backstress law, and a phase transformations law are identified for two materials, a dual phase (DP) steel, DP780, subjected to load reversals and a stainless steel (SS), 316L, subjected to strain rate and temperature sensitive deformation. The latter material undergoes plasticity-induced martensitic phase transformations. The optimization objectives were the quasi static flow stress data for the DP steel case study, while a set of strain-rate and temperature sensitive flow stress and phase volume fraction data for the SS case study. The procedure and results for the two case studies are presented and discussed illustrating advantages and versatility of the developed methodology. In particular, the efficiency of the developed methodology over an existing genetic algorithm methodology is discussed. Additionally, the parameters identified for the SS case study were utilized to simulate three biaxial tensile loading paths using a finite element implementation of EPSC for further verification.

Pushing nanomaterials up to the kilogram scale – An accelerated approach for synthesizing antimicrobial ZnO with high shear reactors, machine learning and high-throughput analysis

Novel materials are the backbone of major technological advances. However, the development and wide-scale introduction of new materials, such as nanomaterials, is limited by three main factors—the expense of experiments, inefficiency of synthesis methods and complexity of scale-up. Reaching the kilogram scale is a hurdle that takes years of effort for many nanomaterials. We introduce an improved methodology for materials development, combining state-of-the-art techniques—multi-objective machine learning optimization, high yield microreactors and high throughput analysis. We demonstrate this approach through the optimization of ZnO nanoparticle synthesis, simultaneously targeting high yield and high antibacterial activity. In fewer than 100 experiments, we developed a 1 kg day−1 continuous synthesis for ZnO (with a space-time-yield of 62.4 kg day−1 m−3), having an antibacterial activity comparable to hydrothermally synthesized nano-ZnO and cetrimonium bromide. Following this, we provide insights into the mechanistic factors underlying the performance-yield tradeoffs of synthesis and highlight the need for benchmarking machine learning models with traditional chemical engineering methods. Methods for increasing model accuracy at steep pareto fronts, in this case at yields close to 1 kg per day, should also be improved. To project the next steps for process scale-up and the potential advantages of this methodology, we conduct a scalability analysis in comparison to conventional batch production methods, in which there is a significant reduction in degrees of freedom. The proposed method has the potential to significantly reduce experimental costs, increase process efficiency and enhance material performance, which culminate to form a new pathway for materials discovery.

Abstract 236: Genomic loss in cancers enable discovery of metabolic targets for precision cancer therapy via multiobjective flux analysis and machine learning

Abstract Large-scale chromosomal alterations, particularly chromosomal deletions in cancer genomes confer functional advantages to cancer cells via the loss of tumor suppressor genes (TSGs). However, due to the nature of these focal and arm-level deletions, essential house-keeping genes in the neighborhood of TSGs are potentially lost. We explore the emergence of metabolic adaptations and vulnerabilities that arise due to the collateral loss of essential metabolic genes. In our previous work, we showed that genomic loss in the locus containing SMAD4 and ME2 in pancreatic ductal adenocarcinomas forces these cells to rely on ME3 to compensate for the collateral loss of ME2; thereby revealing a highly selective metabolic target in these cells. Cancer cells can not only exploit such genetic redundancies but also rely on redundancies built into their complex metabolic network to compensate for the loss of metabolic function. Importantly, there is an unexplored landscape of these genomic loss events beyond well-characterized TSGs. To address these challenges, we have developed a platform to identify patient-specific metabolic vulnerabilities emerging due to distinct patterns of genomic loss events across tumors. Our platform presents opportunities for precision-based therapeutic intervention by targeting metabolic vulnerabilities in cancer patients. It uses genomic and clinical data available in cancer patient databases to obtain candidate metabolic genes that are lost to genomic deletions in an unbiased manner. To delineate metabolic redundances and tackle the complexity of genome-scale metabolic models, we employ an innovative multi-objective metabolic flux analysis approach. The utility of this platform is demonstrated via the discovery of a novel metabolic target in a cohort of ovarian cancer patients. The predicted collateral lethal target is validated in vitro using RNA interference and small-molecule inhibitors. Furthermore, we verify the metabolic mechanism of vulnerability predicted by the algorithm using deuterium tracing experiments. The target is also validated in vivo with mice containing ovarian tumors derived from cancer cells with and without the genomic deletion. Surprisingly, the collateral lethal metabolic target was also found to exist in a subset of aggressive endometrial cancers. Finally, we developed a multi-layer machine learning model to predict occurrence of the particular genomic deletion in ovarian and endometrial cancer patients with minimal molecular information to remove the need for whole-genome sequencing data. The model was trained and tested using the publicly-available molecular data from TCGA and AACR GENIE datasets. Citation Format: Abhinav Achreja, Tao Yu, Anjali Mittal, Srinadh Choppara, Noah Meurs, Olamide Animasahun, Jin Heon Jeon, Aradhana Mohan, Anusha Jayaraman, Reva Kulkarni, Justin Reinhold, Michele Cusato, Analisa Difeo, Xiongbin Lu, Deepak Nagrath. Genomic loss in cancers enable discovery of metabolic targets for precision cancer therapy via multiobjective flux analysis and machine learning [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2021; 2021 Apr 10-15 and May 17-21. Philadelphia (PA): AACR; Cancer Res 2021;81(13_Suppl):Abstract nr 236.

A Multiple Gradient Descent Design for Multi-Task Learning on Edge Computing: Multi-Objective Machine Learning Approach

Multi-task learning technique is widely utilized in machine learning modeling where commonalities and differences across multiple tasks are exploited. However, multiple conflicting objectives often occur in multi-task learning. Conventionally, a common compromise is to minimize the weighted sum of multiple objectives which may be invalid if the objectives are competing. In this paper, a novel multi-objective machine learning approach is proposed to solve this challenging issue, which reformulates the multi-task learning as multi-objective optimization. To address the issues contributed by existing multi-objective optimization algorithms, a multi-gradient descent algorithm is introduced for the multi-objective machine learning problem by which an innovative gradient-based optimization is leveraged to converge to an optimal solution of the Pareto set. Moreover, the gradient surgery for the multi-gradient descent algorithm is proposed to obtain a stable Pareto optimal solution. As most of the edge computing devices are computational resource-constrained, the proposed method is implemented for optimizing the edge device's memory, computation and communication demands. The proposed method is applied to the multiple license plate recognition problem. The experimental results show that the proposed method outperforms state-of-the-art learning methods and can successfully find solutions that balance multiple objectives of the learning task over different datasets.

Multi-objective machine learning of four mechanical properties of steels

Multi-objective machine learning of four mechanical properties of steels

Learning comprehensible and accurate hybrid trees

Finding the best classifiers according to different criteria is often performed by a multi-objective machine learning algorithm. This study considers two criteria that are usually treated as the most important when deciding which classifier to apply in practice: comprehensibility and accuracy. A model that offers a broad range of trade-offs between the two criteria is introduced because they conflict; i.e., increasing one decreases the other. The choice of the model is motivated by the fact that domain experts often formalize decisions based on knowledge that can be represented by comprehensible rules and some tacit knowledge. This approach is mimicked by a hybrid tree that consists of comprehensible parts that originate from a regular classification tree and incomprehensible parts that originate from an accurate black-box classifier. An empirical evaluation on 23 UCI datasets shows that the hybrid trees provide trade-offs between the accuracy and comprehensibility that are not possible using traditional machine learning models. A corresponding hybrid-tree comprehensibility metric is also proposed. Furthermore, the paper presents a novel algorithm for learning MAchine LeArning Classifiers with HybrId TrEes (MALACHITE), and it proves that the algorithm finds a complete set of nondominated hybrid trees with regard to their accuracy and comprehensibility. The algorithm is shown to be faster than the well-known multi-objective evolutionary optimization algorithm NSGA-II for trees with moderate size, which is a prerequisite for comprehensibility. On the other hand, the MALACHITE algorithm can generate considerably larger hybrid-trees than a naïve exhaustive search algorithm in a reasonable amount of time. In addition, an interactive iterative data mining process based on the algorithm is proposed that enables inspection of the Pareto set of hybrid trees. In each iteration, the domain expert analyzes the current set of nondominated hybrid trees, infers domain relations, and sets the parameters for the next machine learning step accordingly.

Fine-grained Powercap Allocation for Power-constrained Systems based on Multi-objective Machine Learning

Power capping is an important solution to keep the system within a fixed power constraint. However, for the over-provisioned and power-constrained systems, especially the future exascale supercomputers, powercap needs to be reasonably allocated according to the workloads of compute nodes to achieve trade-offs among performance, energy and powercap. Thus it is necessary to model performance and energy and to predict the optimal powercap allocation strategies. Existing power allocation approaches have insufficient granularity within nodes. Modeling approaches usually model performance and energy separately, ignoring the correlation between objectives, and do not expose the Pareto-optimal powercap configurations. Therefore, this article combines the powercap with uncore frequency scaling and proposes an approach to predict the Pareto-optimal powercap configurations on the power-constrained system for input MPI and OpenMP parallel applications. Our approach first uses the elaborately designed micro-benchmarks and a small number of existing benchmarks to build the training set, and then applies a multi-objective machine learning algorithm which combines the stacked single-target method with extreme gradient boosting to build multi-objective models of performance and energy. The models can be used to predict the optimal processor and memory powercap settings, helping compute nodes perform fine-grained powercap allocation. When the optimal powercap configuration is determined, the uncore frequency scaling is used to further optimize the energy consumption. Compared with the reference powercap configuration, the predicted optimal configurations predicted by our method can achieve an average powercap reduction of 31.35 percent, an average energy reduction of 12.32 percent, and average performance degradation of only 2.43 percent.

Quantitative PET Imaging and Clinical Parameters as Predictive Factors for Patients With Cervical Carcinoma: Implications of a Prediction Model Generated Using Multi-Objective Support Vector Machine Learning.

Purpose:Quantitative features from pre-treatment positron emission tomography (PET) have been used to predict treatment outcomes for patients with cervical carcinoma. The purpose of this study is to use quantitative PET imaging features and clinical parameters to construct a multi-objective machine learning predictive model.Materials/Methods:Seventy-five patients with stage IB2-IVA disease treated at our institution from 2009–2012 were analyzed. Models predicting locoregional and distant failure were generated using clinical parameters (age, race, stage, histology, tumor size, nodal status) and imaging features (12 textural, 9 intensity, 8 geometric features, 2 additional imaging features) from pre-treatment PET. Model features were selected based on a multi-objective evolutionary algorithm to maximize specificity given a fixed moderately high sensitivity using support vector machine learning methods. Model 1 used clinical parameters only (C), Model 2 used imaging features only (I), and Model 3 used clinical and imaging features (C+I). Sensitivity, specificity, area under a receiver-operating characteristic curve (AUC), and p-values were compared to assess ability to predict locoregional and distant failure.Results:C+I had the highest performance for both locoregional failure (AUC 0.84, p < 0.01; specificity: 0.86; sensitivity: 0.79) and distant failure (AUC 0.75, p < 0.01; specificity: 0.75; sensitivity: 0.75).Conclusions:Based on a moderately high fixed sensitivity and optimized for specificity, the model using both clinical parameters and imaging features (C+I) had the best performance in predicting both locoregional failure and distant failure.

Machine learning meets continuous flow chemistry: Automated optimization towards the Pareto front of multiple objectives

Automated development of chemical processes requires access to sophisticated algorithms for multi-objective optimization, since single-objective optimization fails to identify the trade-offs between conflicting performance criteria. Herein we report the implementation of a new multi-objective machine learning optimization algorithm for self-optimization, and demonstrate it in two exemplar chemical reactions performed in continuous flow. The algorithm successfully identified a set of optimal conditions corresponding to the trade-off curve (Pareto front) between environmental and economic objectives in both cases. Thus, it reveals the complete underlying trade-off and is not limited to one compromise as is the case in many other studies. The machine learning algorithm proved to be extremely data efficient, identifying the optimal conditions for the objectives in a lower number of experiments compared to single-objective optimizations. The complete underlying trade-off between multiple objectives is identified without arbitrary weighting factors, but via true multi-objective optimization.

An efficient multi-objective learning algorithm for RBF neural network

Most of modern multi-objective machine learning methods are based on evolutionary optimization algorithms. They are known to be global convergent, however, usually deliver nondeterministic results. In this work we propose the deterministic global solution to a multi-objective problem of supervised learning with the methodology of nonlinear programming. As the result, the proposed multi-objective algorithm performs a global search of Pareto-optimal hypotheses in the space of RBF networks, determining their weights and basis functions. In combination with the Akaike and Bayesian information criteria, the algorithm demonstrates a high generalization efficiency on several synthetic and real-world benchmark problems.

Corrections to “Pareto-Based Multiobjective Machine Learning: An Overview and Case Studies” [May 08 397-415

In the above titled paper (ibid., vol. 38, no. 3, pp. 397-415, May 08), there are three sites where an inequality is put wrongly. The corrections are presented here.

Pareto-Based Multiobjective Machine Learning: An Overview and Case Studies

Machine learning is inherently a multiobjective task. Traditionally, however, either only one of the objectives is adopted as the cost function or multiple objectives are aggregated to a scalar cost function. This can be mainly attributed to the fact that most conventional learning algorithms can only deal with a scalar cost function. Over the last decade, efforts on solving machine learning problems using the Pareto-based multiobjective optimization methodology have gained increasing impetus, particularly due to the great success of multiobjective optimization using evolutionary algorithms and other population-based stochastic search methods. It has been shown that Pareto-based multiobjective learning approaches are more powerful compared to learning algorithms with a scalar cost function in addressing various topics of machine learning, such as clustering, feature selection, improvement of generalization ability, knowledge extraction, and ensemble generation. One common benefit of the different multiobjective learning approaches is that a deeper insight into the learning problem can be gained by analyzing the Pareto front composed of multiple Pareto-optimal solutions. This paper provides an overview of the existing research on multiobjective machine learning, focusing on supervised learning. In addition, a number of case studies are provided to illustrate the major benefits of the Pareto-based approach to machine learning, e.g., how to identify interpretable models and models that can generalize on unseen data from the obtained Pareto-optimal solutions. Three approaches to Pareto-based multiobjective ensemble generation are compared and discussed in detail. Finally, potentially interesting topics in multiobjective machine learning are suggested.