Fair Resource Allocation Research Articles

Distributive Justice as a Means of Combating Systemic Racism in Healthcare

Distributive Justice as a Means of Combating Systemic Racism in Healthcare

Fair resource allocation for demands with sharp lower tail inequalities

We consider a fairness problem in resource allocation where multiple groups demand resources from a common source with the total fixed amount. We show that for many common demand distributions satisfying sharp lower-tail inequalities, a natural allocation that provides resources proportional to each group's average demand performs very well. More specifically, this natural allocation is approximately fair and efficient (i.e., it provides near maximum utilization). We also show that, when a small amount of unfairness is allowed, the Price of Fairness (PoF), in this case, is close to 1.

Ang Isyu ng Katarungan sa Panahon ng Pandemya

Given the usually limited supply of resources needed to meet our healthcare needs during a pandemic, coming up with a fair system of prioritization by which such resources are to be distributed is both extremely urgent and morally obligatory. In the area of political morality, the issue of what counts as a fair allocation of resources, which may generally come in the form of being benefits or burdens, is referred to as the issue of distributive justice. In this essay, I look into the issue of distributive justice in the context of the current COVID-19 pandemic. After examining the basic claims of the various theories of distributive justice and how they apply to healthcare resource allocations, I then advance a framework for evaluating the moral justifiability of prioritization schemes for a fair allocation of healthcare resources, especially the vaccines, during the pandemic. The framework basically incorporates moral considerations based on welfare, capabilities, and opportunities. © 2021. Napoleon M. Mabaquiao, Jr.

Twelve years of European cancer drug approval—a systematic investigation of the ‘magnitude of clinical benefit’

BackgroundThe magnitude of clinical benefit of solid cancer drugs can be standardly assessed via the Magnitude of Clinical Benefit Scale (MCBS) developed by the European Society for Medical Oncology (ESMO). We applied two ESMO-MCBS versions to the last 12 years of European cancer drug approval and compared two predefined marketing authorisation timeframes to identify potential score changes over time.Material and methodsOriginator solid cancer drugs and indication extensions that were approved between 1 January 2009 and 31 October 2020 by the European Medicines Agency (EMA) were included in our analyses. To evaluate the clinical benefit of these cancer indications, the original ESMO-MCBS (v 1.1) and a locally adapted ESMO-MCBS version were applied to the study sample. Thus, two ESMO-MCBS versions were compared, and an additional analysis was conducted to identify potential score differences between two approval timeframes 2009-2014 versus 2015-2020.ResultsA total of 144 cancer indications intended as curative (n = 9) or non-curative (n = 135) treatment options were eligible for an ESMO-MCBS assessment. Solely a minority of the assessed cancer indications met the meaningful clinical benefit (MCB) criteria independent of the applied version of the scale and treatment intention (original: n = 48/144, 33.3% versus adapted: n = 27/144, 18.8%). Comparing the two EMA approval timeframes, a growing number of approved cancer indications could be observed: 2009-2014: n = 9/year versus 2015-2020: n = 14/year. In addition, almost no difference in the proportion of cancer indications that have met the MCB criteria was detectable when comparing the predefined authorisation timeframes (MCB increase original: +4.1% and adapted: +3.9%).ConclusionApplying both versions of the ESMO-MCBS can help to identify potentially beneficial cancer indications, but also those with rather uncertain or low clinical benefit and thus, support the fair allocation of limited health care resources.

An Opportunity Downlink Interference Alignment Algorithm Based on Punch Scheduling in Cognitive Heterogeneous Networks

For the hierarchical structure and frequency reuse of cognitive heterogeneous networks, it is inevitable to effectively deal with interference while gaining capacity. In this article, we consider the downlink cognitive heterogeneous network, in which a macro base station can serve a certain number of macro users, and each macro cell is composed of a certain number of micro cells. An opportunistic downlink interference alignment (ODIA) algorithm for cognitive heterogeneous networks based on puncture scheduling is proposed. First, we designed a pair of precoding and decoding matrices to eliminate co-layer interference. Second, in order to reduce the overhead required to transmit system state information(CSI), we proposed a user scheduling algorithm that not only can obtain a larger interference-free size, but also improvements in fairness, system capacity and overflow probability. The simulation results show that the proposed ODIA scheme is superior to the existing interference management algorithms in terms of interference, system capacity and fairness of system resource allocation.

Toddlers' interventions toward fair and unfair individuals

Cooperative societies rely on reward and punishment for norm enforcement. We examined the developmental origin of these interventions in the context of distributive fairness: past research has shown that infants expect resources to be distributed fairly, prefer to interact with fair distributors, and evaluate others based on their fair and unfair resource allocations. In order to determine whether infants would intervene in third-party resource distributions by use of reward and punishment we developed a novel task. Sixteen-month-old infants were taught that one side of a touch screen produces reward (vocal statements expressing praise; giving a cookie), whereas the other side produces punishment when touched (vocal statements expressing admonishment; taking away a cookie). After watching videos in which one actor distributed resources fairly and another actor distributed resources unfairly, participants' screen touches on the reward and punishment panels while the fair and unfair distributors appeared on screen were recorded. Infants touched the reward side significantly more than the punishment side when presented with the fair distributor but touched the screen sides equally when the unfair distributor was shown. Control experiments revealed no evidence of reward or punishment when infants saw food items they liked and disliked, or individuals uninvolved in the resource distribution events. These results provide the earliest evidence that infants are able to spontaneously intervene in socio-moral situations by rewarding positive actions.

Fair resource allocation: Using welfare-based dominance constraints

In this paper we consider the problem of supporting resource allocation decisions affecting multiple beneficiaries. Such problems inherently involve efficiency-fairness trade-offs. We introduce a new approach based on the paradigm of maximizing efficiency subject to constraints to ensure that the decision is acceptably fair. In contrast to existing literature, we incorporate fairness in the form of welfare dominance, ensuring that the resultant distribution of benefits to beneficiaries is at least as good as some reference distribution with respect to a set of social welfare functions that satisfy commonly accepted efficiency and fairness related axioms. We introduce a practical means to parameterize the problem, which allows for excluding welfare functions that are deemed insufficiently or overly sensitive to inequality. This allows for analyzing the impact of changes in inequality aversion on efficiency, thus revealing the trade-off between efficiency and fairness. We develop tractable reformulations for the resulting non-linear multi-level optimization problems. We then extend this approach for cases where resources are allocated to groups of individuals with different sizes. We demonstrate the potential use of the suggested framework on two case studies: a workload allocation problem and a healthcare provisioning problem.

Maximin Fairness with Mixed Divisible and Indivisible Goods

We study fair resource allocation when the resources contain a mixture of divisible and indivisible goods, focusing on the well-studied fairness notion of maximin share fairness (MMS). With only indivisible goods, a full MMS allocation may not exist, but a constant multiplicative approximate allocation always does. We analyze how the MMS approximation guarantee would be affected when the resources to be allocated also contain divisible goods. In particular, we show that the worst-case MMS approximation guarantee with mixed goods is no worse than that with only indivisible goods. However, there exist problem instances to which adding some divisible resources would strictly decrease the MMS approximation ratios of the instances. On the algorithmic front, we propose a constructive algorithm that will always produce an \alpha-MMS allocation for any number of agents, where \alpha takes values between 1/2 and 1 and is a monotonically increasing function determined by how agents value the divisible goods relative to their MMS values.

Deep Reinforcement Learning-Based Dynamic Spectrum Access for D2D Communication Underlay Cellular Networks

This letter investigates a deep reinforcement learning (DRL)-based spectrum access scheme for device-to-device (D2D) communication underlay cellular networks. Specifically, cellular users (CUEs) and D2D pairs attempt to access the time slots (TSs) of a shared spectrum, and TSs are dynamically scheduled to CUEs in different frames. Based on the DRL theory, D2D pairs can be seen as a centralized agent which aims to learn an optimal spectrum access strategy to maximize the sum throughput without any prior information. In particular, with different locations of CUEs, the spectrum access manners for D2D communication are changed to ensure the communication quality of CUEs at the cell edge. Then, a double deep Q-network (DDQN) based D2D spectrum access (D <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">4</sup> SA) algorithm is proposed, which makes D2D pairs learn to decide whether to access the spectrum in different TSs. Moreover, to ensure the fairness of resource allocation among D2D pairs, we improve the proposed algorithm and incorporate fairness into the objective function. Simulation results show that our proposed algorithm can achieve an optimal sum throughput close to the theoretical upper bound, where the performance is significantly improved compared to the scheme based on base station cooperation.

Willingness to Pay for Health-Related Quality of Life Gains in Relation to Disease Severity and the Age of Patients

ObjectivesDecision-making frameworks that draw on economic evaluations increasingly use equity weights to facilitate a more equitable and fair allocation of healthcare resources. These weights can be attached to health gains or reflected in the monetary threshold against which the incremental cost-effectiveness ratios of (new) health technologies are evaluated. Currently applied weights are based on different definitions of disease severity and do not account for age-related preferences in society. However, age has been shown to be an important equity-relevant characteristic. This study examines the willingness to pay (WTP) for health-related quality of life (QOL) gains in relation to the disease severity and age of patients, and the outcome of the disease. MethodsWe obtained WTP estimates by applying contingent-valuation tasks in a representative sample of the public in The Netherlands (n = 2023). We applied random-effects generalized least squares regression models to estimate the effect of patients’ disease severity and age, size of QOL gains, disease outcome (full recovery/death 1 year after falling ill), and respondent characteristics on the WTP. ResultsRespondents’ WTP was higher for more severely ill and younger patients and for larger-sized QOL gains, but lower for patients who died. However, the relations were nonlinear and context dependent. Respondents with a lower age, who were male, had a higher household income, and a higher QOL stated a higher WTP for QOL gains. ConclusionsOur results suggest that—if the aim is to align resource-allocation decisions in healthcare with societal preferences—currently applied equity weights do not suffice.

A Proficient Fair Resource Allocation in the Channel of Multiuser Orthogonal Frequency Division Multiplexing using a Novel Hybrid Bat-Krill Herd Optimization

The wireless communication system in the present generation address more challenges because of priority based Resource Allocation (RA) to Real-Time (RT), Best Effort (BE), as well as Non-Real Time (NRT) users in the Wi-Max (Worldwide Inter-operability for Microwave Access) network. Furthermore, the fall of packet and data overflow became an excessive problem in the network. To address this problem, Multi-Input and Multi-Output based Orthogonal Frequency Division Multiplexing (MIMO-OFDM) channel is introduced but in some cases, it also met some challenges in resource allocation strategy. Therefore, this research is to emphasize the Quality of Service (QoS) and prevents data flooding in nodes by the proposed Detain Permissive Network (DPN) model. Furthermore, a novel Hybrid Bat and Krill Herd Optimization (HB-KHO) is proposed for the DPN to allocate the resources based on the priority of users in the MIMO-OFDM channel based Wi-Max network. The execution of the proposed method is carried out in NS-2 platform. The simulation outcome has been attained the finest priority-based resource allocation in terms of better fairness, throughput, and Packet Delivery Ratio (PDR). Moreover, the outcome from the novel proposed technique is compared with existing resource allocation techniques and the comparison shows that the proposed technique is effectively allocated resources to all users in the network.

Fair and Fit Resource Allocation Algorithm for Network-Controlled Device-to-Device (D2D) Communication

Fair and Fit Resource Allocation Algorithm for Network-Controlled Device-to-Device (D2D) Communication

Strategic Planning of an Integrated Fuel Production System with a Fair-Sustainable Approach

This work introduces a systematic approach for the optimization of an integrated system to produce fuels and simultaneously capture emissions while accounting for fairness schemes and sustainability metrics. The proposed fair-sustainable approach is relevant because a fair allocation of resources is key for long-term sustainability. The considered stakeholders in the integrated system include refineries, biorefineries, and eco-industries (taken in this work as endeavors for forest plantations resulting in capturing emissions). The optimal allocation of income among these stakeholders is analyzed by evaluating different fairness schemes (social welfare, Rawlsian welfare, and Nash approaches) and a sustainability metric (sustainability weighted return on investment). To incentivize the installation of eco-industries, economic compensation options are included. The applicability of the optimization formulation is demonstrated using a case study of Mexico. However, the model is general and can be applied to different case studies. The results show distinct income allocations under the fairness schemes and the sustainability metric. These allocations are compared to their utopic points. Additionally, the avoided emissions and the generated jobs obtained through the distinct schemes are analyzed. The results also show that the Nash scheme can provide fair trade-offs among the stakeholders’ income and environmental and social functions.

Clinical Priority Setting and Decision-Making in Sweden: A Cross-sectional Survey Among Physicians.

Priority setting in healthcare that aims to achieve a fair and efficient allocation of limited resources is a worldwide challenge. Sweden has developed a sophisticated approach. Still, there is a need for a more detailed insight on how measures permeate clinical life. This study aimed to assess physicians' views regarding (1) impact of scarce resources on patient care, (2) clinical decision-making, and (3) the ethical platform and national guidelines for healthcare by the National Board of Health and Welfare (NBHW). An online cross-sectional questionnaire was sent to two groups in Sweden, 2016 and 2017. Group 1 represented 331 physicians from different departments at one University hospital and group 2 consisted of 923 members of the Society of Cardiology. Overall, a 26% (328/1254) response rate was achieved, 49% in group 1 (162/331), 18% in group 2 (166/923). Scarcity of resources was perceived by 59% more often than 'at least once per month,' whilst 60% felt less than 'well-prepared' to address this issue. Guidelines in general had a lot of influence and 19% perceived them as limiting decision-making. 86% professed to be mostly independent in decision-making. 36% knew the ethical platform 'well' and 'very well' and 64% NBHW's national guidelines. 57% expressed a wish for further knowledge and training regarding the ethical platform and 51% for support in applying NBHW's national guidelines. There was a need for more support to deal with scarcity of resources and for increased knowledge about the ethical platform and NBHW's national guidelines. Independence in clinical decision-making was perceived as high and guidelines in general as important. Priority setting as one potential pathway to fair and transparent decision-making should be highlighted more in Swedish clinical settings, with special emphasis on the ethical platform.

Game-based approach of fair resource allocation in wireless powered cooperative cognitive radio networks

A resource allocation approach is proposed for wireless powered cooperative cognitive radio network (WP-CCRN) in this paper, where multiple secondary users (SUs) capture both the wireless energy and the authorized spectrum from primary user (PU) to communicate with a secondary access point (SAP). In return, SUs serve as relay nodes to help the primary transmitter (PT) forwarding data to the primary receiver (PR). For the sake of the stable cooperation between PU and SUs, we present a game-based fair resource allocation algorithm (GFRAA) that takes into account the utilities of both sides. The Stackelberg game is applied for solving the resource allocation problem between PU and SUs, where PU acts as a leader while SUs are followers. To solve the non-convex optimization problem in the primary network, a new variable namely allocated energy is introduced to transform the primal problem into a convex problem. Meanwhile, the relationship among SUs is modeled as a Supermodular game. Since SUs are rational and selfish, a fairness evaluation mechanism is designed to adjust the fairness among SUs. Simulation results unveil the trade-off between the fairness of SUs, the throughput of PU, and the sum-throughput of SUs. The superiority of the GFRAA has been validated by comparing with the other two algorithms.

Resource Allocation in Uplink NOMA-IoT Networks: A Reinforcement-Learning Approach

Non-orthogonal multiple access (NOMA) exploits the potential of the power domain to enhance the connectivity for the Internet of Things (IoT). Due to time-varying communication channels, dynamic user clustering is a promising method to increase the throughput of NOMA-IoT networks. This article develops an intelligent resource allocation scheme for uplink NOMA-IoT communications. To maximise the average performance of sum rates, this work designs an efficient optimization approach based on two reinforcement learning algorithms, namely deep reinforcement learning (DRL) and SARSA-learning. For light traffic, SARSA-learning is used to explore the safest resource allocation policy with low cost. For heavy traffic, DRL is used to handle traffic-introduced huge variables. With the aid of the considered approach, this work addresses two main problems of fair resource allocation in NOMA techniques: 1) allocating users dynamically and 2) balancing resource blocks and network traffic. We analytically demonstrate that the rate of convergence is inversely proportional to network sizes. Numerical results show that: 1) Compared with the optimal benchmark scheme, the proposed DRL and SARSA-learning algorithms have lower complexity with acceptable accuracy and 2) NOMA-enabled IoT networks outperform the conventional orthogonal multiple access based IoT networks in terms of system throughput.

Abstract IA-21: AI in an imaging center: Challenges and opportunities

Abstract There is significant and well-founded enthusiasm for the promise of AI to improve healthcare outcomes across diverse clinical applications in oncology. However, careful analyses of the impact of AI tools in actual “real world” clinical settings are sparse in this relatively early phase of scientific discovery. The rigor applied to deep learning model development (training, testing, external validation) must be matched with equal rigor in assessing patient outcomes after implementation of deep learning tools in actual clinical settings. We can learn from our past endeavors in computer-assisted image interpretation. Tools for computer aided detection were approved by the FDA in 1998 to help radiologists find breast cancers on mammograms. However, after widespread clinical implementation (based on promising results in reader studies and reimbursement approvals), clinical trials to assess the impact of CAD on diagnostic accuracy of mammography revealed the tools did not improve human performance in findings cancers, and in many cases increased false positive readings by humans using CAD assistance. Due to severely limited healthcare resources during pandemics, and to protect patients and healthcare workers, state governments and the COVID-19 Pandemic Breast Cancer Consortium urged providers to focus cancer screening efforts on those patients at higher risk. These mandates were and continue to be necessary responses to support fair allocation of scarce resources to maximize benefits for all patients across the full spectrum of healthcare needs and acknowledge the shift in risk-benefit ratios for screening in average risk women. After screening mammography programs were closed due to the COVID-19 pandemic, we implemented an AI image-only risk model strategy at MGH to invite women at higher risk to return to screening for breast cancer. The AI model identified the top 60% of patients at risk, who were diagnosed with 90% of cancers, while the top 60% of at-risk patients based on traditional models harbored less than 50% of cancers. Our AI image-only risk model performed best in identifying women with current breast cancer, followed by strategies which used five-year traditional risk models and included all women with a personal history of breast cancer in invitations to screen. Lifetime risk models performed extremely poorly. Operating at the traditional “high risk” threshold of &amp;gt;20% lifetime risk, these models identified fewer than 10% of patients with cancer in the total population screened. AI-based risk models may support accurate and equitable risk-based screening. It may be possible with early intervention to reduce the widening disparities and later stage breast cancer diagnoses we are experiencing with COVID and mitigate the risk of delayed diagnoses and increased mortality from breast cancer during and after pandemics. Strategies for careful and rigorous assessment of clinical implementation are essential for AI to have the impact in health desired by all. Citation Format: Constance Lehman. AI in an imaging center: Challenges and opportunities [abstract]. In: Proceedings of the AACR Virtual Special Conference on Artificial Intelligence, Diagnosis, and Imaging; 2021 Jan 13-14. Philadelphia (PA): AACR; Clin Cancer Res 2021;27(5_Suppl):Abstract nr IA-21.

Adaptive Multi-Access Algorithm for Multi-Service Edge Users in 5G Ultra-Dense Heterogeneous Networks

In the 5G ultra-dense wireless heterogeneous network system, it is a crucial issue to implement an effective network selection strategy to satisfy the demands of massive edge users and novel 5G services. In this paper, we model the network selection problem of edge users requesting different services as a bipartite graph, and propose a network selection algorithm based on weighted bipartite graph matching for 5G ultra-dense heterogeneous networks, named BGMNS. The proposed algorithm combines Analytic Hierarchy Process (AHP) and Grey Relation Analysis (GRA) to analyze the preferences of multiple services for different network attributes and obtain the Quality of Experience (QoE) of different edge users for each network. Moreover, in order to realize the fair allocation of network resources, we comprehensively consider the importance of the requested services and the obtained QoE by edge users to construct system fairness index. The BGMNS algorithm can optimize the overall QoE of users under the premise of ensuring the system fairness. Simulation results show that compared to the existing network selection algorithms, the proposed BGMNS algorithm can not only provide stable access to users when network status fluctuates randomly, but also effectively reduce user blocking probability as well as total packet loss rate, and significantly improve user average energy efficiency.

Abstract SP080: Hidden clues in the mammogram: How AI can improve early breast cancer detection

Abstract Innovative methods of risk assessment that leverage the strength of Artificial Intelligence (AI) are essential to propel the goals of precision prevention forward. Since the creation of the Gail model in 1989, risk models have supported risk-adjusted screening and prevention, and their continued evolution has been a central pillar of breast cancer research. Prior research has explored multiple risk factors related to hormonal and genetic information. One factor that has received substantial attention is mammographic breast density. Incorporating mammographic breast density into clinically used models such as the Gail and Tyrer-Cuzick risk models significantly improves prediction and discrimination. However, current risk models are limited in that they incorporate only a small fraction of data available on any given patient. Using breast density as a proxy for the detailed information embedded in the mammogram is extremely limited, as breast density assessment is subjective, varies widely across radiologists, and restricts the rich information contained in the digital images to a single crude value. Patients of the same age assigned the same density score can have mammogram images that appear drastically different and can have very different future risk profiles. While previous studies have explored automated methods to assess breast density, these efforts reduce the complex data contained in the mammogram into a few statistics, which are not sufficiently rich to distinguish patients who will and will not develop breast cancer. Deep learning models can operate over full resolution mammogram images to assess a patient’s future breast cancer risk. Rather than manually identifying discriminative image patterns, machine learning models can discover these patterns directly from the data. Specifically, models are trained with full resolution mammograms and the outcome of interest, namely whether the patient developed breast cancer within five years from the date of the examination. Our recent work demonstrates that application of novel artificial intelligence applications to imaging data can significantly improve breast cancer risk prediction. In addition, unlike traditional models, our DL model performs equally well across varied races, ages, and family histories and we have built a clinical platform which is currently in use to support implementation of our risk model into clinical care. The COVID-19 pandemic has revealed severe inequities in healthcare while providing opportunities for essential reform. In breast cancer care, preliminary, conservative estimates predict the disruption of breast cancer screening due to the COVID-19 pandemic will result in a significant upward stage shift of cancers diagnosed and more than 5,000 breast cancer deaths in the U.S. alone. Due to severely limited healthcare resources during pandemics, and to protect patients and healthcare workers, state governments urge providers to focus cancer screening efforts on those patients at higher risk. These mandates are necessary responses to support fair allocation of scarce resources to maximize benefits for all patients across the full spectrum of healthcare needs. AI-based breast cancer risk models have the potential to support more effective and more equitable mammographic screening for breast cancer during these times of severely restricted access to screening. ROC Area Under the Curve Analyses of Traditional vs AI Risk Models Risk ModelTyrer-Cuzick version 8 AUCAI Image Only AUCRaceAfrican American0.58 (0.39, 0.79)0.74 (0.60, 0.90)Asian0.53 (0.35, 0.74)0.79 (0.68, 0.94)White0.64 (0.60, 0.68)0.77 (0.73, 0.80)Age&amp;lt;500.65 (0.57, 0.72)0.75 (0.68, 0.82)50-700.64 (0.60, 0.69)0.76 (0.72, 0.79)&amp;gt;700.52 (0.43, 0.60)0.77 (0.70, 0.84)DensityNon-Dense0.63 (0.58, 0.68)0.77 (0.73, 0.81)Dense0.63 (0.58, 0.69)0.77 (0.73, 0.81) Citation Format: C Lehman, A Yala, L Lamb, R Barzilay. Hidden clues in the mammogram: How AI can improve early breast cancer detection [abstract]. In: Proceedings of the 2020 San Antonio Breast Cancer Virtual Symposium; 2020 Dec 8-11; San Antonio, TX. Philadelphia (PA): AACR; Cancer Res 2021;81(4 Suppl):Abstract nr SP080.

Computation of workflow scheduling using backpropagation neural network in cloud computing: a virtual machine placement approach

For measuring the efficiency of workflow scheduling, determining makespan and execution cost is essential. As estimating makespan and cost is difficult in a Cloud environment, designing an efficient computation of workflow scheduling remains a challenge. The Cloud resources are scaled up and down in accordance with user demand by following a scheduling policy. The scalability of the work environment is achieved through the virtualization process. Based on system experience, this paper proposes the priority-based backfilling backpropagation neural network (PBF-NN) hybrid scheduling algorithm for measuring makespan and execution cost accurately. The backfill algorithm is used to schedule tasks to the available resources. The percentage of migration is reduced when this algorithm is used compared to the First Come First Serve algorithm. Then, the Berger model is used to measure the fairness of resource allocation. The system decides task reallocation based on the fairness value. The backpropagation neural network handles the virtual machine placement process with necessary training and testing. The proposed algorithm dynamically allocates the tasks and reduces the utilization of resources. We use an experimental study to illustrate how the proposed system enables higher efficiency in cost, makespan, and performance.