System-level Objectives Research Articles

Reward function design method to achieve system-level objectives in ambulance diversion problem

This study addresses a reward function design problem in multi-agent systems, using ambulance diversion (AD) problem in emergency medical services (EMS) as an application. While Emergency Departments (EDs) may declare AD to alleviate overcrowding, excessive use of this strategy by individual EDs can have negative consequences, such as limited access and delayed care for emergency patients. To eliminate negative consequences caused by individual decisions, we propose a reward function design method to align individual agents’ decisions with socially optimal strategies. This method ensures that an equilibrium status is achieved under socially optimal operation by removing incentives for deviations from socially optimal behavior while maximizing system welfare without extra investment. The proposed method offers an exact solution for designing reward function of a stochastic game model with finite state and action spaces. We conduct numerical experiments for a 2-ED system and demonstrate that the designed reward function properly guides the EDs toward a socially optimal AD strategy.

POEM: Performance Optimization and Endurance Management for Non-volatile Caches

Non-volatile memories (NVMs) with their high storage density and ultra-low leakage power offer promising potential for redesigning the memory hierarchy in next-generation Multi-Processor Systems-on-Chip (MPSoCs). However, the adoption of NVMs in cache designs introduces challenges such as NVM write overheads and limited NVM endurance. The shared NVM cache in an MPSoC experiences requests from different processor cores and responses from the off-chip memory when the requested data is not present in the cache. Besides, upon evictions of dirty data from higher-level caches, the shared NVM cache experiences another source of write operations, known as writebacks . These sources of write operations: writebacks and responses, further exacerbate the contention for the shared bandwidth of the NVM cache, and create significant performance bottlenecks. Uncontrolled write operations can also affect the endurance of the NVM cache, posing a threat to cache lifetime and system reliability. Existing strategies often address either performance or cache endurance individually, leaving a gap for a holistic solution. This study introduces the Performance Optimization and Endurance Management (POEM) methodology, a novel approach that aggressively bypasses cache writebacks and responses to alleviate the NVM cache contention. Contrary to the existing bypass policies which do not pay adequate attention to the shared NVM cache contention, and focus too much on cache data reuse, POEM’s aggressive bypass significantly improves the overall system performance, even at the expense of data reuse. POEM also employs effective wear leveling to enhance the NVM cache endurance by careful redistribution of write operations across different cache lines. Across diverse workloads, POEM yields an average speedup of \(34\% \) over a naïve baseline and \(28.8\% \) over a state-of-the-art NVM cache bypass technique, while enhancing the cache endurance by \(15\% \) over the baseline. POEM also explores diverse design choices by exploiting a key policy parameter that assigns varying priorities to the two system-level objectives.

Fan Stage Design and Performance Optimization for Low Specific Thrust Turbofans

In modern turbofan engines, the bypass section of the fan stage alone provides the majority of the total thrust required in cruise, and the size of the fan has a considerable effect on the overall engine weight and nacelle drag. Thrust requirements in different parts of the flight envelope must also be satisfied together with sufficient margins towards stalling. An accurate description of the interdependencies between the relevant performance and design attributes of the fan stage alone—such as efficiency, surge margin, fan-face Mach number, stage loading, flow coefficient, and aspect ratio—are therefore necessary to estimate system-level objectives such as mission fuel burn and the direct operating cost with enough confidence during the conceptual design phase. The contribution of this study is to apply a parametric optimization approach to the conceptual design of fan stages for low specific thrust turbofans based on the streamline curvature method. Trade-offs between fan stage attributes for Pareto-optimal solutions are modeled by training Kriging surrogate models on the results from the parametric optimization. A case study is provided in the end to demonstrate the potential implications of including a higher level of fan-stage parameter interdependency in an engine systems model. Results implied that being able to predict the rotor solidity required to maintain a given average blade loading—in addition to stage efficiency—is of significant importance when it comes to evaluating the trade-off between engine weight and thrust-specific fuel consumption.

Optimal Selection and Tracking Of Generalized Nash Equilibria in Monotone Games

A fundamental open problem in monotone game theory is the computation of a specific generalized Nash equilibrium (GNE) among all the available ones, e.g. the optimal equilibrium with respect to a system-level objective. The existing GNE seeking algorithms have in fact convergence guarantees toward an arbitrary, possibly inefficient, equilibrium. In this paper, we solve this open problem by leveraging results from fixed-point selection theory and in turn derive distributed algorithms for the computation of an optimal GNE in monotone games. We then extend the technical results to the time-varying setting and propose an algorithm that tracks the sequence of optimal equilibria up to an asymptotic error, whose bound depends on the local computational capabilities of the agents.

App Deconfliction: Orchestrating Distributed, Multi-Agent, Multi-Objective Operations for Power Systems

Advanced distribution systems need to integrate and orchestrate intelligent subsystems and grid-edge devices that are increasing both in number and sophistication while also serving multiple system-level objectives such as resilience, decarbonization, equity, and profitability. A modular platform-based approach to distribution system operations technology enables operators to deploy a tailored set of best-of-breed algorithms and applications. Combined with the parallel deployment and control of intelligent grid-edge and Internet-of-things devices, this creates a complex environment of distributed-control environment with applications that spans ownership boundaries. Conflicts can emerge between applications that want to control overlapping sets of device setpoints. We propose a formalized approach to resolving these conflicts that can be applied when integrating new algorithms or developing customized solutions. A Deconfliction Pipeline is inserted between the device-controlling applications and the device protocol converter, which transmits control setpoints from the operations platform to the devices. The Deconfliction Pipeline executes a process that sets up, solves, and acts on a formally defined deconfliction problem. The deconfliction problem can be solved using a combination of rules and heuristics, application engagement, and optimization. We demonstrate how a few of the most basic solution strategies can be used to orchestrate harmonious behavior between a pair of simple applications with conflicting greedy optimization objectives.

Data-Driven Revision of Conditional Norms in Multi-Agent Systems

In multi-agent systems, norm enforcement is a mechanism for steering the behavior of individual agents in order to achieve desired system-level objectives. Due to the dynamics of multi-agent systems, however, it is hard to design norms that guarantee the achievement of the objectives in every operating context. Also, these objectives may change over time, thereby making previously defined norms ineffective. In this paper, we investigate the use of system execution data to automatically synthesise and revise conditional prohibitions with deadlines, a type of norms aimed at prohibiting agents from exhibiting certain patterns of behaviors. We propose DDNR (Data-Driven Norm Revision), a data-driven approach to norm revision that synthesises revised norms with respect to a data set of traces describing the behavior of the agents in the system. We evaluate DDNR using a state-of-the-art, off-the-shelf urban traffic simulator. The results show that DDNR synthesises revised norms that are significantly more accurate than the original norms in distinguishing adequate and inadequate behaviors for the achievement of the system-level objectives.

Real-Time Grid and DER Co-Simulation Platform for Testing Large-Scale DER Coordination Schemes

Distributed energy resources (DERs), such as responsive loads and energy storage systems, can help grid operators to more effectively balance supply and demand and manage network constraints. However, consumer acceptance of load coordination schemes depends on ensuring customer quality of service (QoS), thus requiring the careful management of device-level constraints. Simultaneously managing both device-level and system-level objectives is a difficult problem. Most potential solutions to the DER coordination problem are in some way distributed, with decision logic running both at the device (such as a thermostat) and the coordinator, with potentially complex communications between devices and one or more coordinators. It is difficult to effectively test distributed DER control schemes, particularly when there are thousands of devices with both device-level and grid-level constraints and asynchronous communications. This paper addresses this challenge by presenting a novel real-time grid-and-DER cyber-physical co-simulation platform for testing advanced DER coordination schemes and characterizing the behaviors of a DER fleet. The co-simulation platform is particularly suitable for: <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$i$ </tex-math></inline-formula> ) testing the real-time performance of a large fleet of DERs delivering advanced grid services; <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$ii$ </tex-math></inline-formula> ) characterizing the distribution of states for a large fleet of managed DERs; and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$iii$ </tex-math></inline-formula> ) incorporating practical limitations of DERs and communications and analyzing the effects on fleet-wide performance. To illustrate these benefits, we demonstrate how the platform can be used to test a fleet with thousands of DERs coordinated using Packetized Energy Management (PEM) and interacting with a realistic transmission system model.

Robust Networked Multiagent Optimization: Designing Agents to Repair Their Own Utility Functions

We study settings in which autonomous agents are designed to optimize a given system-level objective. In typical approaches to this problem, each agent is endowed with a decision-making rule that specifies the agent’s choice as a function of relevant information pertaining to the system’s state. The choices of other agents in the system comprise a key component of this information. This paper considers a scenario in which the designed decision-making rules are not implementable in the realized system due to discrepancies between the anticipated and realized information available to the agents. The focus of this paper is to develop methods by which the agents can preserve system-level performance guarantees in these unanticipated scenarios through local and independent redesigns of their own decision-making rules. First, we show a general impossibility result which states that in general settings, there are no local redesign methodologies that can offer any preservation of system-level performance guarantees, even when the affected agents satisfy an inconsequentiality criterion. However, we then show that when system-level objectives are submodular, local redesigns of utility functions do exist which allow nominal performance guarantees to degrade gracefully as information is denied to agents. That is, in these submodular settings, agents can adapt to informational inconsistencies independently without incurring much loss in terms of system-level performance.

The Effectiveness of Subsidies and Tolls in Congestion Games

Are rewards or penalties more effective in influencing user behavior? This article compares the effectiveness of subsidies and tolls in incentivizing user behavior in congestion games. The predominantly studied method of influencing user behavior in network routing problems is to institute taxes which alter users’ observed costs in a manner that causes their self-interested choices to more closely align with a system-level objective. Another conceivable method to accomplish the same goal is to subsidize the users’ actions that are preferable from a system-level perspective. We show that, when users behave similarly and predictably, subsidies offer superior performance guarantees to tolls under similar budgetary constraints; however, in the presence of unknown player heterogeneity, subsidies fail to offer the same robustness as tolls.

Forecast Competition in Energy Imbalance Market

Uncertainties in renewable generation make accurate load forecast essential for reliable power system operation. This paper considers energy imbalance market (EIM), where market players are allowed to procure energy ahead of time and trade the mismatch due to forecast error and strategic behaviors. The ISO sets the trading prices according to the market conditions, and pursues various system-level objectives. We first identify the power-law relationship between data volume and forecast accuracy, which enables the formulation of forecast cost model. Then, we cast the interactions in the EIM in the Stackelberg game framework with the ISO acting as the leader. We offer explicit subgame perfect equilibrium among the players in EIM, and derive the sufficient condition for the existence of unique equilibrium. Then, we show that this equilibrium, if exists, supports the maximal social welfare and under certain conditions, minimizes the total mismatch. We further examine the local and global impacts of the forecast errors under mild conditions, together with robustness analysis. Such analysis provides mechanism design guidelines for the ISO to enable the data sharing and forecast method sharing among market players in the EIM. Numerical studies further examine the effectiveness, robustness and sensitivity of the subgame.

Power-flow-based energy management of hierarchically controlled islanded AC microgrids

This paper presents a novel approach of employing hierarchical control to optimize the operation of islanded AC microgrids. The proposed method is an offline, centralized, power-flow-based energy management scheme which includes primary and secondary control dynamics in a modified power-flow formulation. The inner power-flow level maintains bus voltages and system frequency within the desired range and ensures power balance in the network. The outer optimization level ensures that the various components remain within their operational constraints, while optimizing a system-level objective. Two case studies are explored using a modified 14-bus medium voltage (MV) CIGRE benchmark microgrid to validate the proposed energy management algorithm. The first case includes minimization of conventional generator operating cost and renewable energy curtailment, both with linear and non-linear loads, and the second case includes minimization of conventional generator operating cost with load shedding. The results obtained from the case studies show the efficiency of the proposed energy management algorithm, and evidence its reliability for the optimal operation of islanded AC microgrids with multiple renewable energy sources.

A Graph-Based Approach for Mitigating Multi-Sided Exposure Bias in Recommender Systems

Fairness is a critical system-level objective in recommender systems that has been the subject of extensive recent research. A specific form of fairness is supplier exposure fairness, where the objective is to ensure equitable coverage of items across all suppliers in recommendations provided to users. This is especially important in multistakeholder recommendation scenarios where it may be important to optimize utilities not just for the end user but also for other stakeholders such as item sellers or producers who desire a fair representation of their items. This type of supplier fairness is sometimes accomplished by attempting to increase aggregate diversity to mitigate popularity bias and to improve the coverage of long-tail items in recommendations. In this article, we introduce FairMatch, a general graph-based algorithm that works as a post-processing approach after recommendation generation to improve exposure fairness for items and suppliers. The algorithm iteratively adds high-quality items that have low visibility or items from suppliers with low exposure to the users’ final recommendation lists. A comprehensive set of experiments on two datasets and comparison with state-of-the-art baselines show that FairMatch, although it significantly improves exposure fairness and aggregate diversity, maintains an acceptable level of relevance of the recommendations.

Implementation of Energy Management Scenarios in a DC Microgrid Using DC Bus Signaling

A DC microgrid with DC bus signaling (DBS) for power management offers a promising and cost-effective solution for reliable electricity access in rural and remote communities, particularly in developing nations. In this article, a three-layer modularized model of a microgrid is presented, with emphasis on the “Basic DC Microgrid Model” (Layer 1), which ensures effective power management in the microgrid at all times. Through the framework of the Layer 1 model, various energy management scenarios (EMSes) can be implemented, by varying the voltage regulation priorities of the entities. This allows us to achieve different system objectives such as reduced battery requirement or less dependence on the AC Grid. Two of the EMSes are discussed, implemented, and examined through both digital simulations and experimental tests on a laboratory prototype. The results show that the DBS scheme achieves real-time power management in a DC microgrid and that different EMSes can be easily implemented to realize varied system-level objectives.

Planning for resilience in power distribution networks: A multi‐objective decision support

Power grid response against high-impact low-probability (HILP) events could be enhanced by (a) hardening mechanisms to boost its structural resilience and (b) corrective recovery and mitigation analytics to improve its operational resilience. Planning for structural resilience and attempts to find the optimal location of the Tie switches in radially operated power distribution networks that enable harnessing the network topology for maximised resilience against HILP disasters are focussed. This goal is achieved through a novel resilience-oriented multi-objective decision making platform, which employs a k-PEM based probabilistic power flow (PPF) algorithm. The proposed framework offers a decision making analytic embedded with the fuzzy satisfying method (FSM) that characterises the system resilience features, such as robustness, restoration agility, load criticality, and recovered capacity, to assess different network reconfiguration options and select the optimal solution for implementation. The aforementioned resilience features are formulated in nodal level and then aggregated over the entire system to characterise the system-level objective functions. The performance of the suggested framework is analysed on the IEEE 33-Bus test system under a designated HILP event, and the applicability on larger networks has been verified on the IEEE 69-bus test system. The results demonstrate the efficacy and applicability of the proposed framework in boosting the network resilience against future extremes.

Utility Design for Distributed Resource Allocation—Part I: Characterizing and Optimizing the Exact Price of Anarchy

Game theory has emerged as a fruitful paradigm for the design of networked multiagent systems. A fundamental component of this approach is the design of agents' utility functions so that their self-interested maximization results in a desirable collective behavior. In this work we focus on a well-studied class of distributed resource allocation problems where each agent is requested to select a subset of resources with the goal of optimizing a given system-level objective. Our core contribution is the development of a novel framework to tightly characterize the worst case performance of any resulting Nash equilibrium (price of anarchy) as a function of the chosen agents' utility functions. Leveraging this result, we identify how to design such utilities so as to optimize the price of anarchy through a tractable linear program. This provides us with a priori performance certificates applicable to any existing learning algorithm capable of driving the system to an equilibrium. Part II of this work specializes these results to submodular and supermodular objectives, discusses the complexity of computing Nash equilibria, and provides multiple illustrations of the theoretical findings.

Risk-informed sensitivity analysis and optimization of seismic mitigation strategy using Gaussian process surrogate model

This work aims to develop a method to derive the optimal parameters of seismic mitigation devices for achieving the minimum system seismic risk, which is derived by accounting for the correlation between each individual components risk. Gaussian process surrogate model (GPSM) is used to illustrate the functional relationship between seismic risk and the variables of the mitigation devices, and then both sensitivity analysis and parameter optimization are carried out. Specifically, global sensitivity analysis (GSA), which is powerful for assessing the relative importance of parameters of the isolation devices, is achieved through the use of GPSM; the parameter optimization can be efficiently conducted to find the minimum seismic risk. A cable-stayed bridge model is established to demonstrate the application of the proposed GPSM-based method of sensitivity analysis and optimization of seismic isolation devices. The fluid viscous damper (FVD) which provides additional damper and stiffness is selected to illustrate the procedure of the multi-object optimization of seismic isolation devices on cable-stayed bridges. The seismic risk is calculated by integrating probabilistic seismic hazard analysis (PSHA), probabilistic seismic demand analysis (PADA), and probabilistic damage states, incorporating important sources of nonlinearity and uncertainties. Component-level objectives and one system-level object are considered, and the optimal parameters (damping coefficient Cd and exponent α) are determined. The results demonstrate that the GPSM-based is effective and efficient for sensitivity analysis and optimization of seismic isolation devices.

Runtime revision of sanctions in normative multi-agent systems

To achieve system-level properties of a multiagent system, the behavior of individual agents should be controlled and coordinated. One way to control agents without limiting their autonomy is to enforce norms by means of sanctions. The dynamicity and unpredictability of the agents’ interactions in uncertain environments, however, make it hard for designers to specify norms that will guarantee the achievement of the system-level objectives in every operating context. In this paper, we propose a runtime mechanism for the automated revision of norms by altering their sanctions. We use a Bayesian Network to learn, from system execution data, the relationship between the obedience/violation of the norms and the achievement of the system-level objectives. By combining the knowledge acquired at runtime with an estimation of the preferences of rational agents, we devise heuristic strategies that automatically revise the sanctions of the enforced norms. We evaluate our heuristics using a traffic simulator and we show that our mechanism is able to quickly identify optimal revisions of the initially enforced norms.

Individual homogenization in large-scale systems: on the politics of computer and social architectures

One determining characteristic of contemporary sociopolitical systems is their power over increasingly large and diverse populations. This raises questions about power relations between heterogeneous individuals and increasingly dominant and homogenizing system objectives. This article crosses epistemic boundaries by integrating computer engineering and a historicalphilosophical approach making the general organization of individuals within large-scale systems and corresponding individual homogenization intelligible. From a versatile archeological-genealogical perspective, an analysis of computer and social architectures is conducted that reinterprets Foucault’s disciplines and political anatomy to establish the notion of politics for a purely technical system. This permits an understanding of system organization as modern technology with application to technical and social systems alike. Connecting to Heidegger’s notions of the enframing (Gestell) and a more primal truth (anfänglicheren Wahrheit), the recognition of politics in differently developing systems then challenges the immutability of contemporary organization. Following this critique of modernity and within the conceptualization of system organization, Derrida’s democracy to come (à venir) is then reformulated more abstractly as organizations to come. Through the integration of the discussed concepts, the framework of Large-Scale Systems Composed of Homogeneous Individuals (LSSCHI) is proposed, problematizing the relationships between individuals, structure, activity, and power within large-scale systems. The LSSCHI framework highlights the conflict of homogenizing system-level objectives and individual heterogeneity, and outlines power relations and mechanisms of control shared across different social and technical systems.

Distributed Learning of Decentralized Control Policies for Articulated Mobile Robots

State-of-the-art distributed algorithms for reinforcement learning rely on multiple independent agents, which simultaneously learn in parallel environments while asynchronously updating a common, shared policy. Moreover, decentralized control architectures (e.g., CPGs) can coordinate spatially distributed portions of an articulated robot to achieve system-level objectives. In this work, we investigate the relationship between distributed learning and decentralized control by learning decentralized control policies for the locomotion of articulated robots in challenging environments. To this end, we present an approach that leverages the structure of the asynchronous advantage actor-critic (A3C) algorithm to provide a natural means of learning decentralized control policies on a single articulated robot. Our primary contribution shows individual agents in the A3C algorithm can be defined by independently controlled portions of the robot's body, thus enabling distributed learning on a single robot for efficient hardware implementation. We present results of closed-loop locomotion in unstructured terrains on a snake and a hexapod robot, using decentralized controllers learned offline and online respectively. Preprint of the paper submitted to the IEEE Transactions in Robotics (T-RO) journal in October 2018, and accepted for publication as a regular paper in May 2019.

Electrothermal Flexibility for Demand Response Using Inverse Simulation

This paper provides a hierarchical optimization algorithm based on the inverse simulation for the exploitation of the flexibility of electrothermal heating systems in office buildings in a demand response schema led by an aggregator. Inverse simulation, based on the approximate differentiation method, is applied on local building level to turn the customer's comfort specifications, i.e., indoor temperature comfort zone, into the technical power and flexibility specifications and, hence, acceptable power schedules. The aggregator then optimizes for a global system level objective. We prove that the inverse simulation yields the same result as the forward simulation approach, whereas it protects the privacy and thermal comfort of the customer. Moreover, via a general use case, we provide an insight on how to optimally balance demand and supply on the system level.