Cooperative Decision Research Articles

Coordinated Cooperative Distributed Decision-Making Using Synchronization of Local Plans

Centralized decision-making for a Networked Control System (NCS) suffers from a high computational burden on the planning agent. Distributed agents, which compute cooperative decision-making, increase computational performance. In cooperative decision-making, agents locally plan for a subset of all agents. Due to only local system knowledge of the agents, these local plans are inconsistent with the local plans of other agents. This inconsistency leads to the infeasibility of plans. This article introduces an algorithm for synchronizing local plans for cooperative distributed decision-making. The algorithm enables parallel decision-making and achieves feasible decisions. The algorithm consists of two iterative steps: local planning and global synchronization. In the local planning step, the agents parallelly compute local decisions, referred to as plans. Subsequently, consistency of the local plans across agents is achieved using global synchronization. The globally synchronized plans act as reference decisions for the local planning step in the next iteration. In each iteration, the local planning guarantees locally feasible plans, while the synchronization guarantees globally consistent plans in that iteration. The parallel algorithm converges to globally feasible decisions if the coupling topology is feasible. We introduce requirements for the coupling topology to achieve convergence to globally feasible decisions and present the algorithm using a model predictive control example. Our evaluations with car-like robots show that globally feasible decisions are achieved.

Cooperation without punishment

A fundamental question in social and biological sciences is whether self-governance is possible when individual and collective interests are in conflict. Free riding poses a major challenge to self-governance, and a prominent solution to this challenge has been altruistic punishment. However, this solution is ineffective when counter-punishments are possible and when social interactions are noisy. We set out to address these shortcomings, motivated by the fact that most people behave like conditional cooperators—individuals willing to cooperate if a critical number of others do so. In our evolutionary model, the population contains heterogeneous conditional cooperators whose decisions depend on past cooperation levels. The population plays a repeated public goods game in a moderately noisy environment where individuals can occasionally commit mistakes in their cooperative decisions and in their imitation of the role models’ strategies. We show that, under moderate levels of noise, injecting a few altruists into the population triggers positive reciprocity among conditional cooperators, thereby providing a novel mechanism to establish stable cooperation. More broadly, our findings indicate that self-governance is possible while avoiding the detrimental effects of punishment, and suggest that society should focus on creating a critical amount of trust to harness the conditional nature of its members.

Power Quality Improvement and Sustainable operation in A Standalone Micro Grid by Regulating Frequency in A Deregulated Market

Cooperative decision-making has always benefited from achieving objectives subjected to system constraints in a microgrid. This paper considers the power balance between generation and load as a significant power quality issue. Hence, frequency regulation is regarded as a primary challenge in the system. When the system operator works towards profit for the power producer, an integrated operation provides a solution. The sources considered in this system are Solar PhotoVoltaics (SPV), FuelCells (FC), Diesel Generators (DG) and Battery driven Electric Vehicles (BEV), where the BEVs operate in Vehicle microgrid mode. A central controller and local controllers are present to operate the generators at desired levels. A cascaded fuzzy controller is designed that chooses the best suitable BEV to be connected to the microgrid. The system is implemented in a MATLAB Simulation environment, and various scenarios and cases have been considered for evaluating the system response and its sustainability.

Leveraging Autonomous Vehicles to Tally Cooperative Driving Behavior

The number of Autonomous Vehicles (AVs) coexisting with conventional human-driven vehicles is expected to increase significantly in the coming years. This coexistence will last decades before full AV adoption is achieved worldwide. However, the cautious nature of AVs and the aggressive behavior of some human drivers could create unprecedentedly challenging scenarios for AVs, such as being stuck on merge lanes and blocked by human-driven vehicles. On the other hand, the cooperative behavior of other human drivers could assist AVs in avoiding deadlock situations. In this paper, we propose to leverage AVs to tally the cooperative driving behavior of human-driven vehicles. To this end, we model cooperative driving behavior in a “highway merge” scenario, which tends to be challenging for AVs. We vary the percentage of cooperative human-driven vehicles and estimate the percentage of AVs required to tally cooperative acts. Results show that when fifty percent of the human drivers cooperate, cooperation leads to statistically significant reductions of up to 68%, 46%, 38%, and 5% in stop delay, number of stops, vehicle delay, and travel time, respectively. Finally, we demonstrate that a 30% penetration of AVs is sufficient to tally up to 78% of cooperative behavior in highway scenarios. To promote cooperation across the population, our future work revolves around the construction of vehicular profiles based on their cooperative behavior. These profiles will be regularly updated and disseminated among AVs to aid their cooperative decisions toward human-driven vehicles in the upcoming interactions.

Immersive Extended Reality and Remote Sensing Technologies, Simulation Modeling and Spatial Data Acquisition Tools, and Cooperative Decision and Control Algorithms in a Real-Time Interoperable Decentralized Metaverse

Immersive Extended Reality and Remote Sensing Technologies, Simulation Modeling and Spatial Data Acquisition Tools, and Cooperative Decision and Control Algorithms in a Real-Time Interoperable Decentralized Metaverse

Energy Cooperation Among Sustainable Base Stations in Multi-Operator Cellular Networks

Energy Harvesting technology contributes significantly to green cellular networking by ensuring self-sustainability and extinguishing environmental hazards. Due to the imbalance between the harvested energy and traffic load of the base stations (BSs), energy cooperation has become a crucial requirement. However, the decision of optimal energy cooperation among the BSs in a multi-operator cellular network is a challenging task due to the consideration of various factors, such as cost, loss of energy, future information of traffic load, and harvested energy of the BSs, etc. The two conflicting objectives are minimizing the energy buying cost and the loss of energy while transferring through the power links. In this work, we present an optimal energy cooperation framework, formulated as a multi-objective linear programming (MOLP) problem which brings a trade-off between the two above-mentioned conflicting objectives considering the harvested energy and load of the BSs at future time slots. For the prediction of harvested energy of the BSs, we develop a Deep Q-Learning-based prediction method that intelligently increases measurement accuracy through continuous exploration and exploitation. The results of simulation experiments carried out in MATLAB depict that the proposed multi-operator energy cooperation framework outperforms state-of-the-art works in terms of cost, performance, and energy-loss reduction.

A Decision-making Method for Swarm Agents in Attack-defense Confrontation

The cooperative decision-making of swarm agents has attracted extensive attention from researchers due to its potential applications in multidisciplinary engineering problems. This paper studies a confrontation problem called asymmetric attack-defense confrontation (i.e., considering the difference in capability and quantity between agents and targets). The objective is to develop an effective self-organized swarm confrontation decision-making method. The decision-making process consists of task allocation decision and swarm motion decision. At each decision-making step, firstly, each agent forms a coalition with other agents autonomously by using a proposed hedonic coalition formation algorithm according to the attribute of targets. Thus, the agents assigned to the same target form a coalition, and swarm agents form several disjoint coalitions. Then, based on the allocated results, the agents are steered toward the corresponding target by a combat stimulus and a proposed selected interaction swarm algorithm. Finally, while the targets are within the agents’ attack radius, the agents execute the confrontation decision. Simulation results show the effectiveness of the designed method.

Cooperative Threat Engagement Using Drone Swarms

The ability of multiple manned and unmanned aircraft systems to cooperatively engage and disable an aerial threat plays a decisive role in modern warfare scenarios. In this paper, we apply key methods to enable the so-called cooperative threat engagement capability among multiple networked agents, e.g., a swarm of drones, with combat and communication capabilities. In particular, this research combines AI-based decision-making and control techniques for a swarm of loyal wingman drones to coordinate efficient defense actions in a cooperative and autonomous manner. We apply these concepts in a defense scenario that is modeled to analyze the loyal wingman concept, which we consider an interesting testbed for cooperative decision-making and low-level control techniques. The investigated methods were implemented in a realistic 3D UAV simulator for demonstration and evaluation.

Efficient Polar Coded Selective Decode-and-Forward with Cooperative Decision Threshold in Cooperative Multi-Relay Transmissions

In some satellite Internet of Things (IoT) devices with terrain shielding, the qualities of the direct source-destination (S-D) channel are poor, requiring cooperative communications with multi-relays to be employed. In order to solve error propagation of current decode-and-forward (DF) on such occasions, an efficient polar coded selective decode-and-forward (SDF) cooperation method is proposed with a new decision threshold derived from channel state information (CSI). First, the proposed threshold is derived from the CSI by exploiting the channel gain ratio of optimal relay-destination link (R-D) with source-relay (S-R) link. The above R-D link possesses good channel quality among all links in the system. Second, when the channel gain ratio of certain relay links is larger than the aforementioned decision threshold, the source and all these relays cooperatively send messages together to the destination to accomplish perfect SDF transmission. Otherwise, all relays are frozen and the messages are directly transmitted through the S-D link. If it fails anyway, a retransmission is subsequently tried in the next transmission cycle. In addition, a polar code for fading channels is designed and adaptively adjusted to a proper code rate according to channel quality to attain good bit error rate (BER) performance. Simulation results show that the proposed scheme achieves about 0.9 and 0.5 dB gain at BER of 10-4, respectively, in multi-relay cooperative communications with multi-path fading channels compared with those of non-cooperation and existing polar coded cooperation channels. Therefore, the proposed polar coded SDF (PCSDF) scheme can improve both the BER and the outage probability (OP) performance in multi-relay cooperative systems, making it quite suitable for heterogeneous network applications in cooperative satellite IoT systems involving sixth-generation (6G) communications.

Age-Oriented Access Control in GEO/LEO Heterogeneous Network for Marine IoRT: A Deep Reinforcement Learning Approach

With the growing interest in the smart ocean, the satellite-based marine Internet of Remote Things (IoRT) network has been regarded as a promising architecture for sensory data collection and transmission in infrastructure-limited offshore areas. In this article, we investigate the access control problem in the context of GEO/LEO heterogeneous IoRT networks, where multiple gateways are deployed to collect data generated by IoRT devices and then forward them to the terrestrial data center via satellite links. However, most existing access control strategies shed light on the traditional network performance (i.e., transmission delay and communication throughput) in single-layer satellite networks (i.e., low-Earth orbit (LEO) layer or geosynchronous orbit (GEO) layer), whereas the interplay between LEO and GEO layers and the freshness of information are rarely considered. To this end, we first formulate an age-oriented access control problem to minimize the long-term peak Age of Information (AoI) and transform it into a model-free Markov decision process (MDP). Then, a Deep-Double-Dueling- <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$Q$ </tex-math></inline-formula> -Learning (D3QN) policy is trained offline and can be deployed online to make decisions according to dynamic data arrivals and time-varying channels. Simulation results show that the proposed strategy significantly outperforms the state-of-the-art ones in terms of the long-term AoI performance. Furthermore, our strategy could make cooperative decisions for gateways and obtain a proper tradeoff between satellites on different layers.

Cooperative Transmission Mechanism Based on Revenue Learning for Vehicular Networks

With the rapid development of science and technology and the improvement of people’s living standards, vehicles have gradually become the main means of travel. The increase in vehicles has also brought about an increasing incidence of car accidents. In order to reduce traffic accidents, many researchers have proposed the use of vehicular networks to quickly transmit information. As long as these vehicles can receive information from other vehicles or buildings nearby in a timely manner, they can avoid accidents. In vehicular networks, the traditional double connection technique, through interference coordination scheduling strategy based on graph theory, can ensure the fairness of vehicles and obtain suitable neighborhood interference resistance with limited computing resources. However, when a base station transmits data to the vehicular user, the nearby base station and the vehicular network user may be in a state of suspended communication. Thus, the resource utilization of the above double connection vehicular network is not sufficient, resulting in a waste of resources. To solve this issue, this paper presents a study based on earnings learning with a vehicular network multi-point collaborative transmission mechanism, in which the vehicular network users communicate with the surrounding collaborative transmission. We use the Q-learning algorithm in the reinforcement learning process to enable vehicular network users to learn from each other and make cooperative decisions in different environments. In reinforcement learning, the agent makes a decision and changes the state of the environment. Then, the environment feeds back the benefit to the agent through the related algorithm so that the agent gradually learns the optimal decision. Simulation results demonstrate the superiority of our proposed approach with the revenue machine learning model compared with the benchmark schemes.

Shared Decision‐Making in Dementia: A Scoping Review

AbstractBackgroundEvidence supports that older adults with cognitive impairment can reliably communicate their values and choices, even as cognition declines. Shared decision‐making (SDM), including the patient, family members, and healthcare providers, is critical to patient‐centered care. The aim of this scoping review was to synthesize what is known about SDM in dementia.MethodA scoping review was completed in PubMed, Ovid, and CINAHL. Keywords included content areas of dementia and shared decision‐making. Inclusion criteria were as follows: description of shared or cooperative decision making, cognitively impaired patient population, adult patient, and original research. Review articles were excluded, and those for which the formal healthcare provider was the only team member involved in the decision‐making (e.g., physician) and/or the patient sample was not cognitively impaired. Systematically extracted data were organized in tables, compared, and synthesized to identify what is known about SDM in dementia.ResultThe search yielded 263 non‐duplicate articles that were screened by title and abstract. Ninety‐three articles remained, and the full text was reviewed; 32 articles were eligible for this review. Studies were from across Europe (n = 23), North America (n = 7), and Australia (n = 2). The majority of the articles used a qualitative study design (qualitative only, n = 13; mixed‐method, n = 9), and 10 used a quantitative study design (intervention, n = 1; quasi‐experimental, n = 1; descriptive, n = 8). Categories of similar SDM topics emerged, including health promotion, end‐of‐life, advanced care planning, and housing decisions. The majority of articles focused on SDM regarding health promotion for the patient (n = 16). Topics related to health promotion included SDM in general patient care, activities of daily living, clinical appointments, fall prevention, diagnostic testing, and medications. Dementia diagnoses varied within studies and across the literature, ranging from mild cognitive impairment to late‐stage Alzheimer’s disease to dementia with Lewy bodies.ConclusionFindings illustrate that SDM requires deliberate effort and is preferred among family members, healthcare providers, and patients with dementia. Future research should include more robust efficacy testing of decision‐making tools, incorporation of evidence‐based SDM approaches based on cognitive status/diagnosis, and consideration of geographical/cultural differences in healthcare delivery systems.

Intuitive cooperators: Time pressure increases children's cooperative decisions in a modified public goods game.

There is mounting empirical evidence to suggest that adults are intuitively cooperative. When presented with a cooperative dilemma between self-maximizing and benefitting the common good, decisions made quickly are more likely to be cooperative, whereas slow decisions tend to favor self-interest. To investigate the ontogenetic origins of intuitive cooperation, we examined the development of intuitive cooperation in middle childhood. We presented 150 children (7-12 years of age) with an online child-friendly public goods game where participants had a choice between giving two resources to themselves or four to their group. Participants were assigned to one of three decision time conditions; speeded, neutral, or delayed. We found that when decisions were speeded, children were more likely to cooperate compared to when decisions were unconstrained or delayed. Furthermore, children's intuitive choices only favored cooperation if they believed their peers were also cooperative. This pattern of findings held across the age range included in this study. Our findings suggest that in middle and late childhood, children are intuitively cooperative when making decisions to benefit the common good. HIGHLIGHTS: Time pressure increases children's cooperation in a public goods game, compared to when decisions are delayed or unconstrained. Between 7 and 12 years of age children engage in costly cooperation most of the time regardless of decision time. When children believe others are generally cooperative, their intuition is to cooperate. From middle to late childhood, intuitive decisions favor costly cooperation towards the common good.

CEAR: A cooperation based energy aware reward scheme for next generation green cognitive radio networks

Cognitive Radio (CR) networks are envisioned as a key empowering technology of the fifth-generation (5G) wireless communication networks, which solves the major issues of 5G, like high-speed data transmission, seamless connectivity, and increased demand for mobile data. Another significant characteristic of the 5G network is green communications, as energy consumption from the communication field is predicted to rise remarkably by the year 2030. In this work, we are concerned about energy-related issues and propose a cooperation-based energy-aware reward scheme (CEAR) for next-generation green CR networks. The proposed CEAR scheme is based on the antenna and temporal diversity of the primary users (PUs). For providing the service to the PUs, the users of another network called cognitive users (CUs) work as a cooperative relay node, and, in return, they get more spectrum access opportunities as a reward from the primary network. The CUs with delay-tolerant data packets take a cooperative decision by recognizing the availability and traffic load of PUs, channel state information, and data transmission requirements. We utilize the optimal stopping protocol for solving the decision-making problem and use the backward induction method to obtain the optimal cooperative solution. The simulation results reveal notable enhancements in energy efficiency (EE) of CUs compared with other cooperative schemes. The proposed CEAR scheme is more energy-efficient for ultra-dense network deployment because results show that the CU’s EE, spectral efficiency (SE), and throughput improved with the increase of PUs.

Dynamic Cooperation of the O2O Supply Chain Based on Time Delays and Bidirectional Free-Riding

Advertising and service investment can enhance brand goodwill to increase the sales of branded goods. However, the impact of advertising and services on brand goodwill is not immediate but delayed. At the same time, due to the different service characteristics provided by various channels, the phenomenon of bidirectional free-riding occurs. Therefore, this paper studies the dynamic cooperation between service and advertising in the O2O (online to offline) supply chain dominated by brand owners and explores the impacts of advertising, service delay and service free-riding among channels on the dynamic cooperation decisions of the O2O supply chain. A differential game model between brands and retailers is constructed by incorporating the delay effect and the bidirectional free-riding phenomenon. The optimal advertising and service strategies and performance problems of O2O supply chain enterprises under a centralized decision, brand cost-sharing decision and bilateral cost-sharing decision are compared and analyzed. The influence of delay time, showrooming and webrooming effects on the profit of each firm is investigated by example. The results show that the service strategy, advertising strategy and brand goodwill of the O2O supply chain members are optimal under a centralized decision. Still, the supply chain profit is not necessarily optimal under the delay time, showrooming and webrooming effect coefficients. Bilateral cost-sharing contracts can achieve Pareto improvement of supply chain performance. Appropriate setting of a bilateral cost-sharing ratio can adjust the adverse effects of delay and bidirectional free-riding. The long-term strategies to deal with the delay and bidirectional free-riding phenomena are as follows: the bilateral cost-sharing contract can improve corporate profits. Setting the wholesale price, online direct-selling price and service-sharing ratio by brand owners can effectively promote retailers’ investment in service, achieving a win-win situation. Retailers maintain high pricing and service levels to enhance the brand premium ability of physical stores and achieve long-term development.

Cooperative decision making for connected automated vehicles in multiple driving scenarios

AbstractTo improve the application range of decision‐making systems for connected automated vehicles, this paper proposes a cooperative decision‐making approach for multiple driving scenarios based on the combination of multi‐agent reinforcement learning with centralized planning. Specifically, the authors derived driving tasks from driving scenarios and computed the policy functions for different driving scenarios as linear combinations of policy functions for a set of specific driving tasks. Then, the authors classified vehicle coalitions according to the relationships between vehicles and used centralized planning methods to determine the optimal combination of actions for each coalition. Finally, the authors conducted tests in two driving scenarios considering different traffic densities to evaluate the performance of the developed approach. Simulation results demonstrate that the proposed approach exhibits good robustness in multiple driving scenarios while enabling cooperative decision making for connected automated vehicles, thereby ensuring safe and rational decision making.

The primate workplace: Cooperative decision-making in human and non-human primates

The success of group foraging in primates is not only determined by ecological and social factors. It is also influenced by cognition. Group foraging success is constrained, for instance, by the challenges of coordination, synchrony and decision-making, and it is enhanced by the ability to share, learn from others and coordinate actions. However, what we currently know about the cognition of individuals in groups comes primarily from experiments on dyads, and what we know of the effect of ecological factors on group dynamics comes from larger wild groups. Our current knowledge of primate group behaviour is thus incomplete. In this review, we identify a gap in our knowledge of primate group dynamics between the dyadic studies on primate cooperation and the large group observational studies of behavioural ecology. We highlight the potential for controlled experimental studies on coordination and cooperation in primate groups. Currently, these exist primarily as studies of dyads, and these do not go far enough in testing limits of group-level behaviours. Controlled studies on primate groups beyond the dyad would be highly informative regarding the bounds of non-human primate collaboration. We look to the literature on how humans behave in groups, specifically from organisational psychology, draw parallels between human and non-human group dynamics and highlight approaches that could be applied across disciplines. Organisational psychology is explicitly concerned with the interactions between individuals in a group and the emergent properties at the group-level of these decisions. We propose that some of the major shortfalls in our understanding of primate social cognition and group dynamics can be filled by using approaches developed by organisational psychologists, particularly regarding the effects of group size and composition on group-level cooperation. To illustrate the potential applications, we provide a list of research questions drawn from organisational psychology that could be applied to non-human primates.

Carbon emission reductions, pricing and social welfare of three-echelon supply chain considering consumer environmental awareness under carbon tax policy

With the rapid growth in carbon emissions from transport, reducing these emissions is becoming as important as reducing emissions from production. We took a transporter as a member of the supply chain system and constructed a three-echelon supply chain composed of a manufacturer, transporter, and retailer. This study applies the Stackelberg game and Nash game to research the optimal carbon emission reductions, pricing, and social welfare when the government imposes a carbon tax on carbon emitters and consumers are environmentally conscious. The four scenarios were designed as follows: 1) non-cooperative decision-making (NN model), in which neither carbon emission reduction cooperation nor pricing cooperation takes place; 2) local-cooperative decision-making Ⅰ (CN model), in which only cooperation in the field of carbon emission reductions takes place; 3) local-cooperative decision-making Ⅱ (NC model), in which cooperation in the field of pricing without cooperation in the area of carbon emission reductions takes place; and 4) overall-cooperative decision-making (CC model), in which both cooperation in the field of carbon emission reductions and cooperation in the area of pricing take place. The results show that economic (as profit of the supply chain system), social (as pricing and social welfare), and environmental benefits (as the optimal carbon emission reductions) for the three-echelon supply chain will improve with the deepening of cooperation among the supply chain members. Carbon tax policies have different impacts on clean and polluting supply chains. High carbon tax can encourage clean supply chain to increase carbon emission reduction, but will lead to the reduction of carbon emission of polluting supply chain. A cross-shareholding contract is designed to coordinate the supply chain and achieve the optimal state of the overall cooperative decision. Finally, the paper provides suggestions on carbon emission reductions for enterprises and policymaking for the government.

A Multiagent Cooperative Decision-Making Method for Adaptive Intersection Complexity Based on Hierarchical RL

In this paper, we propose a multiagent collaboration decision-making method for adaptive intersection complexity based on hierarchical reinforcement learning—H-CommNet, which uses a two-level structure for collaboration: the upper-level policy network fuses information from all agents and learns how to set a subtask for each agent, and the lower-level policy network relies on the local observation of the agent to control the action targets of the agents from each subtask in the upper layer. H-CommNet allows multiagents to complete collaboration on different time scales, and the scale is controllable. It also uses the computational intelligence of invehicle intelligence and edge nodes to achieve joint optimization of computing resources and communication resources. Through the simulation experiments in the intersection environment without traffic lights, the experimental results show that H-CommNet can achieve better results than baseline in different complexity scenarios when using as few resources as possible, and the scalability, flexibility, and control effects have been improved.

The global cooperation in asteroid mining based on AHP, entropy and TOPSIS

In this paper we introduce an equity evaluation index (EEI) to measure the global equity on asteroid mining, and present a global equity evaluation model by applying the analytic hierarchy process (AHP), entropy weight method and the technique for order preference by similarity to ideal solution (TOPSIS). Based on multi-objective optimization, we provide a global cooperative decision model to allocate the inputs and outputs among countries based on their strengths and demands. Furthermore, we analyze the the trends of EEI of each country under the influence of the outbreak events by using the autoregressive integrated moving average model. Our results show that among all countries with foundations in aerospace, China has the highest EEI while Russia ranks the lowest. Humans mining minerals in a cooperative manner will achieve a win-win situation.