Hierarchical Decision Process Research Articles

Provably Convergent Federated Trilevel Learning

Trilevel learning, also called trilevel optimization (TLO), has been recognized as a powerful modelling tool for hierarchical decision process and widely applied in many machine learning applications, such as robust neural architecture search, hyperparameter optimization, and domain adaptation. Tackling TLO problems has presented a great challenge due to their nested decision-making structure. In addition, existing works on TLO face the following key challenges: 1) they all focus on the non-distributed setting, which may lead to privacy breach; 2) they do not offer any non-asymptotic convergence analysis which characterizes how fast an algorithm converges. To address the aforementioned challenges, this paper proposes an asynchronous federated trilevel optimization method to solve TLO problems. The proposed method utilizes u-cuts to construct a hyper-polyhedral approximation for the TLO problem and solve it in an asynchronous manner. We demonstrate that the proposed u-cuts are applicable to not only convex functions but also a wide range of non-convex functions that meet the u-weakly convex assumption. Furthermore, we theoretically analyze the non-asymptotic convergence rate for the proposed method by showing its iteration complexity to obtain ϵ-stationary point is upper bounded by O(1/ϵ²). Extensive experiments on real-world datasets have been conducted to elucidate the superiority of the proposed method, e.g., it has a faster convergence rate with a maximum acceleration of approximately 80%.

Sliding-mode surface-based approximate optimal control for nonlinear multiplayer Stackelberg-Nash games via adaptive dynamic programming

This paper studies the sliding-mode surface (SMS)-based approximate optimal control issue for a class of nonlinear multiplayer Stackelberg-Nash games (MSNGs). First, considering different roles of the players in MSNGs, a hierarchical decision-making process is expressed as designing different cost functions for the leader and the followers. Therein, the leader makes its decision preferentially with consideration of all followers, while each follower responds optimally to leader’s strategy simultaneously via Nash games. Then, based on the sliding mode control technology, a novel robust control policy is proposed to solve the coupling Hamilton–Jacobi equations of nonlinear MSNGs. Subsequently, SMS-based critic network is utilized to approximate optimal control inputs for all players. Based on the Lyapunov stability theory, it is strictly proven that all signals of the closed-loop nonlinear systems are uniformly ultimately bounded. Finally, the effectiveness of the proposed method is demonstrated via a simulation example.

Science to practice – networked governance of sustainability transitions in an African university

PurposeThis paper aims to present insights on the governance of sustainability transitions in higher education in Africa. The authors interrogate the research literatures on the governance of socio-technical transitions in water, electricity, transport and waste management, and identify barriers and enabling factors that enhance transformative practices in universities.Design/methodology/approachThe analytical framework proposed in this paper combines the elements of governance network theory (GNT) and transition topology. The framework of this study is grounded in an actor-centric approach using GNT to understand networks conducive to sustainability transitions. Events and governance networks were mapped on a transition topology to visualise organisational and institutional changes over time. The study engaged students, management, academic and administrative staff in building a community of practice towards sustainability. This research is based on qualitative content analysis grounded in interview data, focus group discussions, workshops, webinars and secondary data analysis.FindingsThe findings show that the university has consolidated a sustainability vision and targets, but several factors prevent the community from achieving these targets, including hierarchical decision-making processes, a multitude of disjointed committees and fragmentation in the campus community.Originality/valueThis research adds to an emerging body of literature in the field of sustainability in higher education with two contributions. Firstly, the study presents a novel perspective(s) on the governance of sustainability transitions by combining the literatures on governance and sustainability transitions using a new methodological approach of transition topology to show organisational and institutional changes. Secondly, the study presents new empirical evidence for improving the governance of sustainability transitions in a diverse and highly unequal African university community in the process of (de)colonisation of knowledge and governance.

Application of Automatic Driving Task Sequentialisation Monitoring for Band Operator Robots

Abstract Robotics is increasingly advancing across diverse sectors of industry and daily life, with particularly promising applications in autonomous driving. This research develops a robotic autopilot task serialization management system and conducts an in-depth analysis of how robot path planning influences serialization monitoring. The path planning system (PPS) has been enhanced through the integration of a behavioral tree path planning model and hierarchical decision-making processes. The performance of the newly developed PPS algorithm is evaluated through comparative simulation experiments against established path planning algorithms, such as Particle Swarm Optimization (PSO) and Artificial Bee Colony (ABC) algorithms. The findings reveal that the PPS algorithm consistently achieves superior outcomes, planning paths with the shortest average length (188.673) and the lowest standard deviation (1.1547). Furthermore, it demonstrates optimal path efficiency and minimum energy consumption compared to four other migration algorithms. Additionally, the study observes the migration rates for node radii of 60 and 120 mm, which are 26% and 70%, respectively. Notably, as the number of robots and nodes increases, the migration rate stabilizes, underscoring the scalability and effectiveness of the PPS algorithm in complex environments.

Event-Triggered Robust Adaptive Dynamic Programming for Multiplayer Stackelberg-Nash Games of Uncertain Nonlinear Systems.

In this article, an event-triggered robust adaptive dynamic programming (ETRADP) algorithm is developed to solve a class of multiplayer Stackelberg-Nash games (MSNGs) for uncertain nonlinear continuous-time systems. Considering the different roles of players in the MSNG, the hierarchical decision-making process is described as the designed value functions for the leader and all followers, which assist to transform the robust control problem of the uncertain nonlinear system into an optimal regulation problem of the nominal system. Then, an online policy iteration algorithm is formulated to solve the derived coupled Hamilton-Jacobi equation. Meanwhile, an event-triggered mechanism is designed to alleviate computational and communication burdens. Moreover, critic neural networks (NNs) are constructed to obtain the event-triggered approximate optimal control polices for all players, which constitute the Stackelberg-Nash equilibrium of the MSNG. By using Lyapunov's direct method, the stability of the closed-loop uncertain nonlinear system is guaranteed under the ETRADP-based control scheme in the sense of uniform ultimate boundedness. Finally, a numerical simulation is provided to demonstrate the effectiveness of the present ETRADP-based control scheme.

Hamlet: Hierarchical Multi-Objective Reinforcement Learning Method for Charging Power Control of Electric Vehicles With Dynamic Quantity

Electric vehicle (EV) is emerging as an effective choice to reduce carbon emissions in the modern transportation system. However, the large access of EVs brings tremendous passive influence on the stability and performance of power grid operation. Thus, it is valuable to investigate the charging power management method to improve charging experience of EV drivers while maintaining normal bus voltage of the power system. This paper builds the charging power control problem of multiple EVs as a hierarchical Markov decision process (MDP) model and propose a hierarchical multi-objective reinforcement learning method (Hamlet) to obtain the real-time charging power control decisions. Specifically, by treating EVs with the same remaining charging time as the same agent, this paper addresses the dynamic state and action spaces issue and significantly reduce the input dimension of both state and action spaces. Besides, this paper quantifies drivers’ anxiety about the battery volume and distributes the comfort reward over the entire charging interval, which overcomes the delayed comfort reward issue. In order to alleviate the influence of multiple objectives on training stability, this paper develops a multi-objective learning method to dynamically adjust the optimization direction. Simulation results on IEEE 33 bus and 69 bus test feeders prove the validity of the proposed method in mitigating voltage fluctuation, minimizing bill payment and maximizing battery comfort.

Empowering education and workforce planning: A comprehensive approach to university recommendation and profession demand forecasting

This research paper presents a comprehensive approach to empower education and workforce planning through university recommendation and profession demand forecasting. With the increasing student migration for higher education, understanding and addressing the challenges posed by brain-drain and the lack of suitable policies is crucial. To facilitate informed decision-making, a user-centric approach is employed to develop a Personalized University Classification System, providing tailored recommendations. The Prompt Commentary Delivery component utilizes NLP techniques to offer timely feedback and suggestions. Moreover, a hierarchical decision-making process enhances university and program recommendations based on relevant features and user preferences. The Profession Demand Forecasting System employs the ARIMA model and predictive analytics to identify future employment trends, contributing to informed strategies aligned with labor market demands. By addressing these aspects, the research aims to provide stakeholders with valuable insights and strategies to foster economic growth and empower students in their educational journeys.

A Novel Machine-Learning Framework Based on a Hierarchy of Dispute Models for the Identification of Fish Species Using Multi-Mode Spectroscopy.

Seafood mislabeling rates of approximately 20% have been reported globally. Traditional methods for fish species identification, such as DNA analysis and polymerase chain reaction (PCR), are expensive and time-consuming, and require skilled technicians and specialized equipment. The combination of spectroscopy and machine learning presents a promising approach to overcome these challenges. In our study, we took a comprehensive approach by considering a total of 43 different fish species and employing three modes of spectroscopy: fluorescence (Fluor), and reflectance in the visible near-infrared (VNIR) and short-wave near-infrared (SWIR). To achieve higher accuracies, we developed a novel machine-learning framework, where groups of similar fish types were identified and specialized classifiers were trained for each group. The incorporation of global (single artificial intelligence for all species) and dispute classification models created a hierarchical decision process, yielding higher performances. For Fluor, VNIR, and SWIR, accuracies increased from 80%, 75%, and 49% to 83%, 81%, and 58%, respectively. Furthermore, certain species witnessed remarkable performance enhancements of up to 40% in single-mode identification. The fusion of all three spectroscopic modes further boosted the performance of the best single mode, averaged over all species, by 9%. Fish species mislabeling not only poses health-related risks due to contaminants, toxins, and allergens that could be life-threatening, but also gives rise to economic and environmental hazards and loss of nutritional benefits. Our proposed method can detect fish fraud as a real-time alternative to DNA barcoding and other standard methods. The hierarchical system of dispute models proposed in this work is a novel machine-learning tool not limited to this application, and can improve accuracy in any classification problem which contains a large number of classes.

Stackelberg–Nash game approach for price-based demand response in retail electricity trading

This paper studies the price-based demand response problem in a deregulated retail electricity trading, aiming to coordinate the energy consumption behavior of end-users under dynamic retail prices. The challenge here is that in addition to the hierarchical decision-making process between utility company and end-users considered in existing works, the non-cooperative and competitive interdependence among end-users cannot be ignored. To address this issue, we first construct a novel Stackelberg–Nash game, in which the Stackelberg game is used to capture the hierarchical decision-making process between utility company and end-users, while the Nash game is dedicated to describing the interdependence among end-users. Then the existence and uniqueness of the Stackelberg–Nash equilibrium is provided along with theoretical analysis. On the basis of the analysis of equilibrium, we propose a distributed iterative algorithm with an adaptive step size, which is benchmarked with a fixed step-size algorithm. The comparison results on a real-life residential retail electricity market show that our proposed algorithm has better performance in terms of effectiveness and scalability.

Importance Performance Competitor Analysis for Comprehensive Assessment of National Forest Visitor Satisfaction

Public lands are an important source of outdoor recreation opportunities. These opportunities provide a variety of public benefits, including promoting physical and mental well-being, contributing to local economies, and raising conservation awareness. In response to current and potential demand, it is ever more important to meet and anticipate infrastructure and user needs. With aging infrastructure (e.g., trails, access points, parking lots), managers face the need to maintain existing, and establish new, recreation opportunities to meet user expectations and contribute to positive perceptions of recreation offerings within the realities of budgetary challenges. Compounding this difficulty is the frequent presence of hierarchical decision-making process within public agencies providing recreation opportunities. For example, within the U.S. Forest Service National Forest System (NFS), recreation investment decisions, such as those under the Great American Outdoors Act, are often made at the regional administrative level after considering the needs and conditions across multiple NFS units within the region. While certain aspects of infrastructure can be measured for quality (i.e., physical attributes like trail conditions), it is difficult to quantify psychological elements of user satisfaction. Recreationist satisfaction is important to promoting supportive and engaged users of public lands. Therefore, a priority for management is understanding how user experiences compare to expectations. Direct ratings from surveys can be useful proxies of user perceptions. For decades, Importance-Performance Analysis (IPA) has translated importance and satisfaction ratings to action plans, a valuable management tool in valid application. However, IPA treats each unit as an “island,” and doesn’t fully address the challenge of comparing conditions and needs across units. A variation of IPA, known as Importance Performance Competitor Analysis (IPCA), compares a focal unit against a competitor. Using the National Visitor Use Monitoring Program (NVUM) satisfaction survey module, we adapted IPCA to the NFS regional organizations. Rather than analyzing unit versus competitor, we assessed each unit against its complementary regional aggregate over several points in time. A classification-based percentage identified the highest priorities regionally, selecting attributes for follow-up with IPA to identify relevant units for management. For two NFS regions, we present examples of translating survey output to reports meaningful for management, either as a snapshot in time or across multiple time points. These methodologies are adaptable to other unit networks under the same managerial oversight, with Likert scale data collected on multiple time periods. The primary objective is to translate survey data to recommended actions over a hierarchical network, which can guide managers in prioritizing needs and sustainable planning. In our examples, we identified region-level priorities (i.e., attributes) by site type, followed by use of a longstanding methodology (IPA) for site-level identification potentially useful to managers.

End-to-End Entity Linking with Hierarchical Reinforcement Learning

Entity linking (EL) is the task of linking the text segments to the referring entities in the knowledge graph, typically decomposed into mention detection, and entity disambiguation. Compared to traditional methods treating the two tasks separately, recent end-to-end entity linking methods exploit the mutual dependency between mentions and entities to achieve better performance. However, existing end-to-end EL methods have problems utilizing the dependency of mentions and entities in the task. To this end, we propose to model the EL task as a hierarchical decision-making process and design a hierarchical reinforcement learning algorithm to solve the problem. We conduct extensive experiments to show that the proposed method achieves state-of-the-art performance in several EL benchmark datasets. Our code is publicly available at https://github.com/lhlclhl/he2eel.

Two-Timescale Learning-Based Task Offloading for Remote IoT in Integrated Satellite–Terrestrial Networks

In this paper, we propose an integrated satellite-terrestrial network (ISTN) architecture to support delay-sensitive task offloading for remote Internet of Things (IoT), in which satellite networks serve as a complement to terrestrial networks by providing additional communication resources, backhaul capacities, and seamless coverage. Under this architecture, we investigate how to jointly make offloading link selection and bandwidth allocation decisions for BSs and IoT users. Considering the differentiated decision-making time granularities, we formulate a two-timescale stochastic optimization problem to minimize the overall task offloading delay. To accommodate the two-timescale network dynamics and characterize state-action relations, we establish a hierarchical Markov decision process (H-MDP) framework with two separate agents tackling two-timescale network management decisions, and two evolved MDP-based subproblems are formulated accordingly. To efficiently solve the subproblems, we further develop a hybrid proximal policy optimization (H-PPO)-based algorithm. Specifically, a hybrid actor-critic architecture is designed to deal with the mixed discrete and continuous actions. In addition, an action mask layer and an action shaping function are designed to sample feasible task offloading decisions from the time-variant action set. Extensive simulation results have validated the superiority of the proposed ISTN architecture and the H-PPO-based algorithm, especially in scenarios with scarce spectrum resources and heavy traffic loads.

Why there is no need to use a big-M in linear bilevel optimization: a computational study of two ready-to-use approaches

Linear bilevel optimization problems have gained increasing attention both in theory as well as in practical applications of Operations Research (OR) during the last years and decades. The latter is mainly due to the ability of this class of problems to model hierarchical decision processes. However, this ability makes bilevel problems also very hard to solve. Since no general-purpose solvers are available, a “best-practice” has developed in the applied OR community, in which not all people want to develop tailored algorithms but “just use” bilevel optimization as a modeling tool for practice. This best-practice is the big-M reformulation of the Karush–Kuhn–Tucker (KKT) conditions of the lower-level problem—an approach that has been shown to be highly problematic by Pineda and Morales (2019). Choosing invalid values for M yields solutions that may be arbitrarily bad. Checking the validity of the big-Ms is however shown to be as hard as solving the original bilevel problem in Kleinert et al. (2019). Nevertheless, due to its appealing simplicity, especially w.r.t. the required implementation effort, this ready-to-use approach still is the most popular method. Until now, there has been a lack of approaches that are competitive both in terms of implementation effort and computational cost. In this note we demonstrate that there is indeed another competitive ready-to-use approach: If the SOS-1 technique is applied to the KKT complementarity conditions, adding the simple additional root-node inequality developed by Kleinert et al. (2020) leads to a competitive performance—without having all the possible theoretical disadvantages of the big-M approach.

A strategic decision framework using soft-computing for agri-food production: case study living lab in universities

Designing autonomous or semi-autonomous greenhouses that can supply food under extreme environmental conditions or restricted social distances is an endeavor that has to be considered under pandemic conditions such as COVID-19. However, generally advanced greenhouses have been designed using conventional methodologies that are not integrated easily into reconfigurable designs. Moreover, those design methodologies are complex for novice product designers. This paper proposes a novel SDF (Strategic Decision Framework) to support reconfigurable agri-food production systems design. The framework proposed is based on the Integrated Product, Process, and Manufacturing System Development (IPPMD) reference model that uses reconfigurable manufacturing systems (RMS) and Fuzzy Cluster Mean (FCM) algorithms in its decision support system. As a result, the proposed methodology generates fuzzy clusters using degrees of membership that can describe the design constraints straightforwardly. Those fuzzy clusters support a hierarchical decision-making process, so the design process is easily implemented. Besides, the proposed methodology is deployed in a complex, highly non-linear system (a greenhouse) that has an internal ecosystem autonomously controlled by mechanical, electrical, digital, and telecommunication subsystems. Hence, an innovative design methodology implemented for advanced reconfigurable systems is presented. The results confirm that the proposed SDF can be implemented in complex reconfigurable design systems when the manufacturing decisions are unclear to decision-makers and designers. Thus, this methodology provides useful, coherent information regarding the design process that simplifies decision-making when designing a reconfigurable greenhouse. Besides, this research shows an entirely reconfigurable greenhouse as a living lab implemented at Tecnologico de Monterrey, Mexico City campus to validate the proposed SDF.

The market effectiveness of regulatory certification for sustainable food supply: A conjoint analysis approach

The commitment to sustainability follows the consensus of adopting integrated governance to mitigate uncertainty about the whole food system. Thus, a coherent intermediary framework emerges in response to the food system crisis, pledging a “farm to fork” regulatory certification with enhanced credibility and reduced search cost compared with the fragmented intermediation. Yet, the institutional practice is scarce, and the market effectiveness remains underexamined. Drawing upon the heuristic-systematic model from the dual-process theory, this research builds the view of sustainable choice as a hierarchical decision process. The regulatory certification enacts an easy but reliable heuristic judgment, establishing an adaptive base for further systematic decisions. Specifically, common barriers (such as the status quo bias and price sensitivity) to sustainable action are not firmly established as typically assumed; instead, consumer reluctance to action arises from the lacking of a dominant heuristic. Consumers with an available regulatory certification experience less difficult trade-offs and have stronger preferences for technology innovation, such as digital production for food industry 4.0. Potentially, the segment that values regulatory certification as the crucial heuristic accounts for approximately 50%. This research provides insightful implications on how governance reforms spark a shift to sustainable food supply and the market effectiveness of inviting consumers to join the initiatives.

On a Class of Multistage Stochastic Hierarchical Problems

In this paper, following the multistage stochastic approach proposed by Rockafellar and Wets, we analyze a class of multistage stochastic hierarchical problems: the Multistage Stochastic Optimization Problem with Quasi-Variational Inequality Constraints. Such a problem is defined in a suitable functional setting relative to a finite set of possible scenarios and certain information fields. The key of this multistage stochastic hierarchical problem turns out to be the nonanticipativity: some constraints have to be included in the formulation to take into account the partial information progressively revealed. In this way, we are able to study real-world problems in which the hierarchical decision processes are characterized by sequential decisions in response to an increasing level of information. As an application of this class of multistage stochastic hierarchical problems, we focus on the study of a suitable Single-Leader-Multi-Follower game.

A bi-level multi-objective programming model for water resources management under compound uncertainties in Dongjiang River Basin, Greater Bay Area of China

To facilitate regional water resources allocation, an integrated bi-level multi-objective programming (IBMP) model with dual random fuzzy variables was developed in this research The proposed model was derived through incorporating dual random fuzzy variables, multi-objective programming, and interval parameter programming within a bi-level optimization framework. This approach improved upon the previous bi-level programming methods and had two advantages. Firstly, it was capable of reflecting tradeoffs among multiple conflict preferences for water related bi-level hierarchical decision-making processes. Secondly, random fuzzy variables were used to tackle the dual uncertainties in both sides of the constraints, which were characterize as probability density functions and discrete intervals. Then, a real-world water resources planning problem was employed for illustrating feasibility of the application of IBMP model in Dongjiang river watershed of south China. Results reflected the alternative decisions for water allocation schemes under a set of probability levels and fuzzy α - cut levels, which can support in-depth analysis of tradeoffs among multiple levels and objective values. Moreover, modeling comparison analysis was undertaken to illustrate the performances of the proposed model.

Step by step: A hierarchical framework for multi-hop knowledge graph reasoning with reinforcement learning

Recently, knowledge graph reasoning has sparked great interest in research community, which aims at inferring missing information in triples and provides critical support to various tasks (e.g., question answering and recommendation). To date, multi-hop reasoning is a dominant approach which infers the target answer by walking along the path connecting entities and relations, ensuring both accuracy and interpretability. However, in most knowledge graphs, there are multiple relations related to an identical entity, and multiple tail entities for an identical pair of head entity and relation. Due to this one-to-many dilemma, enlarged action space and ignoring logical relationship between entity and relation increase the difficulty of learning. In order to deal with such an issue, this work presents a novel paradigm for knowledge graph reasoning by decomposing it to a two-level hierarchical decision process. We apply the hierarchical reinforcement learning framework which dismantles the task into a high-level process for relation detector and a low-level process for entity reasoning, respectively. In this way, the action space is effectively controlled where the policies can be optimized. The interactions between entity and relation decision enhance the rationality of reasoning. Moreover, we introduce a dynamic prospect mechanism for low-level policy where the information can guide us to a refined and improved action space, assisted by embedding based method. Our proposed model is evaluated on four benchmark datasets and the results validate its superiority over state-of-the-art baselines, showing the interpretability of reasoning process simultaneously.

An Analysis of Pecking Order Theory With the Analytic Hierarchy Process

According to pecking order theory (POT), when a company feels the need for capital, it applies the hierarchies of accounts payable and accruals, retained earnings, debt, and new common stock. Similarly, the analytic hierarchy process (AHP) method calculates the hierarchical priorities for the solution of complex problems. The most important justification explaining the hierarchy implementation of the POT approach are the impacts of asymmetrical information and signaling. Similarly, in the AHP method, the hierarchical decision-making process depends on the justification of the decision maker. The existence of asymmetrical information shows that the two methods complete each other. The aim of the study was to analyze POT with the AHP method, in other words, an examination of the consistency of the POT and AHP approaches with each other. The results showed that the two methods are indeed consistent with each other.

Benders Subproblem Decomposition for Bilevel Problems with Convex Follower

Bilevel optimization formulates hierarchical decision-making processes that arise in many real-world applications, such as pricing, network design, and infrastructure defense planning. In this paper, we consider a class of bilevel optimization problems in which the upper level problem features some integer variables and the lower level problem enjoys strong duality. We propose a dedicated Benders decomposition method for solving this class of bilevel problems, which decomposes the Benders subproblem into two more tractable, sequentially solvable problems that can be interpreted as the upper and lower level problems. We show that the Benders subproblem decomposition carries over to an interesting extension of bilevel problems, which connects the upper level solution with the lower level dual solution, and discuss some special cases of bilevel problems that allow sequence-independent subproblem decomposition. Several novel schemes for generating numerically stable cuts, finding a good incumbent solution, and accelerating the search tree are discussed. A computational study demonstrates the computational benefits of the proposed method over a state-of-the-art, bilevel-tailored, branch-and-cut method; a commercial solver; and the standard Benders method on standard test cases and the motivating applications in sequential energy markets.