Dynamic Problem Research Articles

A reinforcement learning approach for the online dynamic home health care scheduling problem.

Over recent years, home health care has gained significant attention as an efficient solution to the increasing demand for healthcare services. Home health care scheduling is a challenging problem involving multiple complicated assignments and routing decisions subject to various constraints. The problem becomes even more challenging when considered on a rolling horizon with stochastic patient requests. This paper discusses the Online Dynamic Home Health Care Scheduling Problem (ODHHCSP), in which a home health care agency has to decide whether to accept or reject a patient request and determine the visit schedule and routes in case of acceptance. The objective of the problem is to maximize the number of patients served, given the limited resources. When the agency receives a patient's request, a decision must be made on the spot, which poses many challenges, such as stochastic future requests or a limited time budget for decision-making. In this paper, we model the problem as a Markov decision process and propose a reinforcement learning (RL) approach. The experimental results show that the proposed approach outperforms other algorithms in the literature in terms of solution quality. In addition, a constant runtime of less than 0.001 seconds for each decision makes the approach especially suitable for an online setting like our problem.

Dynamic Supplier Selection and Its Optimal Strategy Considering Full Truck Load and Fuzzy Demand using Fuzzy Expected Value-Based Programming

This article proposes a linear integer optimization model incorporating fuzzy parameters to find the optimal solution for a dynamic supplier selection problem with uncertain demand. The uncertain demand value is represented using a fuzzy variable. A fuzzy expected value-based linear optimization solver is used to address the optimization problem, to minimize the total cost under fuzzy demand values. Several computational experiments were conducted to evaluate and analyze the model. The results show that the proposed model effectively identifies the optimal suppliers for each product. Additionally, the model determines the optimal purchase volumes for each product type from the selected suppliers, leading to the minimal total expected cost.

Quantitative Convergence of a Discretization of Dynamic Optimal Transport Using the Dual Formulation

AbstractWe present a discretization of the dynamic optimal transport problem for which we can obtain the convergence rate for the value of the transport cost to its continuous value when the temporal and spatial stepsize vanish. This convergence result does not require any regularity assumption on the measures, though experiments suggest that the rate is not sharp. Via an analysis of the duality gap we also obtain the convergence rates for the gradient of the optimal potentials and the velocity field under mild regularity assumptions. To obtain such rates, we discretize the dual formulation of the dynamic optimal transport problem and use the mature literature related to the error due to discretizing the Hamilton–Jacobi equation.

The Implementation of MBKM at FKIP Muhammadiyah University Bengkulu

The Minister of Education and Culture’s policy through Minister of Education and Culture Regulation Number 3 of 2020 concerning Higher Education Standards regarding Independent Learning on Independent Campuses (MBKM) seeks to give students the freedom to study in higher education as a form of learning innovation to obtain quality learning. The aim of the research is to find out how the existence of independent learning campus management education at the Faculty of Teacher Training and Education, Muhammadiyah University of Bengkulu, can be implemented well and what is the theory and practice in the learning process as seen from the four management functions, namely planning, organizing, implementing, and monitoring, which is usually abbreviated as POAC. This research is research with social and dynamic problems so that the researcher determines the use of descriptive qualitative research to search, collect, process, and analyze research data. The data sources in this research or those referred to as informants, are Deputy Dean 1 of FKIP UMB, Head of Study Program within FKIP, and students. Then non-human data sources include field notes, documents, and recorded interviews. Based on the results of the research and discussion, it is known that overall, it has gone quite well. However, from the evaluation results, there were still several problems encountered, especially regarding the conversion of grades from the MBKM program that students participated in. This is material for evaluation to improve in the future, especially in terms of the implementation of the MBKM Curriculum which is currently running.

Problem of chaotic dynamics of polymer chain with a partly bounded interaction potential

Problem of chaotic dynamics of polymer chain with a partly bounded interaction potential

Dynamic Resource Allocation: The Geometry and Robustness of Constant Regret

We study a family of dynamic resource allocation problems, wherein requests of different types arrive over time and are accepted or rejected. Each request type is characterized by its reward, arrival probability, and resource consumption. An upper bound for the collected reward is given by a linear optimization problem with a random right-hand side. This type of problem, known as packing linear program (LP), is ubiquitous in resource allocation. We provide a detailed characterization of the parametric structure of this packing LP. Relying on this geometric understanding, we revisit and expand on BudgetRatio algorithms that achieve constant regret by resolving this same packing LP in each period and accepting requests scored as sufficiently valuable. We illustrate the benefits of the geometric view in proving that (i) BudgetRatio achieves constant regret relative to the offline (full information) upper bound in the presence of inventory that is (slowly) restocked, and (ii) within explicitly identifiable bounds, the algorithm’s regret is robust to misspecification of the model parameters. This gives bounds for the bandits version of the problem in which the parameters have to be learned. (iii) The algorithm has an equivalent formulation as a generalized bid-price algorithm in which the bid prices can be adaptively and efficiently computed. Our analysis focuses on the evolution of the remaining inventory—in turn of the LP that drives BudgetRatio—as a stochastic process. We prove that it is attracted to sticky regions of the state space in which the online algorithm takes actions consistent with the optimal basis of the offline upper bound, a basis that is revealed only in hindsight at the horizon’s end. Funding: This work was supported by the U.S. Department of Defense [Grant W911NF-20-C-0008].

Reinforcement learning for dynamic pricing and capacity allocation in monetized customer wait-skipping services

ABSTRACT We consider how to facilitate a dynamically-priced premium service option that enables customer parties to shorten their wait in a queue. Offering such a service requires that some of a business’s capacity be reserved continuously and kept ready for premium customers. In tandem with capacity reservation, pricing must be coordinated. Hence, a joint dynamic pricing and capacity allocation problem lies at the heart of this service. We propose a conceptual solution architecture and employ Proximal Policy Optimization (PPO) for dynamic pricing and capacity allocation to maximize total revenue. Simulation experiments over multiple scenarios compared PPO against a human-engineered policy and a baseline policy having no premium option. The human-engineered policy led to significantly greater revenues than the baseline policy in each scenario, illustrating the potential increase in revenues afforded by this concept. The PPO agent substantially improved upon the human-engineered policy advantage, with improvements ranging from 28% to 161%.

A Non-Probabilistic Strategy to Investigate Imprecisely Defined Nonlinear Eigenvalue for Structural Dynamic Problems

Structural dynamic problems are influenced by many parameters which are uncertain in nature. As such, for better estimation, the problems can be considered with epistemic-type uncertainties. Due to the epistemic type of uncertainties, the modeled system appears in the form of an imprecise nonlinear eigenvalue problem. Therefore, this paper proposes a novel nonprobabilistic approach to analyze nonlinear eigenvalue problems in structural dynamics, particularly when dealing with imprecisely defined parameters. In order to solve the fuzzy nonlinear eigenvalue models, the same is to be transformed into a fuzzy eigenvalue problem. Traditional methods often rely on probabilistic frameworks, which may not adequately capture uncertainties inherent in real-world structural systems. The proposed approach leverages concepts from fuzzy set theory to handle imprecise data, providing a more robust and realistic analysis. Then, the developed method is used to solve a case study of structural dynamic problem. Here, a spring-mass system is demonstrated through the developed technique to quantify the uncertain field variables. Based on the imprecise system parameters, different fuzzy models are discussed in various cases and results are reported here. This framework offers valuable insights for engineers and researchers working on dynamic analysis of complex structures, paving the way for more reliable and informed decision-making in structural engineering applications.

Adaptive distributed dynamic average consensus with prescribed performance

AbstractA novel adaptive distributed estimation algorithm is developed for dynamic average consensus problem. The algorithm enables a group of agents to accurately track the average value of signals that are available locally to each agent. Unlike most of existing approaches that rely on error transformation techniques, our algorithm employs an adaptive‐gain based barrier function to simultaneously meet prescribed performance specifications of transient and steady state. It is both simple and easy to implement and is not dependent on bounded reference signals and their derivatives. Additionally, it can operate under less strict assumptions than those that assume absolute bounds on the signals. Simulation results are presented to demonstrate the effectiveness of the developed algorithm.

Dynamic multi-objective optimization based on classification response of decision variables

In recent years, many dynamic multi-objective optimization algorithms (DMOAs) have been proposed to address dynamic multi-objective optimization problems (DMOPs). Most existing DMOAs treat all decision variables uniformly and respond to them in an identical manner. This paper proposes a dynamic multi-objective optimization algorithm based on the classification response of decision variables (CRDV-DMO). Firstly, CRDV-DMO categorizes the decision variables into convergence variables and diversity variables. Different decision variables adopt distinct response strategies. The response strategy of diversity variable (RSDV) uses Latin hypercube sampling to generate the diversity variables of the new environment. For each dimensional convergence variable, the response strategy of convergence variable (RSCV) first evaluates whether the basic center prediction strategy (CPS) yields positive feedback or negative feedback, further determining the predictability of that dimensional convergence variable. RSCV then decides to either use the basic CPS to generate the convergence variable for that dimension or to retain that dimensional convergence variable from the current environment, based on the predictability of that dimensional convergence variable. The proposed algorithm is extensively studied through comparison with several advanced DMOAs, demonstrating its effectiveness in dealing with the benchmark DMOPs and the parameter-tuning problem of the PID controller on a dynamic system.

Path Tracking Control for Four-Wheel Independent Steering and Driving Vehicles Based on Improved Deep Reinforcement Learning

We propose a compound control framework to improve the path tracking accuracy of a four-wheel independent steering and driving (4WISD) vehicle in complex environments. The framework consists of a deep reinforcement learning (DRL)-based auxiliary controller and a dual-layer controller. Samples in the 4WISD vehicle control framework have the issues of skewness and sparsity, which makes it difficult for the DRL to converge. We propose a group intelligent experience replay (GER) mechanism that non-dominantly sorts the samples in the experience buffer, which facilitates within-group and between-group collaboration to achieve a balance between exploration and exploitation. To address the generalization problem in the complex nonlinear dynamics of 4WISD vehicles, we propose an actor-critic architecture based on the method of two-stream information bottleneck (TIB). The TIB method is used to remove redundant information and extract high-dimensional features from the samples, thereby reducing generalization errors. To alleviate the overfitting of DRL to known data caused by IB, the reverse information bottleneck (RIB) alters the optimization objective of IB, preserving the discriminative features that are highly correlated with actions and improving the generalization ability of DRL. The proposed method significantly improves the convergence and generalization capabilities of DRL, while effectively enhancing the path tracking accuracy of 4WISD vehicles in high-speed, large-curvature, and complex environments.

Equilibrium Control in Uncertain Linear Quadratic Differential Games with V-Jumps and State Delays: A Case Study on Carbon Emission Reduction

Uncertainty, time delays, and jumps often coexist in dynamic game problems due to the complexity of the environment. To address such issues, we can utilize uncertain delay differential equations with jumps to depict the dynamic changes in differential game problems that involve uncertain noise, delays, and jumps. In this paper, we first examine a linear quadratic differential game optimistic value problem within an uncertain environment characterized by jumps and delays. By applying the Z(x,y) transform, we convert the infinite-dimensional problem into a finite-dimensional one. We then demonstrate that the condition for the existence of a Nash equilibrium strategy is equivalent to the existence of solutions to two cross-coupled matrix Riccati equations. Furthermore, we explore the saddle point equilibrium strategy of the linear quadratic differential game optimistic value model and derive the corresponding saddle point equilibrium solution. Finally, we apply our results to solve a carbon emission reduction game problem.

Families of Planar Orbits in Polar Coordinates Compatible with Potentials

In light of the planar inverse problem of Newtonian Dynamics, we study the monoparametric family of regular orbits f(r,θ)=c in polar coordinates (where c is the parameter varying along the family of orbits), which are generated by planar potentials V=V(r,θ). The corresponding family of orbits can be uniquely represented by the “slope function” γ=fθfr. By using the basic partial differential equation of the planar inverse problem, which combines families of orbits and potentials, we apply a new methodology in order to find specific potentials, e.g., V=A(r)+B(θ) or V=H(γ) and one-dimensional potentials, e.g., V=A(r) or V=G(θ). In order to determine such potentials, differential conditions on the family of orbits f(r,θ) = c are imposed. If these conditions are fulfilled, then we can find a potential of the above form analytically. For the given families of curves, such as ellipses, parabolas, Bernoulli’s lemniscates, etc., we find potentials that produce them. We present suitable examples for all cases and refer to the case of straight lines.

Dynamic Integrated Scheduling of Production Equipment and Automated Guided Vehicles in a Flexible Job Shop Based on Deep Reinforcement Learning

The high-quality development of the manufacturing industry necessitates accelerating its transformation towards high-end, intelligent, and green development. Considering logistics resource constraints, the impact of dynamic disturbance events on production, and the need for energy-efficient production, the integrated scheduling of production equipment and automated guided vehicles (AGVs) in a flexible job shop environment is investigated in this study. Firstly, a static model for the integrated scheduling of production equipment and AGVs (ISPEA) is developed based on mixed-integer programming, which aims to optimize the maximum completion time and total production energy consumption (EC). In recent years, reinforcement learning, including deep reinforcement learning (DRL), has demonstrated significant advantages in handling workshop scheduling issues with sequential decision-making characteristics, which can fully utilize the vast quantity of historical data accumulated in the workshop and adjust production plans in a timely manner based on changes in production conditions and demand. Accordingly, a DRL-based approach is introduced to address the common production disturbances in emergency order insertions. Combined with the characteristics of the ISPEA problem and an event-driven strategy for handling dynamic events, four types of agents, namely workpiece selection, machine selection, AGV selection, and target selection agents, are set up, which refine workshop production status characteristics as observation inputs and generate rules for selecting workpieces, machines, AGVs, and targets. These agents are trained offline using the QMIX multi-agent reinforcement learning framework, and the trained agents are utilized to solve the dynamic ISPEA problem. Finally, the effectiveness of the proposed model and method is validated through a comparison of the solution performance with other typical optimization algorithms for various cases.

Probing an LSTM-PPO-Based reinforcement learning algorithm to solve dynamic job shop scheduling problem

With the growth of personalized demand and the continuous improvement in social productivity, the large-scale and few-variety centralized production model is gradually transitioning towards a personalized model of small batches and multiple varieties, which makes the manufacturing process of the job shop increasingly complex. Furthermore, disruptive events such as machinery failures and rush orders in the job shop increase the uncertainty and variability of the production environment. Traditional scheduling methods are usually based on fixed rules and heuristic algorithms, which are difficult to adapt to constantly changing production environments and demands. This may lead to inaccurate scheduling decisions and hinder the optimal allocation of job shop resources. To solve the dynamic job shop scheduling problem (JSP) more effectively, this paper proposes a Reinforcement Learning (RL) optimization algorithm integrating long short-term memory (LSTM) neural network and proximal policy optimization (PPO). It can dynamically adjust scheduling strategies according to the changing production environment, achieving comprehensive status awareness of the job shop environment to make optimal scheduling decisions. First, a state-aware network framework based on LSTM-PPO is proposed to achieve real-time perception of job shop state changes. Then, the state and action space of the job shop are described within the context of the state-aware network framework. Finally, an experimental environment is established to verify the algorithm’s effectiveness. Training the LSTM-PPO algorithm makes it feasible to achieve better performance than other scheduling methods. By comparing the initial planning time with the actual completion time of the rescheduling decision under different dynamic disturbances, the efficiency of the proposed algorithm is verified for the dynamic JSP

External ligand-free nickel-catalyzed synthesis of polypyrazines towards stable, high-capacity and low-potential sodium ion storage

Sodium ion batteries (SIBs) have been regarded as a promising alternative to lithium-ion battery owing to low cost and good sustainability, but its competitiveness in energy storage is still limited largely by the development of suitable electrode materials. Traditional inorganic anode materials for SIBs face the problems of sluggish electrochemical kinetics and large volume strain due to the insertion/extraction of large size Na+. By comparison, organic anodes have the advantages of abundance resources, renewability, and designability, but suffer from the lack of stable electrode materials with both high specific capacity and low redox potential. Here, a class of pyrazine rings-rich organic polymers (polypyrazines) were designed and synthesized through external ligand-free nickel-catalyzed carbon–carbon coupling reactions. Among them, poly(2,6-pyrazine) shows the lowest synthetic cost, the highest porosity and the best conductivity. Consequently, as the anode of SIBs, poly(2,6-pyrazine) delivers a low redox potential (about 1 V, vs Na+/Na), high-specific-capacity (617.2 mA h/g at 0.1 A/g), excellent rate-capability (300 mA h/g at 10 A/g) and long-term cycling stability (81 % after 1000 cycles at 1.0 A/g). The low preparation cost and outstanding electrochemical performance of poly(2,6-pyrazine) make it a promising anode candidate for further boosting the energy density of SIBs.

Bismuth-based material with alveolar-like structure as advanced anode for superior potassium storage

Bismuth (Bi)-based anodes have a promising application in potassium-ion batteries (PIBs) due to their high theoretical capacity. However, Bi-based materials undergo unavoidable and drastic volume expansion during charging/discharging. In addition to this, the problem of poor reaction kinetics severely hinders its development. To solve these problems, we designed Bi2O3-Bi/Carbon hollow nanospheres (Bi2O3-Bi@CHNSs) inspired by the structure of alveoli and its contraction/expansion respiration mechanism. Bi2O3-Bi@CHNSs have an alveolus-like hollow core–shell structure and alveolus respiration-like reversible contraction/expansion mechanism for potassium storage, which provides good structural stability. In situ characterizations confirm that the Bi2O3 and Bi hybrid structures synergistically store potassium through a conversion/alloying mechanism. Bi2O3-Bi@CHNSs exhibited excellent rate performance as anode with a capacity of up to 270.4 mAh g−1 at a current density of 40 A g−1. Assembled as a full cell, the capacity reached 106.5 mAh g−1 at a current density of 30 A g−1, which is superior to most of the reported intact PIBs. This work utilized biomimetic design concepts to elaborate a bionic alveolar structure. It provides a new perspective for optimizing the volume expansion of Bi-based anodes and developing high-performance PIBs.

Topology optimization of multi-material structures subjected to dynamic loads

Traditional designs for multi-material structures are mostly based on static loads. However, engineering structures are often subjected to dynamic loads. This paper firstly and systematically conducts an in-depth study on the topological design of multi-material structures under dynamic loads. In this method, the definition of design variable adopts the ordered solid isotropic material with penalization (ordered-SIMP) method to normalize the design variable, without introducing any additional variables, the HHT-α method is employed to solve the dynamic optimization problem. To address the dynamic optimization problems with constraints on mass and cost, a convex optimization method based on the concept of sensitivity separation is used to update the design variables, and search for the structural topologies. Finally, this paper provides some numerical examples with respect to time-varying, material parameters, load amplitude, load direction and multiple loads, to discuss in depth the topological and numerical results of these examples.

Dynamic Multiobjective Optimization Based on Multi-Environment Knowledge Selection and Transfer

Background: Dynamic multiobjective optimization problems (DMOPs) involve multiple conflicting and time-varying objectives, and dynamic multiobjective algorithms (DMOAs) aim to find Pareto optima that are closer to the real one in the new environment as soon as possible. In particular, the introduction of transfer learning in DMOAs has led to good results in solving DMOPs. However, the selection of valuable historical knowledge and the mitigation of negative transfer remain important problems in existing transfer learning-based DMOAs. Method: A DMOA based on multi-environment knowledge selection and transfer (MST-DMOA) is proposed in this article. First, by clustering historical Pareto optima, some representative solutions that can reflect the main evolutionary information are selected as knowledge of the environment. Second, the similarity between the historical and current environments is evaluated, and then the knowledge of multiple similar environments is selected as valuable historical knowledge to construct the source domain. Third, solutions with high quality in the new environment are obtained to form the target domain, which can better help historical knowledge to adapt to the current environment, thus effectively alleviating negative transfer. Conclusions: We compare the proposed MST-DMOA with five state-of-the-art DMOAs on fourteen benchmark test problems, and the experimental results verify the excellent performance of MST-DMOA in solving DMOPs.

A GRASP-based multi-objective approach for the tuna purse seine fishing fleet routing problem

Nowadays, the world’s fishing fleet uses 20% more fuel to catch the same amount of fish compared to 30 years ago. Addressing this negative environmental and economic performance is crucial due to stricter emission regulations, rising fuel costs, and predicted declines in fish biomass and body sizes due to climate change. Investment in more efficient engines, larger ships and better fuel has been the main response, but this is only feasible in the long term at high infrastructure cost. An alternative is to optimize operations such as the routing of a fleet, which is an extremely complex problem due to its dynamic (time-dependent) moving target characteristics. To date, no other scientific work has approached this problem in its full complexity, i.e., as a dynamic vehicle routing problem with multiple time windows and moving targets. In this paper, two bi-objective mixed linear integer programming (MIP) models are presented, one for the static variant and another for the time-dependent variant. The bi-objective approaches allow to trade off the economic (e.g., probability of high catches) and environmental (e.g., fuel consumption) objectives. To overcome the limitations of exact solutions of the MIP models, a greedy randomized adaptive search procedure for the multi-objective problem (MO-GRASP) is proposed. The computational experiments demonstrate the good performance of the MO-GRASP algorithm with clearly different results when the importance of each objective is varied. In addition, computational experiments conducted on historical data prove the feasibility of applying the MO-GRASP algorithm in a real context and explore the benefits of joint planning (collaborative approach) compared to a non-collaborative strategy. Collaborative approaches enable the definition of better routes that may select slightly worse fishing and planting areas (2.9%), but in exchange for a significant reduction in fuel consumption (17.3%) and time at sea (10.1%) compared to non-collaborative strategies. The final experiment examines the importance of the collaborative approach when the number of available drifting fishing aggregation devices (dFADs) per vessel is reduced.