Nonstationary Problem Research Articles

A deep multi-agent reinforcement learning approach to solve dynamic job shop scheduling problem

Manufacturing industry is experiencing a revolution in the creation and utilization of data, the abundance of industrial data creates a need for data-driven techniques to implement real-time production scheduling. However, existing dynamic scheduling techniques have been mainly developed to solve problems of invariable size, and are incapable of addressing the increasing volatility and complexity of practical production scheduling problems. To facilitate near real-time decision-making on the shop floor, we propose a deep multi-agent reinforcement learning-based approach to solve the dynamic job shop scheduling problem. Double deep Q-network algorithm, attached to decentralized scheduling agents, is used to learn the relationships between production information and scheduling objectives, and to make near real-time scheduling decisions. Proposed framework utilizes centralized training and decentralized execution scheme and parameter-sharing technique to tackle the non-stationary problem in the multi-agent reinforcement learning task. Several enhancements are also developed, including the novel state and action representation that can handle size-agnostic dynamic scheduling problems, a chronological joint-action framework to alleviate the credit-assignment difficulty, and knowledge-based reward-shaping techniques to encourage cooperation. Simulation study shows that the proposed architecture significantly improves the learning effectiveness, and delivers superior performance compared to existing scheduling strategies and state-of-the-art deep reinforcement learning-based dynamic scheduling approaches.

Spatiotemporal Sequence Prediction With Point Processes and Self-Organizing Decision Trees.

We study the spatiotemporal prediction problem and introduce a novel point-process-based prediction algorithm. Spatiotemporal prediction is extensively studied in machine learning literature due to its critical real-life applications, such as crime, earthquake, and social event prediction. Despite these thorough studies, specific problems inherent to the application domain are not yet fully explored. Here, we address the nonstationary spatiotemporal prediction problem on both densely and sparsely distributed sequences. We introduce a probabilistic approach that partitions the spatial domain into subregions and models the event arrivals in each region with interacting point processes. Our algorithm can jointly learn the spatial partitioning and the interaction between these regions through a gradient-based optimization procedure. Finally, we demonstrate the performance of our algorithm on both simulated data and two real-life datasets. We compare our approach with baseline and state-of-the-art deep learning-based approaches, where we achieve significant performance improvements. Moreover, we also show the effect of using different parameters on the overall performance through empirical results and explain the procedure for choosing the parameters.

Numerical solution of nonstationary problem for convection of binary mixture in horizontal layer

Nonstationary motion of a liquid binary mixture in a narrow long horizontal channel with rigid walls heated according to a certain law is considered. The possibility of applying the Ostroumov-Birikh solution to the description of the flow under study is used. It reduces the problem to solving a mixed boundary value problem for a system of parabolic equations. A feature of the problem is an additional integral condition on the fluid flow rate. It allows finding the pressure gradient together with the functions of velocity, temperature, and concentration. Applying the constructed numerical procedure, the analysis of the obtained characteristics of motion is carried out using water-ethanol solution as a mixture. The possibilities of stabilizing the unsteady flow and controlling the motion by means of a periodically changing thermal load on the channel wall are shown.

Tensor-Product Space-Time Goal-Oriented Error Control and Adaptivity With Partition-of-Unity Dual-Weighted Residuals for Nonstationary Flow Problems

Abstract In this work, the dual-weighted residual method is applied to a space-time formulation of nonstationary Stokes and Navier–Stokes flow. Tensor-product space-time finite elements are being used to discretize the variational formulation with discontinuous Galerkin finite elements in time and inf-sup stable Taylor–Hood finite element pairs in space. To estimate the error in a quantity of interest and drive adaptive refinement in time and space, we demonstrate how the dual-weighted residual method for incompressible flow can be extended to a partition-of-unity based error localization. We substantiate our methodology on 2D benchmark problems from computational fluid mechanics.

Stochastic dynamic programming heuristic for the [formula omitted] policy parameters computation

This paper introduces a new stochastic dynamic program (SDP) based heuristic to compute the (R,s,S) policy parameters for the non-stationary stochastic lot-sizing problem with backlogging of the excessive demand, fixed order and review costs, and linear holding and penalty costs. Our model combines a greedy relaxation of the problem that considers replenishment cycles independent with a modified version of Scarf’s (s,S) SDP. A simple model implementation requires a prohibitive computational effort to compute the parameters. However, leveraging the K-convexity property and deploying memoisation techniques strongly reduce the computational effort required. The resulting algorithm is considerably faster than the state-of-the-art, extending its applicability by practitioners. An extensive computational study shows that our approach computes the optimal policy in more than 97% of the analysed instances, with a 0.02% average optimality gap.

Thermoelastic State of a Thermosensitive Half Space and a Thermosensitive Layer in Contact Under the Conditions of Complex Heat Exchange

We solve a nonstationary problem of heat conduction for a thermosensitive half space in contact with a thermosensitive layer under the conditions of complex heat exchange with the medium of constant temperature. To construct the solution, we use a numerical-analytic approach characterized by the application of a version of the method of successive approximations, linearizing parameters, the Laplace integral transformation, and its numerical inversion with the help of the Prudnikov formula adapted to the problems of heat conduction. The thermoelastic state of the outlined thermosensitive piecewise homogeneous structure is analyzed by applying the proposed algorithm.

Managing Operations of a Hog Farm Facing Volatile Markets: Inventory and Selling Strategies

Problem definition: We study a dynamic finishing-stage planning problem of a pork producer who at the beginning of each week gets to see how many market-ready hogs she has available for sale and the current market prices. Then, she must decide which hogs to sell to a meatpacker and on the open market and which hogs to hold until the following week. The farmer is contracted to deliver a fixed quantity of hogs to the meatpacker each week priced according to a contractually predetermined market index. If the farmer underdelivers to the meatpacker, she pays a contractually predetermined unit penalty also linked to a market index. Biosecurity protocols prevent the farmer from buying hogs on the open market and selling them to the meatpacker. The farmer can, however, use the open market to sell hogs for prevailing market prices. Methodology/Results: We treat the problem as a dynamic, multiitem, nonstationary inventory problem with multiple sources of uncertainty. The optimal policy is a threshold policy with multiple price-dependent thresholds. The computational complexity required to evaluate the thresholds is the biggest impediment to using the optimal policy as a decision-support tool. So, we utilize an approximate dynamic programming approach that exploits the optimal policy structure and produces a sharp heuristic that is easy to implement. Managerial implications: Numerical experiments calibrated to a pork producer’s data reveal that the optimal policy with the heuristically estimated thresholds substantially improves the existing practice (around 25% on average). The success of the proposed model is attributed to recognizing the value of holding underweight hogs and effectively hedging supply uncertainty and future prices—an insight missed in the planning actions of the current practice.Supplemental Material: The online appendix is available at https://doi.org/10.1287/msom.2023.1216 .

Approximately optimal forest rotation in a nonstationary environment

AbstractThe problem of optimal forest rotation in a nonstationary environment can, in general, not be solved analytically. Even qualitatively characterizing how the solution changes over time is only possible in some special cases. In this paper, we consider an approximation of the true solution to the nonstationary problem. We derive an approximate harvesting rule by solving a sequence of stationary problems that assume the growth conditions at that point in time will prevail indefinitely. Each such problem can be solved using the classic Faustmann rule. We numerically compare this approximate solution to the true solution, both in terms of the harvesting rule and the resulting expected profits, for a wide range of scenarios. We find that the harvesting rules are very similar (mostly % difference) and the profit losses associated with following the approximate rule are very small (less than 0.3%).

Numerical Determination of the Nonsteady Thermal State of a Three-Layer Hollow Thermosensitive Cylinder Under the Conditions of Complex Heat Exchange

By using the straight-line method and conservative difference schemes, we reduce the nonlinear nonstationary boundary-value problem of heat conduction for a three-layer hollow cylinder to a Cauchy problem for a system of ordinary differential equations, which is solved numerically with the help of the formulas of backward differentiation. The conservative discretization of the heat conduction equation and the boundary conditions with respect to the space variable is performed by the integro-interpolation method. We also analyze the influence of temperature dependence of the thermal characteristics of the chosen materials on the temperature field formed in the layers of a three-layer cylinder.

Nonstationary Problems of Heat Conduction for a Thermosensitive Plate with Nonlinear Boundary Condition on One Surface

We propose a method for the determination of temperature fields formed in a plate with regard for the thermal radiation, temperature dependence of thermal characteristics, and densities of surface and volumetric heat sources for a nonuniform distribution of the initial temperature. By using the Kirchhoff transformation, Green’s function, generalized functions, and linear splines, we reduce the problems of heat conduction to the solution of a recurrence nonlinear algebraic equation for the values of the Kirchhoff variable at the spline nodes on the corresponding boundary surface. The results of numerical analysis are presented.

Problem of plane stress state of two-layer body in dynamic elastic-plastic formulation

Composite materials are widely used in industry and everyday life. Many different methods are used to calculate and develop composite materials. Many methods of calculation and design of such materials are successfully used. In this article, for the design of composite and reinforced materials, a technique for solving dynamic contact problems in more precise an elastic-plastic mathematical formulation is used. To consider the physical nonlinearity of the deformation process, the method of successive approximations is used, which makes it possible to reduce the nonlinear problem to a solution of the sequences of linear problems. The problem of a plane stress state (PStS) of a beam made from the composite reinforced double-layered material is being solved in dynamic elastic-plastic mathematical model. The reinforced or armed material consists of two layers: the upper (first) thin layer of solid steel and the lower (second) main layer of glass. This composite base is rigidly attached to an absolutely hard half-space. Rigid adhesion of the layers to each other is assumed. Glass is a very strong and very fragile material at the same time. The fragility of glass is due to the fact that there are many microcracks on the surface, and when a load is applied to the glass surface, these microcracks begin to grow and lead to the destruction of glass products. If we glue or immobilize the tops of microcracks on the surface, we will get a strong reinforced armed material that will be lighter, stronger and not subject to degradation of material properties such as aging, corrosion and creep. The impact process was modelled as a non-stationary plane stress state problem with an even distributed load in the contact area, which changes according to a linear law. The fields of the Odquist parameter and normal stresses were studied and compared to corresponding results of plane strain (PSS) problem with the same material of layers, same their thickness and size of the contact area. The upper reinforcing layer of metal or steel can be applied to the glass surface so that metal or steel atoms penetrate deeply, fill microcracks and bind their tops. The top layer can be quite thin.

Joule heating of an emitter on the cathode surface by field electron emission current with an account of the nonisolation of the apex

This work is devoted to the investigation of the nonstationary problem of the thermal conductivity of a nanoemitter on the surface of a massive copper cathode when a field electron emission current passes through it. At the same time, the dependence of volume resistivity, thermal conductivity on temperature, and size effects have been taken into account. The influence of the Nottingham effect has been considered. The dependence of the equilibrium temperature of the emitter apex on the field enhancement factor for different values of the electric field strength has been found. Based on the assumption that the initial stage of the breakdown begins when the emitter apex melts, the conditions for the occurrence of a vacuum breakdown and the influence of the Nottingham effect have been analyzed.

Nonstationary seismic–well tying with time-varying wavelets

Seismic–well tying is an important technique for correlating well-logging curves in depth with seismic traces in time. An appropriate seismic–well tying technique must account for two types of nonstationarity: the nonstationary time errors in the synthetic seismic trace caused by the inaccurate time–depth relationship established based on sonic-logging velocity and the nonstationary seismic signals due to the time-varying wavelets during wave propagation. The nonstationary problems related to the time–depth relationship and the time-varying wavelets are interrelated in seismic-well tying procedure. We implemented a nonstationary seismic–well tying method by iteratively updating the time-depth relationship and estimating the time-varying wavelets. From the estimated time-varying wavelets, we also estimated a Q-value by assuming that the subsurface medium has a constant Q at depth and used the constant Q to constrain the variation of the seismic wavelet during propagation. Then, we used the improved time–depth relationship and time-varying wavelets with the Q constraint for further iterations. In the iterative procedure, we quantified the accuracy of the seismic–well tying result using the correlation coefficient between the synthetic and the true seismic trace in each iteration and evaluated the reliability using the normalized mean-square errors among the wavelets estimated in different iterations.

On the Bilinear Second Order Differential Realization of an Infinite-Dimensional Dynamical System: An Approach Based on Extensions to M2-Operators

Considering the case of a continual bundle of controlled dynamic processes, the authors have studied the functional-geometric conditions of existence of non-stationary coefficients-operators from the differential realization of this bundle in the class of non-autonomous bilinear second-order differential equations in the separable Hilbert space. The problem under scrutiny belongs to the type of non-stationary coefficient-operator inverse problems for the bilinear evolution equations in the Hilbert space. The solution is constructed on the basis of usage of the functional Relay-Ritz operator. Under this mathematical problem statement, the case has been studied in detail when the operators to be modeled are burdened with the condition, which provides for entire continuity of the integral representation equations of the model of realization. Proposed is the entropy interpretation of the given problem of mathematical modeling of continual bundle dynamic processes in the context of development of the qualitative theory of differential realization of nonlinear state equations of complex infinite-dimensional behavioristic dynamical system.

Problem of plane strain state of two-layer body in dynamic elastic-plastic formulation (Part II)

Composite materials are widely used in industry and everyday life. Mathematical modelling of composite materials began to be actively developed in the 50s and 60s of the last century. Composite materials began to be actively used in industry only at the end of the 70s of the last century. From that time to this day, interest in composite materials has not weakened, and the demands of modern industry and production are constantly increasing. The areas and branches of application of composite materials are expanding. Many different methods are used to calculate and develop composite materials. This article is part two of the previous article, where there is an investigation of the contact problem of the interaction of a striker with a two-layers composite base in a dynamic elastic-plastic mathematical formulation. This composite base is rigidly attached to an absolutely hard half-space. Its first (top) layer is made of steel, and the second (bottom) layer is made of glass. Glass is a widely available cheap amorphous material, the properties of which cannot be degraded as an result of aging, corrosion, and creep processes. The glass layer can be strengthened by reinforcement and hardening. Therefore, composite materials made on the basis of glass are important in modern production; their use gives a great economic benefit. Rigid adhesion of the layers to each other is assumed. The impact process was modelled as a non-stationary plane strain state problem with an even distributed load in the contact area, which changes according to a linear law. The fields of the Odquist parameter and normal stresses were studied depending on the size of the contact area. In this article as in part I for the design of composite and reinforced material the non-stationary contact problem of plane strain state has been solved in more precise elastic-plastic mathematical formulation. To consider the physical nonlinearity of the deformation process, the method of successive approximations is used, which makes it possible to reduce the nonlinear problem to a solution of the sequences of linear problems.
 In contrast to the previous article (Part I), in this papers there is an investigation of the strain-stress state, the fields of the Odquist parameter and normal stresses depending on the thickness of the first (upper) steel layer.

Multi-Microgrid Energy Management Strategy Based on Multi-Agent Deep Reinforcement Learning with Prioritized Experience Replay

The multi-microgrid (MMG) system has attracted more and more attention due to its low carbon emissions and flexibility. This paper proposes a multi-agent reinforcement learning algorithm for real-time energy management of an MMG. In this problem, the MMG is connected to a distribution network (DN). The distribution network operator (DSO) and each microgrid (MG) are modeled as autonomous agents. Each agent makes decisions to suit its interests based on local information. The decision-making problem of multiple agents is modeled as a Markov game and solved by the prioritized multi-agent deep deterministic policy gradient (PMADDPG), where only local observation is required for each agent to make decisions, the centralized training mechanism is applied to learn coordination strategy, and a prioritized experience replay (PER) strategy is adopted to improve learning efficiency. The proposed method can deal with the non-stationary problems in the process of a multi-agent game with partial observable information. In the execution stage, all trained agents are deployed in a distributed manner and make decisions in real time. Simulation results show that according to the proposed method, the training process of a multi-agent game is accelerated, and multiple agents can make optimal decisions only by local information.

SIMULATION OF TEMPERATURE EFFECTS IN MASSIVE BODIES USING THE MODIFIED METHOD OF LINES

This article presents the main ideas and possibilities of the modified straight line method for modeling temperature effects in massive bodies. The calculation model of a rectangular beam with a defined thermal state is adopted. The procedure for reducing the dimensionality of the original equations using the modified method of straight lines and the Bubnov-Galyorkin-Petrov method for boundary conditions is applied. The reduced equation of thermal conductivity and the reduced equation of heat balance are given. Conclusions are made regarding the addition of reduced second-order equations with components from the boundary conditions on the lateral surfaces, and finally reduced second-order equations in spatial variables are given.
 We are considering a beam of rectangular cross-section, the three dimensions of which are of the same order. The thermal state of such an object is three-dimensional and is therefore described systematically using three-dimensional heat conduction equations. When applying the modified method of straight lines to reduce the dimensionality of the original equations, it is convenient to use them in the form of a system of partial differential equations of the first order in spatial coordinates.
 According to the modified method of straight lines, two systems of straight lines parallel to the coordinate axes Oy and Oz are applied to the beam (a decrease in dimensionality in terms of spatial variables y and z is assumed). The choice of such straight lines is due to the fact that the reduced equations will depend on the variables x and t, that is, they will resemble the one-dimensional equations of the one-dimensional non-stationary problem of thermal conductivity, for which efficient and convenient numerical methods have been developed in the literature of mathematical physics at the moment, to which it is easy to adapt the reduced equation.

Numerical Simulation of Heat Transfer and Spread of Virus Particles in the Car Interior

The epidemic caused by the coronavirus infection SARS-CoV-2 at the beginning of 2022 affected approximately 500 million people in all countries. The source of infection is the particles of the virus, which, when breathing, talking, and coughing, are released with the respiratory droplets and aerosol dust of an infected person. Actions aimed at combating and minimizing the consequences of coronavirus infection led to taking measures in scientific areas to investigate the processes of the spread of viral particles in the air, in ventilation, and air conditioning systems of premises and transport, filtration through masks, the effect of partitions, face shields, etc. The article presents a mathematical model of the spread of viral particles in technological transport. Air intake diverters and the operator’s respiratory tract are the sources of the virus. The Euler–Lagrange approach was used to simulate liquid droplets in a flow. Here, the liquid phase is considered as a continuous medium using Navier–Stokes equations, the continuity equation, the energy equation, and the diffusion equation. Accounting for diffusion makes it possible to explicitly model air humidity and is necessary to consider the evaporation of droplets (changes in the mass and size of particles containing the virus). Liquid droplets are modeled using the discrete-phase model (DPM), in which each particle is tracked in a Lagrange coordinate system. The DPM method is effective, since the volume fraction of particles is small relative to the total volume of the medium, and the interaction of particles with each other can be neglected. In this case, the discrete and continuous phases are interconnected through the source terms in the equations. The averaged RANS equations are solved numerically using the k-ω turbulence model in the Ansys Fluent package. The task was solved in a static form and in the time domain. For a non-stationary problem, the stabilization time of the variables is found. The simulation results are obtained in the form of fields of pressures, velocities, temperatures and air densities, and the field of propagation of particles containing the virus. Various regimes were studied at various free flow rates and initial velocities of droplets with viral particles. The results of trajectories and velocities of particles, and particle concentrations depending on time, size, and on the evaporability of particles are obtained.

A Study of The Stochastic Burgers’ Equation Using The Dynamical Orthogonal Method

In the current work, the stochastic Burgers’ equation is studied using the Dynamically Orthogonal (DO) method. The DO presents a low-dimensional representation for the stochastic fields. Unlike many other methods, it has a time-dependent property on both the spatial basis and stochastic coefficients, with flexible representation especially in the strongly transient and nonstationary problems. We consider a computational study and application of the DO method and compare it with the Polynomial Chaos (PC) method. For comparison, both the stochastic viscous and inviscid Burgers’ equations are considered. A hybrid approach, combining the DO and PC is proposed in case of deterministic initial conditions to overcome the singularities that occur in the DO method. The results are verified with the stochastic collocation method. Overall, we observe that the DO method has a higher rate of convergence as the number of modes increases. The DO method is found to be more efficient than PC for the same level of accuracy, especially for the case of high-dimensional parametric spaces. The inviscid Burgers’ equation is analyzed to study the shock wave formation when using the DO after suitable handling of the convective term. The results show that the sinusoidal wave shape is distorted and sharpened as the time evolves till the shock wave occurs.

Dynamics of the oil reservoir depletion

In order to study the dynamics of depletion in heterogeneous oil reservoirs on the base of combined finite-element-difference method for the non-stationary problem of piezoconductivity we have carried out a numerical simulation of the pressure distribution in vicinity of the operating well. At that we have taken into account the heterogeneous distribution of filtration characteristics inside the reservoir and the oil infiltration parameters on the boundaries of the reservoir. The developed method for solving the non-stationary problem of piezoconductivity in deformed oil formations allows us adequately to describe the distribution of pressure near production and injection well systems in real operating conditions. We have shown that depletion processes in vicinity of the active well mainly depend on the intensity of oil production and the degree of oil infiltration at the boundaries of the reservoir’s area and to a lesser extent on the filtration parameters inside the reservoir. Therefore, in order to maintain the proper level of oil production in the reservoir’s area, it is necessary, for example, thanks to the use of modern technologies (system of injection wells), to ensure a sufficient inflow of the oil phase at the borders of the considered area. We have shown that in the cases of low oil infiltration at the boundaries of the reservoir area, the value of depletion is directly proportional to the production power of the well. At the same time, a decreasing of the reservoir permeability leads to a slow downing of depletion processes. The limiting value of the oil boundary infiltration coefficient, which allows achieving industrial oil production, is m. At that, the time of reaching of the stationary productive regime is directly proportional to the value of the oil permeability coefficient inside the reservoir. Before installing a system of production and injection wells in heterogeneous oil reservoirs, it is necessary to carry out a systematic analysis of the degree of depletion of the working reservoir’s areas in order to place them in such a way that would ensure the effective dynamics of filtration processes around these areas.