Total Sojourn Time Research Articles

Deep reinforcement learning for solving steelmaking-continuous casting scheduling problems under time-of-use tariffs

This paper proposes a novel intelligent scheduling method based on deep reinforcement learning (DRL) to solve the multi-objective steelmaking-continuous casting (SCC) scheduling problem, under time-of-use (TOU) tariffs for the first time. The intelligent scheduling system architecture is designed, and a mathematical model is established to minimise the total sojourn time and electricity cost. To effectively reduce production costs by avoiding peak periods of electricity consumption, the ‘start time’ of the system is generated based on the Markov Decision Process (MDP), and heuristic scheduling rules related to power cost are used as the action space, with corresponding reward functions designed according to the characteristics of these two objectives. To satisfy the continuous casting which is a particular SCC constraint, a backward strategy is developed. Additionally, a branching duelling double deep Q-network (BD3QN) is adapted to guide action selection and avoid blind search in the iteration process, and then applied to real-time scheduling. Numerical experiments demonstrate that the proposed method outperforms comparison algorithms in terms of solution quality and CPU times by a large margin.

Monte Carlo Simulation Approach to Shipping Accidents Consequences Assessment

The purpose of this study is to present and apply an innovative technique to model environmental consequences of shipping accidents in relations to events initiating those accidents. The Monte Carlo simulation technique is used to model shipping accidents and chemical release consequences within the world’s sea and ocean waters. The model was created based on the previously designed novel general probabilistic approach to critical infrastructure accident consequences, including three models: the process of initiating events generated by a critical infrastructure accident, the process of environmental threats coming from released chemicals that are a result of initiating events, and the process of environmental degradation stemming from environmental threats. It is a new approach that has never been proposed and applied before. The Monte Carlo simulation method is used under the assumption of the semi-Markov model of these three processes. A procedure for the realization and generation of this process and evaluation of its characteristics is proposed and applied in the preparation of the C# program. Using this program, the processes’ characteristics are predicted for a specific sea area. Namely, for the considered processes, the limit values of transient probabilities between the states and the mean values of total sojourn times at the particular states for the fixed time are determined. The results obtained can be used practically by maritime practitioners involved in making decisions related to the safety of maritime transport and to mitigation actions concerned with maritime accidents.

Probabilistic Approach to Modelling, Identification and Prediction of Environmental Pollution

The probabilistic general model of environmental pollution process based on the semi-Markov one is developed and presented in the paper. The semi-Markov chain model approach is based on using prior information to predict the characteristic of some systems. Now, the semi-Markov process is used for the environmental pollution assessment. The methods and procedures to estimate the environmental pollution process’s basic parameters such as the vector of initial probabilities and the matrix of probabilities of transition between the process’s states as well as the methods and procedures to identify the process conditional sojourn times’ distributions at the particular environmental pollution states and their mean values are proposed and defined. Next, the formulae to predict the main characteristics of the environmental pollution process such as the limit values of transient probabilities and mean total sojourn times in the particular states in the fixed time interval are given. Finally, the application of the presented model and methods for modelling, identification and prediction of the air environmental pollution process generated by sulphur dioxide within the exemplary industrial agglomeration is proposed.

Estimation of Preclinical State Onset Age and Sojourn Time for Heavy Smokers in Lung Cancer.

Estimation of the three key parameters: onset age of the preclinical state, sojourn time and screening sensitivity is critical in cancer screening, since all other terms are functions of the three. A novel link function to connect sensitivity with time in the preclinical state and the likelihood method were used in this project; since sensitivity depends on how long one has entered the preclinical state relative to the total sojourn time. Simulations using Markov Chain Monte Carlo and maximum likelihood estimate were carried out to estimate the key parameters for male and female heavy smokers separately in the low-dose computed tomography group of the National Lung Screening Trial. Sensitivity for male and female heavy smokers were 0.883 and 0.915 respectively at the onset of the preclinical state, and increased to 0.972 and 0.981 at the end. The mean age to make the transition into the preclinical state was 70.94 or 71.15 for male and female heavy smokers respectively, and 90% of heavy smokers at risk for lung cancer would enter the preclinical state in age interval (55.7, 85.8) for males and (54.2, 87.7) for females, and the transition peaked around age 69 for both genders. The mean sojourn time in the preclinical state was 1.43 and 1.49 years, and the 99% credible intervals for the sojourn time were (0.21, 2.96) and (0.37, 2.69) years for male and female heavy smokers correspondingly. Based on the result, low-dose CT should be started at age 55 and ended before 85 for heavy smokers. This provided important information to policy makers.

Consensus of discrete-time multi-agent systems over packet dropouts channels

In this paper, we investigate the consensus problem for discrete linear multi-agent systems (MASs) with Markovian packet dropouts. Both identical and nonidentical packet dropouts are studied. For the discrete-time MASs under identical packet dropouts, we present the expectation of the total sojourn time of packet dropouts and successful message transmission, the switching number from packet dropouts to successful message transmission, and the awaken number of packet dropouts and successful message transmission. Based on these expectations, a linear consensus controller is designed through analyzing the transient properties of the Markov process such that the MASs can reach consensus almost surely for any initial distribution of packet dropouts. When it comes to the nonidentical packet dropouts where all the packets are independent and stochastic, a Markovian-lossy-channel based switching model (MLCBS model) is proposed. Based on the MLCBS model, we also propose an easy-to-implement linear consensus controller such that the MASs with nonidentical packet dropouts can achieve consensus almost surely. Finally, the theoretical results are illustrated by simulation examples.

A functional law of the iterated logarithm for multi-class queues with batch arrivals

A functional law of the iterated logarithm (LIL) and its corresponding LIL are established for a multiclass single-server queue with first come first served (FCFS) service discipline. The functional LIL and its LIL quantify the magnitude of asymptotic stochastic fluctuations of the stochastic processes compensated by their deterministic fluid limits. The functional LIL and LIL are established in three cases: underloaded, critically loaded and overloaded, for performance measures including the total workload, idle time, queue length, workload, busy time, departure and sojourn time processes. The proofs of the functional LIL and LIL are based on a strong approximation approach, which approximates discrete performance processes with reflected Brownian motions. Numerical examples are considered to provide insights on these limit results.

On tandem stochastic networks with time-deteriorating product quality

We consider an n-site tandem stochastic production network where each product moves sequentially through the sites, and the product's quality deteriorates with its sojourn time in the system. At each site the product goes through two stages: the first stage is a processing operation with a generally-distributed random duration. This operation either does or does not conclude successfully; in the latter case, the operation is repeated immediately. Once the processing operation concludes successfully, the product goes through an inspection stage lasting a generally-distributed random duration. At the end of the inspection the product's state is determined as follows: either (i) it requires additional processing and moves forward to the next site; or (ii) it is found ‘good’ and exits the network with quality value depending on its total sojourn time in the system; or (iii) it is declared ‘failed’, discarded, and exits the network with zero quality value. Two scenarios are analysed: (i) a new product enters the system only after the preceding product has exited and (ii) the network is a tandem Jackson-type system. For each scenario, we construct both time-dependent and quality-dependent performance measures. In the case where the sites can be arranged in an arbitrary order, we derive easy to implement optimal index-type policies of ordering the sites so as to maximise the quality rate of the production network.

Monte Carlo simulation of climate-weather change process at maritime ferry operating area

The paper presents a computer simulation technique applied to generating the climate-weather change process at Baltic Sea restricted waters and its characteristics evaluation. The Monte Carlo method is used under the assumption of semi-Markov model of this process. A procedure and an algorithm of climate-weather change process’ realizations generating and its characteristics evaluation are proposed to be applied in C# program preparation. Using this program, the climate-weather change process’ characteristics are predicted for the maritime ferry operating area. Namely, the mean values and standard deviations of the unconditional sojourn times, the limit values of transient probabilities and the mean values of total sojourn times for the fixed time at the climate-weather states are determined.

Approach Integrating Mixed-Integer Programming and Constraint Programming for Planning Rotations of Inland Vessels in a Large Seaport

During the past decade, inland vessels have gained importance in container transport because of their reliability, low environmental impact, and major capacity for increased exploitation. Although inland vessels are crucial in container transport between terminals in the port and the hinterland, in a large seaport like the one in Rotterdam, Netherlands, only 62% of the inland vessels leave the port on time. The other vessels have to stay in the port area for a longer time than planned. This situation leads to uncertainty in waiting times of vessels at terminals and low utilization of terminal quay resources. A two-phase approach is proposed that integrates mixed-integer programming (MIP) and constraint programming (CP) to solve the problem by generating optimal rotation plans for inland vessels. In the first phase, the single-vessel optimization problem is formulated on the basis of MIP and solved with state-of-the-art MIP solvers. In the second phase, the multiple-vessel coordination problem is formulated on the basis of CP, and a large neighborhood search–based heuristic is proposed to solve the problem. Commercial CP solvers are also used for comparison. Simulation results show that the proposed large neighborhood search–based heuristic outperforms the commercial CP solver with regard to both the solution quality and the computation time. Moreover, simulation results with respect to departure time of the last vessel, total sojourn time, and waiting time show significant improvement with earlier departure times and shorter sojourn times and waiting times.

Network Cost Minimization for Mobile Data Gathering in Wireless Sensor Networks

Recent studies have shown that significant benefit can be achieved in wireless sensor networks (WSNs) by employing mobile collectors for data gathering via short-range communications. A typical scenario for such a scheme is that a mobile collector roams over the sensing field and pauses at some anchor points on its moving tour such that it can traverse the transmission range of all the sensors in the field and directly collect data from each sensor. In this paper, we study the performance optimization of such mobile data gathering by formulating it into a cost minimization problem constrained by channel capacity, required minimum data uploads from each sensor and bound of total sojourn time at all anchor points. In order to provide an efficient and distributed algorithm, we decompose this global optimization problem into two subproblems that can be solved by each sensor and the mobile collector, respectively. We show that such decomposition can be characterized as a pricing mechanism, in which each sensor independently adjusts its payment for the data uploading opportunity based on the shadow prices of different anchor points. Correspondingly, we give an efficient algorithm to jointly solve the two subproblems. Our theoretical analysis demonstrates that the proposed algorithm can achieve the optimal data control for each sensor and the optimal sojourn time allocation for the mobile collector, which minimizes the overall network cost. Finally, extensive simulation results further validate that our algorithm achieves lower cost than the compared data gathering strategy.

Loss rates in the single-server queue with complete rejection

Consider the single-server queue in which customers are rejected if their total sojourn time would exceed a certain level $$K$$ . A basic performance measure of this system is the probability $$P_K$$ that a customer gets rejected in steady state. This paper presents asymptotic expansions for $$P_K$$ as $$K\rightarrow \infty $$ . If the service time $$B$$ is light-tailed and inter-arrival times are exponential, it is shown that the loss probability has an exponential tail. The proof of this result heavily relies on results on the two-sided exit problem for Lévy processes with no positive jumps. For heavy-tailed (subexponential) service times and generally distributed inter-arrival times, the loss probability is shown to be asymptotically equivalent to the trivial lower bound $$P(B>K)$$ .

An effective Lagrangian relaxation approach for rescheduling a steelmaking-continuous casting process

This paper studies a steelmaking-continuous casting (SCC) rescheduling problem with machine breakdown and processing time variations. Two objectives are considered in this study: the efficiency objective and the stability objective. The former refers to the total weighted completion time and total sojourn time, whereas the latter refers to the number of operations processed on different machines in the initial and revised schedules. We develop a time-index formulation and an effective Lagrangian relaxation (LR) approach with machine capacity relaxation to address the rescheduling problem. The LR approach decomposes the relaxed problem into batch-level subproblems with variable processing times. A polynomial two-stage dynamic programming algorithm is proposed to solve the batch-level subproblems. An efficient subgradient algorithm with global convergence is presented to solve the corresponding Lagrangian dual (LD) problem. Computational experiments based on practical production data show that the proposed approach not only produces a high quality schedule within an acceptable time but also performs much better than a practical SCC rescheduling method from a large iron and steel enterprise in China.

Explicit Causal Recursive Estimators for Continuous-Time Bivariate Markov Chains

A bivariate Markov chain comprises a pair of random processes which are jointly Markov. In this paper, both processes are assumed to be continuous-time with finite state space. One of the two processes is observable, while the other is an underlying process which affects the statistical properties of the observable process. Neither the observable, nor the underlying process , is required to be a Markov chain. Examples of bivariate Markov chains include the Markov modulated Markov process (MMMP), the Markov modulated Poisson process (MMPP), and the batch Markovian arrival process (BMAP). We develop explicit causal recursions for estimating the number of jumps from one state to another, and the total sojourn time in each state, of a general bivariate Markov chain. Explicit causal recursions of these statistics were previously developed for the MMMP and the MMPP using the transformation of measure approach. We argue that this approach cannot be extended to a general bivariate Markov chain. Instead, we modify the approach developed by Ryden for noncausal estimation of the same statistics of an MMPP, and use the state augmentation approach of Zeitouni and Dembo and a matrix recursion from Stiller and Radons, to derive the causal recursions. The new recursions do not require any numerical integration or sampling scheme of the continuous-time bivariate Markov chain.

Explicit Forward Recursive Estimators for Markov Modulated Markov Processes

A Markov modulated Markov process is a doubly stochastic finite-alphabet continuous-time random process with an underlying hidden Markov chain and an observable conditionally Markov chain. Explicit forward recursions are developed for the conditional mean estimators of the state, number of jumps, and total sojourn time of the underlying chain, and for the conditional number of jumps and total sojourn time of the observable process given any state of the underlying chain. The recursions are derived using the transformation of measure approach, and their explicit forms follow from vectorization and Van Loan's formula for integration of matrix exponentials. The recursions are applicable to Markov modulated Poisson processes. Numerical results from applications of these recursions are also provided.

Two-stage queueing network models for quality control and testing

We study sojourn times in a two-node open queueing network with a processor sharing node and a delay node, with Poisson arrivals at the PS node. Motivated by quality control and blood testing applications, we consider a feedback mechanism in which customers may either leave the system after service at the PS node or move to the delay node; from the delay node, they always return to the PS node for new quality controls or blood tests. We propose various approximations for the distribution of the total sojourn time in the network; each of these approximations yields the exact mean sojourn time, and very accurate results for the variance. The best of the three approximations is used to tackle an optimization problem that is mainly inspired by a blood testing application.

On tandem blocking queues with a common retrial queue

We consider systems of tandem blocking queues having a common retrial queue. The model represents dynamics of short TCP transfers in the Internet. Analytical results are available only for a specific example with two queues in tandem. We propose approximation procedures involving simple analytic expressions, based on mean value analysis (MVA) and on fixed point approach (FPA). The mean sojourn time of a job in the system and the mean number of visits to the orbit queue are estimated by the MVA which needs as an input the fractions of blocked jobs in the primary queues. The fractions of blocked jobs are estimated by FPA. Using a benchmark example of the system with two primary queues, we conclude that the approximation works well in the light traffic regime. We note that our approach becomes exact if the blocking probabilities are fixed. Finally, we consider two optimization problems regarding minimizing mean total sojourn time of a job in the system: (i) finding the best order of queues and (ii) allocating a given capacity among the primary queues.

On the M/G/1 queueing system with multiclass customers and fixed feedback

The M/G/1 queueing system with multiclass customer arrivals, fixed feedback, and first come first served policy is considered, where different classes of customers have different arrival rates, service-time distributions, and feedback numbers. The joint probability generation function of queue size of each class and the Laplace-Stieltjes transform of the total sojourn time of a customer in each class are presented, which extended the results obtained by Choi B D. The mean queue size of each class and mean total sojourn time of a customer in each class are obtained with this result. The results can be used in computer and communication networks for their performance analysis.

Two-boundary problems for a Poisson process with exponentially distributed component

For a Poisson process with exponentially distributed negative component, we obtain integral transforms of the joint distribution of the time of the first exit from an interval and the value of the jump over the boundary at exit time and the joint distribution of the time of the first hit of the interval and the value of the process at this time. On the exponentially distributed time interval, we obtain distributions of the total sojourn time of the process in the interval, the joint distribution of the supremum, infimum, and value of the process, the joint distribution of the number of upward and downward crossings of the interval, and generators of the joint distribution of the number of hits of the interval and the number of jumps over the interval.

The sojourn time and related characteristics of the age-dependent branching process

An age-dependent branching process where disasters occur as a renewal process leading to annihilation or survival of all the cells, is considered. For such a process, the total mean sojourn time of all the cells in the system is analysed using the regeneration point technique. The mean number of cells which die in time t and its asymptotic behaviour are discussed. When the disasters arrival as a Poisson process and the lifetime of the cells follows exponential distribution, elegant inter- relationships are found among the means of (i) the total number of cells which die in time t (ii) the total sojourn time of all cells in the system upto time t and (iii) the number of living cells at time t. Some of the existing results are deduced as special cases for related processes.

Boundary Functionals of a Semicontinuous Process with Independent Increments on an Interval

We investigate boundary functionals of a semicontinuous process with independent increments on an interval with two reflecting boundaries. We determine the transition and ergodic distributions of the process, as well as the distributions of boundary functionals of the process, namely, the time of first hitting the upper (lower) boundary, the number of hittings of the boundaries, the number of intersections of the interval, and the total sojourn time of the process on the boundaries and inside the interval. We also present a limit theorem for the ergodic distribution of the process and asymptotic formulas for the mean values of the distributions considered.