Cumulative Constraint Research Articles

Constraint propagation on GPU: a case study for the cumulative constraint

Constraint propagation on GPU: a case study for the cumulative constraint

Quadratic horizontally elastic not-first/not-last filtering algorithm for cumulative constraint

The not-first/not-last rule is a pendant of the edge finding rule, generally embedded in the cumulative constraint during constraint-based scheduling. It is combined with other filtering rules for more pruning of the tree search. In this paper, the Profile data structure in which tasks are scheduled in a horizontally elastic way is used to strengthen the classic not-first/not-last rule. Potential not-first task intervals are selected using criteria (specified later in the paper), and the Profile data structure is applied to selected task intervals. We prove that this new rule subsumes the classic not-first rule. A quadratic filtering algorithm is proposed for the new rule, thus improving the complexity of the horizontally elastic not-first/not-last algorithm from O(n3) to O(n2). The fixed part of external tasks that overlap with the selected task intervals is considered during the computation of the earliest completion time of task intervals. This improvement increases the filtering power of the algorithm while remaining quadratic. Experimental results, on a well-known suite of benchmark instances of Resource-Constrained Project Scheduling Problems (RCPSPs), show that the propounded algorithms are competitive with the state-of-the-art not-first algorithms in terms of tree search and running time reduction.

Q-Sorting: An Algorithm for Reinforcement Learning Problems with Multiple Cumulative Constraints

Q-Sorting: An Algorithm for Reinforcement Learning Problems with Multiple Cumulative Constraints

Distributed online constrained convex optimization with event-triggered communication

This paper focuses on the distributed online convex optimization problem with time-varying inequality constraints over a network of agents, where each agent collaborates with its neighboring agents to minimize the cumulative network-wide loss over time. To reduce communication overhead between the agents, we propose a distributed event-triggered online primal–dual algorithm over a time-varying directed graph. With several classes of appropriately chose decreasing parameter sequences and non-increasing event-triggered threshold sequences, we establish dynamic network regret and network cumulative constraint violation bounds. Finally, a numerical simulation example is provided to verify the theoretical results.

Oncologic Outcomes and Safety after Spinal Re-Irradiation with Stereotactic Body Radiation Therapy

Management of spinal metastases that fail radiation therapy is a challenge, presenting a fine balance between the risk of pain and neurologic deficits if the tumor is not controlled and the increased risks associated with exceeding the tolerance of the spinal cord and other adjacent critical structures. Data regarding SBRT in the re-irradiation setting is limited. The purpose of this study was to report oncologic outcomes and toxicities for patients that received re-irradiation using SBRT. Patients treated with spine SBRT for re-irradiation at a target which overlapped or abutted a previous conventional RT or SBRT field at a single institution between 2010 and 2021 were retrospectively reviewed. The cumulative constraint to the neural avoidance structures were a BED3 ≤75 Gy (above the conus) or ≤106 Gy (below the conus) accounting for 25% repair at 6 months and 50% repair at 1 year following the first course of RT. Radiographic local recurrence was defined as progressive disease on CT and/or MRI in the treatment volume or at the margin of the treatment field compared with imaging studies before SBRT. Cumulative incidence of local recurrence was reported with death as a competing event, and overall survival was estimated with Kaplan-Meier. Toxicity grades were determined according to NCI CTCAE version 4.0. Ninety patients (225 vertebrae) with a median age of 56.5 years (range: 27-84 years) were included in the analyses. The most common histologies were NSCLC (17.7%), kidney (15.6%), prostate (14.4%), and breast (7.8%). The majority (51.1%) of metastases were in the T-spine, while 31.1% were in the L-spine and 13.3% in the C-spine. The median prescription dose was 27 Gy (range: 14-40 Gy) in a median of 3 fractions (range: 1-5). The median prescription isodose line was 59% (range: 48%-97%). The median time to re-irradiation with SBRT was 14 months (range: 1-89 months), and the most common prior spinal radiation dose was 30 Gy (range: 8-50 Gy) in a median of 5 fractions (range: 1-15). The median maximal BED3 of the spinal cord from prior radiation and re-irradiation were 52.6 Gy and 31.8 Gy, respectively. The median follow-up was 8.7 months (range: 0.4-43.9 months). The 6-month, 1-year, and 2-year local control rates were 88.9%, 83.4%, and 78.9%, respectively. Only 6.7% of patients underwent salvage surgery, at a median of 9 months after SBRT. The median overall survival was 14.0 months, and overall survival was 54.4% at 1 year and 27.8% at 2 years post-SBRT. All toxicities were grade < 2 and no patients developed spinal cord myelopathy. These data suggest excellent local control and low toxicity following SBRT for re-irradiation of spinal metastases. Future prospective and multi-institutional studies are needed to explore the optimal dose fractionation regimen and cumulative normal tissue constraints to maximize local control and minimize toxicity.

Regret and Cumulative Constraint Violation Analysis for Distributed Online Constrained Convex Optimization

This paper considers the distributed online convex optimization problem with time-varying constraints over a network of agents. This is a sequential decision making problem with two sequences of arbitrarily varying convex loss and constraint functions. At each round, each agent selects a decision from the decision set, and then only a portion of the loss function and a coordinate block of the constraint function at this round are privately revealed to this agent. The goal of the network is to minimize the network-wide loss accumulated over time. Two distributed online algorithms with full-information and bandit feedback are proposed. Both dynamic and static network regret bounds are analyzed for the proposed algorithms, and network cumulative constraint violation is used to measure constraint violation, which excludes the situation that strictly feasible constraints can compensate the effects of violated constraints. In particular, we show that the proposed algorithms achieve $\mathcal{O}(T^{\max\{\kappa,1-\kappa\}})$ static network regret and $\mathcal{O}(T^{1-\kappa/2})$ network cumulative constraint violation, where $T$ is the time horizon and $\kappa\in(0,1)$ is a user-defined trade-off parameter. Moreover, if the loss functions are strongly convex, then the static network regret bound can be reduced to $\mathcal{O}(T^{\kappa})$. Finally, numerical simulations are provided to illustrate the effectiveness of the theoretical results.

Horizontally Elastic Edge-Finder Algorithm for Cumulative Resource Constraint Revisited

The success of constraint programming on scheduling problems comes from the low complexity and power of propagators. The data structure Profile recently introduced by Gingras and Quimper in Generalizing the edge-finder rule for the cumulative constraint. In: IJCAI, IJCAI/AAAI Press, pp 3103–3109, 2016, when applied to the edge-finder rule for the cumulative resource constraint (which we call horizontally elastic edge-finder) has improved the filtering power of this rule. In this paper, the algorithm proposed by Gingras and Quimper in Generalizing the edge-finder rule for the cumulative constraint. In: IJCAI, IJCAI/AAAI Press, pp 3103–3109, 2016 is revisited. It is proved that the data structure Profile can be further used for more adjustments. A new formulation of the horizontally elastic edge-finder rule is put forward. Similar to Gingras and Quimper in Generalizing the edge-finder rule for the cumulative constraint. In: IJCAI, IJCAI/AAAI Press, pp 3103–3109, 2016, an \(\mathcal {O}(kn^2)\) algorithm (where n is the number of tasks sharing the resource and \(k\le n\), the number of distinct capacities required by tasks) based on new attributes of the Profile data structure is proposed for the new rule. Experimental results on instances of Resource-Constrained Project Scheduling Problems (RCPSPs) from the benchmark suites highlight that using this new algorithm reduces the number of backtracks for the majority of instances with a slight increase in running time.

Data-driven constrained reinforcement learning for optimal control of a multistage evaporation process

It is challenging for reinforcement learning to solve the optimal control problem of industrial processes with constraints under uncertain operating conditions. In this context, this paper proposes a novel data-driven constrained reinforcement learning algorithm for the optimal control of a multistage evaporation process consisting of multiple evaporators in series with coupled liquid levels. We first formulate the optimal control problem as a constrained Markov decision process. Then, with the cumulative tracking error of the outlet liquor density taken as the cumulative constraint, a Lagrangian-based constrained policy optimization is developed. The fast setpoint tracking is achieved by gradient iteration of the policy and the dual variable. An action correction layer based on the online sequential version of random vector functional-link networks is built on the output of the policy network to address the instantaneous constraints of the liquid levels. The infeasible action is corrected in real-time so as to keep the liquid levels in each evaporator within operating range. Finally, we utilize both on-policy and off-policy data generated by the interaction between the constrained policy and the evaporation environment to update our algorithm, which is more data-efficient. Experiments have been carried out on a multistage evaporation system, and the results validate the effectiveness of the proposed algorithm.

The SoftCumulative Constraint with Quadratic Penalty

The Cumulative constraint greatly contributes to the success of constraint programming at solving scheduling problems. The SoftCumulative, a version of the Cumulative where overloading the resource incurs a penalty is, however, less studied. We introduce a checker and a filtering algorithm for the SoftCumulative, which are inspired by the powerful energetic reasoning rule for the Cumulative. Both algorithms can be used with classic linear penalty function, but also with a quadratic penalty function, where the penalty of overloading the resource increases quadratically with the amount of the overload. We show that these algorithms are more general than existing algorithms and vastly outperform a decomposition of the SoftCumulative in practice.

Safe Online Convex Optimization with Unknown Linear Safety Constraints

We study the problem of safe online convex optimization, where the action at each time step must satisfy a set of linear safety constraints. The goal is to select a sequence of actions to minimize the regret without violating the safety constraints at any time step (with high probability). The parameters that specify the linear safety constraints are unknown to the algorithm. The algorithm has access to only the noisy observations of constraints for the chosen actions. We propose an algorithm, called the Safe Online Projected Gradient Descent (SO-PGD) algorithm, to address this problem. We show that, under the assumption of availability of a safe baseline action, the SO-PGD algorithm achieves a regret O(T^{2/3}). While there are many algorithms for online convex optimization (OCO) problems with safety constraints available in the literature, they allow constraint violations during learning/optimization, and the focus has been on characterizing the cumulative constraint violations. To the best of our knowledge, ours is the first work that provides an algorithm with provable guarantees on the regret, without violating the linear safety constraints (with high probability) at any time step.

Online DR-Submodular Maximization: Minimizing Regret and Constraint Violation

In this paper, we consider online continuous DR-submodular maximization with linear stochastic long-term constraints. Compared to the prior work on online submodular maximization, our setting introduces the extra complication of stochastic linear constraint functions that are i.i.d. generated at each round. In particular, at each time step a DR-submodular utility function and a constraint vector, i.i.d. generated from an unknown distribution, are revealed after committing to an action and we aim to maximize the overall utility while the expected cumulative resource consumption is below a fixed budget. Stochastic long-term constraints arise naturally in applications where there is a limited budget or resource available and resource consumption at each step is governed by stochastically time-varying environments. We propose the Online Lagrangian Frank-Wolfe (OLFW) algorithm to solve this class of online problems. We analyze the performance of the OLFW algorithm and we obtain sub-linear regret bounds as well as sub-linear cumulative constraint violation bounds, both in expectation and with high probability.

Constraint cumulativity in phonotactics: evidence from artificial grammar learning studies

An ongoing debate in phonology concerns the treatment of cumulative constraint interactions, or ‘gang effects’, and by extension the question of which phonological frameworks are suitable models of the grammar. This paper uses a series of artificial grammar learning experiments to examine the inferences that learners draw about cumulative constraint violations in phonotactics in the absence of a confounding natural-language lexicon. I find that learners consistently infer linear counting and ganging cumulativity across a range of phonotactic violations.

Cumulative constraint interaction and the equalizer of OT and HG

We show that, in general, Optimality Theory (OT) grammars containing a restricted family of locally-conjoined constraints (Smolensky 2006) make the same typological predictions as corresponding Harmonic Grammar (HG) grammars. We provide an intuition for the generalization using a simple constrast and neutralization typology, as well as a formal proof. This demonstration adds structure to claims about the (non)equivalence of HG and OT with local conjunction (Legendre et al. 2006, Pater 2016) and provides a tool for understanding how different sets of constraints lead to the same typological predictions in HG and OT.

IPO: Interior-Point Policy Optimization under Constraints

In this paper, we study reinforcement learning (RL) algorithms to solve real-world decision problems with the objective of maximizing the long-term reward as well as satisfying cumulative constraints. We propose a novel first-order policy optimization method, Interior-point Policy Optimization (IPO), which augments the objective with logarithmic barrier functions, inspired by the interior-point method. Our proposed method is easy to implement with performance guarantees and can handle general types of cumulative multi-constraint settings. We conduct extensive evaluations to compare our approach with state-of-the-art baselines. Our algorithm outperforms the baseline algorithms, in terms of reward maximization and constraint satisfaction.

Distributed Online Optimization With Long-Term Constraints

In this article, we consider distributed online convex optimization problems, where the distributed system consists of various computing units connected through a time-varying communication graph. In each time step, each computing unit selects a constrained vector, experiences a loss equal to an arbitrary convex function evaluated at this vector, and may communicate to its neighbors in the graph. The objective is to minimize the system-wide loss accumulated over time. We propose a decentralized algorithm with regret and cumulative constraint violation in <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">${\mathcal O}(T^{\max \lbrace c,1-c\rbrace })$</tex-math></inline-formula> and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">${\mathcal O}(T^{1-c/2})$</tex-math></inline-formula> , respectively, for any <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$c\in (0,1)$</tex-math></inline-formula> , where <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$T$</tex-math></inline-formula> is the time horizon. When the loss functions are strongly convex, we establish improved regret and constraint violation upper bounds in <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">${\mathcal O}(\log (T))$</tex-math></inline-formula> and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">${\mathcal O}(\sqrt{T\log (T)})$</tex-math></inline-formula> . These regret scalings match those obtained by state-of-the-art algorithms and fundamental limits in the corresponding centralized online optimization problem (for both convex and strongly convex loss functions). In the case of bandit feedback, the proposed algorithms achieve a regret and constraint violation in <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">${\mathcal O}(T^{\max \lbrace c,1-c/3 \rbrace })$</tex-math></inline-formula> and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">${\mathcal O}(T^{1-c/2})$</tex-math></inline-formula> for any <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$c\in (0,1)$</tex-math></inline-formula> . We numerically illustrate the performance of our algorithms for the particular case of distributed online regularized linear regression problems on synthetic and real data.

Asynchronous Online Learning in Multi-Agent Systems With Proximity Constraints

We consider the problem of distributed learning from sequential data via online convex optimization. A multi-agent system is considered where each agent has a private objective but is willing to cooperate in order to minimize the network cost, which is the sum of local cost functions. Different from the classical distributed settings, where the agents coordinate through the use of consensus constraints, we allow the neighboring agent actions to be related via a non-linear proximity function. A decentralized saddle point algorithm is proposed that is capable of handling gradient delays arising from computational issues. The proposed online asynchronous algorithm is analyzed under adversarial settings by developing bounds on the regret of O(√T), which measures the cumulative loss incurred by the online algorithm against a clairvoyant, and network discrepancy of O(T <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3/4</sup> ), which measures the cumulative constraint violation or agent disagreement. By allowing the agents to utilize stale gradient information, the proposed algorithm embraces the nuances of distributed learning and serves to be the first distributed online algorithm that can handle adversarial delays. A modified saddle point algorithm is also proposed that explicitly forces the agents to agree as per the constraint function resulting in zero network discrepancy while incurring a slightly higher regret. To showcase the efficacy of the proposed asynchronous algorithm, a spatially correlated random field estimation problem is formulated and solved. Additionally, an application of vision-based target localization with moving cameras demonstrates the benefits of this approach in practice.

Generalized dynamic programming principle and sparse mean-field control problems

In this paper we study optimal control problems in Wasserstein spaces, which are suitable to describe macroscopic dynamics of multi-particle systems. The dynamics is described by a parametrized continuity equation, in which the Eulerian velocity field is affine w.r.t. some variables. Our aim is to minimize a cost functional which includes a control norm, thus enforcing a control sparsity constraint. More precisely, we consider a nonlocal restriction on the total amount of control that can be used depending on the overall state of the evolving mass. We treat in details two main cases: an instantaneous constraint on the control applied to the evolving mass and a cumulative constraint, which depends also on the amount of control used in previous times. For both constraints, we prove the existence of optimal trajectories for general cost functions and that the value function is viscosity solution of a suitable Hamilton-Jacobi-Bellmann equation. Finally, we discuss an abstract Dynamic Programming Principle, providing further applications in the Appendix.

Selecting and scheduling interrelated road projects with uncertain demand

ABSTRACTIn transportation systems, the existence of interrelations among components and uncertainties in various elements such as future demand usually complicates the capital budgeting process. This paper proposes a method for evaluating, selecting and scheduling interrelated road projects in an urban network under demand uncertainties. The objective is to optimally determine the selection, sequence and schedule of capacity improvement projects while minimizing the present value of total system cost, including travel time, vehicle operating and safety costs, subject to a cumulative budget flow constraint. The scheduling problem is formulated as a non-linear integer optimization problem within a genetic algorithm that minimizes the present value of the system cost over a planning horizon. The proposed model also includes a design feature which determines the type of improvement at each location. This study constitutes a useful framework for state planners and regional decision makers for the project prioritization process.

Dative intervention is a gang effect

AbstractThis paper addresses two restrictions regarding agreement with nominative arguments in Icelandic DAT-NOM constructions. The first is the reported asymmetry in intervention effects in mono-clausal versus bi-clausal environments. The second regards the well-known Person Restriction that prohibits agreement with non-3rd person arguments. It is argued that both of these phenomena can be viewed as instances of cumulative constraint interaction, where less important constraints in the grammar ‘gang up’ to block some higher constraint. In order to account for this, I adopt a model of syntax with both weighted constraints and serial optimization that is known asSerial Harmonic Grammarin the phonological literature. It will be demonstrated that such a system can offer a more principled analysis of the construction-specific nature of the aforementioned phenomena.

Linear-time filtering algorithms for the disjunctive constraint and a quadratic filtering algorithm for the cumulative not-first not-last

We present new filtering algorithms for Disjunctive and Cumulative constraints, each of which improves the complexity of the state-of-the-art algorithms by a factor of log n. We show how to perform Time-Tabling and Detectable Precedences in linear time on the Disjunctive constraint. Furthermore, we present a linear-time Overload Checking for the Disjunctive and Cumulative constraints. Finally, we show how the rule of Not-first/Not-last can be enforced in quadratic time for the Cumulative constraint. These algorithms rely on the union find data structure, from which we take advantage to introduce a new data structure that we call it time line. This data structure provides constant time operations that were previously implemented in logarithmic time by the Θ-tree data structure. Experiments show that these new algorithms are competitive even for a small number of tasks and outperform existing algorithms as the number of tasks increases. We also show that the time line can be used to solve specific scheduling problems.