Continuous-time Formulation Research Articles

On the ensemble Kalman inversion under inequality constraints

Abstract The ensemble Kalman inversion (EKI), a recently introduced optimisation method for solving inverse problems, is widely employed for the efficient and derivative-free estimation of unknown parameters. Specifically in cases involving ill-posed inverse problems and high-dimensional parameter spaces, the scheme has shown promising success. However, in its general form, the EKI does not take constraints into account, which are essential and often stem from physical limitations or specific requirements. Based on a log-barrier approach, we suggest adapting the continuous-time formulation of EKI to incorporate convex inequality constraints. We underpin this adaptation with a theoretical analysis that provides lower and upper bounds on the ensemble collapse, as well as convergence to the constraint optimum for general nonlinear forward models. Finally, we showcase our results through two examples involving partial differential equations.

A novel exact formulation for parallel machine scheduling problems

Machine scheduling is one of the most studied problems due to its technical challenges and prevalence in real life. In the literature, continuous- and discrete-time formulations are the two most known formulations for scheduling problems. However, continuous-time formulations often suffer from weak linear relaxations, while discrete-time formulations struggle with large numbers of variables. In contrast, the bucket-indexed formulation is an alternative that mitigates both issues by working with partial time discretization. We propose a mixed-integer linear programming model based on a bucket-indexed formulation to solve a nonpreemptive scheduling problem of identical parallel machines considering release dates, deadlines, precedence, eligibility, and machine availability constraints. We evaluate the proposed formulation against real-world instances comprising more than 400 jobs and 100 machines, comparing its performance against equivalent continuous- and discrete-time formulations. Remarkably, our formulation can be solved to optimality for all instances, outperforming both continuous- and discrete-time formulations.

Continuous-Time Fixed-Lag Smoothing for LiDAR-Inertial-Camera SLAM

Localization and mapping with heterogeneous multisensor fusion have been prevalent in recent years. To adequately fuse multimodal sensor measurements received at different time instants and different frequencies, we estimate the continuous-time trajectory by fixed-lag smoothing within a factor-graph optimization framework. With the continuous-time formulation, we can query poses at any time instants corresponding to the sensor measurements. To bound the computation complexity of the continuous-time fixed-lag smoother, we maintain temporal and keyframe sliding windows with constant size, and probabilistically marginalize out control points of the trajectory and other states, which allows preserving prior information for future sliding-window optimization. Based on continuous-time fixed-lag smoothing, we design tightly coupled multimodal SLAM algorithms with a variety of sensor combinations, like the LiDAR-inertial and LiDAR-inertial-camera SLAM systems, in which online time offset calibration is also naturally supported. More importantly, benefiting from the marginalization and our derived analytical Jacobians for optimization, the proposed continuous-time SLAM systems can achieve real-time performance regardless of the high complexity of continuous-time formulation. The proposed multimodal SLAM systems have been widely evaluated on three public datasets and self-collect datasets. The results demonstrate that the proposed continuous-time SLAM systems can achieve high-accuracy pose estimations and outperform existing state-of-the-art methods. To benefit the research community, we will open source our code at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/APRIL-ZJU/clic</uri> .

A continuous-time formulation for scheduling crude oil operations in a terminal with a refinery pipeline

The reception, mixture, delivery, and processing of crude oil at a refinery are modeled and validated. We employ a continuous-time model based on priority-slots and a multi-operation sequencing approach. The refinery’s desired qualities are defined as equivalent to the terminal mixtures, thus streamlining the formulation and eliminating any bilinear terms in the equations related to pipeline delivery. The goal is to find an optimized schedule that satisfies all constraints, for which an MINLP model is proposed and solved. However, by ruling out mixtures in tanks, the formulation is reduced to a manageable MILP model. Also, the Relax & Fix strategy can be used to determine an optimized schedule of operations over a time horizon in a timely and cost-effective manner. This work presents an innovative solution to the complex challenge of crude oil reception, mixture, pipeline delivery, and refinery processing.

On the Global Convergence of Particle Swarm Optimization Methods

In this paper we provide a rigorous convergence analysis for the renowned particle swarm optimization method by using tools from stochastic calculus and the analysis of partial differential equations. Based on a continuous-time formulation of the particle dynamics as a system of stochastic differential equations, we establish convergence to a global minimizer of a possibly nonconvex and nonsmooth objective function in two steps. First, we prove consensus formation of an associated mean-field dynamics by analyzing the time-evolution of the variance of the particle distribution, which acts as Lyapunov function of the dynamics. We then show that this consensus is close to a global minimizer by employing the asymptotic Laplace principle and a tractability condition on the energy landscape of the objective function. These results allow for the usage of memory mechanisms, and hold for a rich class of objectives provided certain conditions of well-preparation of the hyperparameters and the initial datum. In a second step, at least for the case without memory effects, we provide a quantitative result about the mean-field approximation of particle swarm optimization, which specifies the convergence of the interacting particle system to the associated mean-field limit. Combining these two results allows for global convergence guarantees of the numerical particle swarm optimization method with provable polynomial complexity. To demonstrate the applicability of the method we propose an efficient and parallelizable implementation, which is tested in particular on a competitive and well-understood high-dimensional benchmark problem in machine learning.

A new cumulative scheduling model using effective continuous-time formulation and dominance rules for optimality

The cumulative scheduling problem has been traditionally represented using a discrete time formulation, incorporating the variable resource consumption of each job as decision variables. In our research, we aim to provide a more accessible approach to formulating a specific configuration of this problem. We establish an optimality condition for resource allocation, which enables us to eliminate these decision variables and develop a simpler, more concise, and efficient continuous-time formulation.Furthermore, we introduce two optimality-based dominance rules for the continuous-time model, which are strengthened by valid inequalities. Computational experiments reveal that the continuous-time formulation significantly outperforms existing state-of-the-art approaches. Additionally, the optimality-driven dominance rules and inequalities contribute to enhancing the performance of the continuous-time model, especially in large-scale instances.

Spiking neural predictive coding for continually learning from data streams

For energy-efficient computation in specialized neuromorphic hardware, we present spiking neural coding, an instantiation of a family of artificial neural models grounded in the theory of predictive coding. This model, the first of its kind, works by operating in a never-ending process of “guess-and-check”, where neurons predict the activity values of one another and then adjust their own activities to make better future predictions. The interactive, iterative nature of our system fits well into the continuous time formulation of sensory stream prediction and, as we show, the model’s structure yields a local synaptic update rule, which can be used to complement or as an alternative to online spike-timing dependent plasticity. In this article, we experiment with an instantiation of our model consisting of leaky integrate-and-fire units. However, the framework within which our system is situated can naturally incorporate more complex neurons such as the Hodgkin-Huxley model. Our experimental results in pattern recognition demonstrate the potential of the model when binary spike trains are the primary paradigm for inter-neuron communication. Notably, spiking neural coding is competitive in terms of classification performance and experiences less forgetting when learning from a task sequence, offering a more computationally economical, biologically-motivated alternative to popular artificial neural networks.

Lidar Pose Tracking of a Tumbling Spacecraft Using the Smoothed Normal Distribution Transform

Lidar sensors enable precise pose estimation of an uncooperative spacecraft in close range. In this context, the iterative closest point (ICP) is usually employed as a tracking method. However, when the size of the point clouds increases, the required computation time of the ICP can become a limiting factor. The normal distribution transform (NDT) is an alternative algorithm which can be more efficient than the ICP, but suffers from robustness issues. In addition, lidar sensors are also subject to motion blur effects when tracking a spacecraft tumbling with a high angular velocity, leading to a loss of precision in the relative pose estimation. This work introduces a smoothed formulation of the NDT to improve the algorithm’s robustness while maintaining its efficiency. Additionally, two strategies are investigated to mitigate the effects of motion blur. The first consists in un-distorting the point cloud, while the second is a continuous-time formulation of the NDT. Hardware-in-the-loop tests at the European Proximity Operations Simulator demonstrate the capability of the proposed methods to precisely track an uncooperative spacecraft under realistic conditions within tens of milliseconds, even when the spacecraft tumbles with a significant angular rate.

Continuous-time formulations for multi-mode project scheduling

This paper reviews compact continuous-time formulations for the multi-mode resource-constrained project scheduling problem. Specifically, we first point out a serious flaw in an existing start-end-event-based formulation owing to inconsistent mode choices. We propose two options to formulate the missing constraints and consider an equivalent reformulation with sparser constraint matrix. Second, we formulate an aggregate variant of an existing model that relies on on-off-events, and we clarify the role of mode consistency issues in such models. Third, we suggest two variants of an existing network flow formulation. We enhance our models by adapting several techniques that have been used previously, e.g., in cases with only a single mode. A large set of benchmark instances from the literature provides the basis for an up-to-date and fair computational study with an out-of-the-box solver package. We compare our models against two models from the literature. Our experiments assert confidently that network flow formulations prevail in the test bed, and they provide a hint on why event-based models become less competitive in multi-mode settings.

Simultaneous Scheduling and Synthesis of Industrial Water Allocation Networks

This work addresses integration of batch scheduling with water allocation, recycle and reuse opportunities for freshwater minimization in batch plants via sequential and simultaneous methodologies. The presented scheduling model is based on state task network representation and unit-specific event based continuous time formulation. In the production scheduling model, a three-index finish time variable has been considered for handling multiple states having different processing time durations for the same task in a processing unit. The scheduling model introduces constraints to handle storage violations for production and consumption of the same state in the same unit. In the water network model for freshwater minimization, a regeneration unit along with a central water storage tank has been included to exploit the possibility of water reuse in the washing units. Four case studies are solved with single and multiple contaminants to evaluate the performance of the proposed model, which gives better savings in terms of freshwater consumption and thus also minimizes the effluent generation. Additionally, a preliminary analysis for two-objective optimization is presented where revenue is maximized, and the total water cost is minimized simultaneously using the weighted-sum method.

Tracking MPC Tuning in Continuous Time: A First-Order Approximation of Economic MPC

Economic MPC (EMPC) optimizes closed-loop performance by directly minimizing a given objective function, as opposed to Tracking MPC (TMPC) which instead penalizes deviations from a precalculated optimal reference. The main difference between the two approaches can be observed during transients, as the former always acts optimally, while the latter is only optimal when the reference is accurately tracked. Unfortunately, stability for EMPC is in general difficult to prove, as opposed to TMPC which builds on a rich theory. Additionally, many efficient algorithms are available for TMPC, while solving the EMPC problem can be much harder. In prior works 1, 2, a family of discrete-time TMPC schemes that provide approximate economic optimality has been developed in order to partially overcome these issues. In this paper, we aim at extending such a family of TMPC schemes to the continuous time case. Similarly to the discrete-time case, also in continuous-time we obtain a first-order approximation of the EMPC control law. We demonstrate the theory with a numerical example that confirms the first-order approximation and we show that our continuous-time formulation can be made equivalent to the discrete-time one.

Continuous and Discrete Volterra-Laguerre Models With Delay for Modeling of Smooth Pursuit Eye Movements.

The mathematical modeling of the human smooth pursuit system from eye-tracking data is considered. Recently developed algorithms for the estimation of Volterra-Laguerre (VL) models with explicit time delay are applied in continuous and discrete time formulations to experimental data collected from Parkinsonian patients in different medication states and healthy controls. The discrete VL model with an explicit time delay and the method for its estimation are first introduced in this paper. The estimated parameters of a second-order VL model are shown to capture the ocular dynamics both in health and disease. The possibility of including the estimated time delay, along with the VL kernel parameters, into the set of the model parameters is explored. The results obtained in continuous VL modeling are compared with those in discrete time to discern the effects due to the sampling enforced by the eye tracker used for data acquisition.

Distributed Hybrid Observer With Prescribed Convergence Rate for a Linear Plant Using Multi-Hop Decomposition

With a continuous-time formulation of the multihop decomposition, we propose a distributed hybrid observer for a sensor network where the plant and local observers run in continuous time and the information exchange among the sensing nodes is sampled-data. Process disturbances, measurement noise and communication noise are considered, and we prove that under some necessary detectability assumptions the observer gains can be tuned to guarantee exponential ISS with a prescribeda convergence rate. Simulations illustrate the performance of the proposed observer.

Observability-Aware Intrinsic and Extrinsic Calibration of LiDAR-IMU Systems

Accurate and reliable sensor calibration is essential to fuse LiDAR and inertial measurements, which are usually available in robotic applications. In this article, we propose a novel LiDAR-IMU calibration method within the continuous-time batch-optimization framework, where the intrinsics of both sensors and the spatial-temporal extrinsics between sensors are calibrated without using calibration infrastructure, such as fiducial tags. Compared to discrete-time approaches, the continuous-time formulation has natural advantages for fusing high-rate measurements from LiDAR and IMU sensors. To improve efficiency and address degenerate motions, the following two observability-aware modules are leveraged: first, The information-theoretic data selection policy selects <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">only</i> the most informative segments for calibration during data collection, which significantly improves the calibration efficiency by processing only the selected informative segments. Second, the observability-aware state update mechanism in nonlinear least-squares optimization updates <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">only</i> the identifiable directions in the state space with truncated singular value decomposition, which enables accurate calibration results even under degenerate cases where informative data segments are not available. The proposed LiDAR-IMU calibration approach has been validated extensively in both simulated and real-world experiments with different robot platforms, demonstrating its high accuracy and repeatability in commonly-seen human-made environments.

Refined Bounds for Inter-Carrier Interference in OFDM Due to Time-Varying Channels and Phase Noise

Inter-carrier interference (ICI) due to time-variations of the channel and phase noise is a critical issue in current orthogonal frequency division multiplexing (OFDM) systems. For the discrete-time formulation of OFDM and the general case of partially occupied contiguous subcarriers, we present exact expressions for the average ICI power at each subcarrier. These lead to insightful lower and upper bounds for the bandwidth-averaged ICI power, which bring out the combined impact of Doppler spread and phase noise statistics. The bounds are tighter than those in the literature, which employ a tractable, but less accurate, continuous-time formulation and assume infinitely many subcarriers.

Discrete-Time Distributed Population Dynamics for Optimization and Control

Distributed population dynamics offer a game-theoretical framework for distributed decision making. As such, the application of these methods to the control of networked systems has been widely studied in the literature. The theoretical analyses available in the literature have only considered continuous-time formulations of distributed population dynamics. However, given that modern computers can only process information in a discrete-time fashion, the practical implementation of distributed population dynamics-based methods is inevitably discrete. In consequence, it is paramount to extend the available theory to a more implementable discrete-time approach. For that reason, in this article, we provide a discrete-time analysis of a general class of distributed population dynamics, and we derive sufficient conditions on the discretization time to ensure the asymptotic stability of the discretized dynamics. To illustrate the relevance and performance of the proposed methods, we apply the developed theory to distributed optimization and control problems, including a real multirobot platform, which consider noncomplete communication networks and coupled constraints.

A continuous-time formulation for optimal task scheduling and quality-of-service assurance in nanosatellites

Satellite scheduling concerns an operations research problem that aims at planning the execution of a given set of tasks, either in real-time onboard the spacecraft or offline in the ground station. In particular, the task scheduling problem for nanosatellites has been the focus of recent research, given the unique set of constraints and applications, and considering their limited power resources. This paper applies the continuous-time methodology to deliver a novel scheduling formulation for the nanosatellite scheduling problem with increased flexibility and versatility to the decision-maker. Here, we adopt the concept of priority slots and follow the Multi-Operation Sequencing with Synchronized Start Times (MOS-SST) representation to yield a Mixed-Integer Linear Programming (MILP) formulation that considers constraints on time synchronization, mission quality-of-service, and management of shared resources. By comparing our continuous-time strategy with existing discrete-time alternatives, we showed that the proposed method can solve representative problem instances in a considerably shorter time, while being able to deal with real-valued constraint functions.

Asset pricing under smooth ambiguity in continuous time

We study asset pricing implications of a revealing and tractable formulation of smooth ambiguity investor preferences in a continuous-time environment. Investors do not observe a hidden Markov state and instead make inferences about this state using past data. We show that ambiguity about this hidden state distribution alters investor decisions and equilibrium asset prices. Our continuous-time formulation allows us to apply recursive filtering and Hamilton–Jacobi–Bellman methods to solve the modified decision problem. Using such methods, we show how characterizations of portfolio allocations and local uncertainty-return tradeoffs change when investors are ambiguity-averse.

A New MINLP Continuous Time Formulation for Scheduling Optimization of Oil Refinery with Unreliable CDUs

Short-term scheduling of oil refinery operations is a complicated optimization problem that requires a high level of detail and needs efficient approaches and software tools. Oil refineries are naturally categorized as continuous process industries due to their production processes, which have essentially different characteristics and constraints from discrete manufacturing processes. It is a significant problem to design effective approaches for oil refinery scheduling optimization. Therefore, this paper aims to provide solutions to the above very challenging problem by proposing a novel methodology. The problem is how to optimize crude oil unloading to charging tanks and charging schedules for different types of crude oils to distillation units (CDUs). Several realistic operational features are considered, such as different arrival times for crude oils, continuous operation of CDUs, no rework of crude oils, and the ability to perform preventative maintenance on CDUs. To solve this problem effectively, a new mixed-integer nonlinear programming (MINLP) model is developed. Industrial case studies are used to demonstrate the effectiveness of the proposed formulation. The computational results show that the case studies are effectively solved with the proposed solution approach.

A multiscale domain decomposition approach for parabolic equations using expanded mixed method

We study the multiscale mortar expanded mixed method for second order parabolic partial differential equations. This technique involves non-overlapping domain decomposition which restrict a large problem into smaller pieces. Each block in the domain decomposition is discretized by expanded mixed method. The flux continuity is ensured by constructing a finite element space on the interface between subdomains. A pressure variable is introduced on the interfaces which play the role of Dirichlet boundary condition for each subdomain internal boundary. We demonstrate the unique solvability of discrete problem. We considered both continuous time and discrete time formulation and established a priori error estimates for local subdomain approximations. We derived the optimal order convergence for the appropriate choice of mortar space. An error estimate for mortar pressure variable is also provided. The numerical experiments are performed to show the accuracy and effectiveness of method.