Explicit Constraints Research Articles

DeepSFM: Robust Deep Iterative Refinement for Structure From Motion.

Structure from Motion (SfM) is a fundamental computer vision problem which has not been well handled by deep learning. One of the promising solutions is to apply explicit structural constraint, e.g., 3D cost volume, into the neural network. Obtaining accurate camera poses from images alone can be challenging, especially with complicated environmental factors. Existing methods usually assume accurate camera poses from GT or other methods, which is unrealistic in practice and additional sensors are needed. In this work, we design a physical driven architecture, namely DeepSFM, inspired by traditional Bundle Adjustment, which consists of two cost volume based architectures to iteratively refine depth and pose. The explicit constraints on both depth and pose, when combined with the learning components, bring merit from both traditional BA and emerging deep learning technology. To speed up the learning and inference efficiency, we apply the Gated Recurrent Units (GRUs)-based depth and pose update modules with coarse to fine cost volumes on the iterative refinements. In addition, with the extended residual depth prediction module, our model can be adapted to dynamic scenes effectively. Extensive experiments on various datasets show that our model achieves state-of-the-art performance with superior robustness against challenging inputs.

Norm-Optimal Iterative Learning Control for Wind Turbines During Grid Faults

Due to the increasing share of (offshore) wind turbines, more stringent requirements on power quality have been established. Importantly, the low-voltage ride-through grid requirement states that a wind turbine must remain connected to the electrical grid after a short intermittent grid fault. In the industry mainly gain-scheduled PID-controllers are used to mitigate the effects of grid faults on turbine operation, whereas more advanced solutions have been proposed in the literature such as model predictive control or multiple parallel PI-controllers. Remarkably, all controller implementations mentioned earlier are based on feedback control, where no feedforward strategies have been discussed in the literature. However, feedforward control could improve grid fault recovery performance by exploiting the relatively known fault characteristics by virtue of the specification in the Transmission System Operator requirements. Therefore, for the first time, a norm-optimal Iterative Learning Control (NO-ILC) algorithm is presented that solves these issues by learning the feedforward signal that optimally mitigates the effects of a grid fault. The NO-ILC algorithm applies model-free learning based on iterations, in which the framework of NO-ILC has been extended to include explicit input constraints. The goal of the NO-ILC is to reduce a (quadratic) cost function on specific input and output channels whilst conforming to specific input constraints by solving an optimisation problem, with, for this study blade pitch and rotor speed as respective input and output channels. It is shown that the NO-ILC algorithm can yield improved performance on a high-fidelity model, with a 45% decrease in the cost function used by NO-ILC compared to the nominal feedback control. The optimised feedforward signals resulting from NO-ILC can be used as an analysis tool to closer match the nominal grid fault feedback controllers response with that of NO-ILC, or directly applied as a library that can supplement the feedback controllers output during a grid fault.

First- and second-order optimality conditions for the control of infinite horizon Navier–Stokes equations

First and second-order optimality conditions for optimal control problems over the infinite time horizon subject to the Navier–Stokes equations are derived. The cost functional enhances temporal sparsity of the controls, which implies that the optimal controls shut down in finite time. The problem formulation also includes explicit constraints on the control which may be non-smooth and non-affine.

Experimental validation of a PNS-optimized whole-body gradient coil.

Peripheral nerve stimulation (PNS) limits the usability of state-of-the-art whole-body and head-only MRI gradient coils. We used detailed electromagnetic and neurodynamic modeling to set an explicit PNS constraint during the design of a whole-body gradient coil and constructed it to compare the predicted and experimentally measured PNS thresholds to those of a matched design without PNS constraints. We designed, constructed, and tested two actively shielded whole-body Y-axis gradient coil winding patterns: YG1 is a conventional symmetric design without PNS-optimization, whereas YG2's design used an additional constraint on the allowable PNS threshold in the head-imaging landmark, yielding an asymmetric winding pattern. We measured PNS thresholds in 18 healthy subjects at five landmark positions (head, cardiac, abdominal, pelvic, and knee). The PNS-optimized design YG2 achieved 46% higher average experimental thresholds for a head-imaging landmark than YG1 while incurring a 15% inductance penalty. For cardiac, pelvic, and knee imaging landmarks, the PNS thresholds increased between +22% and +35%. For abdominal imaging, PNS thresholds did not change significantly between YG1 and YG2 (-3.6%). The agreement between predicted and experimental PNS thresholds was within 11.4% normalized root mean square error for both coils and all landmarks. The PNS model also produced plausible predictions of the stimulation sites when compared to the sites of perception reported by the subjects. The PNS-optimization improved the PNS thresholds for the target scan landmark as well as most other studied landmarks, potentially yielding a significant improvement in image encoding performance that can be safely used in humans.

Safe Autonomous Driving with Latent Dynamics and State-Wise Constraints.

Autonomous driving has the potential to revolutionize transportation, but developing safe and reliable systems remains a significant challenge. Reinforcement learning (RL) has emerged as a promising approach for learning optimal control policies in complex driving environments. However, existing RL-based methods often suffer from low sample efficiency and lack explicit safety constraints, leading to unsafe behaviors. In this paper, we propose a novel framework for safe reinforcement learning in autonomous driving that addresses these limitations. Our approach incorporates a latent dynamic model that learns the underlying dynamics of the environment from bird's-eye view images, enabling efficient learning and reducing the risk of safety violations by generating synthetic data. Furthermore, we introduce state-wise safety constraints through a barrier function, ensuring safety at each state by encoding constraints directly into the learning process. Experimental results in the CARLA simulator demonstrate that our framework significantly outperforms baseline methods in terms of both driving performance and safety. Our work advances the development of safe and efficient autonomous driving systems by leveraging the power of reinforcement learning with explicit safety considerations.

Multi-granularity learning of explicit geometric constraint and contrast for label-efficient medical image segmentation and differentiable clinical function assessment

Automated segmentation is a challenging task in medical image analysis that usually requires a large amount of manually labeled data. However, most current supervised learning based algorithms suffer from insufficient manual annotations, posing a significant difficulty for accurate and robust segmentation. In addition, most current semi-supervised methods lack explicit representations of geometric structure and semantic information, restricting segmentation accuracy. In this work, we propose a hybrid framework to learn polygon vertices, region masks, and their boundaries in a weakly/semi-supervised manner that significantly advances geometric and semantic representations. Firstly, we propose multi-granularity learning of explicit geometric structure constraints via polygon vertices (PolyV) and pixel-wise region (PixelR) segmentation masks in a semi-supervised manner. Secondly, we propose eliminating boundary ambiguity by using an explicit contrastive objective to learn a discriminative feature space of boundary contours at the pixel level with limited annotations. Thirdly, we exploit the task-specific clinical domain knowledge to differentiate the clinical function assessment end-to-end. The ground truth of clinical function assessment, on the other hand, can serve as auxiliary weak supervision for PolyV and PixelR learning. We evaluate the proposed framework on two tasks, including optic disc (OD) and cup (OC) segmentation along with vertical cup-to-disc ratio (vCDR) estimation in fundus images; left ventricle (LV) segmentation at end-diastolic and end-systolic frames along with ejection fraction (LVEF) estimation in two-dimensional echocardiography images. Experiments on nine large-scale datasets of the two tasks under different label settings demonstrate our model’s superior performance on segmentation and clinical function assessment.

Abstract Action Scheduling for Optimal Temporal Planning via OMT

Given the model of a system with explicit temporal constraints, optimal temporal planning is the problem of finding a schedule of actions that achieves a certain goal while optimizing an objective function. Recent approaches for optimal planning reduce the problem to a series of queries to an Optimization Modulo Theory (OMT) solver: each query encodes a bounded version of the problem, with additional abstract actions representing an over-approximation of the plans beyond the bound. This technique suffers from performance issues, mainly due to the looseness of the over-approximation, which can include many non-executable plans. In this paper, we propose a refined abstraction for solving optimal temporal planning via OMT by introducing abstract scheduling constraints, which have a double purpose. First, they enforce a partial ordering of abstract actions based on mutual dependencies between them, which leads to a better makespan estimation and allows to prove optimality sooner. Second, they implicitly forbid circular self-enabling of abstract actions, which is a common cause of spurious models that severely affects performance in existing approaches. We prove the soundness and completeness of the resulting approach and empirically demonstrate its superiority with respect to the state of the art.

Benchmarking Large Language Models on Controllable Generation under Diversified Instructions

While large language models (LLMs) have exhibited impressive instruction-following capabilities, it is still unclear whether and to what extent they can respond to explicit constraints that might be entailed in various instructions. As a significant aspect of LLM alignment, it is thus important to formulate such a specialized set of instructions as well as investigate the resulting behavior of LLMs. To address this vacancy, we propose a new benchmark CoDI-Eval to systematically and comprehensively evaluate LLMs' responses to instructions with various constraints. We construct a large collection of constraints-attributed instructions as a test suite focused on both generalization and coverage. Specifically, we advocate an instruction diversification process to synthesize diverse forms of constraint expression and also deliberate the candidate task taxonomy with even finer-grained sub-categories. Finally, we automate the entire evaluation process to facilitate further developments. Different from existing studies on controllable text generation, CoDI-Eval extends the scope to the prevalent instruction-following paradigm for the first time. We provide extensive evaluations of representative LLMs (e.g., ChatGPT, Vicuna) on CoDI-Eval, revealing their limitations in following instructions with specific constraints and there is still a significant gap between open-source and commercial closed-source LLMs. We believe this benchmark will facilitate research into improving the controllability of LLMs' responses to instructions. Our data and code are available at https://github.com/Xt-cyh/CoDI-Eval.

Medical knowledge graph completion via fusion of entity description and type information

Medical Knowledge Graphs (MKGs) are vital in propelling big data technologies in healthcare and facilitating the realization of medical intelligence. However, large-scale MKGs often exhibit characteristics of data sparsity and missing facts. Following the latest advances, knowledge embedding addresses these problems by performing knowledge graph completion. Most knowledge embedding algorithms rely solely on triplet structural information, overlooking the rich information hidden within entity property sets, leading to bottlenecks in performance enhancement when dealing with the intricate relations of MKGs. Inspired by the semantic sensitivity and explicit type constraints unique to the medical domain, we propose BioBERT-based graph embedding model. This model represents an evolvable framework that integrates graph embedding, language embedding, and type information, thereby optimizing the utility of MKGs. Our study utilizes not only WordNet as a benchmark dataset but also incorporates MedicalKG to compare and corroborate the specificity of medical knowledge. Experimental results on these datasets indicate that the proposed fusion framework achieves state-of-art (SOTA) performance compared to other baselines. We believe that this incremental improvement provides promising insights for future medical knowledge graph completion endeavors.

Receding optimal synthesis of robust consensus for multi-agent systems with limited communication

ABSTRACT A novel strategy based on the robust distributed model predictive control (DMPC) is proposed in this paper to address the consensus of multi-agent systems subjected to unknown-but-bounded disturbances with limited communication range. The proposed strategy utilises the formulation of the tube-based predictive control and has the ability to handle the explicit input constraints and achieve the bounded consensus with initial communication topology preserved under the disturbances. In this proposed strategy, a novel constraint is designed to guarantee the connectivity of the system under the disturbances, and the synthesis methods are proposed for the controller. Meanwhile, it is proved that the proposed algorithm in this paper is recursively feasible, the connectivity can always be preserved and the system can finally reach bounded consensus under the disturbances. A simulation example is provided to demonstrate the proposed robust DMPC-based strategy.

Implicit policy constraint for offline reinforcement learning

AbstractOffline reinforcement learning (RL) aims to learn policies entirely from passively collected datasets, making it a data‐driven decision method. One of the main challenges in offline RL is the distribution shift problem, which causes the algorithm to visit out‐of‐distribution (OOD) samples. The distribution shift can be mitigated by constraining the divergence between the target policy and the behaviour policy. However, this method can overly constrain the target policy and impair the algorithm's performance, as it does not directly distinguish between in‐distribution and OOD samples. In addition, it is difficult to learn and represent multi‐modal behaviour policy when the datasets are collected by several different behaviour policies. To overcome these drawbacks, the authors address the distribution shift problem by implicit policy constraints with energy‐based models (EBMs) rather than explicitly modelling the behaviour policy. The EBM is powerful for representing complex multi‐modal distributions as well as the ability to distinguish in‐distribution samples and OODs. Experimental results show that their method significantly outperforms the explicit policy constraint method and other baselines. In addition, the learnt energy model can be used to indicate OOD visits and alert the possible failure.

Signal processing collaborated with deep learning: An interpretable FIRNet for industrial intelligent diagnosis

Due to the neglect of prior characteristics and the lack of explicit constraints on fault knowledge, conventional intelligent diagnosis methods suffer from great hardships in exacting fault-sensitive information and making explainable decisions, resulting in poor interpretability and inferior robustness. Motivated by the excellent multiscale analysis performance of signal processing and the powerful feature mining ability of deep learning, this study proposes an interpretable FIRNet for equipment intelligent diagnosis under strong noise environments. It consists of a well-crafted FIRLayer and a deep learning backbone. Inspired by the modulation principle that fault-sensitive components are normally modulated into multiscale mode characteristics, multiple ex-ante interpretable filters with two learnable parameters, including center frequency and bandwidth, are analytically designed to process the sequence signal sets, represented as signal-processing-based FIRLayer where the extracted multiscale feature maps are taken as an interpretable status information expression. Subsequently, multiscale convolutional kernels are established to extract the high-level feature maps and further make the final diagnostic decisions, represented as the deep learning backbone. The simulated and experimental results show that the proposed FIRNets have higher identification precision compared to the other nine deep learning models. Specially, three aspects, including model interpretability, noise robustness capacity and edge intelligent diagnosis, are further analyzed to illustrate the interpretable advantages of FIRNets. Hereinto, feature visualizations of the FIRLayer and mode decision contributions of the deep learning backbone are in-depth analyzed to verify the interpretable feature representation and decision-making principle of FIRNets. The results indicate that FIRNets have superior ex-post interpretability compared to other methods. Focusing on industrial practices, an efficient edge diagnosis system based on a pruned FIRNet is established, and an online diagnosis accuracy of more than 99% has been achieved. It can be foreseen that the proposed FIRNets show great potential and competitiveness to promote the edge computing application of equipment intelligent diagnosis.

A robot motion skills method with explicit environmental constraints

Purpose This paper aims to solve the problem of the inability to apply learning methods for robot motion skills based on dynamic movement primitives (DMPs) in tasks with explicit environmental constraints, while ensuring the reliability of the robot system. Design/methodology/approach The authors propose a novel DMP that takes into account environmental constraints to enhance the generality of the robot motion skill learning method. First, based on the real-time state of the robot and environmental constraints, the task space is divided into different regions and different control strategies are used in each region. Second, to ensure the effectiveness of the generalized skills (trajectories), the control barrier function is extended to DMP to enforce constraint conditions. Finally, a skill modeling and learning algorithm flow is proposed that takes into account environmental constraints within DMPs. Findings By designing numerical simulation and prototype demonstration experiments to study skill learning and generalization under constrained environments. The experimental results demonstrate that the proposed method is capable of generating motion skills that satisfy environmental constraints. It ensures that robots remain in a safe position throughout the execution of generation skills, thereby avoiding any adverse impact on the surrounding environment. Originality/value This paper explores further applications of generalized motion skill learning methods on robots, enhancing the efficiency of robot operations in constrained environments, particularly in non-point-constrained environments. The improved methods are applicable to different types of robots.

Transfer learning neural network for reconstructing temperature field in film cooling with scarce local measurements

Film cooling is widely adopted for thermal protection of gas turbine blades. Obtaining the surface temperature field is essential for film cooling design. However, it is generally challenging because temperature measurement points are scarce in practice. Although machine learning models can be trained to reconstruct the temperature field, their accuracy is limited when the measurement points are insufficient and the explicit physical constraints are lacking. To address the problem, a transfer learning neural network (TLNN) is proposed in present study. The main idea is to learn the general characteristics of temperature field from abundant data in other working conditions as implicit constraints, and to make fine-tuning of the model for a specific working condition with scarce measurement points. Numerical simulations were conducted under various working conditions to generate dataset for model training and testing. With the blowing ratios of 1 and 0.5 used as the source and target working conditions, the mean relative L2-norm errors are 52 %, 41 %, and 32 % lower for 20, 10, and 5 measurement points, respectively, compared to the model without transfer learning. The results indicate that the proposed TLNN model can accurately reconstruct the temperature field in film cooling with scarce measurement points.

Explicit Effects and Effect Constraints in ReML

An important aspect of building robust systems that execute on dedicated hardware and perhaps in constrained environments is to control and manage the effects performed by program code. We present ReML, a higher-order statically-typed functional language, which allows programmers to be explicit about the effects performed by program code and in particular effects related to memory management. Allowing programmers to be explicit about effects, the regions in which values reside, and the constraints under which code execute, makes programs robust to changes in the program source code and to compiler updates, including compiler optimisations. ReML is integrated with a polymorphic inference system that builds on top of region-inference, as it is implemented in the MLKit, a Standard ML compiler that uses region-based memory management as its primary memory management scheme.

KeyCLD: Learning constrained Lagrangian dynamics in keypoint coordinates from images

We present KeyCLD, a framework to learn Lagrangian dynamics from images. Learned keypoints represent semantic landmarks in images and can directly represent state dynamics. We show that interpreting this state as Cartesian coordinates, coupled with explicit holonomic constraints, allows expressing the dynamics with a constrained Lagrangian. KeyCLD is trained unsupervised end-to-end on sequences of images. Our method explicitly models the mass matrix, potential energy and the input matrix, thus allowing energy based control. We demonstrate learning of Lagrangian dynamics from images on the cl_bnmsqnk pendulum, cartpole and acrobot environments. KeyCLD can be learned on these systems, whether they are unactuated, underactuated or fully actuated. Trained models are able to produce long-term video predictions, showing that the dynamics are accurately learned. We compare with Lag-VAE, Lag-caVAE and HGN, and investigate the benefit of the Lagrangian prior and the constraint function. KeyCLD achieves the highest valid prediction time on all benchmarks. Additionally, a very straightforward energy shaping controller is successfully applied on the fully actuated systems.

On the Solution to Optimal Topology Adjustment in Long-Term Power System Operation With Explicit Short-Circuit Current Limitation Constraints

To restrict the excessive short-circuit current (SCC), this letter proposes an SCC-constrained optimal topology adjustment model considering the N-1 security criterion. Topology adjustment, including transmission switching (TS), bus sectionalization (BS), lines operating out of breaker bay (LOB) and commitment status of units (CU), has nonlinear relationships with SCC and is difficult to express explicitly. In this letter, an explicit formulation of SCC considering topology adjustment is proposed based on the equation of admittance matrix and nodal self-impedance, which is further recast as a mixed-integer linear programming (MILP) model. In addition, the N-1 security criterion is included for avoiding jeopardizing system security by topology adjustment. Numerical case studies validate the effectiveness and advantages of the proposed model in SCC restriction.

Scalable enforcement of geometric non-interference constraints for gradient-based optimization

AbstractMany design optimization problems include constraints to prevent intersection of the geometric shape being optimized with other objects or with domain boundaries. When applying gradient-based optimization to such problems, the constraint function must provide an accurate representation of the domain boundary and be smooth, amenable to numerical differentiation, and fast-to-evaluate for a large number of points. We propose the use of tensor-product B-splines to construct an efficient-to-evaluate level set function that locally approximates the signed distance function for representing geometric non-interference constraints. Adapting ideas from the surface reconstruction methods, we formulate an energy minimization problem to compute the B-spline control points that define the level set function given an oriented point cloud sampled over a geometric shape. Unlike previous explicit non-interference constraint formulations, our method requires an initial setup operation, but results in a more efficient-to-evaluate and scalable representation of geometric non-interference constraints. This paper presents the results of accuracy and scaling studies performed on our formulation. We demonstrate our method by solving a medical robot design optimization problem with non-interference constraints. We achieve constraint evaluation times on the order of $$10^{-6}$$ 10 - 6 seconds per point on a modern desktop workstation, and a maximum on-surface error of less than 1.0% of the minimum bounding box diagonal for all examples studied. Overall, our method provides an effective formulation for non-interference constraint enforcement with high computational efficiency for gradient-based design optimization problems whose solutions require at least hundreds of evaluations of constraints and their derivatives.

A limit analysis-based CASS approach for the in-plane seismic capacity of masonry façades

The present paper proposes a new methodology for the load-bearing capacity analysis of 2D masonry constructions by extending and reformulating the Continuous Airy-based for Stress Singularities (CASS) method, with a particular focus on the incorporation of volume forces, crucial to solve mechanical problems involving masonry constructions accurately. The masonry material is modelled using a Normal, Rigid, No-Tension (NRNT) model, largely adopted by the scientific community as material parameters, often unknowable, are not needed.The load-bearing capacity problem is derived from an energy-based formulation and framed as a classic limit analysis approach, allowing for the direct determination of the maximum incremental load that a masonry structure can withstand, along with the corresponding internal stress pattern.The structural domain is discretised with a simple finite element mesh, and the Airy stress potential is adopted to enforce the internal equilibrium directly. The boundary value problem is then solved as second-order cone programming (SOCP).The CASS method is shown to offer modelling and computational advantages. Indeed, it accurately describes the mechanical response, including the ability to capture singular stress fields typically exhibited by masonry structures, particularly where cracks appear. The adoption of the Airy potential allows for a reduction in the number of explicit constraints from the problem. Moreover, the formulation of the boundary value problem as a SOCP ensures the existence of a unique load multiplier while offering computationally fast solutions even for large problems.Several numerical examples are presented to demonstrate the CASS potential. Specifically, the ability to capture singular stress patterns diagonally crossing the finite elements showcases the CASS mesh independence, providing a straightforward approach to modelling complex geometries and loading conditions.

Dependency-based BERT for Chinese Event Argument Extraction

Existing event extraction methods independently identify and classify each argument role separately, ignoring the interdependence between different parameter roles. Further, these methods rely on simple vectors to represent word embeddings. By embedding explicit syntactic constraints in the attention mechanism, we address these shortcomings by using dependency syntax to guide the text modeling. Specifically, we use dependency syntax to guide the BERT model for Chinese event argument role extraction, which mainly consists of three stages. First, the self-attention method guided by a dependency syntactic parsing tree is embedded in the Transformer computing framework of the BERT model. In addition to obtaining a deep two-way linguistic representation of a word according to its context information, this method also expresses the long-distance syntactic dependency relationships between words based on context information. Secondly, the designed conditional layer normalization method is applied to the event argument extraction model in order to integrate the semantic information of trigger words into the text, thereby improving the accuracy of the argument role extraction. Lastly, conditional random fields (CRFs) are used to determine the optimal sequence of labels at the sentence level based on the dependence relationships between adjacent labels. According to the experimental results, the constructed model outperforms several strong baselines in the Chinese event argument extraction task on the ACE2005 dataset and the iFLYTEKA.I.2020 dataset.