Implicit Constraints Research Articles

Failure-Averse Active Learning for Physics-Constrained Systems

Active learning is a subfield of machine learning that is devised for design and modeling of systems with highly expensive sampling costs. Industrial and engineering systems are generally subject to physics constraints that may induce fatal failures when they are violated, while such constraints are frequently underestimated in active learning. In this paper, we develop a novel active learning method that avoids failures considering implicit physics constraints that govern the system. The proposed approach is driven by two tasks: the safe variance reduction explores the safe region to reduce the variance of the target model, and the safe region expansion aims to extend the explorable region exploiting the probabilistic model of constraints. The global acquisition function is devised to judiciously optimize acquisition functions of two tasks, and its theoretical properties are provided. The proposed method is applied to the composite fuselage assembly process with consideration of material failure using the Tsai-wu criterion, and it is able to achieve zero-failure without the knowledge of explicit failure regions.

HmcNet: A General Approach for Hierarchical Multi-Label Classification

Hierarchical multi-label classification (HMC) deals with the problem of assigning each entity to multiple classes with a taxonomic structure (e.g., tree). Within this structure, classes at different levels tend to have dependencies under the hierarchy constraints. However, most prior studies for HMC tasks tend to ignore the class dependencies within the hierarchy. Moreover, most existing methods generate incoherent predictions and do not satisfy the hierarchy constraint. To this end, based on previously developed HARNN, we propose a general framework, HmcNet, for introducing explicit and implicit class hierarchy constraints to generate coherent predictions. We develop an efficient Prune-based Coherent Prediction (PCP) strategy for the optimal paths selection, which produces coherent predictions in a principled way. HmcNet can be well explained from two perspectives. First, it develops the Hierarchical Attention-based Memory (HAM) unit with implicit class hierarchy constraints to capture class dependencies more intuitively; Second, it subsumes explicit class hierarchy constraints during training and inference phases and generates coherent predictions in a consistent manner. Finally, extensive experimental results on six real-world datasets demonstrate the effectiveness and interpretability of the HmcNet frameworks. To facilitate future research, our code has been made publicly available.

Self-supervised deep learning for joint 3D low-dose PET/CT image denoising

Deep learning (DL)-based denoising of low-dose positron emission tomography (LDPET) and low-dose computed tomography (LDCT) has been widely explored. However, previous methods have focused only on single modality denoising, neglecting the possibility of simultaneously denoising LDPET and LDCT using only one neural network, i.e., joint LDPET/LDCT denoising. Moreover, DL-based denoising methods generally require plenty of well-aligned LD-normal-dose (LD-ND) sample pairs, which can be difficult to obtain. To this end, we propose a self-supervised two-stage training framework named MAsk-then-Cycle (MAC), to achieve self-supervised joint LDPET/LDCT denoising. The first stage of MAC is masked autoencoder (MAE)-based pre-training and the second stage is self-supervised denoising training. Specifically, we propose a self-supervised denoising strategy named cycle self-recombination (CSR), which enables denoising without well-aligned sample pairs. Unlike other methods that treat noise as a homogeneous whole, CSR disentangles noise into signal-dependent and independent noises. This is more in line with the actual imaging process and allows for flexible recombination of noises and signals to generate new samples. These new samples contain implicit constraints that can improve the network’s denoising ability. Based on these constraints, we design multiple loss functions to enable self-supervised training. Then we design a CSR-based denoising network to achieve joint 3D LDPET/LDCT denoising. Existing self-supervised methods generally lack pixel-level constraints on networks, which can easily lead to additional artifacts. Before denoising training, we perform MAE-based pre-training to indirectly impose pixel-level constraints on networks. Experiments on an LDPET/LDCT dataset demonstrate its superiority over existing methods. Our method is the first self-supervised joint LDPET/LDCT denoising method. It does not require any prior assumptions and is therefore more robust.

Cost Optimization of Prestressed U-Shaped Simply Supported Girder Using Box Complex Method

The use of U-shaped girders has become increasingly popular in advanced projects such as metro rail systems due to their ability to provide greater vertical clearance beneath bridges. These girders, characterized by two webs and a bottom flange, contribute essential longitudinal stiffness and strength to the overall structure while effectively countering torsional forces in curved bridges. However, the design and construction of U-shaped girders present challenges, including their relatively higher self-weight compared to other girder types. Consequently, cost optimization has become a crucial focus in structural design studies. This research aims to develop an optimization model for prestressed U-shaped girders using the AASHTO LRFD bridge design specifications. The model is based on the Box complex method, with necessary modifications and improvements to achieve an optimal design. The objective is to minimize the total cost of materials, including concrete, steel reinforcement, and prestressing strands, while satisfying explicit and implicit design constraints. To facilitate the analysis, design, and optimization processes, a program is developed using Visual Studio 2010 and implemented in Visual Basic (VB.NET). The program incorporates separate subroutines for analysis, design, and optimization of the prestressed U-shaped girder, which are integrated to produce the desired results. When running the program, the optimization process required 229 iterations to converge to the optimal cost function value. The results demonstrate that the developed algorithm efficiently explores economically and structurally effective solutions, resulting in cost savings compared to the initial design. The convergence rate of the moment capacity constraint is identified as a key factor in achieving the optimal design. This research makes a significant contribution to the field of civil engineering by applying the classical Box complex method to the optimization of girders, an area where its utilization has been limited. Furthermore, this study specifically addresses the optimization of prestressed U-shaped girders in metro rail projects, where they serve as both the deck and support structure for train loading. By employing the Box complex method, this research aims to fill the research gap and provide valuable insights into the optimization of U-shaped girders. This approach offers a fresh perspective on designing these girders, considering their unique role in supporting metro rail loads. By leveraging the benefits of the Box complex method, researchers can explore new possibilities and uncover optimal design solutions for U-shaped girders in metro rail applications.

Structure invariance-driven collaborative contrastive network for EEG decoding

With a large amount of augmented trainingdata generated by data augmentation (DA) methods, deepneural networks (DNN) have achieved promising performancein decoding electroencephalogram (EEG) neuralpatterns for brain-computer interface (BCI). However, mostcurrent methods neglect the data-structure informationavailable in different views of augmented EEG datasets.In particular, the invariance of data-structure informationcan improve model generalization ability but unexploited.In this paper, we develop the novel Structure Invariance-drivenCollaborative Contrastive Network (SICCNet) forEEG recognition. It is a framework with both implicit andexplicit constraints to exploit invariant data-structure information.SICCNet contains two structure-guided contrastivelearning subnets to perform collaborative contrasts betweenpairs of normalized embeddings derived from differentaugmented views. First, implicit constraint is introducedwith the Inter-Segment Mixup-based (InSegMix)EEG data augmentation strategy to generate new EEGviews without introducing extra noise. Explicit constraint isthen introduced through the collaborative contrastive (co-contrastive)regularization scheme to exploit the invariantdata-structure information from multiple views. The schemecan direct the model to focus on the invariant data-structureinformation and consequently enhance the discriminationability of the model. Extensive experiments on three publicmotor-imagery (MI) based EEG datasets have demonstratedthe outstanding performance of SICCNet when comparedwith other state-of-the-art methods, indicating its promisingpotential in improving BCI system.

Planning for Automated Testing of Implicit Constraints in Behavior Trees

Behavior Trees (BTs) are a formalism increasingly used to control the execution of robotic systems. The strength of BTs resides in their compact, hierarchical and transparent representation. However, when used in practical applications transparency is often hindered by the introduction of implicit run-time relations between nodes, e.g., because of data dependencies or hardware-related ordering constraints. Manually verifying the correctness of a BT with respect to these hidden relations is a tedious and error-prone task. This paper presents a modular planning-based approach for automatically testing BTs offline at design time, to identify possible executions that may violate given data and ordering constraints and to exhibit traces of these executions to help debugging. Our approach supports both basic and advanced BT node types, e.g., supporting parallel behaviors, and can be extended with other node types as needed. We evaluate our approach on BTs used in a commercially deployed robotics system and on a large set of randomly generated trees showing that our approach scales to realistic sizes of more than 3000 nodes.

Height estimation from single aerial imagery using contrastive learning based multi-scale refinement network

ABSTRACT Height map estimation from a single aerial image plays a crucial role in localization, mapping, and 3D object detection. Deep convolutional neural networks have been used to predict height information from single-view remote sensing images, but these methods rely on large volumes of training data and often overlook geometric features present in orthographic images. To address these issues, this study proposes a gradient-based self-supervised learning network with momentum contrastive loss to extract geometric information from non-labeled images in the pretraining stage. Additionally, novel local implicit constraint layers are used at multiple decoding stages in the proposed supervised network to refine high-resolution features in height estimation. The structural-aware loss is also applied to improve the robustness of the network to positional shift and minor structural changes along the boundary area. Experimental evaluation on the ISPRS benchmark datasets shows that the proposed method outperforms other baseline networks, with minimum MAE and RMSE of 0.116 and 0.289 for the Vaihingen dataset and 0.077 and 0.481 for the Potsdam dataset, respectively. The proposed method also shows around threefold data efficiency improvements on the Potsdam dataset and domain generalization on the Enschede datasets. These results demonstrate the effectiveness of the proposed method in height map estimation from single-view remote sensing images.

A randomness device to create the conditions of uncertainty

ABSTRACT The experimental study of decision-making under uncertainty typically builds on Ellsberg-type settings. Yet, as the total number of balls is known in standard Ellsberg’s urns, an implicit constraint is put on the specification of the probability models to consider. In practice, this restricts the ability of Ellsberg’s urns to characterize situations going beyond those of model ambiguity. In this note, I present a simple and easy-to-implement device that creates the initial conditions of uncertainty, which constitute a critical prerequisite for the study of model misspecification.

Construction schedule augmentation with implicit dependency constraints and automated generation of lookahead plan revisions

Revising lookahead plans is practiced in all construction projects to meet contractual deadlines, mitigate activity delays, and disentangle process bottlenecks. We present a new machine learning-based method –which we call Implicit Logic Checker (ILC)– to learn the implicit dependency constraints and the flexibility of construction schedule relationships using a Transformer language model architecture. We then leverage relationships’ flexibilities to provide a basis for creating optimal lookahead plan revisions to help mitigate the effects of unavoidable activity delays. We then deploy a Constrained Conditional (CC) model that applies learned construction planning knowledge to build revised lookahead plans and optimizes them for consistency and duration. Our ILC model is trained and tested on 35,332 manually labeled predecessor-successor relationships from eight real construction projects achieving an F1-Score of 91%. Similarly, the revised lookahead plans generated by the CC model on two case studies show reduced overall plan durations.

A new general class of systems of elliptic quasi-variational–hemivariational inequalities

In this paper we study a general class of systems of strongly coupled quasi-variational–hemivariational inequalities with implicit constraints in reflexive Banach spaces. Each inequality contains a convex potential, a locally Lipschitz superpotential, and a implicit obstacle set. The solvability of the system is established based on a fixed point approach, monotonicity arguments, and nonsmooth analysis. The result generalizes several theorems on various particular cases on variational, variational–hemivariational, and quasi-variational inequalities. We illustrate the applicability of the abstract theory by a nonlinear static thermoelastic frictional contact problem with implicit obstacles for which we provide existence and regularity results.

Built-in wavelet-induced smoothness to reduce plan complexity in intensity modulated radiation therapy (IMRT)

Objective. Reducing plan complexity in intensity modulated radiation therapy (IMRT) to ensure dosimetric accuracy and delivery efficiency of the radiation treatment plans. We propose a novel approach by representing the beamlet intensities using an incomplete wavelet basis that explicitly excludes fluctuating intensity maps from the decision space (explicit hard constraint). This technique provides a built-in wavelet-induced smoothness that improves both dosimetric plan quality and delivery efficiency. Approach. The beamlet intensity maps need to be especially smooth in the leaf travel direction (referred to as the X-direction). We treat the intensity map of each beam as a 2D image and represent it using the wavelets corresponding to low-frequency changes in the X-direction (i.e. approximation and horizontal). The absence of wavelets corresponding to high-frequency changes (i.e. vertical and diagonal) induces built-in smoothness. We still utilize a regularization term in the objective function to promote smoothness in the Y-direction (perpendicular to the X-direction) and further possible smoothness in the X-direction. This technique has been tested on three patient cases of different disease sites (paraspinal, lung, prostate) and all final evaluations and comparisons have been performed on an FDA-approved commercial treatment planning system (Varian EclipseTM). Main results. Wavelet-induced smoothness reduced monitor units by about 10%, 45%, and 14% for paraspinal, lung, and prostate cases, respectively. It also improved organ at risk sparing, especially on the complex paraspinal case where it resulted in about 7%, 13%, and 14% less mean dose to esophagus, lung, and cord, respectively. Moreover, built-in wavelet-induced smoothness desensitizes the results to changing the weight associated to the regularization term, and thereby mitigates the weight fine-tuning difficulty. Significance. Fluence smoothness is often achieved by smoothing the beamlet intensity maps using a proper regularization term in the objective function aiming at disincentivizing fluctuation in the beamlet intensities (implicit soft constraint). This work reports a novel application of wavelets in imposing an explicit smoothness hard constraint in the search space using an incomplete wavelet basis. This idea has been successfully applied to exclude complex and clinically irrelevant radiation plans from the search space. The code and pertained models along with a sample dataset are released on our LowDimRT GitHub (https://github.com/PortPy-Project/LowDimRT).

Model predictive control with implicit terminal ingredients

This paper proposes a new formulation of model predictive control for linear discrete time systems under state and control constraints. The main novelty of the proposed formulation is the introduction, and a computationally adequate utilization, of the implicit terminal constraint set and the implicit terminal cost function. From a numerical point of view, the deployment of the implicit terminal ingredients enables a computationally tractable use of model predictive control for higher dimensional control problems. From a practical point of view, the design of the proposed model predictive control algorithm is very constructive for a class of linear systems with polyhedral constraints. From a structural point of view, the proposed model predictive control retains a priori guarantees of the highly desirable positive invariance and exponential stability properties.

Selected aspects of constrained conforming approximation in higher-order FEM

This paper responds to numerous questions and inquiries related to constrained approximation (approximation with hanging nodes) in higher-order finite element methods that we received during the ESCO 2022 conference. We discuss explicit constraints enforced on linear algebra level as well as implicit constraints which are built into the basis functions of the finite element space, and selected additional mathematical and algorithmic aspects of conforming higher-order finite element approximations with one-level and arbitrary-level hanging nodes.

A Backhaul Adaptation Scheme for IAB Networks Using Deep Reinforcement Learning With Recursive Discrete Choice Model

Challenges such as backhaul availability and backhaul scalability have continued to outweigh the progress of integrated access and backhaul (IAB) networks that enable multi-hop backhauling in 5G networks. These challenges, which are predominant in poor wireless channel conditions such as foliage, may lead to high energy consumption and packet losses. It is essential that the IAB topology enables efficient traffic flow by minimizing congestion and increasing robustness to backhaul failure. This article proposes a backhaul adaptation scheme that is controlled by the load on the access side of the network. The routing problem is formulated as a constrained Markov decision process and solved using a dual decomposition approach due to the existence of explicit and implicit constraints. A deep reinforcement learning (DRL) strategy that takes advantage of a recursive discrete choice model (RDCM) is then proposed. The incorporation of the RDCM helps to address the challenges of backhaul availability and scalability with the aim to increase robustness to backhaul failure in IAB networks. The advantage of applying the RDCM for this problem is that it incorporates choice aversion from prospect theory and the reward is not the only factor affecting the learning rates, but also the punishment. The performance of the proposed algorithm was compared to that of conventional DRL, i.e., without RDCM, and generative model-based learning (GMBL) algorithms. The simulation results of the proposed approach reveal risk perception by introducing certain biases on alternative choices. For instance, applying cumulative prospect theory in the context of route choices reveal different node behaviors as the learning rate and noise levels are varied. The simulation results showed that the proposed algorithm provides better throughput and delay performance over the two baselines.

Small-Signal Angle Stability-Oriented False Data Injection Cyber-Attacks on Power Systems

The small-signal angle stability (SSAS) of a power system is determined by the property of operation points. The widely applied false data injection (FDI) cyber-attack, however, is able to stealthily mislead the optimal power flow (OPF) and thus compromise operation points, leading to damages to the SSAS margin. To provide insights for cyber defenders, this paper proposes and investigates a stealthy SSAS-oriented FDI cyber-attack focusing on two attacking purposes, i.e., the SSAS margin and operation cost, with higher priority on the former one. First, this paper establishes a novel bi-level model with an implicit SSAS constraint based on a structure preserving model to compromise operation points. Then, for the SSAS interarea mode in a typical two-area system, this paper formulates closed-form expressions of how the SSAS margin and operation cost behave with respect to stealthy injections. By comparison, for the SSAS local mode in general power systems, this paper proposes a moving target cyber-attack-based hierarchical solution algorithm. Simulation results on a two-area system, a Kundur 11 bus system, and a modified IEEE 14 bus system demonstrate the significant damaging effects of the proposed SSAS-oriented FDI cyber-attack and the conflict between the two attacking purposes.

Real-Time Surface-Based Volume Constraints on Mass-Spring Model in Unity3D

This paper describes a parallel method to simulate real-time 3D deformable objects using volume preservation constraints on a mass-spring model (MSM) to achieve plausible results in real-time performance. Instead of considering a volumetric mesh which is mostly used to simulate deformable objects, we purely take the surface of the 3D object into account to reduce the time complexity and obtain a high-quality deformable. In the conventional MSM, we can simply control the shape of the deformable object through the stiffness and damping coefficients which is beneficial for our volume constraint. We use the divergence theorem and implicit constraint enforcement scheme to maintain the volume of the object and deform it freely. The surface-based volume constraint is applied to correct the force of the spring network. The proposed algorithm was designed on compute shader in Unity3D and runs outside the normal rendering pipeline in the graphics processing unit (GPU) in order to utilize the massive parallel process to accelerate the performance of the simulation. The performance of the simulation can be accelerated by using the parallel processing method on the GPU with an average speedup factor of 4.27 using the conventional mass-spring method, and an average speedup factor of 2.54 for the volume preservation constraint method. We present several scenes which demonstrate the volume-preserving deformations using the conventional mass-spring method and volume-preservation constraint method and the volume loss of deformable objects for all 3D models is compared. The volume preservation method obtains a volume loss significantly lower than the conventional MSM for all experimental tests.

Multisource Information Fusion Network for Optical Remote Sensing Image Super-Resolution

The super-resolution algorithms based on deep learning can effectively increase optical remote sensing image (ORSI) details for further analysis tasks. Deep unfolding methods have been studied in recent years to bridge the gap between optimization-based and learning-based methods, which can take good advantage of the prior knowledge. However, these unfolding methods usually ignore the utilization of intermediate network features between different iteration stages, thereby limiting the performance of super-resolution results. We propose a multi-source information fusion network (MSFNet) for ORSI super-resolution to address this problem. We mainly consider three strategies to enhance the image super-resolution performance, including feature extraction strategy, information fusion strategy, and the structure of the unfolding network. Firstly, image information of various scales is helpful for mining potential features of images for image super-resolution. Therefore, we introduce multi-scale implicit constraints to the objective function. Secondly, we unfold the optimization process into a neural network by alternating direction method of multipliers (ADMM). The network framework follows the ADMM method's iteration process. This unfolding strategy can effectively utilize the prior information for image reconstruction. Thirdly, we propose a row-column decoupling Transformer module based on this unfolding framework for feature fusion. Specifically, the row Transformer block completes the feature fusion of various scales, and the column Transformer block completes the feature fusion of various channels. The fused features are transmitted to the next iteration stage for feature enhancement. And this fusion strategy enables the network to focus on the global information of the features. We perform experiments on three remote sensing image datasets to fully demonstrate the algorithm's effectiveness. Experiment results show that the proposed algorithm can achieve better image reconstruction performance.

Physics-constrained Gaussian process regression for soil moisture dynamics

Soil moisture (SM) is a crucial variable in the hydrological cycle and several recent studies have been conducted to model it using a hybrid strategy integrating physical-based and data-driven approaches. However, such methods pay little attention to the flaws of the introduced physics constraints, and the inherent nonlinearity of the soil flow process poses a challenge to their real-world application. In this study, a new physics-constrained scheme is proposed to exploit or avoid the adverse effects of the model error. Based on the reasonable assessment of model uncertainty, traditional physical model simulations are dynamically weighted to fuse into a data-driven framework based on Gaussian process regression (GPR) to utilize the respective strengths of both paradigms. For comparison, we also implement a scheme that introduces the gradient from the data as an implicit constraint. The effectiveness of the proposed paradigm is tested with a series of real-world cases. The results show that the physical model based on Richards' equation runs the risk of failing to guarantee even a basic water balance in real soil moisture simulations, which raises a red flag about the introduction of physics constraints, such as in the deep layers of the Goodwell site, the coefficient of determination (R2) predicted by the physical model is even less than 0.2 due to the neglect of the preferential flow process. In contrast, the proposed hybrid scheme significantly improves the accuracy and robustness of the soil moisture retrieval by weighing the reliability of the prediction results, with the R2 of the surface layer at Las Cruses site improving to 0.82 compared to 0.45 for the purely data-driven model. The comparison of the different forms of constraints further demonstrates the generalizability of our proposed constraint scheme.

Improving the performance of the unmeasured latent method construct technique in common method variance detection and correction

SummaryIn behavioral research, common method variance (CMV) is likely to occur when data are obtained from the same sources. Ignoring CMV can lead to biases in parameter estimates. Although the unmeasured latent method construct (ULMC) technique, based on confirmatory factor analysis (CFA), is frequently used for CMV detection and correction, previous research has indicated that its performance is rather poor. In this study, we propose an approach to improve the performance of ULMC by adding identifying indicators together with the use of Rindskopf's reparameterization of the CFA model for error variances and implicit constraints for correlations. The chi‐square test for model fit and the chi‐square difference test are used to test for the existence of CMV and to determine the number of method constructs. The simulation results indicated that, given adequate sample size, the ULMC technique with the proposed approach performs well in CMV detection and correction. Moreover, its performance in CMV correction is superior to the performance of the CFA marker technique. The approach, illustrated with two datasets, is strongly recommended for empirical studies that suffer CMV.

Optimization-Based High-Frequency Circuit Miniaturization through Implicit and Explicit Constraint Handling: Recent Advances

Miniaturization trends in high-frequency electronics have led to accommodation challenges in the integration of the corresponding components. Size reduction thereof has become a practical necessity. At the same time, the increasing performance demands imposed on electronic systems remain in conflict with component miniaturization. On the practical side, the challenges related to handling design constraints are aggravated by the high cost of system evaluation, normally requiring full-wave electromagnetic (EM) analysis. Some of these issues can be alleviated by implicit constraint handling using the penalty function approach. Yet, its performance depends on the arrangement of the penalty factors, necessitating a costly trial-and-error procedure to identify their optimum setup. A workaround is offered by the recently proposed algorithms with automatic adaptation of the penalty factors using different adjustment schemes. However, these intricate strategies require a continuous problem-dependent adaptation of the penalty function throughout the entire optimization process. Alternative methodologies have been proposed by taking an explicit approach to handle the inequality constraints, along with correction-based control over equality conditions, the combination of which proves to be demonstrably competitive for some miniaturization tasks. Nevertheless, optimization-based miniaturization, whether using implicit or explicit constraint handling, remains a computationally expensive task. A reliable way of reducing the aforementioned costs is the incorporation of multi-resolution EM fidelity models into the miniaturization procedure. Therein, the principal operation is based on the simultaneous monitoring of factors such as quality of the constraint satisfaction, as well as algorithm convergence status. This paper provides an overview of the abovementioned size-reduction algorithms, in which theoretical considerations are illustrated using a number of antenna and microwave circuit case studies.