Penalty Term Research Articles

On the regularity of optimal potentials in control problems governed by elliptic equations

Abstract In this paper we consider optimal control problems where the control variable is a potential and the state equation is an elliptic partial differential equation of Schrödinger type, governed by the Laplace operator. The cost functional involves the solution of the state equation and a penalization term for the control variable. While the existence of an optimal solution simply follows by the direct methods of the calculus of variations, the regularity of the optimal potential is a difficult question and under the general assumptions we consider, no better regularity than the BV \mathrm{BV} one can be expected. This happens in particular for the cases in which a bang-bang solution occurs, where optimal potentials are characteristic functions of a domain. We prove the BV \mathrm{BV} regularity of optimal solutions through a regularity result for PDEs. Some numerical simulations show the behavior of optimal potentials in some particular cases.

Within-Class Constraint Based Multi-task Autoencoder for One-Class Classification

Autoencoders (AEs) have attracted much attention in one-class classification (OCC) based unsupervised anomaly detection. The AEs aim to learn the unity features on targets without involving anomalies and thus the targets are expected to obtain smaller reconstruction errors than anomalies. However, AE-based OCC algorithms may suffer from the overgeneralization of AE and fail to detect anomalies that have similar distributions to target data. To address these issues, a novel within-class constraint based multi-task AE (WC-MTAE) is proposed in this paper. WC-MTAE consists of two different task: one for reconstruction and the other for the discrimination-based OCC task. In this way, the encoder is compelled by the OCC task to learn the more compact encoded feature distribution for targets when minimizing OCC loss. Meanwhile, the within-class scatter based penalty term is constructed to further regularize the encoded feature distribution. The aforementioned two improvements enable the unsupervised anomaly detection by the compact encoded features, thereby addressing the issue of the overgeneralization in AEs. Comparisons with several state-of-the-art (SOTA) algorithms on several non-image datasets and an image dataset CIFAR10 are provided where the WC-MTAE is conducted on 3 different network structures including the multilayer perception (MLP), LeNet-type convolution network and full convolution neural network. Extensive experiments demonstrate the superior performance of the proposed WC-MTAE. The source code would be available in future.

Wasserstein Auto-Encoders of Merge Trees (and Persistence Diagrams).

This article presents a computational framework for the Wasserstein auto-encoding of merge trees (MT-WAE), a novel extension of the classical auto-encoder neural network architecture to the Wasserstein metric space of merge trees. In contrast to traditional auto-encoders which operate on vectorized data, our formulation explicitly manipulates merge trees on their associated metric space at each layer of the network, resulting in superior accuracy and interpretability. Our novel neural network approach can be interpreted as a non-linear generalization of previous linear attempts (Pont et al. 2023) at merge tree encoding. It also trivially extends to persistence diagrams. Extensive experiments on public ensembles demonstrate the efficiency of our algorithms, with MT-WAE computations in the orders of minutes on average. We show the utility of our contributions in two applications adapted from previous work on merge tree encoding (Pont et al. 2023). First, we apply MT-WAE to merge tree compression, by concisely representing them with their coordinates in the final layer of our auto-encoder. Second, we document an application to dimensionality reduction, by exploiting the latent space of our auto-encoder, for the visual analysis of ensemble data. We illustrate the versatility of our framework by introducing two penalty terms, to help preserve in the latent space both the Wasserstein distances between merge trees, as well as their clusters. In both applications, quantitative experiments assess the relevance of our framework. Finally, we provide a C++ implementation that can be used for reproducibility.

An ultraweak-local discontinuous Galerkin method for nonlinear biharmonic Schrödinger equations

This paper proposes and analyzes a fully discrete scheme for nonlinear biharmonic Schrödinger equations. We first write the single equation into a system of problems with second-order spatial derivatives and then discretize the space variable with an ultraweak discontinuous Galerkin scheme and the time variable with the Crank–Nicolson method. The proposed scheme proves to be computationally more efficient compared to the local discontinuous Galerkin method in terms of the number of equations needed to be solved at each single time step, and it is unconditionally stable without imposing any penalty terms. It also achieves optimal error convergence in L2 norm both in the solution and in the auxiliary variable with general nonlinear terms. We also prove several physically relevant properties of the discrete schemes, such as the conservation of mass and the Hamiltonian for the nonlinear biharmonic Schrödinger equations. Several numerical studies demonstrate and support our theoretical results.

A General Method for Solving Differential Equations of Motion Using Physics-Informed Neural Networks

The physics-informed neural network (PINN) is an effective alternative method for solving differential equations that do not require grid partitioning, making it easy to implement. In this study, using automatic differentiation techniques, the PINN method is employed to solve differential equations by embedding prior physical information, such as boundary and initial conditions, into the loss function. The differential equation solution is obtained by minimizing the loss function. The PINN method is trained using the Adam algorithm, taking the differential equations of motion in structural dynamics as an example. The time sample set generated by the Sobol sequence is used as the input, while the displacement is considered the output. The initial conditions are incorporated into the loss function as penalty terms using automatic differentiation techniques. The effectiveness of the proposed method is validated through the numerical analysis of a two-degree-of-freedom system, a four-story frame structure, and a cantilever beam. The study also explores the impact of the input samples, the activation functions, the weight coefficients of the loss function, and the width and depth of the neural network on the PINN predictions. The results demonstrate that the PINN method effectively solves the differential equations of motion of damped systems. It is a general approach for solving differential equations of motion.

An Improved Penalty-Based Excited-State Variational Monte Carlo Approach with Deep-Learning Ansatzes.

We introduce several improvements to the penalty-based variational quantum Monte Carlo (VMC) algorithm for computing electronic excited states of Entwistle et al. [Nat. Commun. 14, 274 (2023)] and demonstrate that the accuracy of the updated method is competitive with other available excited-state VMC approaches. A theoretical comparison of the computational aspects of these algorithms is presented, where several benefits of the penalty-based method are identified. Our main contributions include an automatic mechanism for tuning the scale of the penalty terms, an updated form of the overlap penalty with proven convergence properties, and a new term that penalizes the spin of the wave function, enabling the selective computation of states with a given spin. With these improvements, along with the use of the latest self-attention-based ansatz, the penalty-based method achieves a mean absolute error below 1 kcal/mol for the vertical excitation energies of a set of 26 atoms and molecules, without relying on variance matching schemes. Considering excited states along the dissociation of the carbon dimer, the accuracy of the penalty-based method is on par with that of natural-excited-state (NES) VMC, while also providing results for additional sections of the potential energy surface, which were inaccessible with the NES method. Additionally, the accuracy of the penalty-based method is improved for a conical intersection of ethylene, with the predicted angle of the intersection agreeing well with both NES-VMC and multireference configuration interaction.

A formulation for fluid–structure interaction problems with immersed flexible solids: Application to splitters subjected to flow past cylinders with different cross-sections

In the finite element method framework, a fluid–structure formulation is developed by coupling an Eulerian fixed-mesh fluid approach with a Lagrangian deforming-mesh description for a flexible solid. The coupled formulation is solved using a staggered scheme during time. For the fluid solution stage, the solid walls are considered as a time-variable internal boundary. The velocity and pressure fields are obtained by solving the weak form of the fluid dynamic equations in which the solid velocity is imposed on the internal boundary via a penalization term. For the solid solution stage, the displacement field is obtained by solving the discrete solid dynamic equations which consider traction forces computed by integrating pressures and viscous stresses on the nodes belonging to the solid walls. This novel technique is firstly applied to analyze a flexible splitter under the shedding of a flow past square cylinder due to this problem is considered as a benchmark in the literature. The present solutions agree with those computed using body-fitted techniques, thus validating the proposal. Secondly, flexible splitter motions under the shedding of flow past cylinders with different cross-sections and splitter lengths are comprehensively studied. Overall, the computed results confirmed that the hydrodynamic coefficients on the cylinders were reduced because of the presence of the splitter.

Optimal configuration of hydrogen storage capacity of hybrid microgrid considering peak regulation and frequency modulation requirements

This study proposes an innovative hydrogen storage capacity optimization configuration method that considers multiple demand factors, addressing the issue that traditional methods for optimizing hydrogen storage capacity in hybrid microgrids cannot simultaneously balance economic efficiency and grid connection quality. This method breaks through the traditional optimization framework and adopts a double-layer optimization model, combining the peak shaving operation cost of the hybrid microgrid with the interactive power deviation and power fluctuation penalty terms of the interconnection line during the frequency regulation stage, achieving dual optimization of economy and grid connection quality. Through precise mathematical modeling and efficient optimization algorithms, we have successfully obtained the minimum operating cost corresponding to the minimum power fluctuation, the minimum interconnection line interactive power deviation penalty term, and the optimal hydrogen storage capacity configuration scheme. Identify two typical days through cluster analysis. Option 1 (considering peak shaving, frequency regulation, and hydrogen electrolysis) has the lowest net load and a total operating cost of 3.0331 million yuan, which is better than Option 2 (3.0973 million yuan) and Option 3 (5.0398 million yuan). This plan effectively reduces wind and solar power waste, shortens the operating time of thermal power units, and demonstrates the rationality and economy of optimizing hydrogen storage capacity configuration.

Learning under singularity: an information criterion improving WBIC and sBIC

AbstractWe introduce a novel information criterion, termed learning under singularity (LS), designed to enhance the functionality of the widely applicable Bayes information criterion (WBIC) and the singular Bayesian information criterion (sBIC). LS is effective without regularity constraints and demonstrates stability. Watanabe defined a statistical model or a learning machine as regular if the mapping from a parameter to a probability distribution is one-to-one and its Fisher information matrix is positive definite. In contrast, models not meeting these conditions are termed singular. Over the past decade, several information criteria for singular cases have been proposed, including WBIC and sBIC. WBIC is applicable in non-regular scenarios but faces challenges with large sample sizes and redundant estimation of known learning coefficients. Conversely, sBIC is limited in its broader application due to its dependence on maximum likelihood estimates. LS addresses these limitations by enhancing the utility of both WBIC and sBIC. It incorporates the empirical loss from the widely applicable information criterion (WAIC) to represent the goodness of fit to the statistical model, along with a penalty term similar to that of sBIC. This approach offers a flexible and robust method for model selection, free from regularity constraints.

Bridging spatio-temporal discontinuities in global soil moisture mapping by coupling physics in deep learning

The launch of Soil Moisture Active Passive (SMAP) satellite in 2015 has resulted in significant achievements in global soil moisture mapping. Nonetheless, spatiotemporal discontinuities in the soil moisture products have arisen due to the limitations of its orbit scanning gap and retrieval algorithms. To address these issues, this paper presents a physics-constrained gap-filling method, PhyFill for short. The PhyFill method employs a partial convolutional neural network technique to explore spatial domain features of the original SMAP soil moisture data. Then, it incorporates variations in soil moisture induced by precipitation events and dry-down events as penalty terms in the loss function, thereby accounting for monotonicity and boundary constraints in the physical processes governing the dynamic fluctuations of soil moisture. The PhyFill model was applied to SMAP soil moisture data, resulting in continuous spatially daily soil moisture data on a global scale. Three validation strategies are employed: visual inspection through global pattern, simulated missing-region validation, and soil moisture validation with in situ measurements. The results indicated that the reconstructed soil moisture achieved a higher spatial coverage with satisfactory spatial continuity with neighbouring pixels. The simulated validation of the missing regions revealed that the averaged unbiased root mean square difference (ubRMSD) and correlation coefficient (R) were 0.01 m3/m3 and 0.99, respectively versus the gap filled SMAP product. The core validation sites demonstrated that the reconstructed soil moisture data has a consistent ubRMSD compared with the original SMAP soil moisture data (0.04 m3/m3vs. 0.04 m3/m3). The PhyFill method can generate globally continuous, high accurate soil moisture estimates, providing remarkable support for advanced hydrological applications, e.g., global soil moisture dry-down events and patterns.

Kernel Extreme Learning Machine with Discriminative Transfer Feature and Instance Selection for Unsupervised Domain Adaptation

The goal of domain adaptation (DA) is to develop a robust decision model on the source domain effectively generalize to the target domain data. State-of-the-art domain adaptation methods typically focus on finding an optimal inter-domain invariant feature representation or helpful instances from the source domain. In this paper, we propose a kernel extreme learning machine with discriminative transfer features and instance selection (KELM-DTF-IS) for unsupervised domain adaptation tasks, which consists of two steps: discriminative transfer feature extraction and classification with instance selection. At the feature extraction stage, we extend cross domain mean approximation(CDMA) by incorporating a penalty term and develop discriminative cross domain mean approximation (d-CDMA) to optimize the category separability between instances. Subsequently, d-CDMA is integrated into kernel ELM-AutoEncoder(KELM-AE) for extracting inter-domain invariant features. During the classification process, our approach uses CDMA metrics to compute a weights to each source instances based on their impact in reducing distribution differences between domains. Instances with a greater effect receive higher weights and vice versa. These weights are then used to distinguish and select source domain instances before incorporating them into weight KELM for proposing an adaptive classifier. Finally, we apply our approach to conduct classification experiments on publicly available domain adaptation datasets, and the results demonstrate its superiority over KELM and numerous other domain adaptation approaches.

Numerical analysis of a class of penalty discontinuous Galerkin methods for nonlocal diffusion problems

In this paper, we consider a class of discontinuous Galerkin (DG) methods for one-dimensional nonlocal diffusion (ND) problems. The nonlocal models are integral equations that are widely used in describing many physical phenomena with long-range interactions. The ND problem is the nonlocal analog of the classic diffusion problem, and the nonlocality disappears as the interaction radius (horizon) vanishes, then the ND problem recovers the classic diffusion problem. Under certain conditions, the exact solution to the ND problem may contain discontinuities, and this is one of the properties of the ND problem that sets it apart from the classic diffusion problem. It is natural to adopt the DG method to compute the ND problems since the DG method shows its great advantages in resolving problems with discontinuities in the computational fluid dynamics in the past several decades. Based on \cite{DuCAMC2020}, we construct the DG methods with different penalty terms, and the proposed DG methods have their local counterparts as the horizon vanishes. This indicates the proposed methods will converge to the existing DG schemes as the horizon vanishes, which is crucial to obtain the {\it asymptotic compatibility}. Rigorous proofs are provided to demonstrate the stability, error estimates, and asymptotic compatibility of the proposed DG schemes. To see the nonlocal diffusion effect, we also consider the time-dependent convection-diffusion problems with nonlocal diffusion. We conduct several numerical experiments, including accuracy tests and Burgers' equation with nonlocal diffusion, and various horizons are taken to show the good performance of the proposed algorithm and validate the theoretical findings.

Structural response reconstruction of beam-like bridge based on equivalent loads under moving forces

Response reconstruction has advantages in enriching monitoring data with high quality and it is of great importance to bridge structural health monitoring. However, there still are many current difficulties with response reconstruction due to the complex operation of bridges, e.g., hard to deal with time-varying positions of vehicles and inefficient reconstruction for cases with a long time. According to the theory of equivalent load identification, this paper proposes a model-based method for reconstructing responses induced by moving forces. The proposed method introduces moving time windows to extract the measured responses in an orderly manner and re-organizes them into a matrix form. The measured responses, expressed in matrix form, then serve as the input data to identify both equivalent loads and structural initial conditions, but not to estimate the actual moving forces. Herein, the equivalent loads represent some concentrated loads with time-invariant acting positions and are employed to approximately re-express the main information of moving forces. Matrix regularization with a sparse penalty term is used to ensure the robustness of identified results. Since the structural inputs have been estimated, the structural responses happening at the validation sensors then can be calculated as a forward problem. In this process, the role of identified structural inputs is an intermediate variable connecting the measuring sensors and the validation sensors so that, the transmissibility function (TF) discussed in the proposed method is expressed in an implicit pattern. Compared to the response reconstruction methods developed from moving force identification (MFI), an obvious characteristic is that information on the actual moving forces is no longer necessary. Therefore, it can bring convenience in practical application such as the time-varying positions of moving vehicles are not needed in advance. In addition, the application of matrix regularization has the potential for efficient computation. To verify the feasibility and effectiveness of the proposed method, both numerical simulations and experimental studies are finally carried out. The illustrated results indicate that the proposed method can accurately reconstruct the structural response. In addition, some related issues are also discussed.

Roughness regularization for functional data analysis with free knots spline estimation

In the era of big data, an ever-growing volume of information is recorded, either continuously over time or sporadically, at distinct time intervals. Functional Data Analysis (FDA) stands at the cutting edge of this data revolution, offering a powerful framework for handling and extracting meaningful insights from such complex datasets. The currently proposed FDA methods can often encounter challenges, especially when dealing with curves of varying shapes. This can largely be attributed to the method’s strong dependence on data approximation as a key aspect of the analysis process. In this work, we propose a free knots spline estimation method for functional data with two penalty terms and demonstrate its performance by comparing the results of several clustering methods on simulated and real data.

ANALYSIS OF FACTORS AFFECTING LIFE EXPECTANCY IN NORTH SUMATRA PROVINCE USING LASSO AND ADAPTIVE LASSO

Life expectancy is a measure used to evaluate the government's effectiveness in enhancing the overall well-being and health of the population. The life expectancy of a town is impacted by the educational attainment of its residents. LASSO regression is a variant of linear regression that incorporates an extra penalty term into the cost function. adjustable LASSO Regression is a modified version of this technique that incorporates adjustable weighting variables for each coefficient, enhancing its effectiveness.Multicollinearity, a condition of strong correlation between predictor variables, can complicate the identification of independent variables that have a significant impact. The LASSO and Adaptive LASSO are variable selection strategies used in multiple linear regression situations. The LASSO and Adaptive LASSO techniques are effective for identifying important variables and establishing the link between independent and dependent variables

An Improved CH4 Profile Retrieving Method for Ground-Based Differential Absorption Lidar

Range-resolved CH4 concentration measurement is important prior data for atmospheric physical and chemical models. Ground-based differential absorption lidar (DIAL) can measure the vertical distribution of CH4 concentration in the atmosphere. The traditional method uses lidar observational data and the lidar equation to calculate profiles, but the inversion accuracy is greatly affected by noise. Although some denoising methods can improve accuracy at low altitudes, the low signal-to-noise ratio caused by the effect of aerosol Mie scattering and lower aerosol concentrations at high altitudes cannot be solved. Here, an improved cubic smoothing spline fitting CH4 concentration profile inversion method is proposed to address this challenge. By adding a penalty term of the second derivative of the conventional cubic spline function to the objective function, this penalty term acts to smooth the fitting, allowing the fitting function to avoid necessarily passing through those noisy sampling points. This avoids the large fluctuations caused by noisy sampling points, effectively suppresses noise, captures signals with lower noise levels, and thereby enhances the inversion accuracy of the profiles. Simulations and case studies demonstrated the superiority of the proposed method. Compared with the traditional method, cubic smoothing spline fitting can reduce the mean error of the whole CH4 profile by 85.54%. The standard deviation of CH4 concentration retrieved is 3.59 ppb–90.29 ppb and 0.01 ppb–6.75 ppb smaller than the traditional method and Chebyshev fitting, respectively. Three real cases also indicate that the CH4 concentration retrieved by cubic smoothing spline fitting is more consistent with in-situ measurements. In addition, long-term DIAL observations have also revealed notable diurnal and seasonal trends in CH4 concentration at observation sites.

Bayesian reconstruction of 3D particle positions in high-seeding density flows

Abstract Measuring particles' three-dimensional (3D) positions using multi-camera images in fluid dynamics is critical for resolving spatiotemporally complex flows like turbulence and mixing. However, current methods are prone to errors due to camera noise, optical configuration and experimental setup limitations, and high seeding density, which compound to create fake measurements (ghost particles) and add noise and error to velocity estimations. We introduce a Bayesian Volumetric Reconstruction (BVR) method, addressing these challenges by using probability theory to estimate uncertainties in particle position predictions. Our method assumes a uniform distribution of particles within the reconstruction volume and employs a model mapping particle positions to observed camera images. We utilize variational inference with a modified loss function to determine the posterior distribution over particle positions. Key features include a penalty term to reduce ghost particles, provision of uncertainty bounds, and scalability through subsampling. In tests with synthetic data and four cameras, BVR achieved 95% accuracy with less than 3% ghost particles and an RMS error under 0.3 pixels at a density of 0.1 particles per pixel (ppp). This shows 57% to 97% less ghost particles and 1.3 to 2 times lower RMS error than standard MART and IPR reconstructions. In an experimental Poiseuille flow measurement, our method closely matched the theoretical solution. Additionally, in a complex cerebral aneurysm basilar tip geometry flow experiment, our reconstructions were dense and consistent with observed flow patterns. Our BVR method effectively reconstructs particle positions in complex 3D flows, particularly in situations with high particle image densities and camera distortions. It distinguishes itself by providing quantifiable uncertainty estimates and scaling efficiently for larger image dimensions, making it applicable across a range of fluid flow scenarios.

Swarm intelligence for full Stokes dynamic imaging reconstruction of interferometric data

Context. In very long baseline interferometry (VLBI), the combination of multiple antennas permits the synthesis of a virtual telescope with a larger diameter and consequently higher resolution than the individual antennas. However, due to the sparse nature of the array, recovering an image from the observed data is a challenging ill-posed inverse problem. Aims. The VLBI community is interested in not only recovering an image in total intensity from interferometric data, but also in obtaining results in the polarimetric and the temporal domain. Only a few algorithms are able to work in all these domains simultaneously. In particular, the algorithms based on optimization that consider various penalty terms specific to static total intensity imaging, time-variability and polarimetry are restricted to grids in the domain of the objective function. In this work we present a novel algorithm, multiobjective particle swarm optimization (MO-PSO), that is able to recover the optimal weights without any space-gridding, and to obtain the marginal contribution of each of the playing terms. Methods. To this end, we utilized multiobjective optimization together with particle swarm metaheuristics. We let the swarm of weights converge to the best position. Results. We evaluate our algorithm with synthetic data sets that are representative for the main science targets and instrumental configuration of the Event Horizon Telescope Collaboration (EHTC) and its planned successors. We successfully recover the polarimetric, static, and time-dynamic signature of the ground truth movie' even with relative sparsity, and a set of realistic data corruptions. Conclusions. We have built a novel, fast, hyperparameter space gridding-free algorithm that successfully recovers static and dynamic polarimetric reconstructions. Compared to regularized maximum likelihood (RML) methods, it avoids the need for parameter surveys, and it is not limited to the number of pixels, unlike recently proposed multiobjective imaging algorithms. Hence, this technique is a novel useful alternative tool to characterize full Stokes time-(in)dependent signatures in a VLBI data set robustly with a minimal set of user-based choices.

Clean affinity matrix induced hyper-Laplacian regularization for unsupervised multi-view feature selection

Most previous unsupervised multi-view feature selection (UMFS) methods have achieved appealing performance by exploring the consistency among multiple views. However, they have the following shortcomings: (1) They often fail to consider the potential inconsistency that might be caused by view-specific characteristics from the perspective of sparsity. (2) The previously learned hyper-graph might be affected by noise, thereby reducing the quality of the generated graph. To tackle these issues, this paper proposes a clean affinity matrix induced hyper-Laplacian regularization (CAHR) method for UMFS. Firstly, the initial affinity matrix is decomposed into the consistent and inconsistent parts, then a novel diversity penalty term is introduced to enforce the sparsity of the inconsistent part across views, thereby making the consistent part be cleaner. Secondly, a unified affinity matrix is generated by fusing the consistent factors of the initial affinity matrix in a self-weighted manner, thereby considering the consistency of multi-view data. Based on the unified affinity matrix, a hyper-Laplacian matrix is further constructed, which can maintain high-order manifold structure of data. Finally, a loss function is designed to find the best mapping for feature selection. Comprehensive experiments demonstrate that the proposed method significantly outperforms several state-of-the-art UMFS methods.

The stabilized nonconforming virtual element method for the Darcy–Stokes problem

A stabilized nonconforming virtual element method is designed in order to solve the Darcy–Stokes problem, which preserves a divergence-free approximation to the velocity. The same degrees of freedom as the usual Crouzeix–Raviart-type virtual element is used, but a different virtual element space is obtained by modifying the conforming Stokes virtual element with the H1-projection operator. The proposed stabilized scheme contains two jump penalty terms over edges. One is the penalty for jumps of velocity approximation and the other one is the penalty for jumps of its normal component. We analyze this method’s well-posedness and prove its uniform convergence in a discrete energy norm. Finally, we verify the validity of this stabilized scheme by some numerical experiments.