Penalty Term Research Articles

An iterative constraint energy minimizing generalized multiscale finite element method for contact problem

This work presents an Iterative Constraint Energy Minimizing Generalized Multiscale Finite Element Method (ICEM-GMsFEM) for solving the contact problem with high contrast coefficients. The model problem can be characterized by a variational inequality, where we add a penalty term to convert this problem into a non-smooth and non-linear unconstrained minimizing problem. The characterization of the minimizer satisfies the variational form of a mixed Dirilect-Neumann-Robin boundary value problem. So we apply CEM-GMsFEM combined with simismooth Newton method and introduce special boundary correctors along with multiscale spaces to achieve an optimal convergence rate. Numerical results are conducted for different highly heterogeneous permeability fields, validating the fast convergence of the CEM-GMsFEM iteration in handling the contact boundary and illustrating the stability of the proposed method with different sets of parameters. We also prove the fast convergence of the proposed iterative CEM-GMsFEM method and provide an error estimate of the multiscale solution under a mild assumption.

A first-order energy stable scheme for the Allen–Cahn equation with the Allen–Cahn type dynamic boundary condition

In this paper we construct and analyze a first-order time marching scheme for the Allen–Cahn equation with the Allen–Cahn type dynamic boundary condition. To hold the mass conservation in the bulk and on the surface respectively, we add two penalty terms to the model. By using the stabilization method, we verify that the scheme is energy stable and is of first-order in time by the error estimates. Finally we present enough numerical results to validate the convergence and the energy stability of such scheme. Moreover, we simulate the shape deformation of a droplet and the moving process of the contact line.

Bayesian Fictitious Play in Oligopoly: The Case of Risk-Averse Agents

A number of learning models have been suggested to analyze the repeated interaction of boundedly rational agents competing in oligopolistic markets. The agents form a model of the environment that they are competing in, which includes the market demand and price formation process, as well as their expectations of their rivals’ actions. The agents update their model based on the observed output and price realizations and then choose their next period output levels according to an optimization criterion. In previous works, the global dynamics of price movement have been analyzed when risk-neutral agents maximize their expected rewards at each round. However, in many practical settings, agents may be concerned with the risk or uncertainty in their reward stream, in addition to the expected value of the future rewards. Learning in oligopoly models for the case of risk-averse agents has received much less attention. In this paper, we present a novel learning model that extends fictitious play learning to continuous strategy spaces where agents combine their prior beliefs with market price realizations in previous periods to learn the mean and the variance of the aggregate supply function of the rival firms in a Bayesian framework. Next, each firm maximizes a linear combination of the expected value of the profit and a penalty term for the variance of the returns. Specifically, each agent assumes that the aggregate supply of the remaining agents is sampled from a parametric distribution employing a normal-inverse gamma prior. We prove the convergence of the proposed dynamics and present simulation results to compare the proposed learning rule to the traditional best response dynamics.

Methodological Advances with Penalized Structural Equation Models

Penalized structural equation models (PSEM) is a powerful technique that unlocks a variety of new modeling frameworks. PSEM applications have been established previously for standard SEM and ESEM models. In this note we aim to extend these ideas to more general types of models such as finite mixture models, multilevel models as well as models with more general types of outcomes. Maximum likelihood and weighted least squares estimation methods naturally accommodate a penalty term. In Mplus 8.12 the PSEM methodology is implemented for all models that can be estimated with these two estimators. Therefore we can now easily combine the more general models with the features of PSEM such as EFA, Alignment, and parameter invariance. Some additional basic SEM applications are also included.

Location optimization of EV charging stations: A custom K-means cluster algorithm approach

The strategic deployment of EV (EV) charging stations is crucial for promoting sustainable transportation and facilitating the widespread adoption of EVs. However, the lack of readily accessible charging station continues to be a significant barrier to the mainstream adoption of EVs. This study presents a comprehensive approach to optimizing the location of charging stations using a custom K-means clustering algorithm. The algorithm incorporates various factors including charging demand, energy consumption, population density, and existing stations, to identify optimal locations for the charging stations. The proposed methodology aims to minimize the distance between charging stations and areas with high EV demand while considering energy efficiency and avoiding redundancy near low-weighted point locations. The algorithm iteratively assigns data points to clusters and updates the centroids, converging to an optimal solution that balances coverage, accessibility, and sustainability. Folium map in Python and Python code was used for case study analysis. The innovation of the finding shows the effectiveness of the custom K-means clustering algorithm in optimizing charging station placement, by introducing a penalty term to avoid repositioning charging stations close to a low-weighted (charging demand) point charging station, also ensuring convenient access for EV owners, and promoting efficient energy utilization. The study emphasizes on strategic planning and data-driven approaches in developing a comprehensive and sustainable EV charging station network to address the challenge of limited charging station and support the growth of electric mobility.

High-dimensional semiparametric mixed-effects model for longitudinal data with non-normal errors

Difficulties may arise when analyzing longitudinal data using mixed-effects models if nonparametric functions are present in the linear predictor component. This study extends semiparametric mixed-effects modeling in cases when the response variable does not always follow a normal distribution and the nonparametric component is structured as an additive model. A novel approach is proposed to identify significant linear and non-linear components using a double-penalized generalized estimating equation with two penalty terms. Furthermore, the iterative approach intends to enhance the efficiency of estimating regression coefficients by incorporating the calculation of the working covariance matrix. The oracle properties of the resulting estimators are established under certain regularity conditions, where the dimensions of both the parametric and nonparametric components increase as the sample size grows. We perform numerical studies to demonstrate the efficacy of our proposal.

The Use of Slack Variables in the Adjoint Method Handling Inequality Constraints in Optimal Control and the Application to Tumor Drug Dosage

Abstract In this article a modified gradient approach, based on the adjoint method, is introduced, to deal with optimal control problems involving inequality constraints. So far, the only way to incorporate inequality constraints in the adjoint approach, is to introduce additional penalty terms in the cost functional. However, this may distort the optimal control due to weighting factors required for these terms and raise serious concerns about the magnitude of the weighting factors. The method in this article avoids penalty functions and can be used for the iterative computation of optimal controls. In order to demonstrate the key idea, first, a static optimization problem in the Euclidean space is considered. Second, the presented approach is applied to the tumor anti-angiogenesis optimal control problem in medicine, which addresses an innovative cancer treatment approach that aims to inhibit the formation of the tumor blood supply. The tumor anti-angiogenesis optimal control problem with free final time involves inequality and final constraints for control and state variables and is solved by a modified adjoint gradient method introducing slack variables. It has to be emphasized that the novel formulation and the special use of slack variables in this article delivers high accurate solutions without distorting the optimal control.

Modified Bayesian Information Criterion for Item Response Models in Planned Missingness Test Designs

The Bayesian information criterion (BIC) is a widely used statistical tool originally derived for fully observed data. The BIC formula includes the sample size and the number of estimated parameters in the penalty term. However, not all variables are available for every subject in planned missingness designs. This article demonstrates that a modified BIC, tailored for planned missingness designs, outperforms the original BIC. The modification adjusts the penalty term by using the average number of estimable parameters per subject rather than the total number of model parameters. This new criterion was successfully applied to item response theory models in two simulation studies. We recommend that future studies utilizing planned missingness designs adopt the modified BIC formula proposed here.

Nuclear norm regularized loop optimization for tensor network

We propose a loop optimization algorithm based on nuclear norm regularization for the tensor network. The key ingredient of this scheme is to introduce a rank penalty term proposed in the context of data processing. Compared to the standard variational periodic matrix product states method, this algorithm can circumvent the local minima related to short-ranged correlation in a simpler fashion. We demonstrate its performance when used as a part of the tensor network renormalization algorithms [S. Yang, Z.-C. Gu, and X.-G. Wen, ] for the critical two-dimensional Ising model. The scale invariance of the renormalized tensors is attained with higher accuracy while the higher parts of the scaling dimension spectrum are obtained in a more stable fashion. Published by the American Physical Society 2024

Seam estimation based on dense matching for parallax-tolerant image stitching

Image stitching with large parallax poses a significant challenge in the field of computer vision. Existing seam-based approaches attempt to address parallax artifacts by stitching images along seams. However, issues such as object mismatches, disappearances, and duplications still arise occasionally, primarily due to inaccurate alignment of dense pixels or inappropriate seam estimation methods. In this paper, we propose a robust seam-based parallax-tolerant image stitching method that leverages dense flow estimation from state-of-the-art approaches. Firstly, we develop a seam estimation method that does not require pre-estimation of image warping model. Instead, it directly estimates the seam by measuring the local smoothness of the optical flow field and incorporating a penalty term for duplications. Subsequently, we design an iterative algorithm that utilizes the location of estimated seam to solve a spatial smooth warping model and eliminate outlier corresponding pairs. By employing this approach, we effectively address the intertwined challenges of estimating the warping model and seam. Experiment on real-world images shows that our proposed method achieves superior local alignment accuracy near the stitching seam and outperforms other state-of-the-art techniques on visual stitching result. Code is available at https://github.com/zhihao0512/dense-matching-image-stitching.

Flexible Krylov methods for group sparsity regularization

Abstract This paper introduces new solvers for efficiently computing solutions to large-scale inverse problems with group sparsity regularization, including both non-overlapping and overlapping groups. Group sparsity regularization refers to a type of structured sparsity regularization, where the goal is to impose additional structure in the regularization process by assigning variables to predefined groups that may represent graph or network structures. Special cases of group sparsity regularization include ℓ 1 and isotropic total variation regularization. In this work, we develop hybrid projection methods based on flexible Krylov subspaces, where we first recast the group sparsity regularization term as a sequence of 2-norm penalization terms using adaptive regularization matrices in an iterative reweighted norm fashion. Then we exploit flexible preconditioning techniques to efficiently incorporate the weight updates. The main advantages of these methods are that they are computationally efficient (leveraging the advantages of flexible methods), they are general (and therefore very easily adaptable to new regularization term choices), and they are able to select the regularization parameters automatically and adaptively (exploiting the advantages of hybrid methods). Extensions to multiple regularization terms and solution decomposition frameworks (e.g. for anomaly detection) are described, and a variety of numerical examples demonstrate both the efficiency and accuracy of the proposed approaches compared to existing solvers.

RADIANCE: Reliable and interpretable depression detection from speech using transformer

Depression is a common but severe mental disorder that adversely impacts the ability of an individual to function normally in their day-to-day life. A majority of depressed individuals remain undiagnosed due to factors such as social stigma and a shortage of healthcare professionals. Consequently, several Machine Learning and Deep Learning (DL) models based on speech have been proposed for automatic depression detection, with the latter generally outperforming the former. However, DL models are blackbox and offer no transparency. In contrast, healthcare professionals prefer models that provide interpretability besides being accurate. In this direction, we propose a method RADIANCE (Reliable AnD InterpretAble depressioN deteCtion transformErs). RADIANCE incorporates a novel FilterBank VIsion Transformer (FBViT) network, which provides the symptoms of depression as interpretable features. Additionally, we employ a novel loss function that handles the class imbalance issue in the datasets. It also incorporates a penalty term that addresses the hierarchy of misclassification errors. We also propose a reliability predictor based on low-level descriptors that provides a reliability score to indicate the trustworthiness of the prediction by FBViT. Furthermore, in contrast to the conventional averaging and majority pooling, RADIANCE consolidates predictions from multiple clips of the input audio by intricately weighing each prediction based on its reliability score, ensuring a more accurate overall prediction. RADIANCE outperforms the state-of-the-art depression detection methods, achieving an accuracy of 89.36%, 80.36%, and 94.44% over the DAIC-WOZ, E-DAIC, and CMDC datasets, respectively. Further, RADIANCE achieves MAE scores of 3.27 and 5.04 on the DAIC-WOZ and E-DAIC datasets, respectively.

Elastic bodies with kinematic constraints on many small regions

We study the equilibrium of hyperelastic solids subjected to kinematic constraints on many small regions, which we call perforations. Such constraints on the displacement u are given in the quite general form u(x)∈Fx, where Fx is a closed set, and thus comprise confinement conditions, unilateral constraints, controlled displacement conditions, etc., both in the bulk and on the boundary of the body. The regions in which such conditions are active are assumed to be so small that they do not produce an overall rigid constraint, but still large enough so as to produce a non-trivial effect on the behaviour of the body. Mathematically, this is translated in an asymptotic analysis by introducing two small parameters: ɛ, describing the distance between the elements of the perforation, and δ, the size of the element of the perforation. We find the critical scale at which the effect of the perforation is non-trivial and express it in terms of a Γ-limit in which the constraints are relaxed so that, in their place, a penalization term appears in the form of an integral of a function φ(x,u). This function is determined by a blow-up procedure close to the perforation and depends on the shape of the perforation, the constraint Fx, and the asymptotic behaviour at infinity of the strain energy density σ. We give a concise proof of the mathematical result and numerical studies for some simple yet meaningful geometries.

Regularized Regression Can Reintroduce Backdoor Confounding: The Case of Mass Polarization

Regularization can improve statistical estimates made with highly correlated data. However, any regularization procedure embeds assumptions about the data generating process that can have counterintuitive consequences when those assumptions are untenable. We show that rather than simply shrinking estimates, regularization can reopen backdoor causal paths, inflating the estimates of some effects, and in the wrong circumstances, even reversing their direction. Recently, Cavari and Freedman (2023), argued that declining cooperation rates in surveys have inflated measures of mass polarization. We show that this finding is driven by large penalty terms in their regularized regressions, which leads to the estimates being confounded with time. Alternative methods do not show a clear positive or negative effect of declining cooperation on estimated levels of mass polarization.

Energy-efficient resource allocation for UAV-aided full-duplex OFDMA wireless powered IoT communication networks

The rapid development of wireless-powered Internet of Things (IoT) networks, supported by multiple unmanned aerial vehicles (UAVs) and full-duplex technologies, has opened new avenues for simultaneous data transmission and energy harvesting. In this context, optimizing energy efficiency (EE) is crucial for ensuring sustainable and efficient network operation. This paper proposes a novel approach to EE optimization in multi-UAV-aided wireless-powered IoT networks, focusing on balancing the uplink data transmission rates and total system energy consumption within an orthogonal frequency-division multiple access (OFDMA) framework. This involves formulating the EE optimization problem as a Multi-Objective Optimization Problem (MOOP), consisting of the maximization of the uplink total rate and the minimization of the total system energy consumption, which is then transformed into a Single-Objective Optimization Problem (SOOP) using the Tchebycheff method. To address the non-convex nature of the resulting SOOP, characterized by combinatorial variables and coupled constraints, we developed an iterative algorithm that combines Block Coordinate Descent (BCD) with Successive Convex Approximation (SCA). This algorithm decouples the subcarrier assignment and power control subproblems, incorporates a penalty term to relax integer constraints, and alternates between solving each subproblem until convergence is reached. Simulation results demonstrate that our proposed method outperforms baseline approaches in key performance metrics, highlighting the practical applicability and robustness of our framework for enhancing the efficiency and sustainability of real-world UAV-assisted wireless networks. Our findings provide insights for future research on extending the proposed framework to scenarios involving dynamic UAV mobility, multi-hop communication, and enhanced energy management, thereby supporting the development of next-generation sustainable communication systems.

A Three-Block Inexact Heterogeneous Alternating Direction Method of Multipliers for Elliptic PDE-Constrained Optimization Problems with a Control Gradient Penalty Term

Optimization problems with PDE constraints are widely used in engineering and technical fields. In some practical applications, it is necessary to smooth the control variables and suppress their large fluctuations, especially at the boundary. Therefore, we propose an elliptic PDE-constrained optimization model with a control gradient penalty term. However, introducing this penalty term increases the complexity and difficulty of the problems. To solve the problems numerically, we adopt the strategy of “First discretize, then optimize”. First, the finite element method is employed to discretize the optimization problems. Then, a heterogeneous strategy is introduced to formulate the augmented Lagrangian function for the subproblems. Subsequently, we propose a three-block inexact heterogeneous alternating direction method of multipliers (three-block ihADMM). Theoretically, we provide a global convergence analysis of the three-block ihADMM algorithm and discuss the iteration complexity results. Numerical results are provided to demonstrate the efficiency of the proposed algorithm.

Penalized Likelihood Estimation for Current Status Data with Informative Censoring

Current status data occurs when failure time of subjects in a survival study is only known to be either less or greater than the censoring time. Thus, the failure time is either left – or right – censored. Analyzing data of this structure under the Cox Proportional Hazards model with dependent censoring assumption can be challenging. To address this, a Penalized Maximum Likelihood Estimation (PMLE) approach was proposed. The unknown baseline cumulative hazard functions for both the failure time and the censoring time were estimated using splines. The advantage of penalized approach over unpenalized method is that that the desired smoothness level of the functions are controlled by their respective penalty terms. The possible dependence between the failure and censoring times was accounted for using the gamma-frailty model. An easy to implement hybrid computational algorithm is proposed to estimate the PMLEs and the Bayesian technique was employed for the estimation of the variances of the parameters. Extensive simulation studies were conducted to assess the statistical properties of the PMLEs. It was observed that the realized estimators were not only consistent, asymptotically normal and efficient, but also, were robust to the number of knots chosen, the proportion of dependent censoring used and the frailty distribution employed. The proposed PMLE method was further applied to real data obtained from tumorigenicity experiment.

DMFGAN: a multifeature data augmentation method for grape leaf disease identification.

The use of deep learning techniques to identify grape leaf diseases relies on large, high-quality datasets. However, a large number of images occupy more computing resources and are prone to pattern collapse during training. In this paper, a depth-separable multifeature generative adversarial network (DMFGAN) was proposed to enhance grape leaf disease data. First, a multifeature extraction block (MFEB) based on the four-channel feature fusion strategy is designed to improve the quality of the generated image and avoid the problem of poor feature learning ability of the adversarial generation network caused by the single-channel feature extraction method. Second, a depth-based D-discriminator is designed to improve the discriminator capability and reduce the number of model parameters. Third, SeLU activation function was substituted for DCGAN activation function to overcome the problem that DCGAN activation function was not enough to fit grape leaf disease image data. Finally, an MFLoss function with a gradient penalty term is proposed to reduce the mode collapse during the training of generative adversarial networks. By comparing the visual indicators and evaluation indicators of the images generated by different models, and using the recognition network to verify the enhanced grape disease data, the results show that the method is effective in enhancing grape leaf disease data. Under the same experimental conditions, DMFGAN generates higher quality and more diverse images with fewer parameters than other generative adversarial networks. The mode breakdown times of generative adversarial networks in training process are reduced, which is more effective in practical application.

Novel sparse auto-encoder framework with pseudo-labeled reinforcement for cross domain fault diagnosis with imbalanced samples

Abstract By applying diagnostic expertise from one area to another that is closely related, transfer fault diagnosis is an efficient strategy for guaranteeing the secure and dependable operation of mechanical equipment. However, the majority of rotating machinery monitoring data are gathered in industrial applications under normal operating settings, which leads to an imbalance between positive and negative samples. Consequently, performing high-precision fault diagnosis with imbalanced data and different working conditions becomes challenging due to the escalating difficulty and cost of acquiring labeled fault samples. For cross domain fault diagnosis, an enhanced deep transfer sparse auto-encoder (SAE) framework is provided. This approach leverages a SAE to delve deeper into fault features within imbalanced data and reconstruct the samples accordingly. Furthermore, the research proposes a feature domain penalty term to facilitate cross-domain training by aligning the distribution of the source and target domain data which can reduce data distribution deviation. A pseudo-labeled reinforcement training method is presented to further improve cross-domain classification accuracy with imbalanced samples. Extensive experiments were conducted on two datasets to assess the proposed method. The results were compared with other algorithms, demonstrating the effectiveness and superiority of the proposed approach.

A deep learning based assisted analysis approach for Sjogren's syndrome pathology images.

Diagnosing Sjogren's syndrome requires considerable time and effort from physicians, primarily because it necessitates rigorously establishing the presence lymphatic infiltration in the pathological tissue of the labial gland. The aim of this study is to use deep learning techniques to overcome these limitations and improve diagnostic accuracy and efficiency in pathology. We develop an auxiliary diagnostic system for Sjogren's syndrome. The system incorporates the state-of-the-art object detection neural network, YOLOv8, and enables the precise identification and flagging of suspicious lesions. We design the multi-dimensional attention module and S-MPDIoU loss function to improve the detection performance of YOLOv8. By extracting features from multiple dimensions of the feature map, the utilization of the multi-dimensional attention mechanism enhances the feature interaction across disparate positions, enabling the network to proficiently learn and retain salient cell features. S-MPDIoU introduces an angle penalty term that efficiently minimizes the diagonal distance between predicted and ground truth boxes. Additionally, it incorporates a flexible scale factor tailored to different size feature maps, which balances the issue of sudden gradient decrease during high overlap, thereby accelerating the overall convergence rate. To verify the effectiveness of our methods, we create a dataset of lymphocytes using labial gland biopsy pathology images collected from YanTaiShan hospital and trained the model with this dataset. The proposed model is assessed using standard metrics like precision, recall, mAP. The improved model achieves an increase in recall by 9.1%, mAP.5 by 3.2%, and mAP.95 by 2%. The study demonstrated deep learning's potential to analysis pathology images, offering a reference framework for the application of deep learning technology in the medical domain.