Penalty Weights Research Articles

Weight-adaptive augmented Lagrange multiplier sequential convex programming for nonlinear trajectory optimization

Sequential convex programming has become a prominent research focus in aerospace trajectory optimization due to its high computational efficiency. One critical challenge in sequential convex programming is the infeasibility of subproblems, which significantly impacts its stability. To address this issue, certain constraints must be relaxed using relaxation variables, which are then incorporated into the objective function as a penalty term. However, the choice of penalty weight greatly influences the optimality and convergence of sequential convex programming. Therefore, this paper proposes a weight-adaptive augmented Lagrange multiplier sequential convex programming algorithm to tackle the challenge. First, this paper analyzes the difficulty of determining an appropriate penalty weight from the perspective of its penalty effect. Based on this analysis, an augmented penalty term is introduced, and the update rule for the augmented Lagrange multiplier is formulated using the Karush-Kuhn-Tucker conditions. Subsequently, a method with adaptive weight adjustment is designed based on the change in the penalty term to further improve the algorithm's solving speed. Finally, the proposed method is validated through numerical simulations in a Mars-powered landing problem and a small glide vehicle reentry problem. The results show that the proposed method is 20.59 % and 25.16 % faster compared to the general sequential convex programming algorithm, respectively. Furthermore, the proposed method is demonstrated to be robust to the initial choice of penalty weight.

Anterior Cruciate Ligament Tear Detection Based on T-Distribution Slice Attention Framework with Penalty Weight Loss Optimisation.

Anterior cruciate ligament (ACL) plays an important role in stabilising the knee joint, prevents excessive anterior translation of the tibia, and provides rotational stability. ACL injuries commonly occur as a result of rapid deceleration, sudden change in direction, or direct impact to the knee during sports activities. Although several deep learning techniques have recently been applied in the detection of ACL tears, challenges such as effective slice filtering and the nuanced relationship between varying tear grades still remain underexplored. This study used an advanced deep learning model that integrated a T-distribution-based slice attention filtering mechanism with a penalty weight loss function to improve the performance for detection of ACL tears. A T-distribution slice attention module was effectively utilised to develop a robust slice filtering system of the deep learning model. By incorporating class relationships and substituting the conventional cross-entropy loss with a penalty weight loss function, the classification accuracy of our model is markedly increased. The combination of slice filtering and penalty weight loss shows significant improvements in diagnostic performance across six different backbone networks. In particular, the VGG-Slice-Weight model provided an area score of 0.9590 under the receiver operating characteristic curve (AUC). The deep learning framework used in this study offers an effective diagnostic tool that supports better ACL injury detection in clinical diagnosis practice.

Primal-dual hybrid gradient image denoising algorithm based on overlapping group sparsity and fractional-order total variation

This study introduces a non-convex fractional-order hyper-Laplacian variational model for Gaussian noise removal. It employs first the primal-dual hybrid gradient algorithm to solve problems involving overlapping group sparse structures. Additionally, this paper designs a new algorithm leveraging the framework of the Chambolle-Pock algorithm with convergence and aims to recover high-quality images. The model, integrating the overlapping group sparse structure of the hyper-Laplacian prior with the non-convex fractional-order total variation, exhibits superior performance in reducing the staircase effect and maintaining sharp edge contours. To further improve the performance of the algorithm, a semi-adaptive p(x) non-convex penalty weight assignment mechanism is designed by introducing the structure tensor, which according to the characteristics of each region of the image and the noise level. The effectiveness and superiority of the proposed algorithm in image denoising with simulation experiments and comparative analyses are fully verified.

FCAN : Speech emotion recognition network based on focused contrastive learning

Traditional speech emotion recognition (SER) models predominantly utilize cross-entropy loss for supervised optimization. Yet, given the limited size of SER datasets, relying solely on cross-entropy loss does not adequately capture the intricacies within the data. Moreover, conventional contrastive learning techniques, though improving performance, do not completely harness the inherent data structure across samples. To surmount these limitations, we introduce an innovative emotion recognition network founded on focused contrastive learning. Our model employs bidirectional gated recurrent unit (GRU) networks to seize critical contextual dependencies. By integrating an attention mechanism, we adeptly merge the contextual features of dialogues with individual utterance attributes, forming a comprehensive multi-stage feature fusion framework. This strategy extracts more elaborate emotional features from the constrained dataset. Additionally, by amalgamating the focal loss function into our training protocol, we mitigate imbalanced sample issues to a degree. We further refine our contrastive loss function by enacting a weighted strategy for intra-dialogue category assignments, where higher penalty weights are ascribed to harder-to-identify categories, thus incentivizing the model to concentrate on distinguishing subtler category differences. This refinement significantly bolsters the model’s proficiency in learning various emotional feature distributions, thereby enhancing its emotion recognition capabilities. Experimental validation on IEMOCAP and MELD benchmark datasets underpins our model’s superiority.

Penalty weight tuning in high dose rate brachytherapy using multi-objective Bayesian optimization

Objective. Treatment plan optimization in high dose rate brachytherapy often requires manual fine-tuning of penalty weights for each objective, which can be time-consuming and dependent on the planner's experience. To automate this process, this study used a multi-criteria approach called multi-objective Bayesian optimization with q-noisy expected hypervolume improvement as its acquisition function (MOBO-qNEHVI). Approach. The treatment plans of 13 prostate cancer patients were retrospectively imported to a research treatment planning system, RapidBrachyMTPS, where fast mixed integer optimization (FMIO) performs dwell time optimization given a set of penalty weights to deliver 15 Gy to the target volume. MOBO-qNEHVI was used to find patient-specific Pareto optimal penalty weight vectors that yield clinically acceptable dose volume histogram metrics. The relationship between the number of MOBO-qNEHVI iterations and the number of clinically acceptable plans per patient (acceptance rate) was investigated. The performance time was obtained for various parameter configurations. Main results. MOBO-qNEHVI found clinically acceptable treatment plans for all patients. With increasing the number of MOBO-qNEHVI iterations, the acceptance rate grew logarithmically while the performance time grew exponentially. Fixing the penalty weight of the tumour volume to maximum value, adding the target dose as a parameter, initiating MOBO-qNEHVI with 25 parallel sampling of FMIO, and running 6 MOBO-qNEHVI iterations found solutions that delivered 15 Gy to the hottest 95% of the clinical target volume while respecting the dose constraints to the organs at risk. The average acceptance rate for each patient was 89.74% ± 8.11%, and performance time was 66.6 ± 12.6 s. The initiation took 22.47 ± 7.57 s, and each iteration took 7.35 ± 2.45 s to find one Pareto solution.Significance. MOBO-qNEHVI combined with FMIO can automatically explore the trade-offs between treatment plan objectives in a patient specific manner within a minute. This approach can reduce the dependency of plan quality on planner’s experience and reduce dose to the organs at risk.

A multi‐label social short text classification method based on contrastive learning and improved ml‐KNN

AbstractShort texts on social platforms often have the problems of diverse categories and semantic sparsity, making it challenging to identify the diverse intentions of users. To address this issue, this article proposes a multi‐label social short text classification method (IML‐CL) based on contrastive learning and improved ml‐KNN. First, a contrastive learning approach is employed to train a multi‐label text classification model. This approach improves semantic sparsity by leveraging the knowledge from the existing samples to enrich the feature representation of short texts. Simultaneously, an improved ml‐KNN algorithm is developed to enhance the accuracy of label prediction. This algorithm utilizes a two‐layer nearest neighbor rule and introduces a penalty function and weight optimization. Next, the model generates the feature representation for the test sample and predicts its label. Additionally, the improved ml‐KNN algorithm retrieves neighbors of the test sample and uses their label information for prediction. Finally, the two predictions are combined to obtain the final prediction, which accurately identifies the user's intention. The experimental results demonstrate that, on the dataset constructed in this article, the IML‐CL method effectively boosts the performance of the baseline model.

High-dimensional generalized median adaptive lasso with application to omics data.

Recently, there has been a growing interest in variable selection for causal inference within the context of high-dimensional data. However, when the outcome exhibits a skewed distribution, ensuring the accuracy of variable selection and causal effect estimation might be challenging. Here, we introduce the generalized median adaptive lasso (GMAL) for covariate selection to achieve an accurate estimation of causal effect even when the outcome follows skewed distributions. A distinctive feature of our proposed method is that we utilize a linear median regression model for constructing penalty weights, thereby maintaining the accuracy of variable selection and causal effect estimation even when the outcome presents extremely skewed distributions. Simulation results showed that our proposed method performs comparably to existing methods in variable selection when the outcome follows a symmetric distribution. Besides, the proposed method exhibited obvious superiority over the existing methods when the outcome follows a skewed distribution. Meanwhile, our proposed method consistently outperformed the existing methods in causal estimation, as indicated by smaller root-mean-square error. We also utilized the GMAL method on a deoxyribonucleic acid methylation dataset from the Alzheimer's disease (AD) neuroimaging initiative database to investigate the association between cerebrospinal fluid tau protein levels and the severity of AD.

Model-Based Chance-Constrained Reinforcement Learning via Separated Proportional-Integral Lagrangian.

Safety is essential for reinforcement learning (RL) applied in the real world. Adding chance constraints (or probabilistic constraints) is a suitable way to enhance RL safety under uncertainty. Existing chance-constrained RL methods, such as the penalty methods and the Lagrangian methods, either exhibit periodic oscillations or learn an overconservative or unsafe policy. In this article, we address these shortcomings by proposing a separated proportional-integral Lagrangian (SPIL) algorithm. We first review the constrained policy optimization process from a feedback control perspective, which regards the penalty weight as the control input and the safe probability as the control output. Based on this, the penalty method is formulated as a proportional controller, and the Lagrangian method is formulated as an integral controller. We then unify them and present a proportional-integral Lagrangian method to get both their merits with an integral separation technique to limit the integral value to a reasonable range. To accelerate training, the gradient of safe probability is computed in a model-based manner. The convergence of the overall algorithm is analyzed. We demonstrate that our method can reduce the oscillations and conservatism of RL policy in a car-following simulation. To prove its practicality, we also apply our method to a real-world mobile robot navigation task, where our robot successfully avoids a moving obstacle with highly uncertain or even aggressive behaviors.

Distributed non‐linear model predictive control with Gaussian process dynamics for two‐dimensional motion of vehicle platoon

AbstractThe platoon control of connected and automated vehicles is an important topic in transportation research. The characteristics of non‐linearities, external disturbances, and strong coupling are non‐negligible in two‐dimensional motion control. An integrated longitudinal and lateral vehicle dynamics is required. A Gaussian Process‐based Distributed Stochastic Model Predictive Control (GP‐DSMPC) for two‐dimensional motion is proposed. It achieves global longitudinal stability and lateral error suppression. Gaussian process (GP) regression is employed to approximate the unknown model error. For the two‐norm chance constraints, over‐approximating the confidence ellipse to an outer polyhedron is an effective way to reduce the conservativeness and coupling effect in longitudinal and lateral motion. A neighbour‐average target trajectory is designed with an upper‐level optimization for adjustable target coefficients. The sum of the local cost functions is used as a Lyapunov candidate to achieve the global stability of the longitudinal motion. Some conditions on penalty weights and target coefficients among subsystems, and terminal outputs are derived. Simulation results reveal that the proposed method is effective for disturbance attenuation and performs better than distributed non‐linear model predictive control without GP estimation under low‐friction road conditions.

A goal programming model for off-grid power planning with a case of Marinduque island

Off-grid areas pose economic and technical challenges, often overlooked in national power planning. In the Philippines, the National Power Corporation manages mainly oil-based off-grid power assets, with issues of intermittency and reliability. This study proposes an optimal energy mix using non-preemptive Goal Programming combined with a pair-wise comparison to determine the penalty weights to balance energy security, affordability, sustainability, and self-sufficiency goals. Marinduque province serves as a case study, where it is identified that including renewable energy in the grid could help balance energy goals and reduce generation costs and emissions. Increased use of renewables can also satisfy demand and self-sufficiency goals, making electricity more affordable. Once electricity becomes more affordable, subsidies can be reduced and later on be removed. The study's recommendations can be a starting point for private investors and government agencies to improve the energy sector in off-grid areas.

Human Gait Cost Function Varies With Walking Speed: An Inverse Optimal Control Study

This work investigates the optimal cost function composition for human gait at different walking speeds. Kinematic and kinetic data for walking at four walking speeds were collected from five able-bodied individuals. The data was then used to recover optimal cost functions in a predictive simulation environment with musculoskeletal models. 20 inverse optimal control (IOC) problems were solved for cost function weight tuning using the previously developed and validated Adaptive Reference IOC (AR-IOC) algorithm. Given the walking speed range examined (0.6-1.5m/s), the converged cost function weights suggest that the increase in walking speed attributes to a reduction of foot sliding penalty weight and weight increase for the center of mass (CoM) acceleration and stability as confirmed by several experiments. Furthermore, we did not observe any significant weight shift in effort reduction between the upper and the lower body with respect to walking speed. The obtained results from this study can be used in a toolbox for obtaining subject- and task-specific cost functions and assisting the development of personalized rehabilitation technologies.

A Novel Decoupled Model Predictive Control-Based Motion Cueing Algorithm for Driving Simulators

The motion cueing algorithm (MCA) is used to reproduce the driving/flying motion sensation of the real land or air vehicles for the users of motion simulator. Highly accurate MCAs are required for the motion simulators to create realistic sensation for the simulator users, otherwise, they might cause motion sickness and user discomfort. Model predictive control (MPC) is a popular technique in development of the high-fidelity MCAs as it is able to consider the motion sensation as well as the workspace constraints of the simulator platform. Tilt coordination in MCA is in charge of creating sustained linear acceleration feeling through tilting the platform cabin and exploiting gravitational acceleration under human motion threshold, to not allow the user to perceive this as a rotational motion. However, the existing MPC-based MCAs have not considered the rate limit to constrain the tilt motion under human threshold which causes violations of the rotational sensation threshold during generating sustained acceleration feeling. Instead, some researchers increase the penalty weights of rotational motions in order to slow down rotational motions. Using this strategy slows down all the simulator rotational motions not only the ones coming from tilt coordination, to generate sustained acceleration feeling, but also from rotational channel (due to real vehicle rotation) which causes motion sensation error and consequently motion sickness. Therefore, as the existing MPC-based MCAs cannot differentiate the rotations of tilt coordination channel (due to sustained acceleration feeling generation), from rotational channel (due to vehicle rotational motions), they are not able to produce accurate rotational motion sensations because of insufficient rotational motions. In this research, a novel decoupled MPC-based MCA is developed to systematically address the issues related to the existing MPC-based MCAs, in terms of inaccurate motion generation, by redesigning the MPC-based MCA using a series of vestibular system-based MPC models and considering tilt rate limit in tilt coordination. The simulation study using MATLAB software is used to verify and validate the proposed MCA compared to the existing MCAs. The proposed decoupled MPC-based MCA is able to generate accurate motion cues compared to those of the existing MCAs.

Risk state evaluation model for China's food import using G1-LS and variable weight SPA based on bottom-line thinking

PurposeTo maintain the bottom line of food import risk in China, this paper proposes a novel risk state evaluation model based on bottom-line thinking after analyzing the decision-making ideas embedded in the bottom-line thinking method.Design/methodology/approachFirst, the order relation analysis method (G1 method) and Laplacian score (LS) are applied to calculate the constant weights of indexes. Then, the worst-case scenario of food import risk can be estimated to strive for the best result, so the penalty state variable weight function is introduced to obtain variable weights of indexes. Finally, the study measures the risk state of China's food import from the overall situation using the set pair analysis (SPA) method and identifies the key factors affecting food import risk.FindingsThe risk states of food supply in eight countries are in the state of average potential and partial back potential as a whole. The results indicate that China's food import risks are at medium and upper-medium risk levels in most years, fluctuating slightly from 2010 to 2020. In addition, some factors are diagnosed as the primary control objects for holding the bottom line of food import risk in China, including food output level, food export capacity, bilateral relationship and political risk.Originality/valueThis paper proposes a novel risk state evaluation model following bottom-line thinking for food import risk in China. Besides, SPA is first applied to the risk evaluation of food import, expanding the application field of the SPA method.

Sparsity Constrained Joint Activity and Data Detection for Massive Access: A Difference-of-Norms Penalty Framework

Grant-free random access is a promising mechanism to support modern massive machine-type communications in which devices are sporadically active with small payloads. Under this random access, a unique challenge is the detection of device activity without the cooperation from devices. Furthermore, for only a few bits of data, it is more efficient to embed the data to the signature sequences so that the activity and data detection can be jointly carried out. However, compared with the vanilla device activity detection, joint activity and data detection has an extra discontinuous sparsity constraint, which makes the detection problem more challenging. In contrast to the prevalent way of first neglecting the discontinuous sparsity constraint and reinforcing it at the end, this paper proposes a novel approach to incorporate the discontinuous sparsity constraint into the optimization procedure. In particular, we first establish the equivalence between the discontinuous sparsity constraint and a continuous difference-of-norms (DN) form. Then, by introducing a DN penalty term in the objective function, an iterative DN penalty method with an increasing penalty weight is adopted. We prove theoretically that by solving each penalized problem to a stationary solution, the discontinuous sparsity constraint can be exactly satisfied when the penalty weight is sufficiently large, and the resulting solution is guaranteed to be at least a stationary point of the original problem. Due to the superior theoretical guarantee, simulation results demonstrate that the proposed method achieves around 10 times better detection performance than state-of-the-art approaches.

MSC-CSMC: A multi-objective semi-supervised clustering algorithm based on constraints selection and multi-source constraints for gene expression data.

Many clustering techniques have been proposed to group genes based on gene expression data. Among these methods, semi-supervised clustering techniques aim to improve clustering performance by incorporating supervisory information in the form of pairwise constraints. However, noisy constraints inevitably exist in the constraint set obtained on the practical unlabeled dataset, which degenerates the performance of semi-supervised clustering. Moreover, multiple information sources are not integrated into multi-source constraints to improve clustering quality. To this end, the research proposes a new multi-objective semi-supervised clustering algorithm based on constraints selection and multi-source constraints (MSC-CSMC) for unlabeled gene expression data. The proposed method first uses the gene expression data and the gene ontology (GO) that describes gene annotation information to form multi-source constraints. Then, the multi-source constraints are applied to the clustering by improving the constraint violation penalty weight in the semi-supervised clustering objective function. Furthermore, the constraints selection and cluster prototypes are put into the multi-objective evolutionary framework by adopting a mixed chromosome encoding strategy, which can select pairwise constraints suitable for clustering tasks through synergistic optimization to reduce the negative influence of noisy constraints. The proposed MSC-CSMC algorithm is testified using five benchmark gene expression datasets, and the results show that the proposed algorithm achieves superior performance.

Improved area regularization technique for penalty-method-based node-to-segment contact analysis

Among numerous approaches that have been presented for computational contact analysis, the node-to-segment (NTS) approach is widely adopted in industrial applications, particularly with the penalty method. However, the pure penalty-method-based NTS approach fails the patch test even with the area regularization (AR) technique. There have been relatively few solutions for this problem compared to those for Lagrange-multiplier-method-based NTS approaches. This paper presents an improved AR technique and attempts to make the penalty-method-based NTS approach pass the patch test. The proposed approach improves the accuracy without the need to introduce additional geometry to the conventional penalty-method-based NTS approach. The entire competence area of a slave node is not directly utilized as a penalty weighting factor. Instead, a part of the competence area is extracted in accordance with the projection from a master node to a slave segment. In a complementary manner, a two-pass algorithm is used for the selected master nodes to achieve segment-wise force/moment equivalence. The results obtained using the proposed approach are introduced in a two-dimensional frictionless context. It is observed that the proposed approach passes the patch test. Furthermore, a significant improvement in accuracy compared to the conventional NTS and NTS-AR approaches is observed. We aim to extend the approach to three-dimensional applications in the future work.

Critérios de parada para programa de melhoramento genético para seleção de acasalamentos em gado de corte

Abstract The objective of this work was to propose a new stopping criterion to shorten the computing time of the PampaPlus genetic improvement software, while maximizing the genetic qualification index (GQI) of the progeny, controlling inbreeding, and avoiding unintended culling. Data from two beef-cattle herds integrating PampaPlus were used. Five mating scenarios were built using different numbers of sires (9 to 37) and dams (142 to 568). The analyzed algorithm inputs were: expected progeny differences, pedigree information, maximum inbreeding, maximum and minimum number of matches for each sire, and penalty weights for poor performance. The analyzed response variables were computing time and the GQI of the progenies. Three stopping criteria were used: original stopping criterion fixed at 1,000 iterations; saturation stopping criterion (SSC), based on GQI variance; and Bhandari’s stopping criterion (BSC), which includes the generation interval parameter. SSC and BSC reduced processing time in 24.43-53.64% and in 14.32-50.87%, respectively. BSC reaches solution in less time, without losses in GQI quality. BSC is generalizable and effective to reduce the processing time of mating recommendations.

Chance-Constrained Iterative Linear-Quadratic Stochastic Games

Dynamic game arises as a powerful paradigm for multi-robot planning, for which safety constraint satisfaction is crucial. Constrained stochastic games are of particular interest, as real-world robots need to operate and satisfy constraints under uncertainty. Existing methods for solving stochastic games handle chance constraints using exponential penalties with hand-tuned weights. However, finding a suitable penalty weight is nontrivial and requires trial and error. In this letter, we propose the chance-constrained iterative linear-quadratic stochastic games (CCILQGames) algorithm. CCILQGames solves chance-constrained stochastic games using the augmented Lagrangian method. We evaluate our algorithm in three autonomous driving scenarios, including merge, intersection, and roundabout. Experimental results and Monte Carlo tests show that CCILQGames can generate safe and interactive strategies in stochastic environments.

Multi-objective crashworthiness design optimization of a rollover protective structure by an improved constraint-handling technique

<abstract> <p>This study proposes a multi-objective optimization (MOO) strategy with an improved constraint-handling technique to improve the crashworthiness of an excavator rollover protective structure (ROPS). First, the experimental test under the ISO 12117 criteria is conducted and the developed numerical model is verified. Then, the amounts of energy absorption and the cross-sectional forces of components in the ROPS are analyzed. The main energy absorbing and load carrying components are identified. Finally, the thicknesses of the identified components are considered as the design variables. A multi-objective crashworthiness optimization process aims at improving the safety distance and reducing the total mass is designed by the finite element analysis-based surrogate model technique and a modified MOO algorithm. The proposed algorithm modifies the objective function values of an individual with its constraint violations and the true objective function values, of which adaptive penalty weights fed back from the constraint violations are used to keep the balance. Compared with the existing methods, it is found that the optimal solutions obtained by the proposed algorithm show superiority on convergence rate and diversity of distribution. The optimal results show that the safety distance is 27.42% higher while the total mass is 7.06% lower than those of the baseline design when it meets the requirements of ISO 12117. This study provides an alternative crashworthiness design route for the ROPS of the construction machines.</p> </abstract>

A new image decomposition approach using pixel-wise analysis sparsity model

Decomposing an image into two ‘simpler’ layers has been widely used in low-level vision tasks, such as image recovery and enhancement. It is an ill-posed problem since the number of unknowns are larger than the input. In this paper, a two-step strategy is introduced, including task-aware priors estimate and a decomposition model. A pixel-wise analysis sparsity model is proposed to regularize the separation layers, which supposes the transformed image generated with analysis operator is sparse. Unlike regularizing all pixels with one penalty weight, we try to estimate each pixel’s sparsity level with task-aware priors and to achieve pixel-wise sparse penalty. Additionally, one separation layer is regularized with both synthesis sparsity model and pixel-wise analysis sparsity model to exploit their complementary mechanisms. Unlike the analysis one utilizing image local features, the synthesis one exploits an over-complete dictionary and non-local similarity cues to provide flexible prior for regularizing the decomposition results. The proposed model is solved by an alternating optimization algorithm. We evaluate it with two applications, Retinex model and rain streaks removal. Extensive experiments on multiple enhancement datasets, many synthetic and real rainy images demonstrate that our method can remove imaging noise during Retinex decomposition, and can produce high fidelity deraining results. It achieves competing performance in terms of quantitative metrics and visual quality compared with the state-of-the-art methods.