Penalty Function Research Articles

Fuzzy support vector machine using local outlier factor and intuitionistic fuzzy sets for imbalanced datasets

Traditional classifiers are commonly used for solving class balance problems. However, many datasets exhibit class imbalance along with outliers and noise, which affect the classification accuracy, this paper proposes a fuzzy support vector machine algorithm based on local outlier factor and intuitionistic fuzzy sets. First, to highlight the importance of the minority class, the membership degree is set to the maximum value. Meanwhile, local outlier factor is calculated to measure the abnormality and further obtain the membership degree in the majority class. Then, a kernel density estimation method is designed to estimate the sample density accurately. Furthermore, based on the density distribution, intuitionistic fuzzy sets are used to assign membership and non-membership degrees to samples, effectively distinguishing between noise and outliers. Finally, different penalty coefficients for two classes are built to offset the impact of class imbalance on classification accuracy. Experimental results show the advantages of the proposed algorithm.

Stability and Error Analysis for a C$$^0$$ Interior Penalty Method for the Modified Phase Field Crystal Equation

Stability and Error Analysis for a C$$^0$$ Interior Penalty Method for the Modified Phase Field Crystal Equation

Probabilistic branch and bound considering stochastic constraints

In this paper, we investigate a simulation optimization problem that poses challenges due to (i) the inability to evaluate the objective and multiple constraint functions analytically; instead, we rely on stochastic simulation to estimate them, and (ii) a discrete and potentially vast solution space. Rather than providing a single optimal solution, our aim is to identify a set of near-optimal solutions within a specific quantile, such as the top 10%. Our investigation covers two different problem settings or frameworks. The first framework is focused solely on a stochastic objective function, disregarding any stochastic constraints. In this context, we propose employing a probabilistic branch-and-bound algorithm to discover a level set of solutions. Alternatively, the second framework involves stochastic constraints. To address such stochastically constrained problems, we utilize a penalty function methodology in conjunction with a probabilistic branch-and-bound algorithm. Furthermore, we establish a convergence analysis of both algorithms to demonstrate their asymptotic validity and highlight their theoretical properties and behavior. Our experimental results provide evidence of the efficiency of our proposed algorithms, showing that they outperform existing approaches in the field of simulation optimization.

The mixed penalty method for the Signorini problem

We investigate two penalty approaches for the Signorini problem in the mixed form. The well-posedness theory has been established for the two mixed penalty problems in both the continuous and discrete sense. For the continuous case, we obtain the error of the penalty for both two approaches. We also study the error estimates of the discrete mixed penalty problems, which depend on the mesh size and the penalty parameter. Based on the two penalty approaches, we design two algorithms and show their convergence. Several numerical experiments are carried out to confirm the theoretical results.

The solution of Rough Bilevel Nonlinear Programming Problem by using Trust-Region Penalty Method

The solution of Rough Bilevel Nonlinear Programming Problem by using Trust-Region Penalty Method

Bayesian L 1/2 Regression

It is well known that Bridge regression enjoys superior theoretical properties when compared to traditional LASSO. However, the current latent variable representation of its Bayesian counterpart, based on the exponential power prior, is computationally expensive in higher dimensions. In this article, we show that the exponential power prior has a closed form scale mixture of normal decomposition for α = ( 1 2 ) γ , γ ∈ { 1 , 2 , … } . We call these types of priors L 1 2 prior for short. We develop an efficient partially collapsed Gibbs sampling scheme for computation using the L 1 2 prior and study theoretical properties when p > n . In addition, we introduce a non-separable Bridge penalty function inspired by the fully Bayesian formulation and a novel, efficient coordinate descent algorithm. We prove the algorithm’s convergence and show that the local minimizer from our optimization algorithm has an oracle property. Finally, simulation studies were carried out to illustrate the performance of the new algorithms. Supplementary materials for this article are available online.

A novel meshless radial basis reproducing kernel particle approach for 3D thermo-mechanical analysis of mushy zone phase-change problems

This paper presents a three-dimensional thermo-elastoplastic computational model for the analysis of phase change problems involving the mushy zone. The model utilizes a novel meshless radial basis reproducing kernel particle approach, incorporating a high-order kernel that integrates Gaussian and cosine functions. An energy balance equation for the entire domain is formulated using the effective heat capacity method, based on the enthalpy formulation. The governing equations are discretized using the Galerkin weak form to compute thermal residual stresses, with essential boundary conditions enforced through the penalty method. Plastic deformation is characterized using the von Mises criterion with isotropic hardening, and the strain term resulting from temperature-dependent material properties is incorporated into the incremental solution process. Determination of the plastic strain increment is based on the Prandtl–Reuss flow rule. The accuracy and efficiency of the proposed meshless approach were rigorously evaluated through numerical examples encompassing two- and three-dimensional problems. The computational results were compared against available analytical and validated numerical solutions. This comprehensive assessment substantiated the robustness and precision of the developed technique. The case studies presented elucidate the capacity of the novel meshless model to effectively predict temperature distributions and residual stress fields within mushy zone phase change phenomena, accommodating both regular and irregular geometries under various boundary conditions. Notably, the model demonstrated consistent stability and high accuracy across the spectrum of simulated scenarios, thus underscoring its potential as a valuable tool for investigating complex phase change processes.

A modified bi-objective NSGA-II approach to sustainability in reconfiguration planning of dynamic cellular manufacturing systems

Manufacturing plant layouts are developed to facilitate optimal process flow. Modern manufacturing systems must meet present production demands and be adaptable to changes in process flow in the future. Dynamic Cellular Manufacturing Systems (DCMS) increase the flexibility of layouts by reconfiguring cell structure and equipment distribution, to effectively adjust part routings for optimal process flow. Frequent reconfiguring of plant layout may not always be feasible or economical, however, when new product releases are planned, reconfiguring the plant layout to optimise the work-flow may be extremely beneficial. This paper presents a Non-dominated Sorting Genetic Algorithm (NSGA-II) approach to solving a DCMS problem in a sustainable, and responsible manner. A bi-objective integer programming model was developed over multiple planning horizons with fluctuating product demands. This model aims to achieve sustainability by reducing the cost of production, mitigating the environmental impact of production, and minimise negative social impacts on labourers that work in such environments. A penalty function approach was used to enforce the model constraints during optimisation. This study details trade-offs between the economic factors of a DCMS, the environmental implications of reconfiguring such a system, and the social impacts of reconfigurations on the workforce.

Optimal Control Problem with Regular Mixed Constraints via Penalty Functions

AbstractBelow we derive necessary conditions of optimality for problems with mixed constraints (see Dmitruk in Control Cybern 38(4A):923–957, 2009) using the method of penalty functions similar to the one we previously used to solve optimization problems for control sweeping processes (see, e.g., De Pinho et al. in Optimization 71(11):3363–3381, 2022) and, more recently, to solve optimal control problems with pure state constraints (see De Pinho et al. in Syst Control Lett 188:105816, 2024). We intentionally consider a smooth case and the simplest boundary conditions; we consider global minimum and assume that the set of trajectories of the control system is compact. Based on our penalty functions approach we develop a numerical method admitting estimates for its parameters needed to achieve a given precision.

Exploring interpretable evolutionary optimization via significance of each constraint and population diversity

Evolutionary algorithms (EAs) have been widely employed to solve complex constrained optimization problems (COPs). However, numerous EAs treat constraints as a collective black box, employing a uniform processing technique for all constraints. Generally, there exists variability in the significance of each constraint within COPs. To address this issue, this paper is the first attempt to investigate the significance of each constraint spontaneously during the evolution process, and then proposes a co-directed evolutionary algorithm (CdEA-SCPD) for exploring interpretable COPs. First, CdEA-SCPD develops an adaptive penalty function designed to assign different weights to constraints based on their violation severity, thereby varying the significance of each constraint to enhance interpretability and facilitate the algorithm to converge more rapidly toward the global optimum. In addition, a dynamic archiving strategy and a shared replacement mechanism are developed to improve the population diversity of CdEA-SCPD. Extensive experiments on benchmark functions from IEEE CEC2006, CEC2010, and CEC2017 and three engineering problems demonstrate the superiority of the proposed CdEA-SCPD compared to existing competitive EAs. Specifically, on the benchmark functions from IEEE CEC2010, the proposed method yields ρ values lower than 0.05 in the multiple-problem Wilcoxon's signed rank test and ranks first in the Friedman's test. Furthermore, ablation analysis and parameter analysis have demonstrated the beneficial effects of the proposed strategies.

Prediction model of coal seam gas content based on kernel principal component analysis and IDBO-DHKELM

Abstract Accurate coal seam gas content assists in the effective prevention of coal and gas outburst accidents. To solve this problem, an IDBO-DHKELM coal seam gas content prediction model is proposed by combining improved Dung Beetle optimization algorithm (IDBO) with a deep hybrid kernel extreme learning machine (DHKELM). First, the index factors of the coupled gas content are determined according to the influence factors of coal seam gas content and the actual situation of mine production. The correlation of index factors is analyzed by SPSS 27 software via Pearson correlation coefficient matrix. Then, the principal components of the original data are extracted using the principal component analysis method (KPCA). Second, sine chaotic mapping, fusion improved sinusoidal algorithm, and fusion adaptive Gauss–Cauchy hybrid mutation perturbation are introduced to improve the Dung Beetle optimization algorithm (DBO) to enhance its global search capability. Third, IDBO is used to optimize the number of hidden layer nodes, regularization coefficient, penalty coefficient, and kernel parameter in DHKELM, which improves the prediction accuracy and further avoid the phenomenon of overfitting. Finally, the principal component extracted by KPCA is taken as the model’s input, and the gas content as the model’s output. The results are compared and analyzed with those of PSO-BPNN, GA-BPNN, PSO-SVM, and DPO-DHKELM models. The results demonstrate that the IDBO-DHKELM model’s performance is the best in each performance index. Compared with other models, the mean absolute error of test samples in the IDBO-DHKELM model is reduced by 0.402, 0.4407, 0.3554, and 0.0646, respectively. The mean absolute percentage error is decreased by 3.67%, 4.07%, 8.27%, and 6.35%, respectively. The root mean square error decreased by 0.7861, 0.7148, 0.3384, and 0.1186, respectively. The coefficient of determination (R 2) is increased by 0.1544, 0.1404, 0.0955, and 0.0396, respectively. Finally, the IDBO-DHKELM model and other models are applied to an experimental mine. The resulting IDBO-DHKELM model is the closest to the actual value, which further verifies the universality and reliability of the model. Therefore, the model is more suitable for the prediction of coal seam gas content.

Information Gap Decision-Making Theory-Based Medium- and Long-Term Optimal Dispatching of Hydropower-Dominated Power Grids in a Market Environment

In the high-proportion hydropower market, the fairness of the execution of traded electricity and clean energy consumption are two issues that need to be considered in medium- and long-term dispatching. Aiming at the fairness of medium- and long-term optimal dispatching of hydropower-dominated grids and the problem of water abandonment in the power market environment, this paper proposes a medium- and long-term optimal dispatching method for hydropower-dominated grids based on the information gap decision-making theory (IGDT). Firstly, IGDT is used to establish a two-layer model of medium- and long-term optimal dispatching that considers runoff uncertainty, in which the lower layer solves the maximum value of the maximum difference in the contract power completion rate of the power stations, and the upper layer solves the maximum fluctuation range of the interval inflow. Then, a mixed-integer linear programming (MILP)-based single-layer optimization model is obtained through a variety of linearization techniques, and the model is solved via the CPLEX solver (version 12.10.0). The medium- and long-term optimal dispatching of 10 thermal power stations and 22 hydropower stations in Yunnan Power Grid, China, is taken as an example to verify the proposed model. The results show that the maximum difference in the contracted electricity completion rate of each power station is 0.412, and the amount of abandoned hydropower is reduced by 81.33% compared to when the abandoned water penalty function is not considered. It is proved that the proposed model can effectively alleviate the problems of excessive power generation, insufficient power generation and large-scale hydropower abandonment, which are of great significance for realizing the fair dispatching of hydropower-dominated power grids and promoting clean energy consumption in the market environment.

CMA-ES-based topology optimization accelerated by spectral level-set-boundary modeling

Topology optimization commonly encounters several challenges, such as ill-posedness, grayscale issues, interdependencies among design variables, multimodality, and the curse of dimensionality. Furthermore, addressing the latter two concurrently presents considerable difficulty. In this study, we introduce a framework aimed at mitigating all the above obstacles simultaneously. The objective is to achieve optimal configurations in a notably reduced timeframe eliminating the need for the initial trial-and-error iterations. The topology optimization approach we propose is implemented via precise structural boundary modeling utilizing a body-fitted mesh generated using a Fourier series expanded level-set method. This methodology expedites the exploration of optimal solutions. We employ the covariance matrix adaptation-evolution strategy to address multimodality, thereby enhancing the optimization process. The implementation of the Fourier-series-expanded level-set method reduces the number of design variables while maintaining accuracy in finite-element analyses by replacing design variables from discretized level-set functions with the coefficients of the Fourier series expansion. To facilitate the exploration of optimal solutions, a method is also introduced for handling box constraints through an adaptive penalty function. To demonstrate the effectiveness of the proposed scheme, we address three distinct problems: mean compliance minimization, heat flux manipulation, and the control of electromagnetic wave scattering. Despite each system being governed by different equations, topology optimization method consistently yields notable acceleration in computational efficiency across all scenarios, and remarkably without requiring initial guesses.

Missing data recovery based on temporal smoothness and time-varying similarity for wireless sensor network

Wireless Sensor Networks (WSN) play a vital role in the Internet of Things (IoT) and show great potential in monitoring applications. However, due to harsh environmental conditions and unreliable communication links, WSN often encounter partial data loss during data collection, which inevitably affects the quality of service. To address this challenge, researchers have employed matrix completion techniques to recover missing data by exploiting the low-rank features in the data, but its accuracy is not satisfactory. This paper argues that the spatiotemporal characteristics of the data underlie its low-rank nature, enabling a more accurate capture of the intrinsic patterns within the data. Drawing on this insight, we propose a missing data recovery algorithm based on Temporal Smoothness and Time-Varying Similarity (TSTVS). Unlike traditional low-rank methods, the TSTVS algorithm directly utilizes the structural features of data in the spatiotemporal domain to establish a missing data completion model. Subsequently, the model is converted into an unconstrained optimization problem using the penalty function method, and the gradient descent method is applied to solve it, reconstructing the complete data matrix. Finally, simulation experiments were conducted on three real-world monitoring datasets, comparing the TSTVS with three low-rank methods, Efficient Data Collection Approach (EDCA), Matrix factorization with Smoothness constraints (MFS) and Data Recovery Based on Low Rank and Short-Term Stability(DRLRSS). The experimental results indicate that the proposed TSTVS algorithm consistently outperforms the three low-rank based algorithms in terms of recovery accuracy across different datasets and missing rate scenarios.

Zero gradient sum algorithm of arbitrary initial value with constraints and communication delay based on directed graph

The distributed convex optimisation issue with inequality constraints and box constraints on the directed communication topology is examined in this paper. In the case of the communication delay between agents, we propose a piecewise zero-gradient-sum triggered control algorithm that allows arbitrary initial values. Using the log-barrier penalty method, we only need to study an unconstrained approximation problem of the original problem. Firstly, each agent's state converges to the optimal value of the related local objective function in a fixed time. Simultaneously, the upper bound of fixed time in this paper is smaller than some existing results. Then, in the case of communication delay, time-triggered and event-triggered methods with non-uniform sampling interval are proposed in this paper, which make the communication cost lower and the application broader. Using the Lyapunov function method, the sufficient conditions for each agent's state to converge to the optimal point are given, and the Zeno behaviour is excluded. Finally, to confirm the algorithm's effectiveness, we provide two numerical examples and one machine learning example as demonstrations.

A novel behavioral penalty function for interval goal programming with post-optimality analysis

Goal programming (GP) is a multi-objective extension of linear programming. Interval GP (IGP) is one of the earliest methods to expand the range of preferred structures in GP. The decision maker’s (DM’s) utility or preference in IGP is investigated by incorporating a widening range of underlying utility functions, commonly known as penalty functions. The basic idea of these functions is that undesirable deviations from the target levels of the goals are penalized regarding a constant or variable penalty value. The main concern with introducing the penalty functions is providing a wide range of a priori preference structures. Yet, the evaluation of how undesirable deviations are penalized based on DM’s behavioral preferences is not sufficiently addressed in the penalty function types developed in the GP literature. In real-world scenarios involving risk, the achievement levels of decision-making attributes are typically associated with the behavior of the DM. In such scenarios, the DM’s unavoidable attitude toward risk should be integrated into the decision-making process. We introduce the concept of behavioral penalty functions into the IGP approach, incorporating a risk aversion parameter tailored to the nature of each attribute to address this gap. This concept offers an innovative framework for capturing the preferences of the DMs and their various attitudes toward risk within the IGP approach. In this paper, we first introduce the concept of behavioral penalty functions. Next, we develop a behavioral utility-based IGP model. Finally, we present a portfolio selection case study to demonstrate the applicability and efficacy of the proposed procedure, followed by a post-optimality analysis and comparisons with other GP approaches.

Distributed predefined-time optimal economic dispatch for microgrids

With the massive popularization of distributed generators, optimal economic dispatch has been a key optimization problem to maintain stable and efficient work of the whole system. In this paper, a new smooth reconstruction penalty function with continuous and piecewise linear differential is designed to deal with generation power constraints, which promotes to obtain a better suboptimal solution compared with the existing smooth penalty methods. A distributed predefined-time optimal economic dispatch strategy is presented by utilizing a time-based function. By employing the proposed strategy, the minimization of the generation cost with transmission loss under the power balance constraint and generation minimum/maximum constraints can be realized within a predefined settling time. The performance of the proposed optimization strategy is evaluated by simulations and hardware-in-the-loop experiments in terms of validity verification, robustness to load change and topology reconfiguration, plug-and-play functionality, and comparison with the existing results to illustrate the advantages of fast convergence and near optimal results.

Low-carbon operation strategy of virtual power plant considering progressive demand response

Adapting virtual power plants (VPPs) to the development of new power systems has gradually become a trend. Meanwhile, the economical and effective use of demand-side resources in VPPs has become a research hotspot in the field of new energy. This paper proposes a low-carbon operation strategy with progressive demand response (DR). A DR model id established by evaluating the scheduling potential of low-carbon and flexible loads and the DR transfer of different users motivated by incentive mechanisms. Because the changing carbon price in carbon trading also affects the response transfer amount, it is beneficial to reduce the operating cost of the energy system by using consumers’ surplus value according to consumer psychology to guide energy users to participate in DR. Finally, wind, light, user load data and VPP internal unit output are optimized in rolling mode over multiple time scales, and the optimal adjustment results of the previous day are taken as the basis. At the same time, the output deviation penalty function is added to further reduce the system cost. The nonlinear model of DR cost is transformed into a mixed integer linear model, which is modeled on MATLAB software and solved by a CPLEX optimization solver. The case analysis shows that the progressive DR strategy may increase the total operating cost of the system, but reduce the cost of the carbon emission. The proposed model and method can not only improve the feasibility and effectiveness of market supervision strategy, but also provides a useful reference for operators and demand-side users in energy systems to choose strategies.

Topology optimization with graph neural network enabled regularized thresholding

Topology optimization algorithms often employ a smooth density function to implicitly represent geometries in a discretized domain. While this implicit representation offers great flexibility to parametrize the optimized geometry, it also leads to a transition region. Previous approaches, such as the Solid Isotropic Material Penalty (SIMP) method, have been proposed to modify the objective function aiming to converge toward integer density values and eliminate this non-physical transition region. However, the iterative nature of topology optimization renders this process computationally demanding, emphasizing the importance of achieving fast convergence. Accelerating convergence without significantly compromising the final solution can be challenging. In this work, we introduce a machine learning approach that leverages the message-passing Graph Neural Network (GNN) to eliminate the non-physical transition zone for the topology optimization problems. By representing the optimized structures as weighted graphs, we introduce a generalized filtering algorithm based on the topology of the spatial discretization. As such, the resultant algorithm can be applied to two- and three-dimensional space for both Cartesian (structured grid) and non-Cartesian discretizations (e.g. polygon finite element). The numerical experiments indicate that applying this filter throughout the optimization process may avoid excessive iterations and enable a more efficient optimization procedure.

Augmented multi-agent algorithm utilizing intelligent search range detection heuristic to solve assembly line sequencing problem: A case study in the truck industry

Car sequencing is a widely adopted approach to implicitly enhance the short-term balancing of mixed-model assembly lines in practical applications. In this research, we investigate this problem in the context of a real-world case taken from a truck final assembly line. Our study considers cross-ratio constraints that address limitations related to dependent features of the products. Additionally, to manage the distribution of violation errors in cases where predefined congestion level thresholds (soft rules) exist, we propose a piecewise linear penalty function. To tackle these considerations, we formulate a mathematical programming model aimed at minimizing the total weighted penalty incurred from violating both independent and dependent sequencing rules. As the problem falls into the NP-hard category, we develop a multi-start parallel local search algorithm based on multi-agent approach to efficiently solve it. The algorithm is augmented by incorporating a self-adaptive diversification mechanism and a novel intelligent search range detection heuristic. The proposed heuristic enables the searching agents to intelligently explore the solution space rather than relying on a purely random search. The numerical experiments are conducted using 30 real-world instances from three truck assembly lines to validate the quality of the proposed algorithm. The comparison of the results indicates that our algorithm outperforms state-of-the-art algorithms. Furthermore, computational experiments demonstrate the effectiveness of each proposed mechanism in enhancing the algorithm's performance.