Quadratic Penalty Term Research Articles

Order dispatch problem of the inter-city or inter-district ridesplitting service

This study formulates the order dispatch problem of the inter-city/inter-district ridesplitting service as an integer linear programming, and a solution framework that combines Lagrangian relaxation (LR) and alternating direction methods of multipliers is constructed. Specifically, LR problem is constructed by relaxing coupling constraints; based on this, the augmented Lagrangian relaxation (ALR) problem is constructed by adding a quadratic penalty term. Then, the ALR problem can be decomposed into a series of interdependent vehicle route subproblems by using linearisation technique and block coordinate descent method. The upper bound of the model is generated by ALR and a heuristic, and the solution quality can be evaluated by the lower bound provided by LR. Finally, numerous experiments are conducted to verify the effectiveness of proposed algorithms. The results show that the proposed algorithm could achieve an average cost saving of 10% and an average detour reduction of over 20% than the existing algorithm.

Sequential Convex Programming for Reentry Trajectory Optimization Utilizing Modified hp-Adaptive Mesh Refinement and Variable Quadratic Penalty

Due to the strong nonlinearity in the reentry trajectory planning problem for reusable launch vehicles (RLVs), the scale of the problem after high-precision discretization can become significantly large, and the non-convex path constraints are prone to exceed limits. Meanwhile, the objective function oscillation phenomenon may occur due to successive convexification, which results in poor convergence. To address these issues, a novel sequential convex programming (SCP) method utilizing modified hp-adaptive mesh refinement and variable quadratic penalty is proposed in this paper. Firstly, a local mesh refinement algorithm based on constraint violation is proposed. Additional mesh intervals and mesh points are added in the vicinity of the constraint violation points, which improves the satisfaction of non-convex path constraints. Secondly, a sliding window-based mesh reduction algorithm is designed and introduced into the hp-adaptive pseudospectral (PS) method. Unnecessary mesh intervals are merged to reduce the scale of the problem. Thirdly, a variable quadratic penalty-based SCP method is proposed. The quadratic penalty term related to the iteration direction and the weight coefficient updating strategy is designed to eliminate the oscillation. Numerical simulation results show that the proposed method can strictly satisfy path constraints while the computational efficiency and convergence of SCP are improved.

CL-BPUWM: continuous learning with Bayesian parameter updating and weight memory

Catastrophic forgetting in neural networks is a common problem, in which neural networks lose information from previous tasks after training on new tasks. Although adopting a regularization method that preferentially retains the parameters important to the previous task to avoid catastrophic forgetting has a positive effect; existing regularization methods cause the gradient to be near zero because the loss is at the local minimum. To solve this problem, we propose a new continuous learning method with Bayesian parameter updating and weight memory (CL-BPUWM). First, a parameter updating method based on the Bayes criterion is proposed to allow the neural network to gradually obtain new knowledge. The diagonal of the Fisher information matrix is then introduced to significantly minimize computation and increase parameter updating efficiency. Second, we suggest calculating the importance weight by observing how changes in each network parameter affect the model prediction output. In the process of model parameter updating, the Fisher information matrix and the sensitivity of the network are used as the quadratic penalty terms of the loss function. Finally, we apply dropout regularization to reduce model overfitting during training and to improve model generalizability. CL-BPUWM performs very well in continuous learning for classification tasks on CIFAR-100 dataset, CIFAR-10 dataset, and MNIST dataset. On CIFAR-100 dataset, it is 0.8%, 1.03% and 0.75% higher than the best performing regularization method (EWC) in three task partitions. On CIFAR-10 dataset, it is 2.25% higher than the regularization method (EWC) and 0.7% higher than the scaled method (GR). It is 0.66% higher than the regularization method (EWC) on the MNIST dataset. When the CL-BPUWM method was combined with the brain-inspired replay model under the CIFAR-100 and CIFAR-10 datasets, the classification accuracy was 2.35% and 5.38% higher than that of the baseline method, BI-R + SI.

Risk-averse stochastic capacity planning and P2P trading collaborative optimization for multi-energy microgrids considering carbon emission limitations: An asymmetric Nash bargaining approach

The increasing penetration of renewable energy with stochastic characteristics and continuous refinement of carbon emission policies put forward higher requirements for the construction of new power systems. This paper proposes a risk-averse stochastic capacity planning and peer-to-peer (P2P) trading collaborative optimization method for multi-energy microgrids (MEMGs) considering carbon emission limitations. First, a cooperative operation model for MEMGs considering the capacity planning of distributed generation units, uncertainty from renewable energy generations, carbon emission limitations and P2P electricity trading among MEMGs is formulated. Second, a risk-averse stochastic programming method is applied to avoid the potential risk losses caused by randomness and intermittence of renewable energy. Third, an asymmetric Nash bargaining approach is adopted to ensure the fair allocation of benefits and maintain the willingness of individual MEMGs to participate in cooperation. Then, to protect the privacy of individual MEMGs belonging to different stakeholders, the alternating direction method of multipliers is used to solve the two subproblems in a distributed manner. Meanwhile, to further alleviate the computation burdens, a diagonal quadratic approximation method is applied to linearize the quadratic penalty term in the augmented Lagrangian function and realize the parallel solution of all optimization subproblems. Moreover, the influence of different carbon emission targets on the optimal resource combination strategy of the system is investigated by introducing carbon emission factors. Simulations on different models, strategies, and distributed algorithms are conducted to verify the effectiveness and superiority of the proposed method.

Integrating train service route design with passenger flow allocation for an urban rail transit line

Train service design problem considers many operating strategies, i.e., multiple service routes, multiple train compositions, and express/local modes. Incorporating multiple operating strategies, this study first formulates an integer linear programming model for integrating train service route design with the passenger flow allocation problem. Consistency between the two components is enforced by a set of linking constraints that consider the relationship between the number of transit trips assigned to a route and the capacity of a single train. To solve the proposed model on real-life instances, we develop an approach that utilizes the Alternating Direction Method of Multipliers (ADMM). This dualizes the linking constraints and decomposes the problem into two subproblems: a train service route design subproblem and a passenger flow allocation subproblem. The latter can be subdivided into a set of passenger group-specific subproblems and is solved by a label correcting algorithm. Through Lagrangian multipliers, the interplay between the train service route design and passenger flow allocation subproblems is explored. To address the nonlinearities that arise in ADMM, we describe a new linearization technique for the quadratic penalty terms in the two subproblems by exploiting the rolling update mechanism of ADMM. The proposed approach is tested on synthetic and real-life instances from an urban rail company in China. The numerical results show that the proposed ADMM approach provides objective values that are on average 7.63% better than the conventional sequential approach. We also demonstrate that ADMM provides smaller optimality gaps in general when compared to a Lagrangian relaxation approach.

Regularization Strategy Aided Robust Unsupervised Learning for Wireless Resource Allocation

Unsupervised learning (UL) is widely used in the wireless resource allocation problems due to its lower computational complexity and better performance compared with traditional optimization algorithms. Since wireless resource allocation problems usually have several constraints, primal-dual learning based UL framework are widely adopted. However, the primal-dual learning approach has the problem of oscillation around the constraint threshold while training and there may be serious constraint violations when deployment. In addition, although the output of the neural network can also be restricted to the feasible region by the penalty function method, the optimality of such training methods cannot be guaranteed. In this paper, we combine the primal dual learning method with the penalty function method and propose a regularized unsupervised learning (RUL) framework to enhance the robustness of the primal-dual learning based UL framework. In the proposed RUL framework, we use regularization techniques to improve the robustness of primal-dual learning by reducing the risk of constraint violations while training. A quadratic penalty term is introduced into the Lagrangian function of the wireless optimization problem where the constraints can be equivalent to equality constraints to form its augmented Lagrangian function. In the simulation, we give a simple point to point power optimization problem as an example to show that the proposed RUL can improve the robustness of constraint convergence, and can also accelerate training speed.

Peer-to-peer energy trading among multiple microgrids considering risks over uncertainty and distribution network reconfiguration: A fully distributed optimization method

This paper proposes a two-layer optimization framework to co-optimize the P2P energy trading among multiple microgrids (MMGs) under uncertainty and optimal topology planning of the distribution networks (DNs). At the upper layer, the traditional verification optimal power flow model of DNs is transformed into a prosumer-focused and transaction-oriented dynamic network reconfiguration model. At the lower layer, uncertainty from wind power generations is integrated into the operating model of individual MGs and addressed by the stochastic programming (SP) method. Meanwhile, the conditional value at risk technique is introduced to find a trade-off between cost minimization and risk aversion flexibly. To establish the global negotiation mechanism among all participants (not only between distribution system operators and MGs, but also among MMGs), a fully distributed method is developed by combining an analytical target cascading algorithm and an alternating direction multiplier method. Furthermore, a diagonal quadratic approximation method is utilized to linearize the quadratic penalty term so that achieving parallel computing for all independent optimization subproblems. Simulations of different strategies, models, and distributed algorithms are implemented to verify the rationality and validity of the proposed method. The results of these case studies demonstrate that the proposed risk-averse SP approach can avoid over-optimistic solutions, the obtained P2P trading strategies are immune to uncertainty and P2P trading behaviors among MMGs can help reduce network losses of DNs. In addition, comparisons with other distributed algorithms verify the high performance of the proposed fully distributed method.

Single-Sensor Engine Multi-Type Fault Detection.

Engine fault detection is conducive to improving equipment reliability and reducing maintenance costs. In practical scenarios, high-quality data is difficult to obtain. Usually, only single-sensor data is available. This paper proposes a fault detection method combining Variational Mode Decomposition (VMD) and Random Forest (RF). At first, the spectral energy distribution is obtained by decomposing and statistic the engine data of multiple working conditions. Based on the spectral energy distribution, the overall optimal mode number was identified, and the quadratic penalty term was optimized using SNR. The improved VMD (IVMD) improves mode aliasing and iterative efficiency and unifies feature dimensions. Decomposition of real signals demonstrates the effectiveness. The paper designs a feature vector composed of seven types of attributes, including unit bandwidth energy, center frequency, maximum singular value and so on. The feature vector is then fed to RF for classification. Features are selected in order of importance to classification to improve the training efficiency. By comparing with various algorithms, the proposed method has higher accuracy and faster training efficiency in single-speed, multi-speed and cross-speed single-sensor data diagnosis. The results show that the method has application prospects with little training data and low hardware requirements.

Inverse stochastic optimal controls

We study an inverse problem of the stochastic optimal control of general diffusions with performance index having the quadratic penalty term of the control process. Under mild conditions on the system dynamics, the cost functions, and the optimal control process, we show that our inverse problem is well-posed using a stochastic maximum principle. Then, with the well-posedness, we reduce the inverse problem to some root finding problem of the expectation of a random variable involved with the value function, which has a unique solution. Based on this result, we propose a numerical method for our inverse problem by replacing the expectation above with arithmetic mean of observed optimal control processes and the corresponding state processes. The recent progress of numerical analysis of Hamilton–Jacobi–Bellman equations enables the proposed method to be implementable for multi-dimensional cases. In particular, with the help of the kernel-based collocation method for Hamilton–Jacobi–Bellman equations, our method for the inverse problems still works well even when an explicit form of the value function is unavailable. Several numerical experiments show that the numerical method recovers the unknown penalty parameter with high accuracy.

Lagrangian relaxation-based decomposition approaches for the capacitated arc routing problem in the state-space-time network

ABSTRACT We study the capacitated arc routing problem in the state-space-time network. A multi-commodity network flow model is formulated in the state-space-time network. Two Lagrangian relaxation-based decomposition methods are developed to tackle this problem. By dualizing the coupling constraints to the objective function, the Lagrangian relaxed problem (LR) can be decomposed into several routing subproblems in the state-space-time network. Then we extend quadratic penalty terms to the LR and obtain an augmented Lagrangian relaxed problem (ALR). By applying the alternating direction method of multipliers and linearization techniques, we decompose the ALR into a sequence of routing subproblems. We solve these subproblems of LR and ALR by a time-dependent dynamic programming algorithm. Feasible solutions are produced according to the result of the LR or ALR. The Lagrangian multipliers are updated by the subgradient optimization method. We implement the proposed methods in three networks, showing that the ALR-based method has better performance.

A Novel Denoising Method for Partial Discharge Signal Based on Improved Variational Mode Decomposition

Partial discharge (PD) online monitoring is a common technique for high-voltage equipment diagnosis. However, due to field interference, the monitored PD signal contains a lot of noise. Therefore, this paper proposes a novel method by integrating the flower pollination algorithm, variational mode decomposition, and Savitzky–Golay filter (FPA-VMD-SG) to effectively suppress white noise and narrowband noise in the PD signal. Firstly, based on the mean envelope entropy (MEE), the decomposition number and quadratic penalty term of the VMD were optimized by the FPA. The PD signal containing noise was broken down into intrinsic mode functions (IMFs) by optimized parameters. Secondly, the IMFs were classified as the signal component, the noise dominant component, and the noise component according to the kurtosis value. Thirdly, the noise dominant component was denoised using the SG filter, and the denoised signal was mixed with the signal component to reconstruct a new signal. Finally, threshold denoising was used to eliminate residual white noise. To verify the performance of the FPA-VMD-SG method, compared with empirical mode decomposition with wavelet transform (EMD-WT) and adaptive singular value decomposition (ASVD), the denoising results of simulated and real PD signals indicated that the FPA-VMD-SG method had excellent performance.

Onboard optimization of multi-arc trajectories with constraints on duration of arcs

Autonomous operation of rocket-powered vehicles is a promising technique that could significantly improve the adaptiveness of rocket-powered vehicles. However, to remedy in-flight engine failures and allow mission changes, the autonomous operation requires an onboard method to optimize the powered trajectories. While onboard methods to optimize the powered trajectories have been proposed before, none of them considered the constraints on duration of arcs in multi-arc trajectory optimization problems. The constraints on the duration of burn arcs and coast arcs often arise from the safe operating conditions of rocket engines, and violating these constraints will likely result in physically-infeasible trajectories. Determining the optimal duration of each arc while satisfying the constraints is a challenging problem for onboard trajectory optimization methods. In this paper, an onboard method to optimize multi-arc trajectories is proposed that is capable of enforcing the constraints on the duration of burn arcs and coast arcs. The proposed method employs a hybrid regularization approach to facilitate the convergence of the duration of arcs. Quadratic penalty terms and fixed-radius trust-region terms are simultaneously used to limit the change of duration of arcs between iterations. The two terms complement each other, contributing to the efficiency of the hybrid regularization approach. Moreover, we propose an optimality-based stopping criterion that certifies the optimality of optimized duration of arcs and helps reduce unnecessary iterations. Numerical results on an onboard computational platform are provided to validate the efficiency of the proposed method.

Adaptive Recursive Variational Mode Decomposition for Multiple Engine Faults Detection

Engine fault detection is critical to enhancing the reliability of modern equipment. However, it is challenging to obtain a large number of high-quality labeled data for engines, which is not conducive to improving the training accuracy of deep learning methods. Therefore, this article proposes a fault detection method combining adaptive recursive variational mode decomposition (ARVMD) and component energy distribution spectrum (CEDS). The paper first introduces recursive mode into VMD. Then, the mode number is dynamically selected according to the energy distribution of the power spectral density to extract the intrinsic mode functions (IMFs) continuously. The quadratic penalty term is optimized correspondingly using SNR. The decomposition results of artificial and real signals demonstrated that ARVMD has higher SNR and efficiency than VMD. Next, the center frequency and unit bandwidth energy of IMF are used to construct CEDS. Fault diagnosis is realized by the CEDS correlation ranking of various faults. Finally, two case studies are performed to illustrate the effectiveness of the proposed method. The results show that the proposed ARVMD-CEDS method provides an efficient and effective solution for single-channel engine fault diagnosis.

Output-only modal identification based on the variational mode decomposition (VMD) framework

This paper proposes a novel modal identification variational mode decomposition (MIVMD) for output-only modal identification (OMI). The proposed MIVMD is more direct and elegant than most existing VMD-related OMI works that repeatedly use the conventional VMD to each channel of multivariate vibration signals and need post-processing to extract mode shapes. Firstly, OMI is expressed as a constrained variational problem by using the modal superposition equation. Secondly, as in VMD, the squared L2-norm of the gradient of the modal response is employed to evaluate its bandwidth. Then, the quadratic penalty term and Lagrangian multipliers are used to render the problem unconstrained. Finally, the alternate direction method of multipliers (ADMM) is utilized to solve this unconstrained optimization problem. The results demonstrate that the proposed MIVMD can concurrently identify natural frequencies, mode shapes, and modal responses from multivariate signals without multiple decompositions and post-processing. Because VMD-based methods are restricted to decompose narrowband modes, a short-time counterpart of MIVMD (ST-MIVMD), which can identify time-varying systems involving wideband modes or closely-spaced modes, is also presented. In the end, a series of numerical and experimental examples are performed to verify the effectiveness and advantages of the proposed method in addressing the OMI problem.

Integrated line planning and train timetabling through price-based cross-resolution feedback mechanism

Railway line planning and train timetabling are two key planning steps that determine the operating cost and passenger service quality of a railway operator under the infrastructure capacity limitations. Traditionally, the line planning and train timetabling problems are solved sequentially at the strategic and tactical level, respectively. In this study, by introducing two types of binary decision variables, we first propose a unified integer linear programming (ILP) model for the integrated optimization of line planning and train timetabling. The line planning problem is modeled using ILP to satisfy passenger demand, whereas the cyclic train timetabling problem is formulated as a multi-commodity network flow model with a side track capacity constraint. The two types of binary decision variables are coupled by a cross-resolution consistency constraint, which ensures the conformity of the line planning and train timetabling decisions. Furthermore, a dual decomposition mechanism based on the Alternating Direction Method of Multipliers (ADMM) is developed to dualize the cross-resolution consistency and track capacity constraints, such that the original ILP model is decomposed into a line planning sub-problem and a set of train-specific sub-problems. After the linearization of the quadratic penalty terms in the ADMM, each sub-problem contains the Lagrangian relaxation price information based on the cross-resolution consistency constraint. Moreover, the primal and dual solutions are obtained by iteratively and efficiently solving the line planning sub-problem using a commercial solver, and each train-specific sub-problem through a tailored forward dynamic programming algorithm. Furthermore, a real-life case study is conducted based on the Beijing–Shanghai high-speed railway corridor to verify the efficiency and effectiveness of the proposed model and algorithm. The results of the numerical experiments demonstrate that the ADMM can achieve significantly smaller optimality gaps than Lagrangian relaxation, and the integrated optimization approach can improve the objective value by 5.78% on average compared with the sequential optimization approach.

An open-source FEniCS-based framework for hyperelastic parameter estimation from noisy full-field data: Application to heterogeneous soft tissues

We introduce a finite-element-model-updating-based open-source framework to identify mechanical parameters of heterogeneous hyperelastic materials from in silico generated full-field data which can be downloaded here https://github.com/aflahelouneg/inverse_identification_soft_tissue. The numerical process consists in simulating an extensometer performing in vivo uniaxial tensile experiment on a soft tissue. The reaction forces and displacement fields are respectively captured by force sensor and Digital Image Correlation techniques. By means of a forward nonlinear FEM model and an inverse solver, the model parameters are estimated through a constrained optimization function with no quadratic penalty term. As a case study, our Finite Element Model Updating (FEMU) tool has been applied on a model composed of a keloid scar surrounded by healthy skin. The results show that at least 4 parameters can be accurately identified from an uniaxial test only. The originality of this work lies in two major elements. Firstly, we develop a low-cost technique able to characterize the mechanical properties of heterogeneous nonlinear hyperelastic materials. Secondly, we explore the model accuracy via a detailed study of the interplay between discretization error and the error due to measurement uncertainty. Next steps consist in identifying the real parameters and so finding the matching preferential directions of keloid scars growth.

A Novel Parameter-Adaptive VMD Method Based on Grey Wolf Optimization with Minimum Average Mutual Information for Incipient Fault Detection

Recently, variational mode decomposition (VMD) has attracted wide attention on mechanical vibration signal analysis. However, there are still some dilemmas in the application of VMD, such as the determination of the number of mode decomposition K and quadratic penalty term α. In order to acquire appropriate parameters of VMD, an improved parameter-adaptive VMD method based on grey wolf optimizer (GWO) is developed by taking the minimum average mutual information into consideration (GWOMI). Firstly, the parameters (K, α) are adaptively determined through GWOMI. Then, the vibration signal is decomposed by the developed method and effective modes are extracted according to the maximum kurtosis. Finally, the extracted modes are processed by Hilbert envelope analysis to acquire the incipient fault features. With the simulation and experimental analysis, it is clearly found that the developed method is effective and performs better than some existing ones.

A Convex Variational Model for Restoring SAR Images Corrupted by Multiplicative Noise

This paper studies a new convex variational model for denoising and deblurring images with multiplicative noise. Considering the statistical property of the multiplicative noise following Nakagami distribution, the denoising model consists of a data fidelity term, a quadratic penalty term, and a total variation regularization term. Here, the quadratic penalty term is mainly designed to guarantee the model to be strictly convex under a mild condition. Furthermore, the model is extended for the simultaneous denoising and deblurring case by introducing a blurring operator. We also study some mathematical properties of the proposed model. In addition, the model is solved by applying the primal-dual algorithm. The experimental results show that the proposed method is promising in restoring (blurred) images with multiplicative noise.

Shearlet-TGV based model for restoring noisy images corrupted by Cauchy noise

By combining with the shearlet transform and the second-order total generalized variation (TGV) regularization, a strictly convex shearlet-TGV based model is proposed for restoring images corrupted by Cauchy noise. The shearlet-TGV based model can be taken as a minimization problem for which the objective function is composed of a second-order TGV regularization term, a $$l_{1}$$-norm to the shearlet transform, a data fidelity term to the Cauchy noise, and a quadratic penalty term to guarantee the uniqueness of the solution. Computationally, the shearlet-TGV based model is transformed into a minimax problem by using the dual technique of optimization. Then, a high efficient Chambolle–Pock’s first-order primal–dual algorithm is developed to solve the transformed minimax problem. At last, compared with several existing state-of-the-art methods, experimental results demonstrate the effectiveness of our proposed method, in terms of the signal to noise ratio, the peak signal to noise ratio, the mean square error and the structural similarity index.

Parameter-Adaptive VMD Method Based on BAS Optimization Algorithm for Incipient Bearing Fault Diagnosis

In view of the incipient fault characteristics are difficult to be extracted from the raw bearing fault signals, an incipient bearing fault diagnosis method based on parameter-adaptive variational mode decomposition (VMD) is proposed. The beetle antennae search (BAS) algorithm is adopted to seek for the optimal combination of the VMD parameters. The reciprocals of the calculated kurtosis values of intrinsic mode functions (IMFs) decomposed via VMD are employed as a fitness function in the searching process. The optimal mode number and the quadratic penalty term of VMD are adaptively set after the search. Afterwards, a vibration signal is decomposed into a set of IMFs using the parameter-adaptive VMD, and the IMF with the maximal kurtosis value is selected as the sensitive one. The selected IMF is further analyzed by Hilbert envelope demodulation. The resulting envelope spectrum can show the significant fault impulse characteristics which are highly helpful to diagnose incipient bearing faults. The kurtosis and the proportion of fault energy are introduced as the input vector of the extreme learning machine (ELM). Comparisons have been conducted via ELM to evaluate the performance by using EMD and the fixed-parameter VMD. The experimental results demonstrate that the proposed method is more effective in extracting the incipient bearing fault characteristics.