Regularization Penalty Term Research Articles

Improving Graph Convolutional Network with Learnable Edge Weights and Edge-Node Co-Embedding for Graph Anomaly Detection.

The era of Industry 4.0 is gradually transforming our society into a data-driven one, which can help us uncover valuable information from accumulated data, thereby improving the level of social governance. The detection of anomalies, is crucial for maintaining societal trust and fairness, yet it poses significant challenges due to the ubiquity of anomalies and the difficulty in identifying them accurately. This paper aims to enhance the performance of the current Graph Convolutional Network (GCN)-based Graph Anomaly Detection (GAD) algorithm on datasets with extremely low proportions of anomalous labels. This goal is achieved through modifying the GCN network structure and conducting feature extraction, thus fully utilizing three types of information in the graph: node label information, node feature information, and edge information. Firstly, we theoretically demonstrate the relationship between label propagation and feature convolution, indicating that the Label Propagation Algorithm (LPA) can serve as a regularization penalty term for GCN, aiding in training and enabling learnable edge weights, providing a basis for incorporating node label information into GCN networks. Secondly, we introduce a method to aggregate node and edge features, thereby incorporating edge information into GCN networks. Finally, we design different GCN trainable weights for node features and co-embedding features. This design allows different features to be projected into different spaces, greatly enhancing model expressiveness. Experimental results on the DGraph dataset demonstrate superior AUC performance compared to baseline models, highlighting the feasibility and efficacy of the proposed approach in addressing GAD tasks in the scene with extremely low proportions of anomalous data.

Research on the service condition monitoring method of rolling bearings based on adaptive deconvolution and transformer

Aiming at the complexity of the rolling bearing operation state in actual working conditions and the limitation of the feature extraction capability of single-scale model, this paper proposes a fault diagnosis method based on two-channel adaptive deconvolution and transformer module (TADAT). First, a frequency domain dataset is built and multiscale feature information is extracted using two-channel adaptive deconvolution at the beginning stage of the model; the global features of the signal are further extracted by combining them in series with the transformer (multi-head attention). Next, the loss function is optimized by introducing a regularized loss function with label smoothing and L2 regularization penalty term. Finally, the model is validated against an open-source dataset and operating condition data collected from an unbalanced bearing load test bed with an accuracy of over 98.5%. The model also maintains high accuracy under noisy samples. The test shows that the rolling bearing service condition monitoring method based on TADAT feature extraction model proposed by this method is characterized by simple model structure, high diagnostic accuracy and strong generalization performance, which provides a useful reference for the research of rolling bearing service condition monitoring.

Cattle face detection method based on channel pruning YOLOv5 network and mobile deployment

The detection and identification of individual cattle plays an integral role in precision feeding and insurance claims, among others. Most current research is based on high-performance computing devices, which limits the application of deep learning techniques to mobile terminals. To this end, in this paper, we propose a channel-pruned YOLOv5 network-based method for cattle face detection on mobile terminals, referred to as NS-YOLO. First, the original model is sparsely trained and a sparse regularization penalty term is applied to the BN layers, then the corresponding mask values are labeled according to different weight thresholds, and the channels are pruned with global thresholds. Second, the detection precision is recovered by fine-tuning the model. Finally, the NCNN forward inference framework is used to quantize the model and an Android-based cattle face detection application is developed. Experimental results show that the model size, number of parameters and FLOPs are reduced by 86.10%, 88.19% and 63.25%, respectively, and the inference time is reduced by 35.53% compared to the original model, while mAP0.5 is reduced by only 1.6%. In particular, the 16-bit quantized model reduces the model size by 93.97% and the inference time by 34.97% compared to the original model on the mobile side. The results show that the proposed method can be easily deployed in resource-constrained mobile devices and has great advantages in practical engineering applications.

A 2D-DOA Sparse Estimation Method with Total Variation Regularization for Spatially Extended Sources

In this paper, a novel two-dimensional direction of arrival (2D-DOA) estimation method with total variation regularization is proposed to deal with the problem of sparse DOA estimation for spatially extended sources. In a general sparse framework, the sparse 2D-DOA estimation problem is formulated with the regularization of extended source characteristics including spatial position grouping, acoustic signal block sparse, and correlation features. An extended sources acoustic model, two-dimensional array manifold and its complete representation, total variation regularization penalty term, and the regularization equation are built, and are utilized to seek the solutions where the non-zero coefficients are grouped together with optimum sparseness. A total variation sparse 2D-DOA estimation model is constructed by combining total variation regularization with LASSO. The model can be easily solved by the convex optimization algorithm, and the solving process can promote the sparsity of the solution on the spatial derivatives and the solution itself. The theoretical analysis results show that the steps of decorrelation processing and angle matching of traditional 2D-DOA estimation methods could be avoided when adopting the proposed method. The proposed method has better robustness to noise, better sparsity, and faster estimation speed with higher resolution than traditional methods. It is promising to provide a coherent sources sparse representation of a non-strictly sparse field.

A modified FGL sparse canonical correlation analysis for the identification of Alzheimer's disease biomarkers

<abstract> <p>Imaging genetics mainly finds the correlation between multiple datasets, such as imaging and genomics. Sparse canonical correlation analysis (SCCA) is regarded as a useful method that can find connections between specific genes, SNPs, and diseased brain regions. Fused pairwise group lasso-SCCA (FGL-SCCA) can discover the chain relationship of genetic variables within the same modality or the graphical relationship between images. However, it can only handle genetic and imaging data from a single modality. As Alzheimer's disease is a kind of complex and comprehensive disease, a single clinical indicator cannot accurately reflect the physiological process of the disease. It is urgent to find biomarkers that can reflect AD and more synthetically reflect the physiological function of disease development. In this study, we proposed a multimodal sparse canonical correlation analysis model FGL-JSCCAGNR combined FGL-SCCA and Joint SCCA (JSCCA) method which can process multimodal data. Based on the JSCCA algorithm, it imposes a GraphNet regularization penalty term and introduces a fusion pairwise group lasso (FGL), and a graph-guided pairwise group lasso (GGL) penalty term, the algorithm in this paper can combine data between different modalities, Finally, the Annual Depression Level Total Score (GDSCALE), Clinical Dementia Rating Scale (GLOBAL CDR), Functional Activity Questionnaire (FAQ) and Neuropsychiatric Symptom Questionnaire (NPI-Q), these four clinical data are embedded in the model by linear regression as compensation information. Both simulation data and real data analysis show that when FGI-JSCCAGNR is applied to the imaging genetics study of Alzheimer's patients, the model presented here can detect more significant genetic variants and diseased brain regions. It provides a more robust theoretical basis for clinical researchers.</p> </abstract>

Clothing image classification algorithm based on convolutional neural network and optimized regularized extreme learning machine

This paper proposes a new method that uses Alexnet with ImageNet transfer learning as the feature extractor and optimized and regularized extreme learning as the classifier. We keep the first five convolutional layers and the first two fully connected layers of Alexnet, and then train the network. Then, the mutual information between each dimension of the feature and its category is calculated and sorted, and the feature with the highest ranking is selected for feature dimensionality reduction. The regularization penalty term is introduced to the extreme learning machine to control its algorithm complexity and solve the problem of overfitting. Finally, the Runge Kutta optimization algorithm is employed to ameliorate the hidden layer bias and input weight of the regularized extreme learning machine, and the optimized regularized extreme learning machine is used to classify the dimensionality-reduced clothing image traits. The test outcome illustrates that on some apparel classification with style (ACWS) datasets, the precision, recall, F1-score, and accuracy of the proposed algorithm are 93.06%, 93.17%, 92.82%, and 93.14%, respectively, which are better than those of other clothing image classification algorithms. The results verify that the raised algorithm significantly ameliorates the classification property of clothing image algorithms.

A Barzilai and Borwein regularization feasible direction algorithm for convex nonlinear SOC programming with linear constraints

A Barzilai and Borwein regularization feasible direction algorithm is proposed for convex nonlinear second-order cone programming with linear constraints. In the algorithm, a regularization penalty term based on the Barzilai and Borwein parameters is added to the Frank–Wolfe linearization objective function of the direction generating subproblem. The Barzilai and Borwein subproblem is transformed into a multi-block separable convex quadratic second-order cone programming (CQSOCP) subproblem. A parallel inexact alternating direction method is applied to solve the multi-block separable CQSOCP subproblem. The global convergence is given. Numerical results demonstrate that our method is efficient for some random convex nonlinear second-order cone programming problems with low accuracy.

A New Approach for Estimating Dissolved Oxygen Based on a High-Accuracy Surface Modeling Method.

Dissolved oxygen (DO) is a direct indicator of water pollution and an important water quality parameter that affects aquatic life. Based on the fundamental theorem of surfaces in differential geometry, the present study proposes a new modeling approach to estimate DO concentrations with high accuracy by assessing the spatial correlation and heterogeneity of DO with respect to explanatory variables. Specifically, a regularization penalty term is integrated into the high-accuracy surface modeling (HASM) method by applying geographically weighted regression (GWR) with some covariates. A modified version of HASM, namely HASM_MOD, is illustrated through a case study of Poyang Lake, China, by comparing the results of HASM, a support vector machine (SVM), and cokriging. The results indicate that HASM_MOD yields the best performance, with a mean absolute error (MAE) that is 38%, 45%, and 42% lower than those of HASM, the SVM, and cokriging, respectively, by using the cross-validation method. The introduction of a regularization penalty term by using GWR with respect to covariates can effectively improve the quality of the DO estimates. The results also suggest that HASM_MOD is able to effectively estimate nonlinear and nonstationary time series and outperforms three other methods using cross-validation, with a root-mean-square error (RMSE) of 0.20 mg/L and R2 of 0.93 for the two study sites (Sanshan and Outlet_A stations). The proposed method, HASM_MOD, provides a new way to estimate the DO concentration with high accuracy.

Design of Extreme Learning Machine with Smoothed ℓ0 Regularization

In extreme learning machine (ELM), a large number of hidden nodes are required due to the randomly generated hidden layer. To improve network compactness, the ELM with smoothed l0 regularizer (ELM-SL0 for short) is studied in this paper. Firstly, the l0 regularization penalty term is introduced into the conventional error function, such that the unimportant output weights are gradually forced to zeros. Secondly, the batch gradient method and the smoothed l0 regularizer are combined for training and pruning ELM. Furthermore, both the weak convergence and strong convergence of ELM-SL0 are investigated. Compared with other existing ELMs, the proposed algorithm obtains better performance in terms of estimation accuracy and network sparsity.

Maximum Correntropy Criterion Based l1-Iterative Wiener Filter for Sparse Channel Estimation Robust to Impulsive Noise

In this paper, we propose a new sparse channel estimator robust to impulsive noise environments. For this kind of estimator, the convex regularized recursive maximum correntropy (CR-RMC) algorithm has been proposed. However, this method requires information about the true sparse channel to find the regularization coefficient for the convex regularization penalty term. In addition, the CR-RMC has a numerical instability in the finite-precision cases that is linked to the inversion of the auto-covariance matrix. We propose a new method for sparse channel estimation robust to impulsive noise environments using an iterative Wiener filter. The proposed algorithm does not need information about the true sparse channel to obtain the regularization coefficient for the convex regularization penalty term. It is also numerically more robust, because it does not require the inverse of the auto-covariance matrix.

A regularized root–quartic mixture of experts for complex classification problems

Mixture of experts is a neural network based ensemble learning approach consisting of several experts and a gating network. In this paper, we introduce regularized root–quartic mixture of experts (R-RTQRT-ME) by incorporating a regularization term into the error function to control the complexity of model and to increase robustness in confronting with over-fitting and noise. The average of the results of R-RTQRT-ME on 20 classification benchmark datasets, shows that this algorithm performs 1.75%, 2.50%, 2.29% better than multi objective regularized negative correlation learning, multi objective negative correlation learning and multi objective neural network, respectively. Also, the average of improvements of R-RTQRT-ME is 1.16%, 2.31%, 3.40%, 3.39% in comparison with root-quartic mixture of experts, mixture of negatively correlated experts, mixture of experts and negative correlation learning, respectively. Furthermore, the effect of the regularization penalty term in R-RTQRT-ME on noisy data is analyzed which shows the robustness of R-RTQRT-ME in these situations.

Convex regularized recursive maximum correntropy algorithm

In this brief, a robust and sparse recursive adaptive filtering algorithm, called convex regularized recursive maximum correntropy (CR-RMC), is derived by adding a general convex regularization penalty term to the maximum correntropy criterion (MCC). An approximate expression for automatically selecting the regularization parameter is also introduced. Simulation results show that the CR-RMC can significantly outperform the original recursive maximum correntropy (RMC) algorithm especially when the underlying system is very sparse. Compared with the convex regularized recursive least squares (CR-RLS) algorithm, the new algorithm also shows strong robustness against impulsive noise. The CR-RMC also performs much better than other LMS-type sparse adaptive filtering algorithms based on MCC.

A Preconditioned Inexact Newton Method for Nonlinear Sparse Electromagnetic Imaging

A nonlinear inversion scheme for the electromagnetic microwave imaging of domains with sparse content is proposed. Scattering equations are constructed using a contrast-source (CS) formulation. The proposed method uses an inexact Newton (IN) scheme to tackle the nonlinearity of these equations. At every IN iteration, a system of equations, which involves the Frechet derivative (FD) matrix of the CS operator, is solved for the IN step. A sparsity constraint is enforced on the solution via thresholded Landweber iterations, and the convergence is significantly increased using a preconditioner that levels the FD matrix's singular values associated with contrast and equivalent currents. To increase the accuracy, the weight of the regularization's penalty term is reduced during the IN iterations consistently with the scheme's quadratic convergence. At the end of each IN iteration, an additional thresholding, which removes small “ripples” that are produced by the IN step, is applied to maintain the solution's sparsity. Numerical results demonstrate the applicability of the proposed method in recovering sparse and discontinuous dielectric profiles with high contrast values.

Using Price-Based Signals to Control Plug-in Electric Vehicle Fleet Charging

We study decentralized plug-in electric vehicle (PEV) charging control, wherein the system operator (SO) sends price-based signals to a load aggregator (LA) that optimizes charging of a PEV fleet. We study a pricing scheme that conveys price and quantity information to the LA and compare it to a simpler price-only scheme. We prove that the price/quantity-based mechanism can yield a socially optimal solution. We also examine several numerical case studies to demonstrate the superior performance of the price/quantity-based scheme. The price/quantity scheme yields nearly identical PEV charging costs compared to the social optima, whereas the price-only scheme is highly sensitive to the choice of a regularization penalty term that is needed to ensure convergence. We also show that the time to compute an equilibrium with the price-only mechanism can be up to two orders of magnitude greater than with the price/quantity scheme and can involve 24 times more information exchange between the SO and LA.

Integrative Approaches in Computational Biomedical Imaging 2013

Increasingly wider availability of biomedical imaging modalities, such as X-ray computed tomography (CT), magnetic resonance imaging (MRI), ultrasonic imaging, positron emission tomography (PET), single-photon emission computed tomography (SPECT), and optical imaging, have led to significant progresses in both biomedical research and clinical practice. Advancements in imaging technology and image informatics have facilitated and enabled quantitative understanding of the biological structures, functional organisms, and pathological mechanisms. This special issue presents 14 latest research contributions in computational biomedical imaging. The focuses have been on the computational principles, strategies, methods, and algorithms that create, extract, and understand imaging data of biomedical significance. The first group of papers has showcased the development and applications of rigorous optimization techniques in biomedical imaging. The work based on dual unscented Kalman filtering provides a reliable and efficient platform to understand the BOLD signals from fMRI. In the other paper, the robust discrete particle swarm optimization algorithm is adopted to solve the branch-cut phase unwrapping problem of MRI data. Without the need for a regularization penalty term, a semigreed method for bioluminescence tomography reconstruction is fast and stable for small animal imaging. The second group of papers has addressed image segmentation and tissue characterization problems, ever popular among image analysis researchers. The work of L. Li et al. applies machine learning techniques to characterize lymph node metastasis in gastric cancer from gemstone spectral imaging with much improved outcomes, while G. V. Sanchez. Ferrero et al. make new advances in ultrasonic speckle characterization using generalized Gama distribution and generalized Gama mixture model. In recent years, physically meaningful modeling and simulation have become integral components for model-constrained imaging and model-based image understanding. Thus, the third group of papers has continued the stride in this direction. The propagation of myocardial electrical activation is simulated using a monodomain model, without the need of explicit mesh constraints. In similar spirit, one work performs forward modeling of 3D optical molecular imaging with simplified spherical harmonics approximation using extended finite element method. The final group of papers has highlighted some new imaging modalities and their potential use in clinical practice. The further development of ultrasound-based quantitative imaging of Young's modulus with high accuracy is presented in this issue. And the novel Diffusion Kurtosis Imaging shows that more detailed classification of brain tissues is clinical significance. We appreciate the many high quality submissions to this special issue from leading researchers in computational biomedical imaging. Because of space limitation, we can only accept a small number of most exciting works. We sincerely hope that the remaining interesting research outcomes will be soon published in other venues.

An iterative thresholding algorithm for the inverse problem of electrical resistance tomography

Image reconstruction in Electrical Resistance Tomography (ERT) is an ill-posed nonlinear inverse problem. Considering the sparsity property of ERT model, in this paper, we replace the conventional l2 regularization penalty term by weighted lp(1≤p<2) penalty term. To overcome the non-quadratic property, a surrogate term is added to the objective function. An interesting condition is that the classical methods (e.g. SVD, Landweber iteration) can be used to solve the lp(1≤p<2) least squares problems. Both typical and complicated distributions (e.g. annular and cross-shape) have been examined using a 16-electrode configuration based on the finite element method (FEM) software COMSOL. The simulated results demonstrate the feasibility of the proposed algorithm, and compared to the l2 regularization method, the proposed algorithms can produce images of higher quality, which are evaluated both qualitatively and quantitatively.

Sparsity and the Bayesian perspective

Sparsity has been recently introduced in cosmology for weak-lensing and CMB data analysis for different applications such as denoising, component separation or inpainting (i.e. filling the missing data or the mask). Although it gives very nice numerical results, CMB sparse inpainting has been severely criticized by top researchers in cosmology, based on arguments derived from a Bayesian perspective. Trying to understand their point of view, we realize that interpreting a regularization penalty term as a prior in a Bayesian framework can lead to erroneous conclusions. This paper is by no means against the Bayesian approach, which has proven to be very useful for many applications, but warns about a Bayesian-only interpretation in data analysis, which can be misleading in some cases.

Sparse Reconstruction for Bioluminescence Tomography Based on the Semigreedy Method

Bioluminescence tomography (BLT) is a molecular imaging modality which can three-dimensionally resolve the molecular processes in small animals in vivo. The ill-posedness nature of BLT problem makes its reconstruction bears nonunique solution and is sensitive to noise. In this paper, we proposed a sparse BLT reconstruction algorithm based on semigreedy method. To reduce the ill-posedness and computational cost, the optimal permissible source region was automatically chosen by using an iterative search tree. The proposed method obtained fast and stable source reconstruction from the whole body and imposed constraint without using a regularization penalty term. Numerical simulations on a mouse atlas, and in vivo mouse experiments were conducted to validate the effectiveness and potential of the method.

Integrative Approaches in Computational Biomedical Imaging

Increasingly wider availability of biomedical imaging modalities, such as X-ray computed tomography (CT), magnetic resonance imaging (MRI), ultrasonic imaging, positron emission tomography (PET), single-photon emission computed tomography (SPECT), and optical imaging, have led to significant progresses in both biomedical research and clinical practice. Advancements in imaging technology and image informatics have facilitated and enabled quantitative understanding of the biological structures, functional organisms, and pathological mechanisms. This special issue presents 14 latest research contributions in computational biomedical imaging. The focuses have been on the computational principles, strategies, methods, and algorithms that create, extract, and understand imaging data of biomedical significance. The first group of papers has showcased the development and applications of rigorous optimization techniques in biomedical imaging. The work based on dual unscented Kalman filtering provides a reliable and efficient platform to understand the BOLD signals from fMRI. In the other paper, the robust discrete particle swarm optimization algorithm is adopted to solve the branch-cut phase unwrapping problem of MRI data. Without the need for a regularization penalty term, a semigreed method for bioluminescence tomography reconstruction is fast and stable for small animal imaging. The second group of papers has addressed image segmentation and tissue characterization problems, ever popular among image analysis researchers. The work of L. Li et al. applies machine learning techniques to characterize lymph node metastasis in gastric cancer from gemstone spectral imaging with much improved outcomes, while G. V. Sanchez. Ferrero et al. make new advances in ultrasonic speckle characterization using generalized Gama distribution and generalized Gama mixture model. In recent years, physically meaningful modeling and simulation have become integral components for model-constrained imaging and model-based image understanding. Thus, the third group of papers has continued the stride in this direction. The propagation of myocardial electrical activation is simulated using a monodomain model, without the need of explicit mesh constraints. In similar spirit, one work performs forward modeling of 3D optical molecular imaging with simplified spherical harmonics approximation using extended finite element method. The final group of papers has highlighted some new imaging modalities and their potential use in clinical practice. The further development of ultrasound-based quantitative imaging of Young's modulus with high accuracy is presented in this issue. And the novel Diffusion Kurtosis Imaging shows that more detailed classification of brain tissues is clinical significance. We appreciate the many high quality submissions to this special issue from leading researchers in computational biomedical imaging. Because of space limitation, we can only accept a small number of most exciting works. We sincerely hope that the remaining interesting research outcomes will be soon published in other venues.

Ill-conditioned GPS compass attitude determination using regularization approach

A new robust motion-based GPS compass attitude determination method is proposed. For vehicles traveling on horizontal surfaces, ill-conditions in matrix inversion often appears in the attitude determination process when using the least square approach and the proposed method is particularly useful in these situations. As a result, noise in the measurement will be amplified leading to unacceptable results in the singular direction in the traditional least square solution. A regularized least square method based on Tikhonov regularization is proposed in this paper to address this problem. By adding a regularization (penalty) term proportional to the desired solution norm in the system minimization cost function, the original ill-conditioned matrix can be replaced by a better-conditioned one to alleviate the perturbations in the solution and produce a regularized solution to the original problem. An error analysis that compares the errors of least square solution and regularized least square solution based on mean square error criterion is presented. It is shown that with an appropriate choice of regularization parameter, regularized least square method effectively reduces the mean square error when compared with the least square method. An algorithm based on baseline length constraint is proposed to select an optimal regularization parameter for regularization. Simulations and experiments are conducted to verify the validity of the proposed method. The results revealed that the accuracy of the regularized least square solution is better than the traditional least square solution based on mean square error criteria for comparison.