Regularization Method Research Articles

Improving Fast Adversarial Training With Prior-Guided Knowledge.

Fast adversarial training (FAT) is an efficient method to improve robustness in white-box attack scenarios. However, the original FAT suffers from catastrophic overfitting, which dramatically and suddenly reduces robustness after a few training epochs. Although various FAT variants have been proposed to prevent overfitting, they require high training time. In this paper, we investigate the relationship between adversarial example quality and catastrophic overfitting by comparing the training processes of standard adversarial training and FAT. We find that catastrophic overfitting occurs when the attack success rate of adversarial examples becomes worse. Based on this observation, we propose a positive prior-guided adversarial initialization to prevent overfitting by improving adversarial example quality without extra training time. This initialization is generated by using high-quality adversarial perturbations from the historical training process. We provide theoretical analysis for the proposed initialization and propose a prior-guided regularization method that boosts the smoothness of the loss function. Additionally, we design a prior-guided ensemble FAT method that averages the different model weights of historical models using different decay rates. Our proposed method, called FGSM-PGK, assembles the prior-guided knowledge, i.e., the prior-guided initialization and model weights, acquired during the historical training process. The proposed method can effectively improve the model's adversarial robustness in white-box attack scenarios. Evaluations of four datasets demonstrate the superiority of the proposed method.

NEURAL NETWORKS AND DEEP LEARNING: ENHANCING AI THROUGH NEURAL NETWORK OPTIMIZATION

Neural networks and deep learning have profoundly impacted artificial intelligence (AI), driving advancements across numerous applications. However, optimizing these networks remains a critical challenge, necessitating sophisticated techniques and methodologies. This article explores the state-of-the-art in neural network optimization, delving into advanced gradient descent variants, regularization methods, learning rate schedulers, batch normalization, and cutting-edge architectures. We discuss their theoretical underpinnings, implementation complexities, and empirical results, providing insights into how these optimization strategies contribute to the development of high-performance AI systems. Case studies in image classification and natural language processing illustrate practical applications and outcomes. The article concludes with an examination of current challenges and future directions in neural network optimization, emphasizing the need for scalable, interpretable, and robust solutions.

A kind regularization method for solving Cauchy problem of the Schrödinger equation

The potential-free field Schrödinger Cauchy problem is the major topic of this research. Because it is gravely ill-posed in the Hadamard sense. The Cauchy problem is modified through an improved boundary method. The regular approximate solution is created based on the priori and posteriori regularization parameter selection rules, and the convergence evidence is provided. The proposed method is practical under the priori and the posteriori regularization methods, according to numerical experiments. It works well and is resistant to data disruption noise.

Monomial-agnostic computation of vanishing ideals

Approximate basis computation of vanishing ideals has recently been studied extensively in computational algebra and data-driven applications such as machine learning. However, symbolic computation and the dependency on term order remain essential gaps between the two fields. In this study, we present the first monomial-agnostic basis computation, which works fully numerically with proper normalization and without term order. This is realized by gradient normalization, a newly proposed data-dependent normalization that normalizes a polynomial with the magnitude of gradients at given points. Its data-dependent nature brings various advantages: i) efficient resolution of the spurious vanishing problem, the scale-variance issue of approximately vanishing polynomials, without accessing coefficients of terms, ii) scaling-consistent basis computation, ensuring that input scaling does not lead to an essential change in the output, and iii) robustness against input perturbations, where the upper bound of error is determined only by the magnitude of the perturbations. Existing studies did not achieve any of these. As further applications of gradient information, we propose a monomial-agnostic basis reduction method and a regularization method to manage positive-dimensional ideals.

Decisions as ill-posed problems: A scoping review of regularization methods in decision science.

Decisions as ill-posed problems: A scoping review of regularization methods in decision science.

Damage identification method based on interval regularization theory

In the field of damage identification, traditional regularization methods neglect the impact of uncertainty factors on the selection of regularization parameters, leading to a decrease in the accuracy of damage identification. Therefore, this study proposes a damage identification based on interval truncated singular value decomposition (DI-ITSVD) method that considers the uncertainty in the selection of regularization parameter. This method treats model errors and measurement noise as interval uncertainties, and introduces the quantified uncertainties into the damage identification solutions through uncertainty propagation methods to determine the interval boundary. Uncertainty regularization parameters are selected to balance residuals and solutions using interval and generalized cross-validation methods. The key aspect of the proposed method in this paper is the integration of interval uncertainty propagation with the truncated singular value decomposition method to ensure the accuracy and stability of the damage identification equation solution. A numerical example of a 29-bar planar truss has been performed to test the effectiveness of the proposed method. The superiority of this method is verified by comparing the identification results with other improved truncated singular value decomposition methods. Finally, the practical application effect of the proposed method was also verified through an experimental work.

Sobolev‐type regularization method for the backward diffusion equation with fractional Laplacian and time‐dependent coefficient

This work is concerned with an ill‐posed problem of reconstructing the historical distribution of a backward diffusion equation with fractional Laplacian and time‐dependent coefficient in multidimensional space. The investigated problem is regularized by a Sobolev‐type equation method. Unlike previous works, to prove the convergence of the regularized solution to the exact one, we only require a very weak and natural a priori condition that the solution belongs to the standard Lebesgue space . This is done by suitably employing the Lebesgue‐dominated convergence theorem. If we go further to impose a stronger a priori condition, one may know how fast the convergence is. Finally, some MATLAB‐based numerical examples are provided to confirm the efficiency of the proposed method.

Z boson emission by an electron and the decays of Z boson into fermions in a de Sitter universe

In this paper we develop a method to obtain the rates for the decay of a Z boson into fermions in de Sitter geometry. Our general results obtained in de Sitter space-time allow us to obtain the Minkowski limits for the transition rates when the expansion parameter vanishes. Another important result reported in the present study is related to the emission of Z bosons by electrons and positrons. All the processes were studied by implementing perturbative methods that allow us to define the transition amplitudes in the first order of perturbation theory. The variation of probabilities and rates in terms of expansion parameter and particles masses is also given, pointing out that the processes that generate particle production are possible only in the early universe. For computing the transition rates and probabilities we use the dimensional regularization method and the minimal substraction method.

Nesterov acceleration‐based iterative method for backward problem of distributed‐order time‐fractional diffusion equation

This paper is concerned with the inverse problem of determining the initial value of the distributed‐order time‐fractional diffusion equation from the final time observation data, which arises in some ultra‐slow diffusion phenomena in applied areas. Since the problem is ill‐posed, we propose an iterated regularization method based on the Nesterov acceleration strategy to deal with it. Convergence rates for the regularized approximation solution are given under both the a priori and a posteriori regularization parameter choice rules. It is shown that the proposed method can always yield the order optimal convergence rates as long as the iteration parameter which appears in the Nesterov acceleration strategy is chosen large enough. In numerical aspect, the main advantage of the proposed method lies in its simplicity. Specifically, due to the Nesterov acceleration strategy, only a few number of iteration steps are required to obtain the approximation solution, and at each iteration step, we only need to numerically solve the standard initial‐boundary value problem for the distributed‐order time‐fractional diffusion equation. Some numerical examples including one‐dimensional and two‐dimensional cases are presented to illustrate the validity and effectiveness of the proposed method.

Super-Oscillating Diffractive Optical Spot Generators

The prior discrete Fourier transform (PDFT) is applied to the design of super-oscillating diffractive optical elements with rotational symmetry. Numerical simulations of the filter response are used to demonstrate the potential of the PDFT-based approach, which includes a regularization method for improved numerical and functional stability of the filter design. For coherent monochromatic illumination, the Strehl ratio of spot generators as a function of the spot radius is compared to the theoretical upper bound. It is shown that the performance of the PDFT design varies significantly depending on the aperture function and the encoding as a phase-only diffractive element. Experimental results are in good agreement with simulations and demonstrate the moderate demands to implement super-oscillating diffractive optical elements.

Elastic reweighted sparsity regularized sparse unmixing for hyperspectral image analysis

Sparse unmixing is a crucial component in the analysis of hyperspectral images because of its ability to sparsely estimate abundances with a spectral library of considerable scale. The existing sparse unmixing techniques tend to employ the sparsity from a single perspective, i.e., ℓ1 or ℓ2,1. In this paper, we accurately estimate abundances from element-wise and channel-wise sparsity perspectives, which forms a new elastic reweighted sparse regularized sparse unmixing method, termed ElSpaSU. The proposed ElSpaSU method leverages the abundance matrix simultaneously using ℓ1 and ℓ2,1 sparsity regularizations. To further promote sparsity in the abundances, the proposed ElSpaSU incorporates two types of iterative reweighting weights. We utilize the alternating direction method of multipliers (ADMM) framework to ensure guaranteed convergence in solving the optimization problem. Experimental outcomes from simulated and real hyperspectral images highlight the remarkable efficacy of the suggested ElSpaSU in contrast to competing approaches.

A combined denoising method for Q-factor compensation of poststack seismic data

Attenuation is a main factor limiting the resolution of seismic data. Earth works as a low-pass filter, which has strong attenuation of the high-frequency data. The loss of high-frequency energy can be compensated by the inverse Q filtering strategy. However, this method will also increase the energy of random noise which limits its application. The inverse Q filtering algorithm also needs the Q-factor as the input parameter, which is not easy to obtain. In this paper, we proposed a three-stage process to correct the attenuation of poststack data. In the first stage, a robust structure-oriented filtering is applied to remove random noise while protecting the structure information to avoid high-frequency noise burst. In the second stage, the local centroid frequency shift (LCFS) method is used to estimate the Q factor along the seismic trace. This method combined shaping regularization and centroid frequency shift (CFS) method to improve the robustness and accuracy of Q estimation to some extent. The final stage is to apply a stable inverse Q-filtering. Synthetic and field data examples demonstrate that time-varying Q-value can be accurately estimated by using the local centroid frequency shift (LCFS) method, and the proposed workflow can compensate the attenuation without bursting of high-frequency random noise.

Nonconvex low-rank regularization method for video snapshot compressive imaging

The reconstruction of snapshot compressive imaging (SCI) presents a significant challenge in signal processing. The primary goal of SCI is to employ a low-dimensional sensor to capture high-dimensional data in a compressed form. As a result, compared to traditional compressive sensing, SCI emphasizes capturing structural information and enhancing the reconstruction quality of high-dimensional videos and hyperspectral images. This paper proposes a novel SCI reconstruction method by integrating non-convex regularization approximation in conjunction with rank minimization. Furthermore, we address the characterization of structural information by leveraging nonlocal self-similarity across video frames to improve the reconstruction quality. We also develop an optimization algorithm based on the alternating direction method of multipliers (ADMM) to solve the model and provide a convergence algorithm analysis. Extensive experiments demonstrate that the proposed approach can potentially reconstruct SCI effectively.

Use of electrical impedance tomography for lung volume reconstruction

The article presents a study of the application of electro-impedance tomography (EIT) in diagnosing lung capacity using the Tikhonov regularization method. The possibility of reconstructing the lungs to monitor the degree of air filling was investigated. The experiment included a series of tests using a torso phantom designed to simulate different states of the lungs - from fully inflated to fully deflated. Lung-filling states were manipulated in controlled scenarios to test nine main experimental conditions reflecting different lung-filling states. In addition, the quality of reconstruction was checked using various types of reference backgrounds. The results show significant differences in lung volume reconstructions depending on the lung filling state. The most successful reconstructions, which were obtained using the 'No phantom' background, provided the most explicit visualization of the lungs, reassuring the method's reliability. The experiments confirm the potential of EIT to distinguish between different lung states and reconstruct the degree of lung filling. The study also underscores the need to optimize the reference background to increase the precision of the images, especially for the left lung.

Active contour model with improved second-order differential driven term

Driven terms in active contour models (ACMs) play a significant role in edge identification and image segmentation. However, in many existing ACMs, the driven terms are iteratively updated, resulting in slower segmentation speed because the image segmentation time increases with the iteration number. To address this problem, an ACM based on an improved second-order differential driven term (ISDDT) is presented, which can extract the edge information of images. The improved second-order differential driven term is computed only once before the iterations. Therefore, the computational complexity of our presented ACM is reduced, leading to a faster image segmentation speed. In addition, an improved regularization method with mean filtering is presented to improve the robustness of our ISDDT model. As an application, a target contour tracking method is developed based on our ISDDT model. Experimental results show that our ISDDT model segments images with inhomogeneous intensities well. The image segmentation speed of our proposed model has obvious advantages.

Radiation absorption, Hall and ion-slip current – driven MHD convection with spanwise cosinusoidally fluctuating temperature: a perturbative study

This study investigates the impact of radiation absorption, Hall and ion-slip current on magnetohydrodynamic free convective fluid flow past an infinite vertical porous plate with viscous dissipation and chemical reaction in the presence of Spanwise Cosinusoidally fluctuating temperature by time. Its significance extends across a wide array of applications, including advanced heat transfer systems for electronics, optimization of material processing kinetics, and fostering innovation for tailored material properties, with far-reaching implications spanning aerospace and renewable energy sectors. The governing equations, subject to stipulated boundary conditions are analyzed using the multiple regular perturbation method. The results are visually presented to evaluate the influence of various parameters on system dynamics. Graphical representations illustrate the physical significance of parameters including velocity, temperature, concentration, skin friction, mass transfer rate and heat transfer coefficients. The findings reveal that increased values of Hall and ion-slip current effects correlate with acceleration of velocity and skin friction. Moreover, higher radiation absorption parameter values lead to enhancements in velocity, temperature and Nusselt number. Conversely, different chemical reaction rates and Schmidt number values result in declines in velocity, concentration and Sherwood number. Additionally, incremental values of the Eckert number improve velocity and Nusselt number but have a reverse impact on temperature. The findings of this research align with those from prior examinations.

Advancing drinking water safety: Facile fabrication of nanofiltration membrane for enhanced antibiotics removal and efficient water softening

Long-term water consumption with contaminated antibiotics and high levels of hardness threatens the health of humans and animals. In this study, a novel positively charged nanofiltration membrane was designed for the dual purpose of efficiently removing antibiotics and softening hard water. The membrane was fabricated by casting P84 polymer solution onto non-woven fabrics, followed by immersion in the polyethyleneimine (PEI) solution for phase inversion and cross-linking, and finalized interfacial polymerization with trimesoyl chloride. Notably, the rapid immersion of the P84 substrate into PEI drastically reduces preparation time by 95 % compared to the regular method while maximizing PEI storage, resulting in uniform pore sizes and a positively charged surface. The optimal membrane showed a high water permeance of 11.3 L m−2 h−1 bar−1 and exceptional antibiotic rejection, i.e., 99.6 % for enrofloxacin, 99.2 % for ciprofloxacin, and 96.3 % for sulfamethoxazole, as well as a highly effective removal in water hardness (over 96 % of Ca2+ and Mg2+) at 4 bar. Simulated water separation tests showed that the optimal membrane effectively controls hard water salinity within optimum drinking water ranges (Ca2+ ≤ 80 mg/L; Mg2+ ≤ 30 mg/L), providing efficient water softening and antibiotic capture ability. Additionally, the optimal membrane demonstrated a superior anti-bacterial and long-term stability performance. This study has significant potential for improving drinking water supply safety and addressing pressing public health and environmental concerns.

Ward identities in a two-dimensional gravitational model: anomalous amplitude revisited using a completely regularization-independent mathematical strategy

We present a detailed investigation of the anomalous gravitational amplitude in a simple two-dimensional model with Weyl fermions. We employ a mathematical strategy that completely avoids any regularization prescription for handling divergent perturbative amplitudes. This strategy relies solely on the validity of the linearity of the integration operation and avoids modifying the amplitudes during intermediate calculations, unlike studies using regularization methods. Additionally, we adopt arbitrary routings for internal loop momenta, representing the most general analysis scenario. As expected, we show that surface terms play a crucial role in both preserving the symmetry properties of the amplitude and ensuring the mathematical consistency of the results. Notably, our final perturbative amplitude can be converted into the form obtained using any specific regularization prescription. We consider three common scenarios, one of which recovers the traditional results for gravitational anomalies. However, we demonstrate that this scenario inevitably breaks the linearity of integration, leading to an undesirable mathematical situation. This clean and transparent conclusion, enabled by the general nature of our strategy, would not be apparent in similar studies using regularization techniques.

Exploring the Feasibility and Educational Suggestions for Integrating OBE Educational Concepts into Higher Vocational English Courses

In response to the English curriculum standards of higher vocational colleges in both society and the nation, this study explores the feasibility of applying Outcome-Based Education (OBE) concepts into higher vocational English courses. A teaching experiment was conducted with 161 students selected from 4 parallel classes at a public higher vocational college in Guangdong Province. The intervention involved teachers in the experimental class utilizing the "Design Plan for the Application of OBE Educational Concepts in Hotel English Courses" provided by the researchers, while the control class proceeded with regular teaching methods. Following an 8-week intervention period, it was found that students in the experimental class demonstrated significantly higher academic performance compared to those in the control class, indicating the positive impact of integrating OBE educational concepts into higher vocational English courses. At the same time, researchers randomly selected 8 students for semi-structured interviews. The results of the interviews revealed that students in the experimental class responded positively and demonstrated recognition and acceptance of the OBE teaching method. Conclusion: The application of the OBE educational concept in higher vocational English courses is deemed feasible. As a result, the researchers have provided educational recommendations for English teachers in higher vocational schools, aiming to foster innovation and enhance educational quality in the field.

A modified iteratively regularized Landweber iteration method: Hölder stability and convergence rates

Abstract In this paper, we formulate the modified iteratively regularized Landweber iteration method in Banach spaces to solve the inverse problems for which the forward operator may be smooth or non-smooth. We study the convergence analysis of the modified method for both the perturbed as well as unperturbed data by utilizing the Hölder stability estimates. In the presence of perturbed data, we terminate the method via a discrepancy principle and show that it is in fact a convergence regularization method that terminates within a few iterations. In the presence of unperturbed data, we show that the iterates converge to the exact solution. Additionally, we deduce the convergence rates in the presence of perturbed as well as unperturbed data. Finally, we discuss two inverse problems on which the method is applicable.