Regularization Method Research Articles

Extracting building outlines based on convolutional neural networks using the property of linear connectivity

Abstract. For years, researchers have been developing an automated method that can replace humans by drawing the outlines of individual buildings in a vector format, which plays an important role in GIS creation, environmental monitoring, urban planning, population density estimation, and energy supply. There is no doubt that this is an extremely difficult task, not only because of the labor required to develop such a highly intelligent algorithm, but also because of the challenges posed by imperfect imaging conditions, different building structures, and the complexity of the background. One of the current challenges in extracting building outlines is to accurately recreate the polygonal boundaries of the building while extracting vectorized building masks as output for direct use in various applications. This work provides a comprehensive workflow for building extraction and improves the predicted area of buildings through boundary regularization. First, a convolutional neural network is used to train instance segmentation model, then regularization and vectorization processes are performed. The main difference from existing methods is a new regularization method based on the concepts of linear connectivity and convexity of a set of points. This approach can effectively identify and remove points that do not belong to the detected building but were incorrectly segmented by the algorithm. Based on the results of experiments, the algorithm showed a high level of efficiency, comparable to leading methods for extracting building boundaries as PolyWorld.

Generalized Shannon entropy sparse wavelet packet transform for fault detection of traction motor bearings in high-speed trains

An effective structural health monitoring method of traction motor bearings is a powerful guarantee for the safety operation of high-speed trains. However, it is exceptionally difficult to detect bearing fault characteristics from the vibration signals of traction motor bearings operating at high rotational speeds. In this scenario, a generalized Shannon entropy sparse wavelet packet transform (GSWPT) for fault detection of motor bearings is proposed in this paper. Firstly, a generalized Shannon entropy sparse regularization method is proposed to obtain sparse wavelet reconstruction coefficients by extending the definition of the Shannon information entropy, and the non-convex sparse regularization function is minimized by synergistic swarm optimization algorithm. Then, the wavelet node coefficients are weighted according to the second-order cyclostationarity index of the wavelet packet node to further enhance the sparsity of the reconstructed signal. Moreover, the optimal decomposition level of GSWPT is adaptively selected by the maximum sparsity and cyclostationarity criterion. Particularly, in order to verify the bearing fault detection performance of GSWPT in practical engineering, a bearing fault dynamic model of traction motor in high-speed train was established based on Hertz contact theory and the fourth-order Runge-Kutta method to obtain simulated data under strong Gaussian white noise, and a corresponding test platform was constructed to collect experimental data under different operating conditions. Finally, the applications on the simulated and experimental signals of traction motor bearings in high-speed trains demonstrate that GSWPT significantly outperforms the conventional wavelet packet transform, dual-tree complex wavelet packet transform, blind deconvolution, modal decomposition, and Infogram methods to some extent for fault detection.

Comprehensive Study of Equilibrium Structure of Trans-Azobenzene: Gas Electron Diffraction and Quantum Chemical Calculations

The geometrical re parameters of trans-azobenzene (E-AB) free molecule were refined by gas electron diffraction (GED) method using available experimental data obtained previously by S. Konaka and coworkers. Structural analysis was carried out by various techniques. First of all, these included the widely used molecular orbital constrained gas electron diffraction method and regularization method. The results of the refinements using different models were also compared—a semirigid model, three variants of one-dimensional dynamic models, and a two-dimensional pseudoconformer model. Several descriptions have been used due to the fact that E-AB has a shallow potential energy surface along the rotation coordinates of phenyl groups. Despite this, it turned out that the semirigid model is suitable for use for E-AB and allows good agreement with experimental data to be achieved. According to the results of GED structural analysis, coupled with the results of DLPNO-CCSD(T0) calculations, E-AB has a planar structure. Based only on GED data, it is impossible to unambiguously determine the rotational angle of the phenyl group due to the facts that (i) with rotation over a wide range of angles, the bonded distances in the molecule change insignificantly and (ii) potential function in a structural analysis within a dynamic model is not determined with the necessary accuracy. This work also examines the sensitivity of the GED method to structural changes caused by trans-cis isomerization. The paper also analyzes the applicability of different variants of density functional theory (DFT) calculations in GED structural analysis using E-AB as an example. There are not enough similar methodological works in the literature. This experimental and methodological information is especially important and relevant for planning and implementing GED experiments and corresponding processing of the results for azobenzene derivatives, in which the conformer and isomeric diversity are even more complicated due to the presence of different substituents.

Muscle fatigue detection based on sEMG signal using autocorrelation function and neural networks

Abstract Feature extraction from an recorded surface electromyography (sEMG) signal plays an important role in identifying and quantifying the characteristics of muscle activities. These features can be used for various applications like muscle function assessment, muscle fatigue detection, etc. Common features extracted from sEMG signal are time-domain or frequency-domain features. However, features which are sensitive to uncertainties in the signal like noise, movement artifacts, and outliers should be avoided. Autocorrelation function (ACF), which is a measure of similarity between a signal and its time delayed version, is considered in this work as a feature to overcome the impact of noise, artifacts, and outliers. An artificial neural network (ANN) is developed to differentiate between fatigue and non-fatigue conditions using the calculated ACF from sEMG segments. The performance of an ANN model that can be adapted by means of various regularization methods was investigated. The proposed ANN model achieved an accuracy of about 97.62 %, a precision of about 95.50 % and a sensitivity of about 100 % in the classification of fatigue and non-fatigue sEMG segments, outperforming k-means and linear support vector machine approaches that served as references.

Enhancing Open-World Bacterial Raman Spectra Identification by Feature Regularization for Improved Resilience against Unknown Classes.

The combination of deep learning techniques and Raman spectroscopy shows great potential offering precise and prompt identification of pathogenic bacteria in clinical settings. However, the traditional closed-set classification approaches assume that all test samples belong to one of the known pathogens, and their applicability is limited since the clinical environment is inherently unpredictable and dynamic, unknown, or emerging pathogens may not be included in the available catalogs. We demonstrate that the current state-of-the-art neural networks identifying pathogens through Raman spectra are vulnerable to unknown inputs, resulting in an uncontrollable false positive rate. To address this issue, first we developed an ensemble of ResNet architectures combined with the attention mechanism that achieves a 30-isolate accuracy of 87.8 ± 0.1%. Second, through the integration of feature regularization by the Objectosphere loss function, our model both achieves high accuracy in identifying known pathogens from the catalog and effectively separates unknown samples drastically reducing the false positive rate. Finally, the proposed feature regularization method during training significantly enhances the performance of out-of-distribution detectors during the inference phase improving the reliability of the detection of unknown classes. Our algorithm for Raman spectroscopy empowers the identification of previously unknown, uncataloged, and emerging pathogens ensuring adaptability to future pathogens that may surface. Moreover, it can be extended to enhance open-set medical image classification, bolstering its reliability in dynamic operational settings.

Improved quantification in CEST-MRI by joint spatial total generalized variation.

In this work, the use of joint Total Generalized Variation (TGV) regularization to improve Multipool-Lorentzian fitting of chemical exchange saturation transfer (CEST) Spectra in terms of stability and parameter signal-to-noise ratio (SNR) was investigated. The joint TGV term was integrated into the nonlinear parameter fitting problem. To increase convergence and weight the gradients, preconditioning using a voxel-wise singular value decomposition was applied to the problem, which was then solved using the iteratively regularized Gauss-Newton method combined with a Primal-Dual splitting algorithm. The TGV method was evaluated on simulated numerical phantoms, 3T phantom data and 7T in vivo data with respect to systematic errors and robustness. Three reference methods were also implemented: The standard nonlinear fitting, a method using a nonlocal-means filter for denoising and the pyramid scheme, which uses downsampled images to acquire accurate start values. The proposed regularized fitting method showed significantly improved robustness (compared to the reference methods). In testing, over a range of SNR values the TGV fit outperformed the other methods and showed accurate results even for large amounts of added noise. Parameter values found were closer or comparable to the ground truth. For in vivo datasets, the added regularization increased the parameter map SNR and prevented instabilities. The proposed fitting method using TGV regularization leads to improved results over a range of different data-sets and noise levels. Furthermore, it can be applied to all Z-spectrum data, with different amounts of pools, where the improved SNR and stability can increase diagnostic confidence.

On the numerical solution of weakly singular integral equations of the first kind using a regularized projection method

This study investigates a numerical method based on the Jacobi–Gauss quadrature for solving Fredholm integral equations of the first kind with a weakly singular kernel by combining the Tikhonov regularization and projection methods. This numerical method reduces the solution of the weakly singular integral equations of the first kind to the solution of a linear system of algebraic equations. The theoretical analysis of the proposed technique is provided. Finally, several tests are presented to show the validity and efficiency of this approach.

Investigation of convective peristaltic flow of non-Newtonian fluids through a non-uniform tapered porous conduit with Ohmic heating and viscous dissipation

ABSTRACT The study of convective magnetohydrodynamic (MHD) biofluid transport through a non-uniform duct is of interest due to its applications to biomedical devices and equipment. Because of this, a comparative study of free convective peristaltic flow of non-Newtonian (Casson and Jeffrey) fluids in a tapered porous symmetric duct with Hall current and viscous dissipation effects is carried out. The proposed dimensional governing equations are converted into a system of non-dimensional partial differential equations by invoking adequate non-dimensional parameters. Furthermore, a long-wavelength approximation is used to reduce the complexity of the system. Then the reduced system of partial differential equations is solved analytically using the regular perturbation method. The fascinating features of the influential parameters on the flow characteristics are demonstrated and analyzed in depth through graphs and tables. The results demonstrate that the velocity and temperature in the Jeffrey fluid flow are larger compared to those in the Casson fluid flow. The thermal Grashof number and the Hall current parameter enhanced the pressure rise profile. This research has significant implications in biomedical devices (MRI, vitamins A and D injections, heart-lung machines, etc.) and pharmaceutical fluid preparation in biomedical sciences.

Constancy of stabilized Randall-Sundrum model with respect to quantum corrections

In the Randall-Sundrum model stabilized with the help of the Goldberger-Wise scalar field, the lowest order vacuum quantum corrections to the energy-momentum tensor of the scalar and tensor fields are calculated. It is shown that the corresponding vacuum energy density of the fields in the bulk leads to the Casimir effect. Using the dimensional regularization method and the Abel-Plana formula, the divergences have been isolated and the vacuum energy density has been renormalized. An analytical expression for the Casimir force is obtained and an estimate of its influence on the model parameters is made. It was found that the quantum corrections lead only to a negligibly small decrease in the distance between the branes, i.e. the model possesses constancy with respect to quantum corrections.

Multi-sensor data fusion-enabled lightweight convolutional double regularization contrast transformer for aerospace bearing small samples fault diagnosis

Aiming at the problems of low information utilization and lack of feature mining capability in multi-sensor fusion networks, this study presents a multi-sensor data fusion-enabled lightweight convolutional double regularization contrast transformer for aerospace bearing small samples fault diagnosis. Firstly, a metric termed integrated cliff entropy is devised to assign weights to vibration signals from diverse sensor channels. It aims to enhance the cyclic impulse characteristics within the fused signals, thereby facilitating more precise fault identification. Secondly, a lightweight Diwaveformer architecture is constructed as the backbone of contrast learning. It enables the global and local features of faulty signals to be comprehensively extracted with less computational effort. Finally, a double contrast loss is constructed to optimize the distribution of intra-class and inter-class features to improve the fault identification ability of the network with small samples. Additionally, a discard regularization method is designed to remove the projection head during the contrast learning process, further advancing the model lightweight. Our method achieved accuracies of 95.54% and 92.56% on two aerospace bearing datasets with extremely sparse training samples, which proved its superior performance.

Structured Pruning for Deep Convolutional Neural Networks: A Survey.

The remarkable performance of deep Convolutional neural networks (CNNs) is generally attributed to their deeper and wider architectures, which can come with significant computational costs. Pruning neural networks has thus gained interest since it effectively lowers storage and computational costs. In contrast to weight pruning, which results in unstructured models, structured pruning provides the benefit of realistic acceleration by producing models that are friendly to hardware implementation. The special requirements of structured pruning have led to the discovery of numerous new challenges and the development of innovative solutions. This article surveys the recent progress towards structured pruning of deep CNNs. We summarize and compare the state-of-the-art structured pruning techniques with respect to filter ranking methods, regularization methods, dynamic execution, neural architecture search, the lottery ticket hypothesis, and the applications of pruning. While discussing structured pruning algorithms, we briefly introduce the unstructured pruning counterpart to emphasize their differences. Furthermore, we provide insights into potential research opportunities in the field of structured pruning. A curated list of neural network pruning papers can be found at: https://github.com/he-y/Awesome-Pruning. A dedicated website offering a more interactive comparison of structured pruning methods can be found at: https://huggingface.co/spaces/he-yang/Structured-Pruning-Survey.

Joint input–output identification of unstable systems with kernel regularization

This paper discusses closed-loop identification of unstable systems. In particular, we first apply the joint input–output identification method and then convert the identification problem of unstable systems into that of stable systems, which can be tackled by using kernel-based regularization methods. We propose to identify two transfer functions by kernel regularization, the one from the reference signal to the input, and the one from the reference signal to the output. Since these transfer functions are stable, kernel regularization methods can construct their accurate models. Then the model of unstable system is constructed by ratio of these functions. The effectiveness of the proposed method is demonstrated by a numerical example and a practical experiment with a DC motor.

Evaluation of the spatial structure of multiline emission in a capacitively coupled plasma using tomographic reconstruction

By imaging a capacitively coupled plasma from multiple directions using telecentric lens cameras and optical bandpass filters, the spatial structure of emission at specific wavelengths was reconstructed using the Tikhonov–Phillips regularization method. Camera parameters, crucial for relating three-dimensional structures to two-dimensional images, were evaluated experimentally to avoid a complex analytical approach. Assuming an axisymmetric emission profile, 750.4 nm Ar and 585.2 nm Ne emissions from Ar/Ne mixture plasma were reconstructed. The pressure dependence of the reconstructed Ar profile showed a similar trend to that of the two-dimensional emission images. The spatial structure of the emission intensity ratio of Ne to Ar from the reconstructed Ar and Ne profiles agreed well with a spatial distribution of electron temperatures measured with a Langmuir probe.

Regularizing stock return covariance matrices via multiple testing of correlations

This paper develops a large-scale inference approach for the regularization of stock return covariance matrices. The framework allows for the presence of heavy tails and multivariate GARCH-type effects of unknown form among the stock returns. The approach involves simultaneous testing of all pairwise correlations, followed by setting non-statistically significant elements to zero. This adaptive thresholding is achieved through sign-based Monte Carlo resampling within multiple testing procedures, controlling either the traditional familywise error rate, a generalized familywise error rate, or the false discovery proportion. Subsequent shrinkage ensures that the final covariance matrix estimate is positive definite and well-conditioned while preserving the achieved sparsity. Compared to alternative estimators, this new regularization method demonstrates strong performance in simulation experiments and real portfolio optimization.

Skin Penetration of Corn Silk (Zea mays L.) Transdermal Patch on Wistar Mice Skin Using Franz Diffusion Cell

Background: Corn silk (Zea mays L.) contains many active compounds, especially the flavonoid quercetin which has pharmacological activity as an antihyperlipidemic agent by reducing cholesterol and triglyceride levels in the body. Antihyperlipidemic treatment by oral route, such as statin drugs, has the disadvantage of experiencing a first-pass effect in the liver, which reduces the bioavailability of the drug. In addition to avoiding the first-pass effect, transdermal patches can improve patient compliance because they are easy to use. Objective: This study aims to optimize the transdermal patch formula of corn silk extract and test the penetration of the optimum formula by in vitro. Method: Optimization of the formula using the Regular Two-Level Factorial Design method on Design Expert®. This study used 2 factors, namely HPMC with a concentration of 3%-4% and PVP with a concentration of 1%-2%. The optimum formula obtained was subjected to in vitro penetration test using Franz diffusion cell. Results: Based on the results of factorial design analysis, the optimum formula of transdermal patches is at HPMC and PVP concentrations of 3.49% and 1% with moisture content, moisture uptake, percentage of elongation, and folding endurance respectively of 7.79%, 4.19%, 13.26% and 470.58 fold. The optimum formula of corn silk extract transdermal patch preparation also had an optimum percent cumulative amount of penetrated flavonoids of 96.06% and flux of 6.17 µg/cm2.hour at 3 hours. Conclusion: Transdermal patch dosage of corn silk extract with HPMC and PVP concentrations of 3.49% and 1% proved to have good characteristics and penetration rate.

Optimized tuberculosis classification system for chest X‐ray images: Fusing hyperparameter tuning with transfer learning approaches

AbstractAdvanced diagnostic methods are necessary for the prompt and reliable identification of tuberculosis (TB), which continues to be a worldwide health problem. Globally, there were projected to be 10 million new cases of tuberculosis in 2021, of which 9.8 million affected adults and 0.2 million children. About 15% of fatalities worldwide are attributable to tuberculosis (1.5 million deaths for every 10 million infections). To create a reliable model for tuberculosis (TB) identification using chest X‐ray pictures, we use deep learning approaches in this work, namely Convolutional Neural Networks (CNNs) and a combination of transfer learning and hyperparameter tuning. The dataset provides a varied selection of 3500 normal and 700 TB‐infected patients. It consists of 4200 photos that were obtained from the “Tuberculosis (TB) Chest X‐ray Database” on Kaggle. By utilizing the benefits of a trained model, the suggested methodological approach incorporates transfer learning. To maximize the performance of the suggested model, hyperparameter adjustment is also used. Using the VGG19 pre‐trained neural network, the model design is based on the concepts of transfer learning. The architecture makes use of task‐specific layers, regularization methods, and deliberate layer freezing to enable sophisticated categorization. Training and assessment stages demonstrate encouraging outcomes, with an accuracy of almost 98% attained on a different test dataset. A more thorough examination highlights the need for caution when interpreting high accuracy, nevertheless, by highlighting possible difficulties.

Evaluating the Regular Sharaf Learning Program at the Foundation of Islamic and Arabic Language Learning In Indonesia

This research aims to scrutinize and assess the learning dynamics within the regular Sharaf learning program conducted by the Islamic and Arabic Learning Foundation. Employing a descriptive qualitative approach, data for this study were obtained through interviews with the program director, supervisors, and administrators. Data collection involved observation, interviews, and document analysis. Data analysis followed Miles and Huberman's framework, comprising data collection, reduction, presentation, and drawing conclusions. Findings indicate that the regular Sharaf learning program features effective teaching methods, sequential curriculum delivery, adequate instructional materials, and proficient instructors. The researchers aspire that this study contributes to advancing the management of Arabic language programs and facilitates the development of comprehensive Arabic language activities for enhanced outcomes.

Diversity supporting robustness: Enhancing adversarial robustness via differentiated ensemble predictions

Deep learning models remain susceptible to adversarial attacks, prompting a primary focus on individual defense strategies. Previous studies have shown that increasing diversity among ensemble models can enhance their adversarial robustness. However, existing diversity metrics often lack a direct association with robustness, potentially limiting ensemble methods’ effectiveness in countering adversarial attacks. This paper presents a new diversity training method, termed Enhancing Adversarial Robustness through Diversity that Supports Robustness (EADSR), which is strongly linked to increased adversarial robustness. We developed a safety space hypothesis and a simultaneous training method based on the models’ responsiveness to non-natural perturbations which amplifies ensemble model diversity. By categorizing the ensemble models’ prediction behaviors into four classes, four regularization methods are applied for diversity training throughout the training process. We conducted comparative analysis with existing diversity methods to find a remarkable 30%–100% improvement on multiple benchmark datasets. The EADSR method incurs manageable training costs and is applicable to complex datasets such as ImageNet, demonstrating its effectiveness and practicality as a defense approach.

Multiparametric MRI-based radiomic nomogram for predicting HER-2 2+ status of breast cancer

ObjectiveTo explore the application of multiparametric MRI-based radiomic nomogram for assessing HER-2 2+ status of breast cancer (BC) MethodsPatients with pathology-proven HER-2 2+ invasive BC, who underwent preoperative MRI were divided into training (72 patients, 21 HER-2-positive and 51 HER-2-negative) and validation (32 patients, 9 HER-2-positive and 23 HER-2-negative) sets by randomization. All were classified as HER-2 2+ FISH-positive (HER-2-positive) or -negative (HER-2-negative) according to IHC and FISH. The 3D VOI was drawn on MR images by two radiologists. ADC, T2WI, and DCE images were analyzed separately to extract features (n = 1906). L1 regularization, F-test, and other methods were used to reduce dimensionality. Binary radiomics prediction models using features from single or combined imaging sequences were constructed using logistic regression (LR) classifier then and validated on a validation dataset. To build a radiomics nomogram, multivariate LR analysis was conducted to identify independent indicators. An evaluation of the model's predictive efficacy was made using AUC. ResultsOn the basis of combined ADC, T2WI, and DCE images, ten radiomic features were extracted following feature dimensionality reduction. There was superior diagnostic efficiency of radiomic signature using all three sequences compared to either one or two sequences (AUC for training group: 0.883; AUC for validation group: 0.816). Based on multivariate LR analysis, radiomic signature and peritumoral edema were independent predictors for identifying HER-2 2 +. In both training and validation datasets, nomograms combining peritumoral edema and radiomics signature demonstrated an effective discrimination (AUCs were respectively 0.966 and 0. 884). ConclusionThe nomogram that incorporated peritumoral edema and multiparametric MRI-based radiomic signature can be used to effectively predict the HER-2 2+ status of BC.

Rethinking Regularization with Random Label Smoothing

Regularization helps to improve machine learning techniques by penalizing the models during training. Such approaches act in either the input, internal, or output layers. Regarding the latter, label smoothing is widely used to introduce noise in the label vector, making learning more challenging. This work proposes a new label regularization method, Random Label Smoothing, that attributes random values to the labels while preserving their semantics during training. The idea is to change the entire label into fixed arbitrary values. Results show improvements in image classification and super-resolution tasks, outperforming state-of-the-art techniques for such purposes.