Regularization Technique Research Articles

Pediatric physiotherapy management of airway clearance therapy and exercise: Data from the Australian Bronchiectasis Registry.

Regular airway clearance techniques (ACTs) and exercise are recommended for children with bronchiectasis, but current clinical practice and their predictors are unknown. We aimed to describe current use of ACTs and exercise among Australian children with bronchiectasis and identify associated predictors. Physiotherapy-specific data of 397 children (median age = 8 were extracted from the Australian Bronchiectasis Registry. A multivariate analysis was undertaken to identify predictors associated with the use of regular ACTs and physical exercise. Regular ACTs were undertaken by 118 (30%) children while 192 (48%) engaged in regular exercise. Physical exercise was the most common ACT modality (n = 83, 20%). The likelihood of regular ACT increased in children whose sputum isolated Pseudomonas aeruginosa (OR = 3.88, 95% CI 1.69-8.89) and was 50% higher for every respiratory exacerbation in the previous 12-months that required hospitalization (OR = 1.50, 95% CI 1.15-1.95). For every year older in age, children had increased odds of engaging in physical exercise (OR = 1.21, 95% CI 1.08-1.34) or using an ACT device (OR = 1.21, 95% CI 1.05-1.34). Regular exercise was twice as likely in the presence of bibasal bronchiectasis (OR = 2.43, 95% CI 1.14-5.16), yet less likely in those with ≥1 hospitalizations in the previous 12-months (OR = 0.76, 0.95% CI 0.57-1.03). Approximately one-third of children with bronchiectasis undertake regular ACTs while physical exercise was undertaken in approximately one in two children. Age, frequent respiratory exacerbations requiring hospitalization and the extent of disease are predictors of undertaking regular ACTs and exercise. Identification of these factors may assist in tailoring ACT, exercise and ACT modality prescription in clinical practice.

Impacts of Buoyancy and Joule Heating on Unsteady MHD Fluid Flow Along a Semi‐Infinite Vertical Porous Plate With Dufour, Chemical Reaction, and Radiation Effect

ABSTRACTAn analytical solution for unsteady, incompressible, laminar MHD fluid flow involving mass and heat transfer along a semi‐infinite vertical moving flat plate in a porous medium have been studied in this paper. Effects of heat radiation and absorption, Joule heating, Dufour, thermal and solutal buoyancy, and first order chemical reaction of are discussed under suitable physical conditions. It is postulated that the plate will migrate in the fluid's motion's direction due to the presence of a uniform magnetic field normal to the porous surface. The regular perturbation technique is used to solve the dimensionless governing equations. The mathematical derivations for fluid velocity, temperature, and concentration are evaluated; apart from that, skin friction, rate of mass and heat transfer at the plate are also expressed. The present outcomes are compared with previously obtained results and are found to be in excellent agreement. It is observed that the fluid velocity and temperature enhanced with thermal and solutal buoyancy forces as well as the Dufour effect. Also, with an increase in chemical reaction parameter, there is a decrease in fluid velocity, temperature, and concentration. Skin friction and Nusselt number increase with higher heat absorption parameter values. Schmidt number and chemical reaction parameter both lead to a rise in Sherwood number. Applications for these models have been observed in a number of industrial and technical methods.

X-DoF: Automatic Degree of Freedom Subset Selection for Inverse Blocked Force Characterization

Abstract Selecting the proper set of Degrees of Freedom is essential in inverse (Blocked) Force calculation. Including too many Degrees of Freedom in the computation can lead to overfitting, resulting in inaccurate force estimations and poor prediction quality. The discrepancy arises from errors within the dataset, such as measurement noise or other artifacts. This paper presents a solution to the overfitting problem, introducing the X-DoF procedure to automatically identify the relevant subset of Blocked Force Degrees of Freedom. Its effectiveness is showcased through numerical and experimental validation and compared against regularization techniques.

A hybrid multiscale model for fluid flow in fractured rocks using homogenization method with discrete fracture networks

Fluid flow in subsurface tight reservoirs containing pores, microcracks and macrocracks is notably influenced by the characteristics of macro/micro-cracks. A novel hybrid multiscale model is proposed to address the response of macrocracks and pores/microcracks in different spatial scales. Specifically, an equivalent macroscopic model (EMM) deduced from locally periodic representative element volume (REV) is developed using the asymptotic homogenization method to represent the poroelastic behavior of porous medium with microcracks. Simultaneously, the macrocracks are modeled explicitly using the discrete fracture model (DFM), where the hydraulic properties of cracks influenced by fluid pressure gradient is represented by the nonlinear opening/closure behavior. The obtained hybrid model takes into account the heterogeneous nature of fractured rock masses containing pores, micro/macro-cracks, which is fundamental to describe fluid flow behavior in fracture-matrix system. Specialized finite elements, regular meshing technique and adaptive time stepping algorithm are adopted to improve the computational efficiency. The hybrid multiscale model is firstly validated step by step to demonstrate the accuracy and then used to simulate fluid flow in fractured rock reservoir, shedding light on the underlying mechanisms of the enhanced flow capacity resulting from microcrack distribution, connectivity, and macrocrack stimulation.

Tunable enhancement of the cylindrical Luneburg lens focusing ability with the aid of a conformal graphene strip

We consider the plane wave focusing characteristics of the layered cylindrical Luneburg lens equipped with a conformal strip of graphene, in the H-polarization case. The angular width and location of the strip is arbitrary, and its surface impedance is characterized with the aid of the quantum-physics Kubo formalism. We use a mathematically accurate full-wave analytical regularization technique, which is based on the explicit inversion of the problem static part and yields a Fredholm second-kind matrix equation. This guarantees the convergence of the resulting meshless numerical algorithm. We compute the focusing ability of a microsize lens as a function of the frequency in the wide range up to 60 THz. This analysis shows that a graphene strip, placed into the focal area of the Luneburg lens, enhances its focusing ability at the resonance frequency of the strip plasmon mode proportionally to the quality factor. This frequency is defined by the strip width and is tunable with the aid of graphene’s chemical potential.

Machine learning-assisted DFT-prediction of pristine and endohedral doped (O and Se) Ge12C12 and Si12C12 nanostructures as anode materials for lithium-ion batteries

Nanostructured materials have gained significant attention as anode material in rechargeable lithium-ion batteries due to their large surface-to-volume ratio and efficient lithium-ion intercalation. Herein, we systematically investigated the electronic and electrochemical performance of pristine and endohedral doped (O and Se) Ge12C12 and Si12C12 nanocages as a prospective negative electrode for lithium-ion batteries using high-level density functional theory at the DFT/B3LYP-GD3(BJ)/6-311 + G(d, p)/GEN/LanL2DZ level of theory. Key findings from frontier molecular orbital (FMO) and density of states (DOS) revealed that endohedral doping of the studied nanocages with O and Se tremendously enhances their electrical conductivity. Furthermore, the pristine Si12C12 nanocage brilliantly exhibited the highest Vcell (1.49 V) and theoretical capacity (668.42 mAh g− 1) among the investigated nanocages and, hence, the most suitable negative electrode material for lithium-ion batteries. Moreover, we utilized four machine learning regression algorithms, namely, Linear, Lasso, Ridge, and ElasticNet regression, to predict the Vcell of the nanocages obtained from DFT simulation, achieving R2 scores close to 1 (R2 = 0.99) and lower RMSE values (RMSE < 0.05). Among the regression algorithms, Lasso regression demonstrated the best performance in predicting the Vcell of the nanocages, owing to its L1 regularization technique.

Accurate identification of moving vehicle loads on beam-like bridge structures integrating novel PCA-based dictionary with grouping and weighting strategy

Most moving load identification (MLI) methods incorporate the dictionary theory as a regularization technique to address the ill-posedness of the problem. However, the selection of atoms in the dictionary often inadequately represents the actual vehicle loads in existing methods, and the prior information from these atoms is disregarded in the subsequent MLI computations. Therefore, a novel MLI framework is proposed for beam-like bridge structures based on a novel dictionary derived from principal component analysis (PCA) and newly grouping and weighting strategy in this study. At first, a vehicle-bridge coupling system (VBCS) is established to obtain the interaction forces between the moving vehicles and bridge deck. The system matrices between the interaction forces and structural responses are derived in the time domain. The PCA technique is then employed to extract information from these interaction forces and to subsequently construct a PCA-based dictionary. Based on the eigenvalues of each principal component in the dictionary, a weighted group sparse model, incorporating the newly grouping and weighting strategy, is defined to obtain the coefficients of each atom. The solution to this model is obtained using the alternating direction method (ADM). Finally, the proposed method is validated in numerical simulations comparing with some existing methods. The effects of noise levels, road surface roughness, number of training data, response combinations and the tolerances in ADM were also studied. Similarity law for the VBCS is conducted in experimental verifications to construct the PCA-based dictionary, allowing for a reasonable identification of gross vehicle weights. The results indicate that the MLI accuracy has been enhanced by the proposed method, and the similarity law employed in experiments is reasonable. Settingthe tolerance in ADM as ε = 1 × 10−5 can lead to a higher accuracy and reduce the computation cost in both numerical simulations and experimental verifications.

Non-Negative Matrix Factorization with Averaged Kurtosis and Manifold Constraints for Blind Hyperspectral Unmixing

The Nonnegative Matrix Factorization (NMF) algorithm and its variants have gained widespread popularity across various domains, including neural networks, text clustering, image processing, and signal analysis. In the context of hyperspectral unmixing (HU), an important task involving the accurate extraction of endmembers from mixed spectra, researchers have been actively exploring different regularization techniques within the traditional NMF framework. These techniques aim to improve the precision and reliability of the endmember extraction process in HU. In this study, we propose a novel HU algorithm called KMBNMF, which introduces an average kurtosis regularization term based on endmember spectra to enhance endmember extraction, additionally, it integrates a manifold regularization term into the average kurtosis-constrained NMF by constructing a symmetric weight matrix. This combination of these two regularization techniques not only optimizes the extraction process of independent endmembers but also improves the part-based representation capability of hyperspectral data. Experimental results obtained from simulated and real-world hyperspectral datasets demonstrate the competitive performance of the proposed KMBNMF algorithm when compared to state-of-the-art algorithms.

Topology modification against membership inference attack in Graph Neural Networks

Graph Neural Networks (GNNs) can effectively leverage information from graph data and apply to various downstream tasks, such as recommendation system, or anomaly detection. Previous studies have shown that node-level GNNs are susceptible to membership inference attacks, potentially leaking private information about the nodes. However, current studies on defending against such attacks primarily focuses on image and text data. They may not be directly applicable to graph data. In addition, classic defense methods such as differential privacy, regularization techniques, and adversarial training. They may reduce model availability or require model retraining while protecting privacy. To deal with these problems, we propose a novel defense strategy against membership inference attacks in graph neural networks. The strategy involves augmenting the graph with additional nodes and modifying its topology to create a new privacy-preserving graph, thereby protecting the privacy of the original nodes. We conducted extensive experiments on three representative GNN models and compared them with state-of-the-art baseline methods to support our research. The experimental results demonstrate that our method significantly reduces the success rate of membership inference attacks while maintaining the basic performance of the target model.

ConvAtt Network: A Low Parameter Approach For Sign Language Recognition

Despite recent advances in Large Language Models in text processing, Sign Language Recognition (SLR) remains an unresolved task. This is, in part, due to limitations in the available data. In this paper, we investigate combining 1D convolutions with transformer layers to capture local features and global interactions in a low-parameter SLR model. We experimented using multiple data augmentation and regularization techniques to categorize signs of the French Belgian Sign Language. We achieved a top-1 accuracy of 42.7% and a top-10 accuracy of 81.9% in 600 different signs. This model is competitive with the current state of the art while using a significantly lower number of parameters.

An Improved Hybrid Model Based on Ensemble Features and Regularization Selection for Classification

Feature selection is a pivotal process in machine learning, essential for enhancing model performance by reducing dimensionality, improving generalization, and mitigating overfitting. By eliminating irrelevant or redundant features, simpler and more interpretable models are achieved, which generally perform better. In this study, we introduce an advanced hybrid method combining ensemble feature selection and regularization techniques, designed to optimize model accuracy while significantly reducing the number of features required. Applied to a customer satisfaction dataset, our method was first tested without feature selection, where the model achieved a ROC AUC value of 0.946 on the test set using all 369 features. However, after applying our proposed feature selection method, the model achieved a higher ROC AUC value of 0.954, utilizing only 12 key features and completing the task in approximately 43% less time. These findings demonstrate the effectiveness of our approach in producing a more efficient and superior-performing model.

Leveraging survival analysis in cost-aware deepnet for efficient hard drive failure prediction

AbstractMany real-world datasets, such as those used for failure and anomaly detection, are severely imbalanced, with a relatively small number of failed instances compared to the number of normal instances. To address these issues, this paper leverages the Backblaze hard disk drives (HDDs) data and makes several contributions to hard drive failure prediction. This research explores 1D convolutional neural networks (CNN) to utilize the sequential nature of hard drive sensor data. The performance of 1D CNN models is compared to traditional machine learning (ML) algorithms, such as the synthetic minority over-sampling technique (SMOTE) and weighted logistic regression (WLR), demonstrating superior results, suggesting the potential effectiveness of the proposed approaches. In addition to these efforts, this paper aims to provide a comprehensive understanding of hard drive longevity and the critical factors contributing to their eventual failure through survival analysis. The 1D CNN models employ weighted binary cross-entropy (WCE) loss and modified focal loss (MFL) functions to manage class imbalanced issues commonly observed in hard drive data. The findings suggest that 1D CNN models outperform traditional ML models, with regularization techniques like dropout and early stopping proving effective in controlling overfitting. Notably, the 1D CNN model with WCE loss demonstrated the best overall performance with a $$G_{mean}$$ G mean of 0.692, successfully maximizing the FDR without increasing the FAR. In parallel, the research also employs Cox regression to identify key SMART parameters influencing drive failure. The high concordance index (c-index) of the Cox model (0.958) adds confidence to the insights derived. The research thus sets a solid groundwork for data center management strategies, with a future focus on practical implementation and evaluation of these findings.

Tree smoothing: Post-hoc regularization of tree ensembles for interpretable machine learning

Random Forests (RFs) are powerful ensemble learning algorithms that are widely used in various machine learning tasks. However, they tend to overfit noisy or irrelevant features, which can result in decreased generalization performance. Post-hoc regularization techniques aim to solve this problem by modifying the structure of the learned ensemble after training. We propose a novel post-hoc regularization via tree smoothing for classification tasks to leverage the reliable class distributions closer to the root node whilst reducing the impact of more specific and potentially noisy splits deeper in the tree. Our novel approach allows for a form of pruning that does not alter the general structure of the trees, adjusting the influence of nodes based on their proximity to the root node. We evaluated the performance of our method on various machine learning benchmark data sets and on cancer data from The Cancer Genome Atlas (TCGA). Our approach demonstrates competitive performance compared to the state-of-the-art and, in the majority of cases, and outperforms it in most cases in terms of prediction accuracy, generalization, and interpretability.

Exploring noise-induced techniques to strengthen deep learning model resilience

AbstractIn artificial intelligence, combating overfitting and enhancing model generalization is crucial. This research explores innovative noise-induced regularization techniques, focusing on natural language processing tasks. Inspired by gradient noise and Dropout, this study investigates the interplay between controlled noise, model complexity, and overfitting prevention. Utilizing long short-term memory and bidirectional long short term memory architectures, this study examines the impact of noise-induced regularization on robustness to noisy input data. Through extensive experimentation, this study shows that introducing controlled noise improves model generalization, especially in language understanding. This contributes to the theoretical understanding of noise-induced regularization, advancing reliable and adaptable artificial intelligence systems for natural language processing.

StructNet-DDI: Molecular Structure Characterization-Based ResNet for Prediction of Drug-Drug Interactions.

This study introduces a deep learning framework based on SMILES representations of chemical structures to predict drug-drug interactions (DDIs). The model extracts Morgan fingerprints and key molecular descriptors, transforming them into raw graphical features for input into a modified ResNet18 architecture. The deep residual network, enhanced with regularization techniques, efficiently addresses training issues such as gradient vanishing and exploding, resulting in superior predictive performance. Experimental results show that StructNet-DDI achieved an AUC of 99.7%, an accuracy of 94.4%, and an AUPR of 99.9%, demonstrating the model's effectiveness and reliability. These findings highlight that StructNet-DDI can effectively extract crucial features from molecular structures, offering a simple yet robust tool for DDI prediction.

A regularisation technique to precisely infer limb darkening using transit measurements: Can we estimate stellar surface magnetic fields?

Abstract The high-precision measurements of exoplanet transit light curves that are now available contain information about the planet properties, their orbital parameters, and stellar limb darkening (LD). Recent 3D magneto-hydrodynamical (MHD) simulations of stellar atmospheres have shown that LD depends on the photospheric magnetic field, and hence its precise determination can be used to estimate the field strength. Among existing LD laws, the uses of the simplest ones may lead to biased inferences, whereas the uses of complex laws typically lead to a large degeneracy among the LD parameters. We have developed a novel approach in which we use a complex LD model but with second derivative regularisation during the fitting process. Regularisation controls the complexity of the model appropriately and reduces the degeneracy among LD parameters, thus resulting in precise inferences. The tests on simulated data suggest that our inferences are not only precise but also accurate. This technique is used to re-analyse 43 transit light curves measured by the NASA Kepler and TESS missions. Comparisons of our LD inferences with the corresponding literature values show good agreement, while the precisions of our measurements are better by up to a factor of 2. We find that 1D non-magnetic model atmospheres fail to reproduce the observations while 3D MHD simulations are qualitatively consistent. The LD measurements, together with MHD simulations, confirm that Kepler-17, WASP-18, and KELT-24 have relatively high magnetic fields (&amp;gt;200 G). This study paves the way for estimating the stellar surface magnetic field using the LD measurements.

Machine learning-based football match prediction system

This study develops a machine learning-based system to predict English Premier League (EPL) outcomes, employing models such as Principal Component Analysis (PCA), K-Nearest Neighbors (KNN), Random Forests, and Support Vector Machines (SVM). The analysis covered a large dataset of matches, with the data normalized to ensure consistency and accuracy across models. Among the methods used, Random Forests showed the most robust performance in predicting match outcomes, particularly in forecasting wins and losses. However, both Random Forests and SVM encountered difficulties in accurately predicting draws, which points to areas where further refinement is needed. The prediction probabilities largely fell within a specific range, indicating the models' ability to identify patterns, but significant overfitting was observed in the models. This overfitting suggests that while the models performed well on the training data, they struggled to generalize to new, unseen data, highlighting the importance of implementing more effective regularization techniques to prevent overfitting and improve the models' overall predictive accuracy in real-world scenarios.

Tumor Detection in MRI Data using Deep Learning Techniques for Image Classification and Semantic Segmentation

This study aims to apply image classification and semantic segmentation methods using deep learning for accurate and efficient tumor detection in brain MRI data. It looks at ways to get over issues including interpretability of the model and a lack of annotated data. A pre-trained VGG19 model and a basic convolutional neural network (CNN) were trained using a public dataset of 273 brain MRI images for the purpose of image classification. Techniques for regularization, transfer learning, and data augmentation were used. A U-Net architecture was trained using manually created masks from the same dataset for semantic segmentation, with several training setups investigated. With a mean test accuracy of 77.19%, the pre-trained VGG19 model surpassed the basic CNN in picture categorization. The U-Net model performed better for semantic segmentation after training on the entire dataset for two epochs as opposed to training on a subset for ten epochs. The work acknowledges issues including data scarcity and the requirement for model interpretability and scalability, which call for additional research, but also underlines the potential of deep learning for brain tumor identification.

The deep learning-based physical education course recommendation system under the internet of things

This study aims to propose a deep learning (DL)-based physical education course recommendation system by combining the Internet of Things (IoT) technology and DL, to improve the accuracy and personalization of recommendation. Firstly, IoT devices such as smart bracelets and smart clothing are used to monitor students' physiological data in real-time, and IoT sensors are utilized to sense the environment around students. Secondly, IoT devices capture students' social interactions with their peers, recommending socially oriented courses. Meanwhile, by integrating IoT data with students' academic data, course recommendations are optimized to match students' learning progress and schedule. Finally, Generative Adversarial Network (GAN) models, especially the improved Regularization Penalty Conditional Feature Generative Adversarial Network (RP-CFGAN) model, deal with data sparsity and cold start problems. The experimental results show that this model performs well in TopN evaluation and is markedly enhanced compared with traditional models. This study denotes that integrating IoT technology and GAN models can more accurately understand student needs and provide personalized recommendations. Although the model performs well, there is still room for improvement, such as exploring more regularization techniques, protecting user privacy, and extending the system to diverse platforms and scenarios.

Transfer graph feature alignment guided multi-source domain adaptation network for machinery fault diagnosis

In recent years, the application of unsupervised multi-source domain adaptation (MSDA) techniques for fault diagnosis has gained significant traction. Current research typically overlooks or fails to effectively capture critical data structure information during feature extraction. Another challenge is optimising the integration of information from multiple source domains to diagnose the target domain while avoiding negative transfer. To address these challenges, this study proposes a transfer graph feature alignment guided multi-source domain adaptation network (MDTGAL). In the proposed method, a transfer graph sample generator module (GSG) is constructed to model the data structure between source and target domains, and multiple graph feature extractors are employed to learn the data structure information from different domain combinations. A regularisation technique is introduced to extract the domain-invariant features by aligning the parameters across multiple independent graph feature extractors. In addition, a weighted soft-voting mechanism based on the polynomial kernel-induced maximum mean difference metric (PK-MMD) is designed to fuse the outputs from multiple classifiers, to comprehensively account for the influence of each source domain. The proposed method was tested on multi-source domain transfer tasks involving various operating conditions of rotating machinery. The experimental results demonstrate that the MDTGAL exhibits superior cross-domain diagnostic performance, outperforming existing mainstream methods. In addition, this study explored the impact of varying numbers of source domains on the diagnostic accuracy of the target domain, providing insights into selecting the correct number of source domains for specific MSDA tasks.