Generalization Capacity Research Articles

Predicting crack behavior in holed plates using deep learning

In this study, we introduce a methodology for predicting the behavior of cracks in holed plates. The foundation of our methodology lies in the generation of rich dataset finite element simulations. These simulations capture the complex mechanical responses exhibited by holed plates under varying initial geometries. Using this dataset as training input, we employ a multilayer perceptron deep learning model to discern the underlying relationships between the plate’s initial geometry and its subsequent crack behavior. Through meticulous data preprocessing and fine-tuning of the model architecture, the MLP undergoes rigorous training and validation to optimize its predictive capabilities. Mean squared errors are utilized to assess the accuracy and generalization capacity of the trained model. The results suggest that this model can serve as a powerful rapid predictive tool, capable of analyzing crack behavior in new instances of the plate’s geometry with remarkable efficiency and accuracy.

A simple self-supervised learning framework with patch-based data augmentation in diagnosis of Alzheimer’s disease

Alzheimer’s disease (AD) stands as a prominent age-related disorder with significant global impact. Utilizing computer-aided diagnosis aids in the timely identification of mild cognitive impairment, facilitating early intervention and management. Self-supervised learning models have attracted much attention due to their advantages of no manual labeling, so they are very suitable for solving the problems of difficulty in data acquisition and high cost of manual labeling in medical image processing. However, the existing self-supervised algorithms applied to medical images often have poor diagnostic effects and consume large resources, as a consequence, the model’s ability to learn meaningful representations from medical images is hampered, leading to suboptimal performance. Subsequently, diverse patches extracted from the same brain are amalgamated to create contrast views, and attention weights are then employed to enhance the fitting and generalization capacity of the model. Experimental results on the ADNI dataset with 1365 subjects show that PD-SIM has been improved in the diagnosis of different diseases(such as the classification ACC of AD and CN reached 0.797, and the classification ACC of early cognitive impairment reached 0.7036), and also alleviates the problem of large consumption of computer resources, downstream tasks require only 52ms per image. The proposed method performs well in atrophic structure identification and AD diagnosis. Therefore, PD-SIM has a wide range of application prospects and is of great practical significance within the realm of medical image analysis. The data and code can be found at https://github.com/Z1Ting/PD-SIM.

Representations and generalization in artificial and brain neural networks

Humans and animals excel at generalizing from limited data, a capability yet to be fully replicated in artificial intelligence. This perspective investigates generalization in biological and artificial deep neural networks (DNNs), in both in-distribution and out-of-distribution contexts. We introduce two hypotheses: First, the geometric properties of the neural manifolds associated with discrete cognitive entities, such as objects, words, and concepts, are powerful order parameters. They link the neural substrate to the generalization capabilities and provide a unified methodology bridging gaps between neuroscience, machine learning, and cognitive science. We overview recent progress in studying the geometry of neural manifolds, particularly in visual object recognition, and discuss theories connecting manifold dimension and radius to generalization capacity. Second, we suggest that the theory of learning in wide DNNs, especially in the thermodynamic limit, provides mechanistic insights into the learning processes generating desired neural representational geometries and generalization. This includes the role of weight norm regularization, network architecture, and hyper-parameters. We will explore recent advances in this theory and ongoing challenges. We also discuss the dynamics of learning and its relevance to the issue of representational drift in the brain.

Cross‐scale resolution consistent network for salient object detection

AbstractThe salient object detection task tries to simulate the human visual system for most eye‐catching objects or regions detection. However, due to the complexity of the visual mechanisms, current methods will suffer from severe performance degradation, leading to inconsistent prediction results for the same regions, when directly adopting a model trained on a fixed resolution to evaluate at other different resolutions. Considering that consistency in predictions is essential for salient object detection, a cross‐scale resolution consistent salient object detection method, called RCNet, is proposed. Specifically, to enhance the model's capacity for generalization across images of varying resolutions and make the model implicitly learn the scale invariance, a multi‐resolution data enhancement module is constructed to generate images with arbitrary resolutions for the same scene. Moreover, to accomplish better multi‐level feature fusion, a cross‐scale fusion module is developed to fuse high‐level semantic features and low‐level detail features. Additionally, to explicitly learn the scale invariance of the salient scores, a hybrid salient consistency loss is formulated on salient object detection with different resolutions. Comprehensive evaluations on five benchmark datasets show that RCNet achieves a highly competitive result.

Intuitionistic fuzzy twin proximal SVM with fuzzy hyperplane and its application in EEG signal classification

Twin support vector machine (TSVM) is a contemporary machine learning technique to tackle classification and regression problems. However, TSVM lacks the ability to differentiate between support vectors and noises since it neglects the positional information of input data samples, and hence it is sensitive to noises. Additionally, it fails to consider uncertainties associated with the data, which reduces its capacity for generalization. To address these drawbacks, we propose a novel fuzzy hyperplane based intuitionistic fuzzy twin proximal support vector machine. The first significant feature of the proposed approach is that it gives an intuitionistic fuzzy number based on the relevance to each data vector. This efficiently reduces the impact of noise and outliers by leveraging the local neighborhood information within the data points by incorporating membership and non-membership weights. Secondly, all the parameters present in the model are fuzzy variables, including the offset term and the elements of the normal vector. The suggested fuzzy hyperplane successfully reflects the inherent ambiguity prevalent in real-world categorization problems by reflecting vagueness in the input data through the use of fuzzy variables. The model’s efficiency is enhanced by solving two systems of linear equations to obtain two non-parallel classifiers rather than solving two quadratic programming problems as in standard TSVM. Utilizing non-linear kernel functions within the feature space enables the method to effectively identify complex patterns or non-linear relationships within the datasets. In order to demonstrate the effectiveness of the suggested approach, extensive computer experiments have been conducted on a set of eighteen benchmark datasets with both linear and non-linear kernels. In addition, rigorous statistical analysis, including Friedman and post-hoc Nemenyi tests, have been employed to assess the significance of the observed performance differences. Furthermore, we also performed numerical experiments utilizing linear, Gaussian, and polynomial kernels to classify electroencephalogram (EEG) signals. The outcomes of the experiment are analyzed in terms of average accuracy, processing time, and F-measure. The results demonstrate that the proposed method outperforms existing methods and achieves better generalization.

Multiclass Classification using Enhanced MobileNet V2 Architecture

In computer vision applications including object detection, face recognition, image classification, etc., the Convolutional Neural Network (CNN) predominates. MobileNet v2 is one such CNN architecture which can significantly cut down the parameters and cost of calculation while giving comparatively higher accuracy. In this paper, we propose the enhanced MobileNet architecture which shows an improvement in the accuracy. We have applied transfer learning techniques to fine-tune the MobileNet V2 model on the food dataset. The proposed model’s generalization capacity, it’s robustness and recognition accuracy shows a considerable amount of improvement with respect to the base architecture. The findings demonstrate that the enhanced MobileNet architecture’s accuracy for food images recognition and classification has increased to 97.16%. Key Words: Convolutional Neural Network (CNN), MobileNet v2, transfer learning, image recognition and classification

Convolutional Neural Networks for Road Detection: An Unsupervised Domain Adaptation Approach

Abstract. Due to the frequent road network changes, keeping them updated is fundamental for several purposes. Currently, models based on Deep Learning (DL), specifically, Convolutional Neural Networks (CNNs), such as encoder-decoder type, are state-of-the-art for this purpose. In this context, the high performance in CNNs has two aspects involved: the model needs a large labeled dataset, and the dataset belongs to the same probability distribution. In practical applications, however, this may not hold, since there is a domain shift effect, and it is not customary for the availability of labeled data. To approach these challenges, we propose to adapt the U-Net architecture (encoder-decoder) to the Unsupervised Domain Adaptation (UDA) that does not need labeling data to minimize the domain shift effect. Our results demonstrate that the proposed method contributes to road segmentation, whose model reaches 74.31% (IoU) and 85.04% (F1), against the same model without UDA that reaches 67.36% (IoU) and 80.02% (F1). This implies that the information that comes from the target domain, even unsupervised, contributes to adversarial learning, improving the generalization capacity of the model, enhancing aspects such as better discrimination surrounding classes, and in the geometric delineation of the road network.

Feature selection based on contradictory state sequence for multi-scale interval valued decision table

In the context of selecting features for multi-scale interval valued decision table (MIVDT), conventional research approaches encounter difficulties including elevated data complexity, diminished computational efficiency, and limited model generalization capacity. To overcome these difficulties, feature selection methods based on contradictory state sequence (CSS) and fuzzy contradictory state sequence (FCSS) are proposed in this paper. Initially, MIVDT is established. According to the characteristics of interval value, a more thorough and impartial metric for assessing the inclusion degree is suggested, along with similarity relation based on the metric. Subsequently, the introduction of the first contradictory object allows for the delineation of contradictory state and fuzzy contradictory state, which serve to characterize the consistency of MIVDT. These two concepts are proposed to enhance the accuracy and efficiency of capturing key information in intricate data. Additionally, rapid feature selection algorithms are suggested, which rely on CSS and FCSS. In contrast to conventional feature selection approaches, the algorithms introduced in this study demonstrate superior computational efficiency and enhanced generalization capabilities when confronted with intricate datasets. In twelve open source datasets, the upper limit for the quantity of objects is 110341, and the average accuracy values of experiments under two parameter combinations are 99.54% and 97.08% respectively. The results of experiments demonstrate that the algorithms introduced are capable of efficiently identifying a novel feature subset from intricate datasets within a shorter timeframe.

MOC-Z Model of Transient Cavitating Flow in Viscoelastic Pipe

In this paper, a unitary method for the solution of transient cavitating flow in viscoelastic pipes is proposed in the framework of the method of characteristics (MOC) and a Z-mirror numerical scheme (MOC-Z model). Assuming a standard form of the continuity equation allows the unified treatment of both viscoelasticity and cavitation. An extension of the MOC-Z is used for Courant numbers less than 1 to overcome a few cases with numerical instabilities. Four viscoelastic models were considered: a Kelvin–Voigt (KV) model without the instantaneous strain, and three generalised Kelvin–Voigt models with one, two, and three KV elements (GKV1, GKV2, and GKV3, respectively). The use of viscoelastic parameters of KV and GKV models calibrated for transient flow tests without cavitation allows good comparisons between experimental and numerical pressure versus time for transient tests with cavitation. Whereas for tests without cavitation, the mean absolute error (MAE) always decreases when the complexity of the model increases (from KV to GKV1, GKV2, and GKV3) for all the considered tests, this does not happen for tests with cavitation, probably because the decreasing capacity of parameter generalization for the increasing complexity of the model. In particular, in the examined cases, the KV model performs better than the GKV1 and the GKV3 models in three cases out of five, and the GKV2 model performs better than the GKV3 model in three cases out of five. Furthermore, the GKV2 model performs better than the KV model only in three cases out of five.

Efficient augmented radial basis function for reliability analysis with dynamic importance sampling

Conducting reliability analysis on complex systems, particularly those characterized by minimal failure probabilities and computationally intensive models, presents significant challenges. This article introduces a reliability analysis method combining the adaptive augmented radial basis function and the dynamic importance sampling (ARBF-DIS) algorithm. This method improves the generalization capacity of the augmented radial basis function at both local and global scales through the application of an anisotropic technique and orthogonal conditions. By using efficient cross-validation, the methodology facilitates the training of the metamodel, enabling it to provide predictive variance essential for constructing an adaptive sampling learning function. A key aspect of this approach is its capability to address the issue of low failure probabilities through dynamic importance sampling, which involves calculating an optimal importance sampling function derived from the current metamodel. The effectiveness and precision of the ARBF-DIS method are demonstrated through two analytical examples and a solid rocket motor reliability problem.

Enhancing Molecular Energy Predictions with Physically Constrained Modifications to the Neural Network Potential.

Exclusively prioritizing the precision of energy prediction frequently proves inadequate in satisfying multifaceted requirements. A heightened focus is warranted on assessing the rationality of potential energy curves predicted by machine learning-based force fields (MLFFs), alongside evaluating the pragmatic utility of these MLFFs. This study introduces SWANI, an optimized neural network potential stemming from the ANI framework. Through the incorporation of supplementary physical constraints, SWANI aligns more cohesively with chemical expectations, yielding rational potential energy profiles. It also exhibits superior predictive precision compared with that of the ANI model. Additionally, a comprehensive comparison is conducted between SWANI and a prominent graph neural network-based model. The findings indicate that SWANI outperforms the latter, particularly for molecules exceeding the dimensions of the training set. This outcome underscores SWANI's exceptional capacity for generalization and its proficiency in handling larger molecular systems.

Deep neural network model for predicting thermal-hydraulic performance of a solar air heater with artificial roughness: Sensitivity, generalization capacity, and computational efficiency

A novel coupled computational fluid dynamics-deep neural network approach is proposed to accurately predict the performance of an artificially roughened solar air heater. Computational fluid dynamics data sets are used to develop an optimized neural network model for the prediction of thermal-hydraulic performance factor. The optimized neural network model with softmax transfer function in hidden layers has an architecture of 3‐37‐37‐1 and it predicts the thermal-hydraulic performance factor of the artificially roughened solar air heater with maximum, mean, and minimum error of 2.54 %, 0.245 %, and 0.0009 %, respectively. The concept of Shapley values is adopted to evaluate the global sensitivity analysis of the optimized deep neural network model and it is found that rib height is the most influential design parameter for the artificially roughened solar air heater. The generalization capacity of the neural network model is evaluated by comparing its predictions with experimental data, demonstrating superior predictive performance over traditional computational fluid dynamics simulations. The optimized artificial neural network model achieves a computational time gain of approximately 100 % against computational fluid dynamics.

User-agnostic adaptation of human locomotion intent: Leveraging Teacher-Student-Learning and ensemble modeling

Wearable robot control in complicated contexts requires an understanding of human locomotion intent and behaviors. However, signals from human–robot interfaces are typically reliant on the user, leading to poor performance of the model trained on training user when applied to new end-point users. This study aims to address this problem by developing a novel Teacher-Student-Learning (TSL) approach using Heterogeneous Ensemble Hypotheses (HEH) to achieve unsupervised user-agnostic adaptation. The motivation behind using the Teacher-Student-Learning architecture is to leverage the diverse features extracted by HEH while maintaining computational efficiency. HEH reduces the difference between labeled training users and unlabeled end-point users by incorporating multiple diverse feature generators to extract a wide range of features, thereby increasing classification accuracy and reducing variance. However, this approach sacrifices efficiency due to the ensemble nature of HEH. To address this trade-off, the knowledge from HEH is distilled into a single network using TSL, ensuring precision and efficiency. The proposed approach is evaluated on two publicly available human locomotion datasets and a 2D moon dataset. Experimental results demonstrate the effectiveness of the proposed method, outperforming all other methods. Tested with three datasets, the proposed method can classify end-point users’ data with an average accuracy of 98.9%, 96.7%, and 96.9% while yielding a low processing time (1 ms). Compared to a benchmark method, the suggested strategy improves the average accuracy by 1.5% and 7.2% for categorizing the target users’ locomotor modes and stabilizes the learning curves. The proposed method represents a significant contribution to the field of human activity recognition and human intent prediction for human–robot interaction systems, enhancing the efficiency and generalization capacity of these systems.

A Method for Elevated Ducts Refinement Based on Convolutional Neural Network

AbstractElevated duct (EleD) is an abnormal atmospheric refraction structure with a suspended trapped layer. The precise and highly resolved elevated duct‐height‐based data (EleDH) is crucial for radio communication systems, especially in electromagnetic wave path loss prediction and EleDH‐producing systems. However, producing high‐resolution EleDH is challenging because of the massive details in the EleDH data. Direct and high‐time refinement procedures mostly lead to unrealistic outcomes. The study provides a Dense‐Linear convolutional neural network (DLCNN)‐based EleDH refinement technique based on the development of statistical downscaling and super‐resolution technologies. Additionally, the stack approach is used, and the refining order is taken into consideration to ensure precision in high‐time refinement and provide reliable outcomes. To demonstrate the strength of DLCNN in capturing complex internal characteristics of EleDH, a new EleD data set is first funded, which only contains the duct height. From this data set, we use the duct height as the core refinement of the EleD's trapped layer and the thickness of the trapped layer to ensure reliable duct height. Seven super‐resolution models are utilized for fair comparisons. The experimental results prove that the DLCNN has the highest refinement performance; also, it obtained excellent generalization capacity, where the minimum and maximum obtained Accuracy(20%), MAE, and RMSE were 85.22% ∓ 88.30%, 36.09 ∓ 45.97 and 8.68 ∓ 10.14, respectively. High‐resolution EleDH improves path loss prediction, where the minimum and maximum obtained bias were 2.37 ∓ 9.51 dB.

Interpretable machine learning decodes soil microbiome’s response to drought stress

BackgroundExtreme weather events induced by climate change, particularly droughts, have detrimental consequences for crop yields and food security. Concurrently, these conditions provoke substantial changes in the soil bacterial microbiota and affect plant health. Early recognition of soil affected by drought enables farmers to implement appropriate agricultural management practices. In this context, interpretable machine learning holds immense potential for drought stress classification of soil based on marker taxa.ResultsThis study demonstrates that the 16S rRNA-based metagenomic approach of Differential Abundance Analysis methods and machine learning-based Shapley Additive Explanation values provide similar information. They exhibit their potential as complementary approaches for identifying marker taxa and investigating their enrichment or depletion under drought stress in grass lineages. Additionally, the Random Forest Classifier trained on a diverse range of relative abundance data from the soil bacterial micobiome of various plant species achieves a high accuracy of 92.3 % at the genus rank for drought stress prediction. It demonstrates its generalization capacity for the lineages tested.ConclusionsIn the detection of drought stress in soil bacterial microbiota, this study emphasizes the potential of an optimized and generalized location-based ML classifier. By identifying marker taxa, this approach holds promising implications for microbe-assisted plant breeding programs and contributes to the development of sustainable agriculture practices. These findings are crucial for preserving global food security in the face of climate change.

Quine’s Tarskian Angle on Truth: Immanence, Semantic Ascent and the Importance of Generality

There are two main parts to this article:  (1) Quine's view of truth is substantive in a way that is not generally recognized.  There are elements in the view of deflationism, minimalism, and of course disquotationalism to be sure, but from Quine's perspective the capacity for generalization - ascribing truth not to explicitly given sentences but to kinds of sentences - requires a full-bore Tarskian apparatus.  This is necessary in order for truth to play what for Quine are its vital roles in science, philosophy (epistemology and those parts of metaphysics that Quine recognizes), and crucially in logic itself.  (These points have not been made with adequate care in the literature). (2) But is Tarskianism sufficient for playing all of those roles? Contrary to an interesting article by Lars Bergström - I argue that it is indeed sufficient, at least from Quine's point of view, and to see the point is to appreciate how thoroughgoing and powerful the notion of truth is, despite its being, 'in itself', only a device of sentential generalization.  This dovetails with Quine's famous naturalism, a connection which I bring out at the end.

Enhancing rectangular side weir discharge prediction using stacking technique

In this paper, the impact of different variables on discharge coefficients in sharp-crested rectangular side weirs was investigated utilizing 559 experimental data points. Various methodologies, including De Marchi, Traditional weir equation (TWE), Domínguez, adjusted Domínguez, and Schmidt procedures, were employed for the analysis. Novel equations were developed to predict the discharge coefficients with greater accurately, surpassing the predictive capabilities of previous studies' equations. Additionally, three supervised machine learning algorithms, namely extreme gradient boosting (XGBoost), Light gradient boosting machine (LightGBM), and support vector regression (SVR), were employed to enhance the precision of discharge coefficient estimation. Among these algorithms, the XGBoost model exhibited superior performance compared to LightGBM and SVR models for forecasting discharge coefficients. Furthermore, using a stacking technique, hybrid models were constructed by combining XGBoost, LightGBM, and SVR. The hybrid models showed enhanced performance and generalization capacities than the individual models. Notably, the HYBRID5 model, specifically designed for estimated the Schmidt coefficient, achieved outstanding statistical indices (R2 = 0.995, MSE = 1.378, MRPE = 3.73 %) for calculating discharge of rectangular side weirs.

AttABseq: an attention-based deep learning prediction method for antigen-antibody binding affinity changes based on protein sequences.

The optimization of therapeutic antibodies through traditional techniques, such as candidate screening via hybridoma or phage display, is resource-intensive and time-consuming. In recent years, computational and artificial intelligence-based methods have been actively developed to accelerate and improve the development of therapeutic antibodies. In this study, we developed an end-to-end sequence-based deep learning model, termed AttABseq, for the predictions of the antigen-antibody binding affinity changes connected with antibody mutations. AttABseq is a highly efficient and generic attention-based model by utilizing diverse antigen-antibody complex sequences as the input to predict the binding affinity changes of residue mutations. The assessment on the three benchmark datasets illustrates that AttABseq is 120% more accurate than other sequence-based models in terms of the Pearson correlation coefficient between the predicted and experimental binding affinity changes. Moreover, AttABseq also either outperforms or competes favorably with the structure-based approaches. Furthermore, AttABseq consistently demonstrates robust predictive capabilities across a diverse array of conditions, underscoring its remarkable capacity for generalization across a wide spectrum of antigen-antibody complexes. It imposes no constraints on the quantity of altered residues, rendering it particularly applicable in scenarios where crystallographic structures remain unavailable. The attention-based interpretability analysis indicates that the causal effects of point mutations on antibody-antigen binding affinity changes can be visualized at the residue level, which might assist automated antibody sequence optimization. We believe that AttABseq provides a fiercely competitive answer to therapeutic antibody optimization.

A hybrid data-driven method optimized by physical rules for online state collaborative estimation of lithium-ion batteries

Accurate online estimation of state of charge (SOC) and state of health (SOH) is crucial for safe operation and reasonable planning of battery management system (BMS). However, battery internal states are unmeasurable directly and coupled with each other, which results in serious difficulties on multi-states accurate estimation. Considering SOC estimation is deeply influenced by SOH, this paper proposes a collaborative estimation method combined machine learning algorithm with simple physical rule. Integrated the merits of the above methods, it can not only improve the estimating accuracy but also low computational burden to a large extent. Firstly, SOH acts as input of SOC in collaboration, which means SOH should have been already known before SOC estimation operation. Thus, SOH is creatively predicted based on gate recurrent unit network in ahead of SOC estimation. Subsequently, the well-trained SOC data-driven model is combined with the ampere hour (Ah) integration, which act as observation and state equation respectively in the particle filtering algorithm to realize the final estimation of SOC. This way, the sole data-driven model is well improved under physics restriction without any other computational procedure, such as online parameter identification. Simultaneously, the predicted SOH is used to fill the Ah integration expression to realize the collaboration between SOC and SOH. What's more, the SOH is further corrected by the incremental capacity peak at the checkpoint, thereby further reducing the estimation deviation. The experimental result on battery shows that the SOH estimated error is below 0.3Ah (the rated capacity is 32.5Ah) and SOC error is less than 1.5 %. What's more, the generalization capacity of the proposed method is verified on different battery types based on public datasets. Obviously, the proposed method has fairly satisfactory property and performance and can totally support its promotion into state online evaluation of battery energy storage.

Room Interior Generation Using Stable Diffusion Models

The present paper puts forward an innovative method for the fine tuning of Stable Diffusion with the use of Dreambooth, which is a very fast technique in the image generation task. In particular, our approach covers the topic of room interior generation. It offers a quick option to instruct the algorithm with the most recent concepts, without the need to retrain it. Using a shortened list of instance and class prompts as examples we introduce a revolutionary training pipeline where class and instance information interacts to order a model's learning process. By conducting elaborate trials where we demonstrate that our methodology out-perform the competitors in the realistic visualization, which is based on the given theme. We use an extensive evaluation process to prove the effectiveness of the method on many datasets, which can ensure its generalization capacity to unseen various layouts of the rooms and their interior design. On one hand, precise ablation studies are carried out in order to evaluate the influence of the distinguished components in the given model. This paper shows the encompassing work of Dreambooth as a tool of choice for the personalized room interior synthesis. The new possibilities to fine-tune the generative models are also becoming a subject for futurological research in the interior design area.