Superior Generalization Research Articles

Learning spatial‐frequency interaction for generalizable deepfake detection

AbstractIn recent years, face forgery detection has gained significant attention, resulting in considerable advancements. However, most existing methods rely on CNNs to extract artefacts from the spatial domain, overlooking the pervasive frequency‐domain artefacts present in deepfake content, which poses challenges in achieving robust and generalized detection. To address these issues, we propose the dual‐stream frequency—spatial fusion network is proposed for deepfake detection. The dual‐stream frequency‐spatial fusion network consists of three components: the spatial forgery feature extraction module, the frequency forgery feature extraction module, and the spatial–frequency feature fusion module. The spatial forgery feature extraction module employs spatial‐channel attention to extract spatial domain features, targeting artefacts in the spatial domain. The frequency forgery feature extraction module leverages the focused linear attention to detect frequency domain anomalies in internal regions, enabling the identification of generated content. The spatial–frequency feature fusion module then fuses forgery features extracted from both the spatial and frequency domains, facilitating accurate detection of splicing artefacts and internally generated forgeries. This approach enhances the model's ability to more accurately capture forgery characteristics. Extensive experiments on several widely‐used benchmarks demonstrate that our carefully designed network exhibits superior generalization and robustness, significantly improving deepfake detection performance.

Mf-net: multi-feature fusion network based on two-stream extraction and multi-scale enhancement for face forgery detection

Due to the increasing sophistication of face forgery techniques, the images generated are becoming more and more realistic and difficult for human eyes to distinguish. These face forgery techniques can cause problems such as fraud and social engineering attacks in facial recognition and identity verification areas. Therefore, researchers have worked on face forgery detection studies and have made significant progress. Current face forgery detection algorithms achieve high detection accuracy within-dataset. However, it is difficult to achieve satisfactory generalization performance in cross-dataset scenarios. In order to improve the cross-dataset detection performance of the model, this paper proposes a multi-feature fusion network based on two-stream extraction and multi-scale enhancement. First, we design a two-stream feature extraction module to obtain richer feature information. Secondly, the multi-scale feature enhancement module is proposed to focus the model more on information related to the current sub-region from different scales. Finally, the forgery detection module calculates the overlap between the features of the input image and real images during the training phase to determine the forgery regions. The method encourages the model to mine forgery features and learns generic and robust features not limited to a particular feature. Thus, the model achieves high detection accuracy and performance. We achieve the AUC of 99.70% and 90.71% on FaceForensics++ and WildDeepfake datasets. The generalization experiments on Celeb-DF-v2 and WildDeepfake datasets achieve the AUC of 80.16% and 65.15%. Comparison experiments with multiple methods on other benchmark datasets confirm the superior generalization performance of our proposed method while ensuring model detection accuracy. Our code can be found at: https://github.com/1241128239/MFNet.

DeepJEB: 3D Deep Learning-based Synthetic Jet Engine Bracket Dataset

Abstract Recent advances in artificial intelligence (AI) have impacted various fields, including mechanical engineering. However, the development of diverse, high-quality datasets for structural analysis remains a challenge. Traditional datasets, like the jet engine bracket dataset, are limited by small sample sizes, hindering the creation of robust surrogate models. This study introduces the Deep- JEB dataset, generated through deep generative models and automated simulation pipelines, to address these limitations. DeepJEB offers comprehensive 3D geometries and corresponding structural analysis data. Key experiments validated its effectiveness, showing significant improvements in surrogate model performance. Models trained on DeepJEB achieved up to a 23% increase in the coefficient of determination and over a 70% reduction in mean absolute percentage error (MAPE) compared to those trained on traditional datasets. These results underscore the superior generalization capabilities of DeepJEB. By supporting advanced modeling techniques, such as graph neural networks (GNNs) and convolutional neural networks (CNNs), DeepJEB enables more accurate predictions in structural performance. The DeepJEB dataset is publicly accessible at: https://www.narnia.ai/dataset.

Robust Health Condition Prediction of Helicopter Turboshaft Engines Using Ensemble Machine Learning Models

This paper presents a novel ensemble approach that combines multiple machine-learning algorithms to deliver robust predictions of helicopter turboshaft engine health status (nominal or faulty) using operational data. Engine health is evaluated through the torque margin, defined as the percentage difference between the measured and target torque values. A Gaussian process model is used to estimate the torque margin as a probability distribution function (PDF), and these predictions are incorporated as features into various machine-learning models. These models are then employed to perform binary classification, determining the engine's health state. To enhance performance, a reference set is defined for each unseen data point, allowing a comparison of the relative performances of the models, with the best performer selected for the final prediction. Our ensemble method achieves high accuracy in health classification while providing precise torque margin estimates. The results demonstrate that ensemble models offer superior generalization and reliability compared to individual machine-learning algorithms, especially when applied to complex, multivariate datasets like those from helicopter turboshaft engines.

CSTAN: A Deepfake Detection Network with CST Attention for Superior Generalization

With the advancement of deepfake forgery technology, highly realistic fake faces have posed serious security risks to sensor-based facial recognition systems. Recent deepfake detection models mainly use binary classification models based on deep learning. Despite achieving high detection accuracy on intra-datasets, these models lack generalization ability when applied to cross-datasets. We propose a deepfake detection model named Channel-Spatial-Triplet Attention Network (CSTAN), which focuses on the difference between real and fake features, thereby enhancing the generality of the detection model. To enhance the feature-learning ability of the model for image forgery regions, we have designed the Channel-Spatial-Triplet (CST) attention mechanism, which extracts subtle local information by capturing feature channels and the spatial correlation of three different scales. Additionally, we propose a novel feature extraction method, OD-ResNet-34, by embedding ODConv into the feature extraction network to enhance its dynamic adaptability to data features. Trained on the FF++ dataset and tested on the Celeb-DF-v1 and Celeb-DF-v2 datasets, the experimental results show that our model has stronger generalization ability in cross-datasets than similar models.

Lightweight plant disease recognition model based on small datasets in complex contexts

ABSTRACT We propose a lightweight plant disease recognition model, CDS-FSSRSNet, leveraging contrastive deep supervised learning and feature separation techniques on ShuffleNetV2. To combat background interference, we introduce a channel weight-based feature separation module (CWFS). By combining structural reparameterization and ShuffleNetV2, our model achieves high-performance recognition while remaining resource-efficient. Utilizing FSSRSNet as a feature extractor with contrastive learning enhances disease feature differentiation, particularly on small datasets. Tested on apple and wheat datasets, CDS-FSSRSNet outperforms existing lightweight models, demonstrating superior generalization capabilities.

A deep transfer learning model for online monitoring of surface roughness in milling with variable parameters

Surface roughness is crucial for the functional and aesthetic properties of mechanical components and must be carefully controlled during machining. However, predicting it under varying machining parameters is challenging due to limited experimental data and fluctuating factors like tool wear and vibration. This study develops a deep transfer learning model that incorporates the correlation alignment method and tool wear to enhance model generalization and reduce data acquisition costs. It utilizes multi-sensor data and the ResNet18 with a convolutional block attention module (CBAM-ResNet) to extract features with improved generalization and accuracy for monitoring milled surface roughness under varying conditions. The performance of the model is evaluated from different perspectives. First, the proposed model achieves high accuracy with fewer than 500 experimental samples from the target domain by using the CORAL module in the CBAM-ResNet model. This demonstrates the model's strong generalization capability by minimizing second-order statistical discrepancies between different datasets. Second, ablation experiments reveal a significant reduction in test error when incorporating CORAL and tool wear, highlighting their contributions to improved model generalization. Integrating tool wear information significantly reduces test errors across various transfer conditions, as it reflects changes in cutting force, vibration, and built-up edge formation. Third, comparisons with existing deep transfer models further emphasize the advantages of the proposed approach in improving model generalization. In summary, the proposed surface roughness model, which incorporates tool wear and multi-sensor signal features as inputs and employs feature transfer and CBAM-ResNet, demonstrates superior generalization and accuracy across various machining parameters.

The Adversarial Robust and Generalizable Stereo Matching for Infrared Binocular Based on Deep Learning

Despite the considerable success of deep learning methods in stereo matching for binocular images, the generalizability and robustness of these algorithms, particularly under challenging conditions such as occlusions or degraded infrared textures, remain uncertain. This paper presents a novel deep-learning-based depth optimization method that obviates the need for large infrared image datasets and adapts seamlessly to any specific infrared camera. Moreover, this adaptability extends to standard binocular images, allowing the method to work effectively on both infrared and visible light stereo images. We further investigate the role of infrared textures in a deep learning framework, demonstrating their continued utility for stereo matching even in complex lighting environments. To compute the matching cost volume, we apply the multi-scale census transform to the input stereo images. A stacked sand leak subnetwork is subsequently employed to address the matching task. Our approach substantially improves adversarial robustness while maintaining accuracy on comparison with state-of-the-art methods which decrease nearly a half in EPE for quantitative results on widely used autonomous driving datasets. Furthermore, the proposed method exhibits superior generalization capabilities, transitioning from simulated datasets to real-world datasets without the need for fine-tuning.

Channel Interaction and Transformer Depth Estimation Network: Robust Self-Supervised Depth Estimation Under Varied Weather Conditions

Monocular depth estimation provides low-cost environmental information for intelligent systems such as autonomous vehicles and robots, supporting sustainable development by reducing reliance on expensive, energy-intensive sensors and making technology more accessible and efficient. However, in practical applications, monocular vision is highly susceptible to adverse weather conditions, significantly reducing depth perception accuracy and limiting its ability to deliver reliable environmental information. To improve the robustness of monocular depth estimation in challenging weather, this paper first utilizes generative models to adjust image exposure and generate synthetic images of rainy, foggy, and nighttime scenes, enriching the diversity of the training data. Next, a channel interaction module and Multi-Scale Fusion Module are introduced. The former enhances information exchange between channels, while the latter effectively integrates multi-level feature information. Finally, an enhanced consistency loss is added to the loss function to prevent the depth estimation bias caused by data augmentation. Experiments on datasets such as DrivingStereo, Foggy CityScapes, and NuScenes-Night demonstrate that our method, CIT-Depth, exhibits superior generalization across various complex conditions.

A lightweight model design approach for few-shot malicious traffic classification

Classifying malicious traffic, which can trace the lineage of attackers’ malicious families, is fundamental to safeguarding cybersecurity. However, the deep learning approaches currently employed require substantial volumes of data, conflicting with the challenges in acquiring and accurately labeling malicious traffic data. Additionally, edge network devices vulnerable to cyber-attacks often cannot meet the computational demands required to deploy deep learning models. The rapid mutation of malicious activities further underscores the need for models with strong generalization capabilities to adapt to evolving threats. This paper introduces an innovative few-shot malicious traffic classification method that is precise, lightweight, and exhibits enhanced generalization. By refining traditional transfer learning, the source model is segmented into public and private feature extractors for stepwise transfer, enhancing parameter alignment with specific target tasks. Neuron importance is then sorted based on the task of each feature extractor, enabling precise pruning to create an optimal lightweight model. An adversarial network guiding principle is adopted for retraining the public feature extractor parameters, thus strengthening the model’s generalization power. This method achieves an accuracy of over 97% on few-shot datasets with no more than 15 samples per class, has fewer than 50 K model parameters, and exhibits superior generalization compared to baseline methods.

Multi-domain improves classification in out-of-distribution and data-limited scenarios for medical image analysis

Current machine learning methods for medical image analysis primarily focus on developing models tailored for their specific tasks, utilizing data within their target domain. These specialized models tend to be data-hungry and often exhibit limitations in generalizing to out-of-distribution samples. In this work, we show that employing models that incorporate multiple domains instead of specialized ones significantly alleviates the limitations observed in specialized models. We refer to this approach as multi-domain model and compare its performance to that of specialized models. For this, we introduce the incorporation of diverse medical image domains, including different imaging modalities like X-ray, MRI, CT, and ultrasound images, as well as various viewpoints such as axial, coronal, and sagittal views. Our findings underscore the superior generalization capabilities of multi-domain models, particularly in scenarios characterized by limited data availability and out-of-distribution, frequently encountered in healthcare applications. The integration of diverse data allows multi-domain models to utilize information across domains, enhancing the overall outcomes substantially. To illustrate, for organ recognition, multi-domain model can enhance accuracy by up to 8% compared to conventional specialized models.

PST-Diff: Achieving High-Consistency Stain Transfer by Diffusion Models With Pathological and Structural Constraints.

Histopathological examinations heavily rely on hematoxylin and eosin (HE) and immunohistochemistry (IHC) staining. IHC staining can offer more accurate diagnostic details but it brings significant financial and time costs. Furthermore, either re-staining HE-stained slides or using adjacent slides for IHC may compromise the accuracy of pathological diagnosis due to information loss. To address these challenges, we develop PST-Diff, a method for generating virtual IHC images from HE images based on diffusion models, which allows pathologists to simultaneously view multiple staining results from the same tissue slide. To maintain the pathological consistency of the stain transfer, we propose the asymmetric attention mechanism (AAM) and latent transfer (LT) module in PST-Diff. Specifically, the AAM can retain more local pathological information of the source domain images, while ensuring the model's flexibility in generating virtual stained images that highly confirm to the target domain. Subsequently, the LT module transfers the implicit representations across different domains, effectively alleviating the bias introduced by direct connection and further enhancing the pathological consistency of PST-Diff. Furthermore, to maintain the structural consistency of the stain transfer, the conditional frequency guidance (CFG) module is proposed to precisely control image generation and preserve structural details according to the frequency recovery process. To conclude, the pathological and structural consistency constraints provide PST-Diff with effectiveness and superior generalization in generating stable and functionally pathological IHC images with the best evaluation score. In general, PST-Diff offers prospective application in clinical virtual staining and pathological image analysis.

A virtual calibration chamber for cone penetration test based on deep-learning approaches

The interpretation of the cone penetration test (CPT) still relies largely on empirical correlations that have been predominantly developed in resource-intensive and time-consuming calibration chambers. This paper presents a CPT virtual calibration chamber using deep learning (DL) approaches, which allow for the consideration of depth-dependent cone resistance profiles through the implementation of two proposed strategies: (1) depth-resistance mapping using a multilayer perceptron (MLP) and (2) sequence-to-sequence training using a long short-term memory (LSTM) neural network. Two DL models are developed to predict cone resistance profiles (qc) under various soil states and testing conditions, where Bayesian optimization (BO) is adopted to identify the optimal hyperparameters. Subsequently, the BO-MLP and BO-LSTM networks are trained using the available data from published datasets. The results show that the models with BO can effectively improve the prediction accuracy and efficiency of neural networks compared to those without BO. The two training strategies yielded comparable results in the testing set, and both can be used to reproduce the whole cone resistance profile. An extended comparison and validation of the prediction results are carried out against numerical results obtained from a coupled Eulerian-Lagrangian (CEL) model, demonstrating a high degree of agreement between the DL and CEL models. Ultimately, to demonstrate the usability of this new virtual calibration chamber, the predicted qc is used to enhance the preceding correlations with the relative density (Dr) of the sand. The improved correlation with superior generalization has an R2 of 82% when considering all data, and 89.6% when examining the pure experimental data.

Underwater image restoration via spatially adaptive polarization imaging and color correction

Polarization imaging is extensively employed in underwater image restoration owing to its effectiveness in eliminating backscattered light. However, the existing polarization-based imaging methods rely on the strong assumption that the degrees of polarization (DoPs) of the target and backscattered light remain constant. Additionally, these methods ignore wavelength-specific absorption phenomena. To address these challenges, we propose an underwater image restoration method via spatially adaptive polarization imaging and color correction. First, based on optimal polarization difference images, we propose a method for calculating the spatial DoP of the target. In this method, the polarization characteristics of the target and background are fully exploited, and a scoring function is designed to achieve an optimal differential image. Second, we propose an adaptive local optimization method to estimate the spatial DoP of backscattering, which involves dividing the image into multiple local regions, and then employing improved particle swarm optimization (IPSO) algorithms to obtain the optimal value of DoP for each region. Finally, we introduce an adaptive compensation method based on the channel with minimal color absorption to correct color shifts during underwater polarization imaging. Extensive experiments demonstrate that our method exhibits superior generalization capability and outstanding restoration performance compared to existing methods.

Physical prior-guided deep learning for SIM reconstruction: modeling object-to-image degradation

Structured illumination microscopy (SIM) provides an enhanced spatial resolution of up to twice the conventional capacity. Recently, many approaches have attempted to combine deep learning frameworks with SIM reconstruction for better capability. Nonetheless, the inadequacy of training samples highlights the challenge of limited credibility and low generalization ability of deep learning, thus significantly constraining the application in biology. To tackle this issue, we propose an object-to-image plane degradation network (OIDN) guided by the physical process of optical imaging. Specifically, the proposed OIDN embeds the object-to-image plane degradation process into the reconstruction network to provide explicit guidance. With a set of learnable point spread function (PSF) parameters constrained by physical prior, OIDN successfully converts the conventional image-to-image data pattern mapping into the object-to-image plane degradation mapping that highly aligns with the optical processes of SIM imaging. Comprehensive experiments demonstrate that the proposed method reliably yields high-quality images across signal-to-noise ratio conditions, exhibiting superior generalization ability across diverse datasets and sample types.

Identification modeling of ship manoeuvring motion based on ISSA-SVR

This paper introduces the sparrow search algorithm–support vector regression (SSA-SVR) for the first time to improve the precision of mathematical modelling for ship motion through parameter identification. To address the nonsmoothness in ship motion, this study proposes a multistrategy enhanced modelling method named the improved SSA-SVR (ISSA-SVR) for predicting ship motion. Using the KRISO container ship (KCS) test data, this study establishes first-order linear response, second-order nonlinear response, and manoeuvring model group mathematical models through ISSA-SVR identification. The generalisability of the models derived from support vector regression, backpropagation neural networks, particle swarm optimisation-SVR, SSA-SVR, and ISSA-SVR is validated using distinct data from different samples. The results show that the algorithmic model proposed herein excels in prediction compared with other algorithms, thus highlighting its superior generalisation. Furthermore, the proposed method offers solutions with increased accuracy and improved stability for modelling ship motion.

DELR-Net: a network for 3D multimodal medical image registration in more lightweight application scenarios.

3D multimodal medical image deformable registration plays a significant role in medical image analysis and diagnosis. However, due to the substantial differences between images of different modalities, registration is challenging and requires high computational costs. Deep learning-based registration methods face these challenges. The primary aim of this paper is to design a 3D multimodal registration network that ensures high-quality registration results while reducing the number of parameters. This study designed a Dual-Encoder More Lightweight Registration Network (DELR-Net). DELR-Net is a low-complexity network that integrates Mamba and ConvNet. The State Space Sequence Module and the Dynamic Large Kernel block are used as the main components of the dual encoders, while the Dynamic Feature Fusion block is used as the main component of the decoder. This study conducted experiments on 3D brain MR images and abdominal MR and CT images. Compared to existing registration methods, DELR-Net achieved better registration results while maintaining a lower number of parameters. Additionally, generalization experiments on other modalities showed that DELR-Net has superior generalization capabilities. DELR-Net significantly improves the limitations of 3D multimodal medical image deformable registration, achieving better registration performance with fewer parameters.

Foundation models in robotics: Applications, challenges, and the future

We survey applications of pretrained foundation models in robotics. Traditional deep learning models in robotics are trained on small datasets tailored for specific tasks, which limits their adaptability across diverse applications. In contrast, foundation models pretrained on internet-scale data appear to have superior generalization capabilities, and in some instances display an emergent ability to find zero-shot solutions to problems that are not present in the training data. Foundation models may hold the potential to enhance various components of the robot autonomy stack, from perception to decision-making and control. For example, large language models can generate code or provide common sense reasoning, while vision-language models enable open-vocabulary visual recognition. However, significant open research challenges remain, particularly around the scarcity of robot-relevant training data, safety guarantees and uncertainty quantification, and real-time execution. In this survey, we study recent papers that have used or built foundation models to solve robotics problems. We explore how foundation models contribute to improving robot capabilities in the domains of perception, decision-making, and control. We discuss the challenges hindering the adoption of foundation models in robot autonomy and provide opportunities and potential pathways for future advancements. The GitHub project corresponding to this paper can be found here: https://github.com/robotics-survey/Awesome-Robotics-Foundation-Models .

Synthetic data at scale: a development model to efficiently leverage machine learning in agriculture.

The rise of artificial intelligence (AI) and in particular modern machine learning (ML) algorithms during the last decade has been met with great interest in the agricultural industry. While undisputedly powerful, their main drawback remains the need for sufficient and diverse training data. The collection of real datasets and their annotation are the main cost drivers of ML developments, and while promising results on synthetically generated training data have been shown, their generation is not without difficulties on their own. In this paper, we present a development model for the iterative, cost-efficient generation of synthetic training data. Its application is demonstrated by developing a low-cost early disease detector for tomato plants (Solanum lycopersicum) using synthetic training data. A neural classifier is trained by exclusively using synthetic images, whose generation process is iteratively refined to obtain optimal performance. In contrast to other approaches that rely on a human assessment of similarity between real and synthetic data, we instead introduce a structured, quantitative approach. Our evaluation shows superior generalization results when compared to using non-task-specific real training data and a higher cost efficiency of development compared to traditional synthetic training data. We believe that our approach will help to reduce the cost of synthetic data generation in future applications.

Classification of cervical vertebral maturation stages with machine learning models: leveraging datasets with high inter- and intra-observer agreement

ObjectivesThis study aimed to assess the accuracy of machine learning (ML) models with feature selection technique in classifying cervical vertebral maturation stages (CVMS). Consensus-based datasets were used for models training and evaluation for their model generalization capabilities on unseen datasets.MethodsThree clinicians independently rated CVMS on 1380 lateral cephalograms, resulting in the creation of five datasets: two consensus-based datasets (Complete Agreement and Majority Voting), and three datasets based on a single rater’s evaluations. Additionally, landmarks annotation of the second to fourth cervical vertebrae and patients’ information underwent a feature selection process. These datasets were used to train various ML models and identify the top-performing model for each dataset. These models were subsequently tested on their generalization capabilities.ResultsFeatures that considered significant in the consensus-based datasets were consistent with a CVMS guideline. The Support Vector Machine model on the Complete Agreement dataset achieved the highest accuracy (77.4%), followed by the Multi-Layer Perceptron model on the Majority Voting dataset (69.6%). Models from individual ratings showed lower accuracies (60.4–67.9%). The consensus-based training models also exhibited lower coefficient of variation (CV), indicating superior generalization capability compared to models from single raters.ConclusionML models trained on consensus-based datasets for CVMS classification exhibited the highest accuracy, with significant features consistent with the original CVMS guidelines. These models also showed robust generalization capabilities, underscoring the importance of dataset quality.