Generalization Performance Of Model Research Articles

Active learning concerning sampling cost for enhancing AI-enabled building energy system modeling

Machine learning is widely recognized as a promising data-driven modeling technique for the model-based control and optimization of building energy systems. However, the generalizability of data-driven models often faces significant challenges, as the available training data from building operations usually only covers a limited range of working conditions. Active learning can proactively test unseen and informative working conditions to enrich training set by adding new data samples, leading to improved generalization performance of data-driven models. A novel distance and information density-based sample strategy is developed which accounts for the real-time status of building operation and outdoor environment. Based on Mahalanobis distance, this strategy determines the sampling value of an unlabeled sample (unseen working condition) by assessing its similarity to both the training samples and other unlabeled samples. As collecting sufficiently representative samples can be difficult, costly and time-consuming, a distance-based sampling cost metric is proposed to compare the efficiency of different sampling methods, considering the detrimental effects of the actively sampling process on the normal operation of building energy systems. This paper presents a comprehensive and in-depth comparison on five active learning methods, including one incorporating the distance-based sampling strategy, by conducting data experiments on the data collected from the cooling towers of a real high-rise building. The results show that active learning can effectively identify informative data samples and improve the generalization performance of data-driven models. The research outcomes are valuable for enhancing AI-enabled data-driven modeling of building energy systems with substantial decreases in costs on data sampling.

A multipoint prediction model for the deformation of concrete dams considering climatic features of high-altitude regions

High-altitude regions are characterized by natural climatic features such as low temperatures and high radiation. Concrete dams built in these regions face more complex operating conditions. Traditional single-output deformation prediction models ignore the correlations between different monitoring points, making it difficult to assess the overall deformation behavior of concrete dams. The black-box nature of some machine learning models can lead to a lack of interpretability, affecting the practical application. Therefore, a multipoint prediction model (MPM) is proposed to analyze the deformation behavior of concrete dams in high-altitude regions. Firstly, the deformation monitoring points are divided based on the climatic features of high-altitude regions. Subsequently, the MPM is constructed using the multi-output CatBoost model and Quasi-Monte Carlo algorithm to achieve deformation prediction across different partitioned monitoring points. Finally, the elucidation of the MPM optimization procedure in a transparent manner alongside the quantification of the influence exerted by input variables on output outcomes notably augments the interpretive capacity of the MPM. The proposed model is validated using monitoring data from a concrete dam in a high-altitude region. The disparities in chaotic and entropy variation features among deformation monitoring points in different zones corroborate the appropriateness of the zoning in this study. Comparisons between the single-output CatBoost model and MPM verify the feasibility and efficiency of multipoint predictions. Comparisons with other multi-output prediction models validate the superior predictive performance of the MPM. Additionally, the proposed model's generalization performance is validated using data from another concrete arch dam in high-altitude regions.

1D -FNO - Residual convolutional neural network autoregressive prediction model for deep water wave train evolution based on high-order spectral (HOS) data

Accurate prediction of deep-water wave dynamics is essential for ocean engineering, meteorology, and maritime safety. Traditional physical models, though effective, face significant computational and time challenges with complex nonlinear dynamics. This study introduces an autoregressive prediction model using a one-dimensional adaptive Fourier Neural Operator-Residual Convolutional Neural Network (1D-FNO-ResNet) to enhance prediction accuracy. We developed a high-precision single-step prediction model capable of capturing wave characteristics such as height and fluid particle velocity. A fine-tuning training strategy based on multi-step prediction errors was proposed for autoregressive evolution. Performance comparisons between the ResNet and FNO-ResNet models were made under various fine-tuning steps. A correction model was also introduced to control error accumulation, improving autoregressive prediction steps. The model's generalization performance was validated with modulated and random wave trains under different conditions, optimizing predictive stability and accuracy. This approach demonstrates efficient 1D deep-water wave evolution prediction, offering a novel deep learning solution for wave dynamics prediction in ocean engineering, supporting wave energy conversion device design and optimization.

Collaborative-driven reservoir formation pressure prediction using GAN-ML models and well logging data

Formation pressure is the foundation and prerequisite for achieving efficient drilling and risk control in complex geological formations. However, intelligent prediction of formation pressure faces challenges such as insufficient sample sizes, poor prediction performance, and low prediction accuracy, which severely restrict the realization of safe and efficient drilling. To address this, this paper proposes a new method based on Generative Adversarial Networks (GAN) and Machine Learning (ML), aiming to improve adaptation to complex geological conditions and effectively capture the nonlinear relationship between logging data and formation pressure for intelligent prediction of formation pressure. The research results show that the GAN technology successfully amplified the 125 cable-measured formation pressure data by 204.8 times. Compared to machine learning predictions without using GAN technology, the constructed GAN-ML model reduced the average root mean square error by 0.59 MPa and the average absolute error by 0.71 MPa in the training set. The GAN-ML model predicted the formation pressure of adjacent wells with an average root mean square error of 1.59 MPa and an average absolute error of 1.96 MPa compared to the cable-measured formation pressure values. This achievement significantly enhances the robustness and generalization performance of intelligent models, demonstrating the potential of GAN-ML in addressing small sample data problems. It opens a new technical path for data-driven intelligent prediction and provides innovative data analysis and prediction methods for the field of geological exploration.

A radial basis deformable residual convolutional neural model embedded with local multi-modal feature knowledge and its application in cross-subject classification

In the spatial cognition and emotion recognition tasks based on electroencephalography (EEG), the signal information representation of the single modal is incomplete because of the significant inter-subject differences in the EEG signals, resulting in low generalization performance of classification models. To address this issue, we propose a radial basis deformable residual convolutional neural networks model embedded with local multi-modal feature knowledge (RBDRCNN-LMFK). The RBDRCNN leverages Euclidean distance alignment, deformable convolution, depth-separable convolution, and residual connection modules for effective EEG feature extraction. The embedded local feature knowledge algorithm enables the effective combination of multi-modal information. To further validate the effectiveness of the algorithm, we used the left-one-subject-out cross-validation algorithm on the virtual city walking (VCW), SJTU Emotion EEG Dataset IV (SEED IV), and Building block gesture recognition (BBGR) datasets for spatial cognition and emotion recognition tasks. The average accuracy of the VCW was 97.84 % using the kNN classifier, surpassing the Concate fusion method’s 96.62 %. The average accuracy for the SEED IV dataset was 87.56 %, higher than the Concate Fusion’s 69.97 %. On the BBGR dataset, the kNN classifier achieved an average accuracy of 87.80 %, compared to 85.31 % with the Concate fusion. The results show that the model enhances the recognition of biologically significant features in EEG signals by embedding local features and increases the correlation between different brain regions associated with the task. This work illustrates a promising direction of using deep learning models to discover effective task-related features from highly diverse EEG signals and enhance brain regions’ correlation through multi-modal knowledge embedding.

Enhancing Robot Learning with Transformer-based Morphology Modeling

Abstract The transformer model has made significant progress in various areas through large-scale training. In contrast, the traditional robot performs a single task, and there is an issue with migrating the strategic model. In this study, a Robot Morphology Learning (RML) method is proposed to enhance efficiency and generalization performance by learning multiple tasks in a transformer model. RML constructs the robot’s morphology as a graph and utilizes a graph neural network to handle graphs of arbitrary connections and sizes, addressing the disparity in state and action space dimensions. RML breaks through the limitation of non-migration of models, realizes efficient training, and improves the generalization performance of models, enabling quick adaptation to new tasks. Experimental results show that the proposed method outperforms previous methods in both multi-task learning and transfer learning experiments.

Classification of recurrent depression using brain CT images through feature fusion

ObjectiveThe objective of the study is to address the issues of high workload and low diagnostic efficiency in clinical medicine by employing computer-aided diagnostic analysis methods. The focus is on the identification of depressed patients using medical image data, particularly brain CT images. MethodsThe study proposes a CT image classification method for identifying depression disorder based on deep learning theory. Key methods include migration learning and feature fusion. Preprocessing and data enhancement techniques are utilized to filter out unnecessary features. An attention module is employed to better extract deep feature information. Additionally, a binary classification focus loss function is applied to address the unbalanced distribution of the dataset. ResultsExperimental results indicate that the proposed method achieves an image classification accuracy of 97.79%. Compared to a single model, there is an increase in accuracy by 2.61% and 1.81%, respectively. This improvement effectively enhances the classification accuracy of brain CT images with depressive disorders. The classification model also demonstrates better generalization performance. ConclusionThe study concludes that the proposed CT image classification method, incorporating migration learning, feature fusion, preprocessing, and attention modules, significantly improves the accuracy and classification speed of identifying depressive disorders in brain CT images. The model's generalization performance is highlighted, providing effective support for the enhancement of medical image classification accuracy.

DiffuPrompter: Pixel-Level Automatic Annotation for High-Resolution Remote Sensing Images with Foundation Models

Instance segmentation is pivotal in remote sensing image (RSI) analysis, aiding in many downstream tasks. However, annotating images with pixel-wise annotations is time-consuming and laborious. Despite some progress in automatic annotation, the performance of existing methods still needs improvement due to the high precision requirements for pixel-level annotation and the complexity of RSIs. With the support of large-scale data, some foundational models have made significant progress in semantic understanding and generalization capabilities. In this paper, we delve deep into the potential of the foundational models in automatic annotation and propose a training-free automatic annotation method called DiffuPrompter, achieving pixel-level automatic annotation of RSIs. Extensive experimental results indicate that the proposed method can provide reliable pseudo-labels, significantly reducing the annotation costs of the segmentation task. Additionally, the cross-domain validation experiments confirm the powerful effectiveness of large-scale pseudo-data in improving model generalization performance.

A multi-scale multi-channel CNN introducing a channel-spatial attention mechanism hyperspectral remote sensing image classification method

ABSTRACT Aiming the problems that the classification performance of hyperspectral images in existing classification algorithms is highly dependent on spatial-spectral information and that detailed features are ignored in single convolutional channel feature extraction, resulting in poor generalization performance of the feature extraction model, a multi-scale multi-channel convolutional neural network (MMC-CNN) model is proposed in this paper. First, the data set is divided into two kinds of pixel module, and then different channels are used for feature extraction. A channel-space attention mechanism module is also introduced, and a multi-scale multichannel convolutional neural network (CSAM-MMC) model with the introduction of channel-space attention mechanism is proposed for deeper spatial-spectral feature extraction of hyperspectral image elements while reducing the redundancy of convolutional pooling parameters to achieve better HSI classification. To evaluate the effectiveness of the proposed model, experiments were conducted on Indian Pines, Pavia Center and KSC datasets respectively, and the overall classification accuracies of this paper’s MMC-CNN model in the HSI dataset were 97.23%, 98.50%, and 97.85%, thus verifying the model’s high feature extraction accuracy. The CSAM-MMC model in this paper further improves 0.13%, 0.35%, and 0.71% relative to the MMC-CNN model, which provides higher overall accuracies relative to other state-of-the-art algorithms.

An intelligent MRI assisted diagnosis and treatment system for osteosarcoma based on super-resolution

AbstractMagnetic resonance imaging (MRI) examinations are a routine part of the cancer treatment process. In developing countries, disease diagnosis is often time-consuming and associated with serious prognostic problems. Moreover, MRI is characterized by high noise and low resolution. This creates difficulties in automatic segmentation of the lesion region, leading to a decrease in the segmentation performance of the model. This paper proposes a deep convolutional neural network osteosarcoma image segmentation system based on noise reduction and super-resolution reconstruction, which is the first time to introduce super-resolution methods in the task of osteosarcoma MRI image segmentation, effectively improving the Model generalization performance. We first refined the initial osteosarcoma dataset using a Differential Activation Filter, separating those image data that had little effect on model training. At the same time, we carry out rough initial denoising of the image. Then, an improved information multi-distillation network based on adaptive cropping is proposed to reconstruct the original image and improve the resolution of the image. Finally, a high-resolution network is used to segment the image, and the segmentation boundary is optimized to provide a reference for doctors. Experimental results show that this algorithm has a stronger segmentation effect and anti-noise ability than existing methods. Code: https://github.com/GFF1228/NSRDN.

An open flame and smoke detection dataset for deep learning in remote sensing based fire detection

ABSTRACT The early warning of fires is pivotal for preventing substantial economic losses and ecological damage. However, the enhancement of fire model generalization performance still faces challenges due to limitations in existing fire datasets, such as image quantity and heterogeneity. Taking the advantage of rapid advancements in remote sensing technology and artificial intelligence, we introduce the Flame And Smoke Detection Dataset (FASDD), a pioneering collection comprising over 120,000 heterogeneous images covering various fire scenarios, aiming to promote the evolution of fire detection models. FASDD serves as a challenging benchmark for object detection tasks and features three sub-datasets: FASDD_CV, FASDD_UAV, and FASDD_RS. These sub-datasets comprise cross-domain images from ground-based, airborne, and spaceborne sensors, respectively. Extensive experiments using advanced Swin Transformer models demonstrate satisfactory fire detection performance, with mAP scores of 84.9%, 89.7%, and 74.0% on the respective sub-datasets. Notably, the application case in wildfire localization highlights the proficiency of FASDD_RS models for monitoring and tracking forest fires. Trained on FASDD_RS, FASDD_UAV, and FASDD_CV, deep learning models can be individually deployed on edge devices including satellites, drones, and terrestrial sensors, thus empowering collaborative fire warning within the space-air-ground integrated observation network. Additionally, a standardized procedure for data preparation, data procedure, and quality assurance is established, providing off-the-shelf annotations with four common configurations to facilitate machine learning model training. FASDD stands as a pivotal benchmark, providing valuable resources for training and evaluating deep learning fire detection models, especially Large Vision Models (LVM). It holds the potential to accelerate research progress in urban firefighting or forest fire detection, providing vital security measures for early fire warning and emergency response tasks.

Contralateral Hypertrophy Post Yttrium-90 Transarterial Radioembolization in Patients With Hepatocellular Carcinoma and Portal Vein Tumor Thrombus.

Objectives Contralateral hypertrophy of non-irradiated liver following Yttrium-90 (90Y) transarterial radioembolization (TARE) is increasingly recognized as an option to facilitate curative surgical resection in patients that would otherwise not be surgical candidates due to a small future liver remnant (FLR). This study aimed to investigate the correlation between patient features and liver hypertrophy and identify potential predictors for liver growth in patients with hepatocellular carcinoma (HCC) and portal vein tumor thrombus (PVTT) undergoing TARE. Methodology Twenty-three patients with HCC and PVTT were included. Contralateral liver hypertrophy was assessed at six months posttreatment based on CT or MRI imaging. Thirteen patient features were selected for statistical and prediction analysis. Univariate Spearman correlation and analysis of variance (ANOVA) tests were performed. Subsequently, four feature-selection methods based on multivariate analysis were used to improve model generalization performance. The selected features were applied to train linear regression models, with fivefold cross-validation to assess the performance of the predicted models. Results The ratio of disease-free target liver volume to spared liver volume and total liver volume showed the highest correlations with contralateral hypertrophy (P-values = 0.03 and 0.05, respectively). In three out of four feature-selection methods, the feature of disease-free target liver volume to total liver volume ratio was selected, having positive correlations with the outcome and suggesting that more hypertrophy may be expected when more volume of disease-free liver is irradiated. Conclusions Contralateral hypertrophy post-90Y TARE can be an option for facilitating surgical resection in patients with otherwise small FLR.

G-YOLO: A YOLOv7-based target detection algorithm for lightweight hazardous chemical vehicles.

Hazardous chemical vehicles are specialized vehicles used for transporting flammable gases, medical waste, and liquid chemicals, among other dangerous chemical substances. During their transportation, there are risks of fire, explosion, and leakage of hazardous materials, posing serious threats to human safety and the environment. To mitigate these possible hazards and decrease their probability, this study proposes a lightweight object detection method for hazardous chemical vehicles based on the YOLOv7-tiny model.The method first introduces a lightweight feature extraction structure, E-GhostV2 network, into the trunk and neck of the model to achieve effective feature extraction while reducing the burden of the model. Additionally, the PConv is used in the model's backbone to effectively reduce redundant computations and memory access, thereby enhancing efficiency and feature extraction capabilities. Furthermore, to address the problem of performance degradation caused by overemphasizing high-quality samples, the model adopts the WIoU loss function, which balances the training effect of high-quality and low-quality samples, enhancing the model's robustness and generalization performance. Experimental results demonstrate that the improved model achieves satisfactory detection accuracy while reducing the number of model parameters, providing robust support for theoretical research and practical applications in the field of hazardous chemical vehicle object detection.

Sex classification from functional brain connectivity: Generalization to multiple datasets.

Machine learning (ML) approaches are increasingly being applied to neuroimaging data. Studies in neuroscience typically have to rely on a limited set of training data which may impair the generalizability of ML models. However, it is still unclear which kind of training sample is best suited to optimize generalization performance. In the present study, we systematically investigated the generalization performance of sex classification models trained on the parcelwise connectivity profile of either single samples or compound samples of two different sizes. Generalization performance was quantified in terms of mean across-sample classification accuracy and spatial consistency of accurately classifying parcels. Our results indicate that the generalization performance of parcelwise classifiers (pwCs) trained on single dataset samples is dependent on the specific test samples. Certain datasets seem to "match" in the sense that classifiers trained on a sample from one dataset achieved a high accuracy when tested on the respected other one and vice versa. The pwCs trained on the compound samples demonstrated overall highest generalization performance for all test samples, including one derived from a dataset not included in building the training samples. Thus, our results indicate that both a large sample size and a heterogeneous data composition of a training sample have a central role in achieving generalizable results.

The optimization of evaporation rate in graphene-water system by machine learning algorithm

Solar interfacial evaporation, as a novel practical freshwater production method, requires continuous research on how to improve the evaporation rates to increase water production. In this study, sets of data were obtained from molecule dynamics simulation and literature, in which the parameters included height, diameter, height–radius ratio, evaporation efficiency, and evaporation rate. Initially, the correlation between the four input parameters and the output of the evaporation rate was examined through traditional pairwise plots and Pearson correlation analysis, revealing weak correlations. Subsequently, the accuracy and generalization performance of the evaporation rate prediction models established by neural network and random forest were compared, with the latter demonstrating superior performance and reliability confirmed via random data extraction. Furthermore, the impact of different percentages (10%, 20%, and 30%) of the data on the model performance was explored, and the result indicated that the model performance is better when the test set is 20% and all the constructed model converge. Moreover, the mean absolute error and mean squared error of the evaporation rate prediction model for the three ratios were calculated to evaluate their performance. However, the relationship between the height- radius ratio and optimal evaporation rate was investigated using the enumeration method, and it was determined that the evaporation efficiency was optimal when the height–radius ratio was 6. Finally, the importance of height, diameter, height– radius ratio, and evaporation efficiency were calculated to optimize evaporator structure, increase evaporation rate, and facilitate the application of interfacial evaporation in solar desalination.

Prompt guidance query with cascaded constraint decoders for human–object interaction detection

AbstractHuman–object interaction (HOI) detection, which localises and recognises interactions between human and object, requires high‐level image and scene understanding. Recent methods for HOI detection typically utilise transformer‐based architecture to build unified future representation. However, these methods use random initial queries to predict interactive human–object pairs, leading to a lack of prior knowledge. Furthermore, most methods provide unified features to forecast interactions using conventional decoder structures, but they lack the ability to build efficient multi‐task representations. To address these problems, we propose a novel two‐stage HOI detector called PGCD, mainly consisting of prompt guidance query and cascaded constraint decoders. Firstly, the authors propose a novel prompt guidance query generation module (PGQ) to introduce the guidance‐semantic features. In PGQ, the authors build visual‐semantic transfer to obtain fuller semantic representations. In addition, a cascaded constraint decoder architecture (CD) with random masks is designed to build fine‐grained interaction features and improve the model's generalisation performance. Experimental results demonstrate that the authors’ proposed approach obtains significant performance on the two widely used benchmarks, that is, HICO‐DET and V‐COCO.

Stable Unlearnable Example: Enhancing the Robustness of Unlearnable Examples via Stable Error-Minimizing Noise

The open sourcing of large amounts of image data promotes the development of deep learning techniques. Along with this comes the privacy risk of these image datasets being exploited by unauthorized third parties to train deep learning models for commercial or illegal purposes. To avoid the abuse of data, a poisoning-based technique, "unlearnable example", has been proposed to significantly degrade the generalization performance of models by adding imperceptible noise to the data. To further enhance its robustness against adversarial training, existing works leverage iterative adversarial training on both the defensive noise and the surrogate model. However, it still remains unknown whether the robustness of unlearnable examples primarily comes from the effect of enhancement in the surrogate model or the defensive noise. Observing that simply removing the adversarial perturbation on the training process of the defensive noise can improve the performance of robust unlearnable examples, we identify that solely the surrogate model's robustness contributes to the performance. Furthermore, we found a negative correlation exists between the robustness of defensive noise and the protection performance, indicating defensive noise's instability issue. Motivated by this, to further boost the robust unlearnable example, we introduce Stable Error-Minimizing noise (SEM), which trains the defensive noise against random perturbation instead of the time-consuming adversarial perturbation to improve the stability of defensive noise. Through comprehensive experiments, we demonstrate that SEM achieves a new state-of-the-art performance on CIFAR-10, CIFAR-100, and ImageNet Subset regarding both effectiveness and efficiency.

TDLC: Tensor decomposition‐based direct learning‐compression algorithm for DNN model compression

SummaryAs a deep neural networks (DNNs) model compression method, learning‐compression (LC) algorithm based on pre‐trained models and matrix decomposition increases training time and ignores the structural information of models. In this manuscript, a tensor decomposition‐based direct LC (TDLC) algorithm without pre‐trained models is proposed. In TDLC, the pre‐trained model is eliminated, and tensor decomposition is first applied to LC algorithm to preserve the structural features of the model. There are two key steps in TDLC. An optimal rank selection method is first proposed in compression‐step (C‐step) of TDLC to find global optimal ranks of tensor decomposition. Second, TDLC utilizes cyclical learning rate, which is different from traditional monotonically learning rates schedule, to improve the generalization performance of uncompressed models in learning‐step (L‐step). TDLC obtains the optimal compression model by alternately optimizing L‐step and C‐step. TDLC is compared with 16 state‐of‐the‐art compression methods in experiments part. Extensive experimental results show that TDLC produces high‐accuracy compression models with high compression rate. Comparing with TDLC‐pre‐trained, TDLC notably achieves 30% training time shorten and 11% parameter reduction in Resnet32, while improving accuracy by 0.2%.

Generalization challenges inelectrocardiogram deep learning: insights from dataset characteristics and attention mechanism.

Aim: Deep learning's widespread use prompts heightened scrutiny, particularly in the biomedical fields, with a specific focus on model generalizability. This study delves into the influence of training data characteristics on the generalization performance of models, specifically in cardiac abnormality detection. Materials & methods: Leveraging diverse electrocardiogram datasets, models are trained on subsets with varying characteristics and subsequently compared for performance. Additionally, the introduction of the attention mechanism aims to improve generalizability. Results: Experiments reveal that using a balanced dataset, just 1% of a large dataset, leads to equal performance in generalization tasks, notably in detecting cardiology abnormalities. Conclusion: This balanced training data notably enhances model generalizability, while the integration of the attention mechanism further refines the model's ability to generalize effectively.

Vocoder Detection of Spoofing Speech Based on GAN Fingerprints and Domain Generalization

As an important part of the text-to-speech (TTS) system, vocoders convert acoustic features into speech waveforms. The difference in vocoders is key to producing different types of forged speech in the TTS system. With the rapid development of general adversarial networks (GANs), an increasing number of GAN vocoders have been proposed. Detectors often encounter vocoders of unknown types, which leads to a decline in the generalization performance of models. However, existing studies lack research on detection generalization based on GAN vocoders. To solve this problem, this study proposes vocoder detection of spoofed speech based on GAN fingerprints and domain generalization. The framework can widen the distance between real speech and forged speech in feature space, improving the detection model’s performance. Specifically, we utilize a fingerprint extractor based on an autoencoder to extract GAN fingerprints from vocoders. We then weight them to the forged speech for subsequent classification to learn the forged speech features with high differentiation. Subsequently, domain generalization is used to further improve the generalization ability of the model for unseen forgery types. We achieve domain generalization using domain-adversarial learning and asymmetric triplet loss to learn a better generalized feature space in which real speech is compact and forged speech synthesized by different vocoders is dispersed. Finally, to optimize the training process, curriculum learning is used to dynamically adjust the contributions of the samples with different difficulties in the training process. Experimental results show that the proposed method achieves the most advanced detection results among four GAN vocoders. The code is available at https://github.com/multimedia-infomation-security/GAN-Vocoder-detection .