Data Augmentation Strategy Research Articles

Seg2Tunnel: A hierarchical point cloud dataset and benchmarks for segmentation of segmental tunnel linings

Point clouds provide a novel and effective alternative to understanding the structural behaviours of segmental tunnel linings. 3D deep learning (DL) has emerged as a promising technology capable of automatically deriving point-wise semantic and instance labels from point clouds. The utilisation of 3D DL in segment segmentation of tunnel point clouds has not been explored and the development of tailored 3D DL networks has been hindered by the absence of specialised datasets and benchmarks. To bridge this gap, this paper introduces a richly annotated hierarchical dataset: ‘Seg2Tunnel’, acquired from five tunnels and including 1,300 tunnel rings. Using the Seg2Tunnel dataset, the feasibility of applying 3D DL to the segment segmentation is demonstrated for the first time. Experiments are conducted to investigate the influences of training set size, data augmentation strategy, input size, and hyperparameter on the performance of trained 3D DL models and to provide benchmarks and insights for future uses of the Seg2Tunnel dataset. The 3D DL models trained by the Seg2Tunnel dataset outperform currently existing image- and voxel-based DL methods. The Seg2Tunnel dataset and benchmarks are fundamental in shaping the design of 3D DL networks tailored for tunnel point clouds. The study provides a novel paradigm for automatically understanding the tunnel structural elements in the point clouds, paving the way for unmanned construction and intelligent evaluation of segmental tunnel linings.

Cutout as augmentation in contrastive learning for detecting burn marks in plastic granules

Abstract. Plastic granules are a common delivery form for creating products in industries such as the plastic manufacturing, construction and automotive ones. In the corresponding sorting process of plastic granules, diverse defect types could appear. Burn marks, which potentially lead to weakened structural integrity of the plastic, are one of the most common types. Thus, plastic granules with burn marks should be filtered out during the sorting process. Artificial intelligence (AI)-based anomaly detection approaches are widely used in the field of visual-based sorting due to the higher accuracy and lower requirement of expert knowledge compared with classic rule-based algorithms (Chandola et al., 2009). In this contribution, a simple data augmentation strategy, cutout, is implemented as a way of simulating defects when combined with a contrastive learning-based methodology and is proven to improve the accuracy of the anomaly detection of burn marks. Different variants of cutout are also evaluated. Specifically, synthetic image data are used due to the lack of real data.

Offline Data-Driven Multiobjective Optimization Evolutionary Algorithm Based on Generative Adversarial Network

Usually, data-driven multi-objective optimiza -tion problems (DD-MOPs) are indirectly solved by evolutionary algorithms through the built surrogate model which is well-trained from sample data. However, in most DD-MOPs, only a few available data can be practicably collected from real engineering experiments due to the unaffordable cost and time. The key challenge in such a DD-MOP is to prevent the serious deterioration on the accuracy of the obtained approximate Pareto front. In this paper, two novel strategies, critical fitness for evolutionary algorithms and data augmentation for a surrogate model, are complementarily imposed by a generative adversarial network (GAN) to tackle with the challenges in DD-MOPs. In the critical fitness strategy, a new critical fitness, composed of the critical score from the discriminator of GAN and the prediction value of surrogate model, is proposed to improve the accuracy of approximate Pareto front of a DD-MOP. In the data augmentation strategy, some new samples are synthetized by the generator of GAN to build a better-trained surrogate model. As a result, the GAN concurrently serves the critical fitness strategy and the data augmentation strategy as the roles of “killing two birds with one stone”. The performance of the proposed algorithm for DD-MOPs was well-verified over 26 benchmark problems and successfully applied to discover new NdFeB materials.

Text data augmentation and pre-trained Language Model for enhancing text classification of low-resource languages.

In the domain of natural language processing (NLP), the development and success of advanced language models are predominantly anchored in the richness of available linguistic resources. Languages such as Azerbaijani, which is classified as a low-resource, often face challenges arising from limited labeled datasets, consequently hindering effective model training. The primary objective of this study was to enhance the effectiveness and generalization capabilities of news text classification models using text augmentation techniques. In this study, we solve the problem of working with low-resource languages using translations using the Facebook mBart50 model, as well as the Google Translate API and a combination of mBart50 and Google Translate thus expanding the capabilities when working with text. The experimental outcomes reveal a promising uptick in classification performance when models are trained on the augmented dataset compared with their counterparts using the original data. This investigation underscores the immense potential of combined data augmentation strategies to bolster the NLP capabilities of underrepresented languages. As a result of our research, we have published our labeled text classification dataset and pre-trained RoBERTa model for the Azerbaijani language.

SADSNet: A robust 3D synchronous segmentation network for liver and liver tumors based on spatial attention mechanism and deep supervision.

• Introduce a data augmentation strategy to expand the required different morphological data during the training and learning phase, and improve the algorithm's feature learning ability for complex and diverse tumor morphology CT images.• Design attention mechanisms for encoding and decoding paths to extract fine pixel level features, improve feature extraction capabilities, and achieve efficient spatial channel feature fusion.• The deep supervision layer is used to correct and decode the final image data to provide high accuracy of results.• The effectiveness of this method has been affirmed through validation on the LITS, 3DIRCADb, and SLIVER datasets. Accurately extracting liver and liver tumors from medical images is an important step in lesion localization and diagnosis, surgical planning, and postoperative monitoring. However, the limited number of radiation therapists and a great number of images make this work time-consuming. This study designs a spatial attention deep supervised network (SADSNet) for simultaneous automatic segmentation of liver and tumors. Firstly, self-designed spatial attention modules are introduced at each layer of the encoder and decoder to extract image features at different scales and resolutions, helping the model better capture liver tumors and fine structures. The designed spatial attention module is implemented through two gate signals related to liver and tumors, as well as changing the size of convolutional kernels; Secondly, deep supervision is added behind the three layers of the decoder to assist the backbone network in feature learning and improve gradient propagation, enhancing robustness. The method was testing on LITS, 3DIRCADb, and SLIVER datasets. For the liver, it obtained dice similarity coefficients of 97.03%, 96.11%, and 97.40%, surface dice of 81.98%, 82.53%, and 86.29%, 95% hausdorff distances of 8.96 mm, 8.26 mm, and 3.79 mm, and average surface distances of 1.54 mm, 1.19 mm, and 0.81 mm. Additionally, it also achieved precise tumor segmentation, which with dice scores of 87.81% and 87.50%, surface dice of 89.63% and 84.26%, 95% hausdorff distance of 12.96 mm and 16.55 mm, and average surface distances of 1.11 mm and 3.04 mm on LITS and 3DIRCADb, respectively. The experimental results show that the proposed method is effective and superior to some other methods. Therefore, this method can provide technical support for liver and liver tumor segmentation in clinical practice.

A feature‐enhanced hybrid attention network for traffic sign recognition in real scenes

AbstractCurrently, traffic sign recognition techniques have been brought into the assistive driving of automobiles. However, small traffic sign recognition in real scenes is still a challenging task due to the class imbalance issue and the size limit of the traffic signs. To address the above issues, a feature‐enhanced hybrid attention network is proposed based on YOLOv5s for a small, fast, and accurate traffic sign detector. First, a series of online data augmentation strategies are designed in the preprocessing module for the model training. Second, the hybrid channel and spatial attention module CSAM are integrated into the backbone for a better feature extraction ability. Third, the channel attention module CAM is used in the detection head for a more efficient feature fusion ability. To validate the approach, extensive experiments are conducted based on the Tsinghua‐Tencent 100K dataset. It is found that the novel method achieves state‐of‐the‐art performance with only negligible increases in the model parameter and computational overhead. Specifically, the , parameters, and FLOPs are 85.8%, 7.13 M, and 16.1 G, respectively.

CR-LCRP: Course recommendation based on Learner–Course Relation Prediction with data augmentation in a heterogeneous view

Course recommendation aims to offer suitable courses for learners and alleviate dropout issue in Massive Open Online Course (MOOC) learning scenarios. The present studies focus on learning semantic representations by modeling the learner’s personal information and the courses’ teaching attributes. However, with massive courses offered and various learners involved, the data sparsity and cold start are still two dominant challenges hindering the booming of online learning, which roughly results from the scarce exploitation on various relationships between learners and courses. To alleviate the above issues, we leverage the strong ability of Heterogeneous Information Network (HIN) in modeling heterogeneous data types and complex structural characteristics, and propose a Course Recommendation method based on Learner–Course Relation Prediction (CR-LCRP) with multi-granularity data augmentation strategy. Specifically, the multi-source interactive data on courses, the learner–learner similarity and the course–course similarity are integrated in building a heterogeneous information network. With both explicit and implicit features extracted and structural information from meta-paths enhanced, the CR-LCRP method learns high-quality representations of learners and courses, and eventually performs learner–course relation prediction in MOOC learning scenarios. Finally, extensive experiments are conducted on real-world MOOCs datasets and the experimental results demonstrate that the proposed CR-LCRP method outperforms the state-of-the-art baseline models with superior performance on several widely used evaluation metrics.

Breast cancer classification method based on improved VGG16 using mammography images

Background and objectiveBreast cancer has become the leading global cancer, and early detection and diagnosis of breast cancer are of paramount importance for treatment. MethodologyThis paper proposes a breast X-ray mammography image classification model based on Convolutional Neural Network (CNN). The model categorizes breast X-ray mammography images into benign and malignant classes. Built upon the VGG network, the model adjusts the network structure and conducts experiments on the dataset collected and organized by the Medical Imaging Department of Ganzhou People's Hospital and The Sixth Affiliated Hospital of Jinan University. To address the issue of imbalanced data in the dataset, the model employs a focal loss function for optimization and combines transfer learning and data augmentation strategies during network training. ResultsExperimental results demonstrate that the model achieves an average recognition rate of 96.945% across four different magnification levels. Notably, recognition rates exceed 95.5% for the 50X, 100X, and 200× magnification levels, demonstrating excellent classification capabilities. ConclusionThis model ignificantly improving classification accuracy compared to previous models, which provides meaningful insights into the classification of breast X-ray mammography images.

Toward Robustness in Multi-Label Classification: A Data Augmentation Strategy against Imbalance and Noise

Multi-label classification poses challenges due to imbalanced and noisy labels in training data. In this paper, we propose a unified data augmentation method, named BalanceMix, to address these challenges. Our approach includes two samplers for imbalanced labels, generating minority-augmented instances with high diversity. It also refines multi-labels at the label-wise granularity, categorizing noisy labels as clean, re-labeled, or ambiguous for robust optimization. Extensive experiments on three benchmark datasets demonstrate that BalanceMix outperforms existing state-of-the-art methods. We release the code at https://github.com/DISL-Lab/BalanceMix.

Semi-supervised Class-Agnostic Motion Prediction with Pseudo Label Regeneration and BEVMix

Class-agnostic motion prediction methods aim to comprehend motion within open-world scenarios, holding significance for autonomous driving systems. However, training a high-performance model in a fully-supervised manner always requires substantial amounts of manually annotated data, which can be both expensive and time-consuming to obtain. To address this challenge, our study explores the potential of semi-supervised learning (SSL) for class-agnostic motion prediction. Our SSL framework adopts a consistency-based self-training paradigm, enabling the model to learn from unlabeled data by generating pseudo labels through test-time inference. To improve the quality of pseudo labels, we propose a novel motion selection and re-generation module. This module effectively selects reliable pseudo labels and re-generates unreliable ones. Furthermore, we propose two data augmentation strategies: temporal sampling and BEVMix. These strategies facilitate consistency regularization in SSL. Experiments conducted on nuScenes demonstrate that our SSL method can surpass the self-supervised approach by a large margin by utilizing only a tiny fraction of labeled data. Furthermore, our method exhibits comparable performance to weakly and some fully supervised methods. These results highlight the ability of our method to strike a favorable balance between annotation costs and performance. Code will be available at https://github.com/kwwcv/SSMP.

Are You Concerned about Limited Function Evaluations: Data-Augmented Pareto Set Learning for Expensive Multi-Objective Optimization

Optimizing multiple conflicting black-box objectives simultaneously is a prevalent occurrence in many real-world applications, such as neural architecture search, and machine learning. These problems are known as expensive multi-objective optimization problems (EMOPs) when the function evaluations are computationally or financially costly. Multi-objective Bayesian optimization (MOBO) offers an efficient approach to discovering a set of Pareto optimal solutions. However, the data deficiency issue caused by limited function evaluations has posed a great challenge to current optimization methods. Moreover, most current methods tend to prioritize the quality of candidate solutions, while ignoring the quantity of promising samples. In order to tackle these issues, our paper proposes a novel multi-objective Bayesian optimization algorithm with a data augmentation strategy that provides ample high-quality samples for Pareto set learning (PSL). Specifically, it utilizes Generative Adversarial Networks (GANs) to enrich data and a dominance prediction model to screen out high-quality samples, mitigating the predicament of limited function evaluations in EMOPs. Additionally, we adopt the regularity model to expensive multi-objective Bayesian optimization for PSL. Experimental results on both synthetic and real-world problems demonstrate that our algorithm outperforms several state-of-the-art and classical algorithms.

Object-Aware Domain Generalization for Object Detection

Single-domain generalization (S-DG) aims to generalize a model to unseen environments with a single-source domain. However, most S-DG approaches have been conducted in the field of classification. When these approaches are applied to object detection, the semantic features of some objects can be damaged, which can lead to imprecise object localization and misclassification. To address these problems, we propose an object-aware domain generalization (OA-DG) method for single-domain generalization in object detection. Our method consists of data augmentation and training strategy, which are called OA-Mix and OA-Loss, respectively. OA-Mix generates multi-domain data with multi-level transformation and object-aware mixing strategy. OA-Loss enables models to learn domain-invariant representations for objects and backgrounds from the original and OA-Mixed images. Our proposed method outperforms state-of-the-art works on standard benchmarks. Our code is available at https://github.com/WoojuLee24/OA-DG.

Controllable 3D Face Generation with Conditional Style Code Diffusion

Generating photorealistic 3D faces from given conditions is a challenging task. Existing methods often rely on time-consuming one-by-one optimization approaches, which are not efficient for modeling the same distribution content, e.g., faces. Additionally, an ideal controllable 3D face generation model should consider both facial attributes and expressions. Thus we propose a novel approach called TEx-Face(TExt & Expression-to-Face) that addresses these challenges by dividing the task into three components, i.e., 3D GAN Inversion, Conditional Style Code Diffusion, and 3D Face Decoding. For 3D GAN inversion, we introduce two methods, which aim to enhance the representation of style codes and alleviate 3D inconsistencies. Furthermore, we design a style code denoiser to incorporate multiple conditions into the style code and propose a data augmentation strategy to address the issue of insufficient paired visual-language data. Extensive experiments conducted on FFHQ, CelebA-HQ, and CelebA-Dialog demonstrate the promising performance of our TEx-Face in achieving the efficient and controllable generation of photorealistic 3D faces. The code will be publicly available.

Semi-supervised 3D Object Detection with PatchTeacher and PillarMix

Semi-supervised learning aims to leverage numerous unlabeled data to improve the model performance. Current semi-supervised 3D object detection methods typically use a teacher to generate pseudo labels for a student, and the quality of the pseudo labels is essential for the final performance. In this paper, we propose PatchTeacher, which focuses on partial scene 3D object detection to provide high-quality pseudo labels for the student. Specifically, we divide a complete scene into a series of patches and feed them to our PatchTeacher sequentially. PatchTeacher leverages the low memory consumption advantage of partial scene detection to process point clouds with a high-resolution voxelization, which can minimize the information loss of quantization and extract more fine-grained features. However, it is non-trivial to train a detector on fractions of the scene. Therefore, we introduce three key techniques, i.e., Patch Normalizer, Quadrant Align, and Fovea Selection, to improve the performance of PatchTeacher. Moreover, we devise PillarMix, a strong data augmentation strategy that mixes truncated pillars from different LiDAR scans to generate diverse training samples and thus help the model learn more general representation. Extensive experiments conducted on Waymo and ONCE datasets verify the effectiveness and superiority of our method and we achieve new state-of-the-art results, surpassing existing methods by a large margin. Codes are available at https://github.com/LittlePey/PTPM.

ISP-Teacher:Image Signal Process with Disentanglement Regularization for Unsupervised Domain Adaptive Dark Object Detection

Object detection in dark conditions has always been a great challenge due to the complex formation process of low-light images. Currently, the mainstream methods usually adopt domain adaptation with Teacher-Student architecture to solve the dark object detection problem, and they imitate the dark conditions by using non-learnable data augmentation strategies on the annotated source daytime images. Note that these methods neglected to model the intrinsic imaging process, i.e. image signal processing (ISP), which is important for camera sensors to generate low-light images. To solve the above problems, in this paper, we propose a novel method named ISP-Teacher for dark object detection by exploring Teacher-Student architecture from a new perspective (i.e. self-supervised learning based ISP degradation). Specifically, we first design a day-to-night transformation module that consistent with the ISP pipeline of the camera sensors (ISP-DTM) to make the augmented images look more in line with the natural low-light images captured by cameras, and the ISP-related parameters are learned in a self-supervised manner. Moreover, to avoid the conflict between the ISP degradation and detection tasks in a shared encoder, we propose a disentanglement regularization (DR) that minimizes the absolute value of cosine similarity to disentangle two tasks and push two gradients vectors as orthogonal as possible. Extensive experiments conducted on two benchmarks show the effectiveness of our method in dark object detection. In particular, ISP-Teacher achieves an improvement of +2.4% AP and +3.3% AP over the SOTA method on BDD100k and SHIFT datasets, respectively. The code can be found at https://github.com/zhangyin1996/ISP-Teacher.

Self-Supervised Representation Learning with Meta Comprehensive Regularization

Self-Supervised Learning (SSL) methods harness the concept of semantic invariance by utilizing data augmentation strategies to produce similar representations for different deformations of the same input. Essentially, the model captures the shared information among multiple augmented views of samples, while disregarding the non-shared information that may be beneficial for downstream tasks. To address this issue, we introduce a module called CompMod with Meta Comprehensive Regularization (MCR), embedded into existing self-supervised frameworks, to make the learned representations more comprehensive. Specifically, we update our proposed model through a bi-level optimization mechanism, enabling it to capture comprehensive features. Additionally, guided by the constrained extraction of features using maximum entropy coding, the self-supervised learning model learns more comprehensive features on top of learning consistent features. In addition, we provide theoretical support for our proposed method from information theory and causal counterfactual perspective. Experimental results show that our method achieves significant improvement in classification, object detection and semantic segmentation tasks on multiple benchmark datasets.

I Prefer Not to Say: Protecting User Consent in Models with Optional Personal Data

We examine machine learning models in a setup where individuals have the choice to share optional personal information with a decision-making system, as seen in modern insurance pricing models. Some users consent to their data being used whereas others object and keep their data undisclosed. In this work, we show that the decision not to share data can be considered as information in itself that should be protected to respect users' privacy. This observation raises the overlooked problem of how to ensure that users who protect their personal data do not suffer any disadvantages as a result. To address this problem, we formalize protection requirements for models which only use the information for which active user consent was obtained. This excludes implicit information contained in the decision to share data or not. We offer the first solution to this problem by proposing the notion of Protected User Consent (PUC), which we prove to be loss-optimal under our protection requirement. We observe that privacy and performance are not fundamentally at odds with each other and that it is possible for a decision maker to benefit from additional data while respecting users' consent. To learn PUC-compliant models, we devise a model-agnostic data augmentation strategy with finite sample convergence guarantees. Finally, we analyze the implications of PUC on challenging real datasets, tasks, and models.

An RGB‐D object detection model with high‐generalization ability applied to tea harvesting robot for outdoor cross‐variety tea shoots detection

AbstractDetecting tea shoots is the first and most crucial step in achieving intelligent tea harvesting. However, when faced with thousands of tea varieties, establishing a high‐quality and comprehensive database comes with significant costs. Therefore, it has become an urgent challenge to improve the model's generalization ability and train it with minimal samples to develop a model capable of achieving optimal detection performance in various environments and tea varieties. This paper introduces a model named You Only See Tea (YOST) which utilizes depth maps to enhance model's generalization ability. It is applied to detect tea shoots in complex environments and to perform cross‐variety tea shoots detection. Our approach differs from common data augmentation strategies aimed at enhancing model generalization by diversifying the data set. Instead, we enhance the model's learning capability by strategically amplifying its attention towards core target features while simultaneously reducing attention towards noncore features. The proposed module YOST is developed upon the You Only Look Once version 7 (YOLOv7) model, utilizing two shared‐weight backbone networks to process both RGB and depth images. Then further integrate two modalities with feature layers at the same scale into our designed Ultra‐attention Fusion and Activation Module. By utilizing this approach, the model can proficiently detect targets by capturing core features, even when encountering complex environments or unfamiliar tea leaf varieties. The experimental results indicate that YOST displayed faster and more consistent convergence compared with YOLOv7 in training. Additionally, YOST demonstrated a 6.58% enhancement in AP50 for detecting tea shoots in complex environments. Moreover, when faced with a cross‐variety tea shoots detection task involving multiple unfamiliar varieties, YOST showcased impressive generalization abilities, achieving a significant maximum AP50 improvement of 33.31% compared with YOLOv7. These findings establish its superior performance. Our research departs from the heavy reliance on high‐generalization models on a large number of training samples, making it easier to train small‐scale, high‐generalization models. This approach significantly alleviates the pressure associated with data collection and model training.

AI-supported estimation of safety critical wind shear-induced aircraft go-around events utilizing pilot reports

The occurrence of wind shear and severe thunderstorms during the final approach phase contributes to nearly half of all aviation accidents. Pilots usually employ the go-around procedure in order to lower the likelihood of an unsafe landing. However, multiple factors influence the go-arounds induced by wind shear. In order to predict the wind shear-induced go-around, this study utilized a cutting-edge AI-based Combined Kernel and Tree Boosting (KTBoost) framework with various data augmentation strategies. First, the KTBoost model was trained, tested, and compared to other Machine Learning models using the data extracted from Hong Kong International Airport (HKIA)-based Pilot Reports for the years 2017–2021. The performance evaluation revealed that the KTBoost model with Synthetic Minority Oversampling Technique - Edited Nearest Neighbor (SMOTE-ENN)- augmented data demonstrated superior performance as measured by the F1-Score (94.37%) and G-Mean (94.87%). Subsequently, the SHapley Additive exPlanations (SHAP) approach was employed to elucidate the interpretation of the KTBoost model using data that had been treated with the SMOTE-ENN technique. According to the findings, flight type, wind shear magnitude, and approach runway contributed the most to the wind shear-induced go-around. Compared to international flights, Hong Kong-based airlines endured the highest number of wind shear-induced go-arounds. Shear due to the tailwind contributed more to the go-around than the headwinds. The runways with the most wind shear-induced Go-arounds were 07C and 07R.

Harnessing the power of radiomics and deep learning for improved breast cancer diagnosis with multiparametric breast mammography

Breast cancer, with its high mortality, faces diagnostic challenges due to variability in mammography quality and breast densities, leading to inconsistencies in radiological interpretations. Computer-aided diagnostic (CAD) systems, while helpful, struggle with accurately interpreting lesion characteristics such as morphology, density, and size. To address this, our study developed advanced deep-learning algorithms to improve the detection, localization, risk assessment, and classification of breast lesions, aiming to reduce false positives on human intervention and tackle slow convergence rates. Key innovations of the approach include preprocessing techniques, advanced filtering, and data augmentation strategies to optimize model performance, mitigating over- and under-fitting concerns. A significant development is the Chaotic Leader Selective Filler Swarm Optimization (cLSFSO) method, which effectively detects breast-dense lesions by extracting textural and statistical features. Additionally, the study adapted deep learning models like modified VGGNet and SE-ResNet152 through transfer learning, significantly enhancing their capability to distinguish between normal and suspicious mammography regions. The study also introduces hybrid deep neural network-based approaches, including CNN+LSTM and CNN+SVM, for diagnosing and grading cancerous polyps from the pre-segmented ROIs. Besides, the transfer learning paradigm is employed to boost the efficacy in classifying breast masses and reducing computing time by modifying the final layer of the proposed pre-trained models. The integration of Grad-CAM techniques further refines our analysis, leading to more accurate assessments and improved diagnosis of breast anomalies. Evaluated using benchmark and private datasets, our algorithms demonstrated a sensitivity of 0.99 and an overall AUC of 0.99, indicating significant improvements in mammogram analysis. These advancements aid radiologists, potentially improving patient outcomes and contributing to medical imaging and AI in healthcare.