Data Augmentation Strategy Research Articles

CR-LCRP: Course recommendation based on Learner–Course Relation Prediction with data augmentation in a heterogeneous view

Course recommendation aims to offer suitable courses for learners and alleviate dropout issue in Massive Open Online Course (MOOC) learning scenarios. The present studies focus on learning semantic representations by modeling the learner’s personal information and the courses’ teaching attributes. However, with massive courses offered and various learners involved, the data sparsity and cold start are still two dominant challenges hindering the booming of online learning, which roughly results from the scarce exploitation on various relationships between learners and courses. To alleviate the above issues, we leverage the strong ability of Heterogeneous Information Network (HIN) in modeling heterogeneous data types and complex structural characteristics, and propose a Course Recommendation method based on Learner–Course Relation Prediction (CR-LCRP) with multi-granularity data augmentation strategy. Specifically, the multi-source interactive data on courses, the learner–learner similarity and the course–course similarity are integrated in building a heterogeneous information network. With both explicit and implicit features extracted and structural information from meta-paths enhanced, the CR-LCRP method learns high-quality representations of learners and courses, and eventually performs learner–course relation prediction in MOOC learning scenarios. Finally, extensive experiments are conducted on real-world MOOCs datasets and the experimental results demonstrate that the proposed CR-LCRP method outperforms the state-of-the-art baseline models with superior performance on several widely used evaluation metrics.

Breast cancer classification method based on improved VGG16 using mammography images

Background and objectiveBreast cancer has become the leading global cancer, and early detection and diagnosis of breast cancer are of paramount importance for treatment. MethodologyThis paper proposes a breast X-ray mammography image classification model based on Convolutional Neural Network (CNN). The model categorizes breast X-ray mammography images into benign and malignant classes. Built upon the VGG network, the model adjusts the network structure and conducts experiments on the dataset collected and organized by the Medical Imaging Department of Ganzhou People's Hospital and The Sixth Affiliated Hospital of Jinan University. To address the issue of imbalanced data in the dataset, the model employs a focal loss function for optimization and combines transfer learning and data augmentation strategies during network training. ResultsExperimental results demonstrate that the model achieves an average recognition rate of 96.945% across four different magnification levels. Notably, recognition rates exceed 95.5% for the 50X, 100X, and 200× magnification levels, demonstrating excellent classification capabilities. ConclusionThis model ignificantly improving classification accuracy compared to previous models, which provides meaningful insights into the classification of breast X-ray mammography images.

Toward Robustness in Multi-Label Classification: A Data Augmentation Strategy against Imbalance and Noise

Multi-label classification poses challenges due to imbalanced and noisy labels in training data. In this paper, we propose a unified data augmentation method, named BalanceMix, to address these challenges. Our approach includes two samplers for imbalanced labels, generating minority-augmented instances with high diversity. It also refines multi-labels at the label-wise granularity, categorizing noisy labels as clean, re-labeled, or ambiguous for robust optimization. Extensive experiments on three benchmark datasets demonstrate that BalanceMix outperforms existing state-of-the-art methods. We release the code at https://github.com/DISL-Lab/BalanceMix.

Semi-supervised Class-Agnostic Motion Prediction with Pseudo Label Regeneration and BEVMix

Class-agnostic motion prediction methods aim to comprehend motion within open-world scenarios, holding significance for autonomous driving systems. However, training a high-performance model in a fully-supervised manner always requires substantial amounts of manually annotated data, which can be both expensive and time-consuming to obtain. To address this challenge, our study explores the potential of semi-supervised learning (SSL) for class-agnostic motion prediction. Our SSL framework adopts a consistency-based self-training paradigm, enabling the model to learn from unlabeled data by generating pseudo labels through test-time inference. To improve the quality of pseudo labels, we propose a novel motion selection and re-generation module. This module effectively selects reliable pseudo labels and re-generates unreliable ones. Furthermore, we propose two data augmentation strategies: temporal sampling and BEVMix. These strategies facilitate consistency regularization in SSL. Experiments conducted on nuScenes demonstrate that our SSL method can surpass the self-supervised approach by a large margin by utilizing only a tiny fraction of labeled data. Furthermore, our method exhibits comparable performance to weakly and some fully supervised methods. These results highlight the ability of our method to strike a favorable balance between annotation costs and performance. Code will be available at https://github.com/kwwcv/SSMP.

Are You Concerned about Limited Function Evaluations: Data-Augmented Pareto Set Learning for Expensive Multi-Objective Optimization

Optimizing multiple conflicting black-box objectives simultaneously is a prevalent occurrence in many real-world applications, such as neural architecture search, and machine learning. These problems are known as expensive multi-objective optimization problems (EMOPs) when the function evaluations are computationally or financially costly. Multi-objective Bayesian optimization (MOBO) offers an efficient approach to discovering a set of Pareto optimal solutions. However, the data deficiency issue caused by limited function evaluations has posed a great challenge to current optimization methods. Moreover, most current methods tend to prioritize the quality of candidate solutions, while ignoring the quantity of promising samples. In order to tackle these issues, our paper proposes a novel multi-objective Bayesian optimization algorithm with a data augmentation strategy that provides ample high-quality samples for Pareto set learning (PSL). Specifically, it utilizes Generative Adversarial Networks (GANs) to enrich data and a dominance prediction model to screen out high-quality samples, mitigating the predicament of limited function evaluations in EMOPs. Additionally, we adopt the regularity model to expensive multi-objective Bayesian optimization for PSL. Experimental results on both synthetic and real-world problems demonstrate that our algorithm outperforms several state-of-the-art and classical algorithms.

Object-Aware Domain Generalization for Object Detection

Single-domain generalization (S-DG) aims to generalize a model to unseen environments with a single-source domain. However, most S-DG approaches have been conducted in the field of classification. When these approaches are applied to object detection, the semantic features of some objects can be damaged, which can lead to imprecise object localization and misclassification. To address these problems, we propose an object-aware domain generalization (OA-DG) method for single-domain generalization in object detection. Our method consists of data augmentation and training strategy, which are called OA-Mix and OA-Loss, respectively. OA-Mix generates multi-domain data with multi-level transformation and object-aware mixing strategy. OA-Loss enables models to learn domain-invariant representations for objects and backgrounds from the original and OA-Mixed images. Our proposed method outperforms state-of-the-art works on standard benchmarks. Our code is available at https://github.com/WoojuLee24/OA-DG.

Controllable 3D Face Generation with Conditional Style Code Diffusion

Generating photorealistic 3D faces from given conditions is a challenging task. Existing methods often rely on time-consuming one-by-one optimization approaches, which are not efficient for modeling the same distribution content, e.g., faces. Additionally, an ideal controllable 3D face generation model should consider both facial attributes and expressions. Thus we propose a novel approach called TEx-Face(TExt & Expression-to-Face) that addresses these challenges by dividing the task into three components, i.e., 3D GAN Inversion, Conditional Style Code Diffusion, and 3D Face Decoding. For 3D GAN inversion, we introduce two methods, which aim to enhance the representation of style codes and alleviate 3D inconsistencies. Furthermore, we design a style code denoiser to incorporate multiple conditions into the style code and propose a data augmentation strategy to address the issue of insufficient paired visual-language data. Extensive experiments conducted on FFHQ, CelebA-HQ, and CelebA-Dialog demonstrate the promising performance of our TEx-Face in achieving the efficient and controllable generation of photorealistic 3D faces. The code will be publicly available.

Semi-supervised 3D Object Detection with PatchTeacher and PillarMix

Semi-supervised learning aims to leverage numerous unlabeled data to improve the model performance. Current semi-supervised 3D object detection methods typically use a teacher to generate pseudo labels for a student, and the quality of the pseudo labels is essential for the final performance. In this paper, we propose PatchTeacher, which focuses on partial scene 3D object detection to provide high-quality pseudo labels for the student. Specifically, we divide a complete scene into a series of patches and feed them to our PatchTeacher sequentially. PatchTeacher leverages the low memory consumption advantage of partial scene detection to process point clouds with a high-resolution voxelization, which can minimize the information loss of quantization and extract more fine-grained features. However, it is non-trivial to train a detector on fractions of the scene. Therefore, we introduce three key techniques, i.e., Patch Normalizer, Quadrant Align, and Fovea Selection, to improve the performance of PatchTeacher. Moreover, we devise PillarMix, a strong data augmentation strategy that mixes truncated pillars from different LiDAR scans to generate diverse training samples and thus help the model learn more general representation. Extensive experiments conducted on Waymo and ONCE datasets verify the effectiveness and superiority of our method and we achieve new state-of-the-art results, surpassing existing methods by a large margin. Codes are available at https://github.com/LittlePey/PTPM.

ISP-Teacher:Image Signal Process with Disentanglement Regularization for Unsupervised Domain Adaptive Dark Object Detection

Object detection in dark conditions has always been a great challenge due to the complex formation process of low-light images. Currently, the mainstream methods usually adopt domain adaptation with Teacher-Student architecture to solve the dark object detection problem, and they imitate the dark conditions by using non-learnable data augmentation strategies on the annotated source daytime images. Note that these methods neglected to model the intrinsic imaging process, i.e. image signal processing (ISP), which is important for camera sensors to generate low-light images. To solve the above problems, in this paper, we propose a novel method named ISP-Teacher for dark object detection by exploring Teacher-Student architecture from a new perspective (i.e. self-supervised learning based ISP degradation). Specifically, we first design a day-to-night transformation module that consistent with the ISP pipeline of the camera sensors (ISP-DTM) to make the augmented images look more in line with the natural low-light images captured by cameras, and the ISP-related parameters are learned in a self-supervised manner. Moreover, to avoid the conflict between the ISP degradation and detection tasks in a shared encoder, we propose a disentanglement regularization (DR) that minimizes the absolute value of cosine similarity to disentangle two tasks and push two gradients vectors as orthogonal as possible. Extensive experiments conducted on two benchmarks show the effectiveness of our method in dark object detection. In particular, ISP-Teacher achieves an improvement of +2.4% AP and +3.3% AP over the SOTA method on BDD100k and SHIFT datasets, respectively. The code can be found at https://github.com/zhangyin1996/ISP-Teacher.

Self-Supervised Representation Learning with Meta Comprehensive Regularization

Self-Supervised Learning (SSL) methods harness the concept of semantic invariance by utilizing data augmentation strategies to produce similar representations for different deformations of the same input. Essentially, the model captures the shared information among multiple augmented views of samples, while disregarding the non-shared information that may be beneficial for downstream tasks. To address this issue, we introduce a module called CompMod with Meta Comprehensive Regularization (MCR), embedded into existing self-supervised frameworks, to make the learned representations more comprehensive. Specifically, we update our proposed model through a bi-level optimization mechanism, enabling it to capture comprehensive features. Additionally, guided by the constrained extraction of features using maximum entropy coding, the self-supervised learning model learns more comprehensive features on top of learning consistent features. In addition, we provide theoretical support for our proposed method from information theory and causal counterfactual perspective. Experimental results show that our method achieves significant improvement in classification, object detection and semantic segmentation tasks on multiple benchmark datasets.

I Prefer Not to Say: Protecting User Consent in Models with Optional Personal Data

We examine machine learning models in a setup where individuals have the choice to share optional personal information with a decision-making system, as seen in modern insurance pricing models. Some users consent to their data being used whereas others object and keep their data undisclosed. In this work, we show that the decision not to share data can be considered as information in itself that should be protected to respect users' privacy. This observation raises the overlooked problem of how to ensure that users who protect their personal data do not suffer any disadvantages as a result. To address this problem, we formalize protection requirements for models which only use the information for which active user consent was obtained. This excludes implicit information contained in the decision to share data or not. We offer the first solution to this problem by proposing the notion of Protected User Consent (PUC), which we prove to be loss-optimal under our protection requirement. We observe that privacy and performance are not fundamentally at odds with each other and that it is possible for a decision maker to benefit from additional data while respecting users' consent. To learn PUC-compliant models, we devise a model-agnostic data augmentation strategy with finite sample convergence guarantees. Finally, we analyze the implications of PUC on challenging real datasets, tasks, and models.

An RGB‐D object detection model with high‐generalization ability applied to tea harvesting robot for outdoor cross‐variety tea shoots detection

AbstractDetecting tea shoots is the first and most crucial step in achieving intelligent tea harvesting. However, when faced with thousands of tea varieties, establishing a high‐quality and comprehensive database comes with significant costs. Therefore, it has become an urgent challenge to improve the model's generalization ability and train it with minimal samples to develop a model capable of achieving optimal detection performance in various environments and tea varieties. This paper introduces a model named You Only See Tea (YOST) which utilizes depth maps to enhance model's generalization ability. It is applied to detect tea shoots in complex environments and to perform cross‐variety tea shoots detection. Our approach differs from common data augmentation strategies aimed at enhancing model generalization by diversifying the data set. Instead, we enhance the model's learning capability by strategically amplifying its attention towards core target features while simultaneously reducing attention towards noncore features. The proposed module YOST is developed upon the You Only Look Once version 7 (YOLOv7) model, utilizing two shared‐weight backbone networks to process both RGB and depth images. Then further integrate two modalities with feature layers at the same scale into our designed Ultra‐attention Fusion and Activation Module. By utilizing this approach, the model can proficiently detect targets by capturing core features, even when encountering complex environments or unfamiliar tea leaf varieties. The experimental results indicate that YOST displayed faster and more consistent convergence compared with YOLOv7 in training. Additionally, YOST demonstrated a 6.58% enhancement in AP50 for detecting tea shoots in complex environments. Moreover, when faced with a cross‐variety tea shoots detection task involving multiple unfamiliar varieties, YOST showcased impressive generalization abilities, achieving a significant maximum AP50 improvement of 33.31% compared with YOLOv7. These findings establish its superior performance. Our research departs from the heavy reliance on high‐generalization models on a large number of training samples, making it easier to train small‐scale, high‐generalization models. This approach significantly alleviates the pressure associated with data collection and model training.

AI-supported estimation of safety critical wind shear-induced aircraft go-around events utilizing pilot reports

The occurrence of wind shear and severe thunderstorms during the final approach phase contributes to nearly half of all aviation accidents. Pilots usually employ the go-around procedure in order to lower the likelihood of an unsafe landing. However, multiple factors influence the go-arounds induced by wind shear. In order to predict the wind shear-induced go-around, this study utilized a cutting-edge AI-based Combined Kernel and Tree Boosting (KTBoost) framework with various data augmentation strategies. First, the KTBoost model was trained, tested, and compared to other Machine Learning models using the data extracted from Hong Kong International Airport (HKIA)-based Pilot Reports for the years 2017–2021. The performance evaluation revealed that the KTBoost model with Synthetic Minority Oversampling Technique - Edited Nearest Neighbor (SMOTE-ENN)- augmented data demonstrated superior performance as measured by the F1-Score (94.37%) and G-Mean (94.87%). Subsequently, the SHapley Additive exPlanations (SHAP) approach was employed to elucidate the interpretation of the KTBoost model using data that had been treated with the SMOTE-ENN technique. According to the findings, flight type, wind shear magnitude, and approach runway contributed the most to the wind shear-induced go-around. Compared to international flights, Hong Kong-based airlines endured the highest number of wind shear-induced go-arounds. Shear due to the tailwind contributed more to the go-around than the headwinds. The runways with the most wind shear-induced Go-arounds were 07C and 07R.

Harnessing the power of radiomics and deep learning for improved breast cancer diagnosis with multiparametric breast mammography

Breast cancer, with its high mortality, faces diagnostic challenges due to variability in mammography quality and breast densities, leading to inconsistencies in radiological interpretations. Computer-aided diagnostic (CAD) systems, while helpful, struggle with accurately interpreting lesion characteristics such as morphology, density, and size. To address this, our study developed advanced deep-learning algorithms to improve the detection, localization, risk assessment, and classification of breast lesions, aiming to reduce false positives on human intervention and tackle slow convergence rates. Key innovations of the approach include preprocessing techniques, advanced filtering, and data augmentation strategies to optimize model performance, mitigating over- and under-fitting concerns. A significant development is the Chaotic Leader Selective Filler Swarm Optimization (cLSFSO) method, which effectively detects breast-dense lesions by extracting textural and statistical features. Additionally, the study adapted deep learning models like modified VGGNet and SE-ResNet152 through transfer learning, significantly enhancing their capability to distinguish between normal and suspicious mammography regions. The study also introduces hybrid deep neural network-based approaches, including CNN+LSTM and CNN+SVM, for diagnosing and grading cancerous polyps from the pre-segmented ROIs. Besides, the transfer learning paradigm is employed to boost the efficacy in classifying breast masses and reducing computing time by modifying the final layer of the proposed pre-trained models. The integration of Grad-CAM techniques further refines our analysis, leading to more accurate assessments and improved diagnosis of breast anomalies. Evaluated using benchmark and private datasets, our algorithms demonstrated a sensitivity of 0.99 and an overall AUC of 0.99, indicating significant improvements in mammogram analysis. These advancements aid radiologists, potentially improving patient outcomes and contributing to medical imaging and AI in healthcare.

Face Mask Recognition based on YOLOV3

Wearing masks is considered one of the effective ways to prevent epidemics and protect physical health. In densely populated public places, detecting whether the mask is correctly worn has important application value, which can help managers reduce the potential risk of disease transmission. Previous works mainly rely on manual features, which are limited to fully expressing complex scene information. Thanks to the rapid development of artificial intelligence technology, automatic mask detection methods based on deep learning have achieved breakthroughs in speed and accuracy, while mask detection in complex scenes is still an open issue. To alleviate this issue, this paper proposes a mask detection method based on YOLOV3. Specifically, this study introduces regularization methods such as Drop Block and Label Smoothing, as well as data augmentation strategies such as Gridmask and Mosaic. This study further explores the recognition performance for refined scenarios including no mask, standard mask-wearing and incorrect mask-wearing. Extensive experimental results show the effectiveness of our method. By introducing the improved YOLOV5 algorithm to realize the mask detection function, the data structure is optimized through pruning, the data volume is reduced, the calculation speed is accelerated, and the mask detection is well realized.

End Face Detection of Rebar Based on Improved PP YOLO

This paper introduces an improved PP-YOLO network method to enhance the accuracy of rebar end identification and counting in construction engineering. The network is optimized for the specific characteristics of rebar images, enhancing its recognition capabilities for rebars of various sizes and shapes. Notably, the network structure introduces new pathways in the 4th and 5th layers, enabling more effective learning and identification of rebar features from low-level characteristics, thus improving overall recognition and counting accuracy. Additionally, an optimized data augmentation strategy, tailored to the unique features of rebar images, replaces the traditional Mixup method. Specific algorithms are introduced to enhance the network's efficiency in learning rebar image characteristics. These improvements led to excellent performance in rebar end face recognition tests, achieving an average precision (AP) of 96.23%, a 1.07% increase compared to the original model. This significant performance improvement confirms the effectiveness of our proposed improvements in practical applications, offering new perspectives for the development of rebar detection technology in the construction industry.

The Industrial Application of Artificial Intelligence-Based Optical Character Recognition in Modern Manufacturing Innovations

This paper presents the development of a comprehensive, on-site industrial Optical Character Recognition (OCR) system tailored for reading text on iron plates. Initially, the system utilizes a text region detection network to identify the text area, enabling camera adjustments along the x and y axes and zoom enhancements for clearer text imagery. Subsequently, the detected text region undergoes line-by-line division through a text segmentation network. Each line is then transformed into rectangular patches for character recognition by the text recognition network, comprising a vision-based text recognition model and a language network. The vision network performs preliminary recognition, followed by refinement through the language model. The OCR results are then converted into digital characters and recorded in the iron plate registration system. This paper’s contributions are threefold: (1) the design of a comprehensive, on-site industrial OCR system for autonomous registration of iron plates; (2) the development of a realistic synthetic image generation strategy and a robust data augmentation strategy to address data scarcity; and (3) demonstrated impressive experimental results, indicating potential for on-site industrial applications. The designed autonomous system enhances iron plate registration efficiency and significantly reduces factory time and labor costs.

AbFlex: designing antibody complementarity determining regions with flexible CDR definition.

Antibodies are proteins that the immune system produces in response to foreign pathogens. Designing antibodies that specifically bind to antigens is a key step in developing antibody therapeutics. The complementarity determining regions (CDRs) of the antibody are mainly responsible for binding to the target antigen, and therefore must be designed to recognize the antigen. We develop an antibody design model, AbFlex, that exhibits state-of-the-art performance in terms of structure prediction accuracy and amino acid recovery rate. Furthermore, >38% of newly designed antibody models are estimated to have better binding energies for their antigens than wild types. The effectiveness of the model is attributed to two different strategies that are developed to overcome the difficulty associated with the scarcity of antibody-antigen complex structure data. One strategy is to use an equivariant graph neural network model that is more data-efficient. More importantly, a new data augmentation strategy based on the flexible definition of CDRs significantly increases the performance of the CDR prediction model. The source code and implementation are available at https://github.com/wsjeon92/AbFlex.

Classification of recurrent major depressive disorder using a residual denoising autoencoder framework: Insights from large-scale multisite fMRI data

Background and objectiveRecurrent major depressive disorder (rMDD) has a high recurrence rate, and symptoms often worsen with each episode. Classifying rMDD using functional magnetic resonance imaging (fMRI) can enhance understanding of brain activity and aid diagnosis and treatment of this disorder. MethodsWe developed a Residual Denoising Autoencoder (Res-DAE) framework for the classification of rMDD. The functional connectivity (FC) was extracted from fMRI data as features. The framework addresses site heterogeneity by employing the Combat method to harmonize feature distribution differences. A feature selection method based on Fisher scores was used to reduce redundant information in the features. A data augmentation strategy using a Synthetic Minority Over-sampling Technique algorithm based on Extended Frobenius Norm measure was incorporated to increase the sample size. Furthermore, a residual module was integrated into the autoencoder network to preserve important features and improve the classification accuracy. ResultsWe tested our framework on a large-scale, multisite fMRI dataset, which includes 189 rMDD patients and 427 healthy controls. The Res-DAE achieved an average accuracy of 75.1 % (sensitivity = 69 %, specificity = 77.8 %) in cross-validation, thereby outperforming comparison methods. In a larger dataset that also includes first-episode depression (comprising 832 MDD patients and 779 healthy controls), the accuracy reached 70 %. ConclusionsWe proposed a deep learning framework that can effectively classify rMDD and 33 identify the altered FC associated with rMDD. Our study may reveal changes in brain function 34 associated with rMDD and provide assistance for the diagnosis and treatment of rMDD.

Data Augmentation of a Corrosion Dataset for Defect Growth Prediction of Pipelines Using Conditional Tabular Generative Adversarial Networks.

Due to corrosion characteristics, there are data scarcity and uneven distribution in corrosion datasets, and collecting high-quality data is time-consuming and sometimes difficult. Therefore, this work introduces a novel data augmentation strategy using a conditional tabular generative adversarial network (CTGAN) for enhancing corrosion datasets of pipelines. Firstly, the corrosion dataset is subjected to data cleaning and variable correlation analysis. The CTGAN is then used to generate external environmental factors as input variables for corrosion growth prediction, and a hybrid model based on machine learning is employed to generate corrosion depth as an output variable. The fake data are merged with the original data to form the synthetic dataset. Finally, the proposed data augmentation strategy is verified by analyzing the synthetic dataset using different visualization methods and evaluation indicators. The results show that the synthetic and original datasets have similar distributions, and the data augmentation strategy can learn the distribution of real corrosion data and sample fake data that are highly similar to the real data. Predictive models trained on the synthetic dataset perform better than predictive models trained using only the original dataset. In comparative tests, the proposed strategy outperformed other data generation methods.