Perform Data Augmentation Research Articles

Military Equipment Entity Extraction Based on Large Language Model

The technology of military equipment entity extraction, a crucial component in constructing military knowledge bases, holds significant research value and theoretical importance for guiding the development and improvement of equipment support forces. In the military domain, equipment entities exhibit a phenomenon of nesting, where one entity is contained within another, and abbreviations or codes are frequently used to represent these entities. To address this complexity, this paper proposes a method named CoTNER for extracting entities. Initially, a large-scale language model is used to perform data augmentation with chain-of-thought on the original dataset, providing additional semantic and contextual information. Subsequently, the augmented dataset is fine-tuned on a small-scale language model to adapt it to the task of military equipment entity extraction and to enhance its ability to learn complex rules specific to the domain of military equipment. Additionally, a high-quality data filtering strategy based on instruction-following difficulty scoring is proposed to address the catastrophic forgetting issue that may occur during the fine-tuning of large language models. The experimental results demonstrate that the proposed military equipment entity extraction method outperforms mainstream traditional deep learning methods, validating the effectiveness of CoTNER.

Development of an artificial intelligence model for predicting implant size in total knee arthroplasty using simple X-ray images

BackgroundAccurate estimation of implant size before surgery is crucial in preparing for total knee arthroplasty. However, this task is time-consuming and labor-intensive. To alleviate this burden on surgeons, we developed a reliable artificial intelligence (AI) model to predict implant size.MethodsWe enrolled 714 patients with knee osteoarthritis who underwent total knee arthroplasty from March 2010 to February 2014. All surgeries were performed by the same surgeon using implants from the same manufacturer. We collected 1412 knee anteroposterior (AP) and lateral view x-ray images and retrospectively investigated the implant size. We trained the AI model using both AP and lateral images without any clinical or demographic information and performed data augmentation to resolve issues of uneven distribution and insufficient data. Using data augmentation techniques, we generated 500 images for each size of the femur and tibia, which were then used to train the model. Using data augmentation techniques, we generated 500 images for each size of the femur and tibia, which were then used to train the model. We used ResNet-101 and optimized the model with the aim of minimizing the cross-entropy loss function using both the Stochastic Gradient Descent (SGD) and Adam optimizer.ResultsThe SGD optimizer achieved the best performance in internal validation. The model showed micro F1-score 0.91 for femur and 0.87 for tibia. For predicting within ± one size, micro F1-score was 0.99 for femur and 0.98 for tibia.ConclusionWe developed a deep learning model with high predictive power for implant size using only simple x-ray images. This could help surgeons reduce the time and labor required for preoperative preparation in total knee arthroplasty. While similar studies have been conducted, our work is unique in its use of simple x-ray images without any other data, like demographic features, to achieve a model with strong predictive power.

MvGraphDTA: multi-view-based graph deep model for drug-target affinity prediction by introducing the graphs and line graphs

BackgroundAccurately identifying drug-target affinity (DTA) plays a pivotal role in drug screening, design, and repurposing in pharmaceutical industry. It not only reduces the time, labor, and economic costs associated with biological experiments but also expedites drug development process. However, achieving the desired level of computational accuracy for DTA identification methods remains a significant challenge.ResultsWe proposed a novel multi-view-based graph deep model known as MvGraphDTA for DTA prediction. MvGraphDTA employed a graph convolutional network (GCN) to extract the structural features from original graphs of drugs and targets, respectively. It went a step further by constructing line graphs with edges as vertices based on original graphs of drugs and targets. GCN was also used to extract the relationship features within their line graphs. To enhance the complementarity between the extracted features from original graphs and line graphs, MvGraphDTA fused the extracted multi-view features of drugs and targets, respectively. Finally, these fused features were concatenated and passed through a fully connected (FC) network to predict DTA.ConclusionsDuring the experiments, we performed data augmentation on all the training sets used. Experimental results showed that MvGraphDTA outperformed the competitive state-of-the-art methods on benchmark datasets for DTA prediction. Additionally, we evaluated the universality and generalization performance of MvGraphDTA on additional datasets. Experimental outcomes revealed that MvGraphDTA exhibited good universality and generalization capability, making it a reliable tool for drug-target interaction prediction.

An improved data augmentation approach and its application in medical named entity recognition

Performing data augmentation in medical named entity recognition (NER) is crucial due to the unique challenges posed by this field. Medical data is characterized by high acquisition costs, specialized terminology, imbalanced distributions, and limited training resources. These factors make achieving high performance in medical NER particularly difficult. Data augmentation methods help to mitigate these issues by generating additional training samples, thus balancing data distribution, enriching the training dataset, and improving model generalization. This paper proposes two data augmentation methods—Contextual Random Replacement based on Word2Vec Augmentation (CRR) and Targeted Entity Random Replacement Augmentation (TER)—aimed at addressing the scarcity and imbalance of data in the medical domain. When combined with a deep learning-based Chinese NER model, these methods can significantly enhance performance and recognition accuracy under limited resources. Experimental results demonstrate that both augmentation methods effectively improve the recognition capability of medical named entities. Specifically, the BERT-BiLSTM-CRF model achieved the highest F1 score of 83.587%, representing a 1.49% increase over the baseline model. This validates the importance and effectiveness of data augmentation in medical NER.

GraphCL-DTA: A Graph Contrastive Learning With Molecular Semantics for Drug-Target Binding Affinity Prediction.

Drug-target binding affinity prediction plays an important role in the early stages of drug discovery, which can infer the strength of interactions between new drugs and new targets. However, the performance of previous computational models is limited by the following drawbacks. The learning of drug representation relies only on supervised data without considering the information in the molecular graph itself. Moreover, most previous studies tended to design complicated representation learning modules, while uniformity used to measure representation quality is ignored. In this study, we propose GraphCL-DTA, a graph contrastive learning with molecular semantics for drug-target binding affinity prediction. This graph contrastive learning framework replaces the dropout-based data augmentation strategy by performing data augmentation in the embedding space, thereby better preserving the semantic information of the molecular graph. A more essential and effective drug representation can be learned through this graph contrastive framework without additional supervised data. Next, we design a new loss function that can be directly used to adjust the uniformity of drug and target representations. By directly optimizing the uniformity of representations, the representation quality of drugs and targets can be improved. The effectiveness of the above innovative elements is verified on two real datasets, KIBA and Davis. Compared with the GraphDTA model, the relative improvement of the GraphCL-DTA model on the two datasets is 2.7% and 4.5%. The graph contrastive learning framework and uniformity function in the GraphCL-DTA model can be embedded into other computational models as independent modules to improve their generalization capability.

Tunnel construction worker safety state prediction and management system based on AHP and anomaly detection algorithm model

Tunnels represent complex, high-risk, and technically demanding underground construction projects. The safety of construction workers in tunnels is influenced by various factors, including physiological indicators, tunnel dimensions, and internal environmental conditions. Analyzing safety based solely on static factors is inadequate for modern tunnel engineering safety management requirements. To address this challenge, this paper provides a comprehensive analysis of factors impacting safety and employs the Analytic Hierarchy Process (AHP) to identify seven significant factors with high importance: body temperature, heart rate, internal temperature, internal humidity, CO concentration, chlorine concentration, and the relative positioning of personnel. Considering these factors essential for assessing worker safety, we introduce a novel model named Tunnel-APH-AD. For training models aimed at anomaly detection, we performed data augmentation and utilized four distinct machine learning models. Additionally, ensemble learning techniques were applied to aggregate the predictions from individual models, thereby enhancing the effectiveness of detecting safety states for tunnel workers. We also evaluated the performance of these models on out-of-distribution (OOD) samples to test their robustness and generalizability. The experimental results indicate that, under similar ventilation and tunnel conditions, the ensemble learning model exhibits superior overall performance compared to individual models, underscoring the effectiveness of model combination in improving the accuracy and reliability of safety alerts. Through experimental validation, this study provides interpretable, scalable, and scientifically generalized applications of machine learning theories in systems for tunnel construction worker safety alerts. These findings contribute to advancing safety management practices in tunnel engineering, enabling proactive and effective measures to mitigate potential risks and ensure the well-being of workers.

MDGCL: Graph Contrastive Learning Framework with Multiple Graph Diffusion Methods

In recent years, some classical graph contrastive learning(GCL) frameworks have been proposed to address the problem of sparse labeling of graph data in the real world. However, in node classification tasks, there are two obvious problems with existing GCL frameworks: first, the stochastic augmentation methods they adopt lose a lot of semantic information; second, the local–local contrasting mode selected by most frameworks ignores the global semantic information of the original graph, which limits the node classification performance of these frameworks. To address the above problems, this paper proposes a novel graph contrastive learning framework, MDGCL, which introduces two graph diffusion methods, Markov and PPR, and a deterministic–stochastic data augmentation strategy while retaining the local–local contrasting mode. Specifically, before using the two stochastic augmentation methods (FeatureDrop and EdgeDrop), MDGCL first uses two deterministic augmentation methods (Markov diffusion and PPR diffusion) to perform data augmentation on the original graph to increase the semantic information, this step ensures subsequent stochastic augmentation methods do not lose too much semantic information. Meanwhile, the diffusion matrices carried by the augmented views contain global semantic information of the original graph, allowing the framework to utilize the global semantic information while retaining the local-local contrasting mode, which further enhances the node classification performance of the framework. We conduct extensive comparative experiments on multiple benchmark datasets, and the results show that MDGCL outperforms the representative baseline frameworks on node classification tasks. Among them, compared with COSTA, MDGCL’s node classification accuracy has been improved by 1.07% and 0.41% respectively on two representative datasets, Amazon-Photo and Coauthor-CS. In addition, we also conduct ablation experiments on two datasets, Cora and CiteSeer, to verify the effectiveness of each improvement work of our framework.

Sample-imbalanced wafer map defects classification based on auxiliary classifier denoising diffusion probability model

This paper research on the classification task of wafer map defects under imbalanced sample. To improve the number of imbalanced wafer maps, a denoising diffusion probabilistic model with auxiliary classifier is proposed to generate new wafer maps. The developed model has an U-net structure with multiple inputs and outputs, taking wafer images, noise degree and category features as inputs, and outputs predicted noise and prediction category. When performing data augmentation, the trained model gradually removes prediction noise from the initial random noise and obtains new wafers. Finally, the balanced wafer defect maps are classified using a residual neural network. The proposed auxiliary classifier denoising diffusion probabilistic model and residual neural networks (ACDDPM-ResNet) method has validated on the MIR-WM811K dataset and MixedWM38 dataset, and the defect classification results of the imbalanced wafer map are remarkable improved after data augmentation. In addition, the paper also discusses and analyzes the influence of the maximum noise addition steps and the data augmentation size on the accuracy of wafer classification. It is further validated that the proposed wafer map classification method based on data augmentation can solve the classification problem caused by sample imbalance.

A Novel Data Augmentation Method for Radiomics Analysis Using Image Perturbations

Radiomics extracts hundreds of features from medical images to quantitively characterize a region of interest (ROI). When applying radiomics, imbalanced or small dataset issues are commonly addressed using under or over-sampling, the latter being applied directly to the extracted features. Aim of this study is to propose a novel balancing and data augmentation technique by applying perturbations (erosion, dilation, contour randomization) to the ROI in cardiac computed tomography images. From the perturbed ROIs, radiomic features are extracted, thus creating additional samples. This approach was tested addressing the clinical problem of distinguishing cardiac amyloidosis (CA) from aortic stenosis (AS) and hypertrophic cardiomyopathy (HCM). Twenty-one CA, thirty-two AS and twenty-one HCM patients were included in the study. From each original and perturbed ROI, 107 radiomic features were extracted. The CA-AS dataset was balanced using the perturbation-based method along with random over-sampling, adaptive synthetic (ADASYN) and the synthetic minority oversampling technique (SMOTE). The same methods were tested to perform data augmentation dealing with CA and HCM. Features were submitted to robustness, redundancy, and relevance analysis testing five feature selection methods (p-value, least absolute shrinkage and selection operator (LASSO), semi-supervised LASSO, principal component analysis (PCA), semi-supervised PCA). Support vector machine performed the classification tasks, and its performance were evaluated by means of a 10-fold cross-validation. The perturbation-based approach provided the best performances in terms of f1 score and balanced accuracy in both CA-AS (f1 score: 80%, AUC: 0.91) and CA-HCM (f1 score: 86%, AUC: 0.92) classifications. These results suggest that ROI perturbations represent a powerful approach to address both data balancing and augmentation issues.

WSBCV: A data-driven cross-version defect model via multi-objective optimization and incremental representation learning

Cross-version defect prediction (CVDP) refers to training an excellent model on tagged defect data in a previously released project version, and then performing defect prediction on an unlabeled instance module in the current version. Nevertheless, the complicated internal intrinsic construction hidden behind the code defects makes it difficult for the previous cross-version defect models to capture more discriminative software features, and seriously restrains the CVDP performance. In this study, we propose an intelligent data-driven CVDP model named WSBCV based on multi-objective optimization and incremental representation learning. We firstly leverage an advanced deep generation adversarial network – WGAN-GP (Wasserstein GAN with Gradient Penalty) to perform data augmentation, including balancing defect classes and synthesizing abundant training instances. Secondly, a multi-objective SPEA/R (Strength Pareto-based Evolutionary Algorithm / Reference) feature selection optimization method is built to effectively search the fewest representative feature subsets while achieving the minimum error. Finally, a powerful defect predictor for CVDP based on the BLS (Broad Learning System) with incremental learning is built to learn excellent feature representations and achieve incremental online model update quickly. Experimental results across 32 cross-version pairs from 45 version demonstrate that the proposed SPEA/R, BLS and WSBCV all have statistically significant difference advantages compared to ten multi-objective feature selection approaches, six defect predictors and two CVDP models, respectively.

Zero-shot stance detection based on multi-perspective transferable feature fusion

Zero-shot stance detection involves predicting stances that have not previously been encountered by adapting models to learn transferable features by aligning the source and destination target spaces. The acquisition of transferable target-invariant features is crucial for zero-shot stance detection. This work proposes a stance detection technique that can effectively adapt to new unseen targets, and the essence lies in acquiring fine-grained and easy-to-migrate target-invariant features from multiple perspectives as transferable knowledge. Specifically, we first perform data augmentation by masking topic keywords to mitigate the target dependency introduced by topic keywords in the text. Then, to account for the diversity and granularity of the sample features, we leverage instance-wise contrastive learning to extract transferable meta-features from multiple perspectives. The meta-features bridge features migration from known targets to unseen targets by incorporating different viewpoints. Finally, we incorporate an attention mechanism to fuse the multi-perspective transferable features for predicting the stance of previously unseen targets. The experimental results demonstrate the superiority of our model over competitive baselines across four benchmark datasets.

Supporting Malaria Diagnosis Using Deep Learning and Data Augmentation.

Malaria is an infection caused by the Plasmodium parasite that has a major epidemiological, social, and economic impact worldwide. Conventional diagnosis of the disease is based on microscopic examination of thick blood smears. This analysis can be time-consuming, which is key to generate prevention strategies and adequate treatment to avoid the complications associated with the disease. To address this problem, we propose a deep learning-based approach to detect not only malaria parasites but also leukocytes to perform parasite/μL blood count. We used positive and negative images with parasites and leukocytes. We performed data augmentation to increase the size of the dataset. The YOLOv8 algorithm was used for model training and using the counting formula the parasites were counted. The results showed the ability of the model to detect parasites and leukocytes with 95% and 98% accuracy, respectively. The time spent by the model to report parasitemia is significantly less than the time spent by malaria experts. This type of system would be supportive for areas with poor access to health care. We recommend validation of such approaches on a large scale in health institutions.

Comparison of CNN-based methods for yoga pose classification

Yoga is an exercise developed in ancient India. People perform yoga in order to have mental, physical, and spiritual benefits. While yoga helps build strength in the mind and body, incorrect postures might result in serious injuries. Therefore, yoga exercisers need either an expert or a platform to receive feedback on their performance. Since access to experts is not an option for everyone, a system to provide feedback on the yoga poses is required. To this end, commercial products such as smart yoga mats and smart pants are produced; Kinect cameras, sensors, and wearable devices are used. However, these solutions are either uncomfortable to wear or not affordable for everyone. Nonetheless, a system that employs computer vision techniques is a requirement. In this paper, we propose a deep-learning model for yoga pose classification, which is the first step of a quality assessment and personalized feedback system. We introduce a wavelet-based model that first takes wavelet transform of input images. The acquired subbands, i.e., approximation, horizontal, vertical, and diagonal coefficients of the wavelet transform are then fed into separate convolutional neural networks (CNN). The obtained probability results for each group are fused to predict the final yoga class. A publicly available dataset with 5 yoga poses is used. Since the number of images in the dataset is not enough for a deep learning model, we also perform data augmentation to increase the number of images. We compare our results to a CNN model and the three models that employ the subbands separately. Results obtained using the proposed model outperforms the accuracy output achieved with the compared models. While the regular CNN model has 61% and 50% accuracy for the training and test data, the proposed model achieves 91% and 80%, respectively.

MobileNet-Based Architecture for Distracted Human Driver Detection of Autonomous Cars

Distracted human driver detection is an important feature that should be included in most levels of autonomous cars, because most of these are still under development. Hereby, this paper proposes an architecture to perform this task in a fast and accurate way, with a full declaration of its details. The proposed architecture is mainly based on the MobileNet transfer learning model as a backbone feature extractor, then the extracted features are averaged by using a global average pooling layer, and then the outputs are fed into a combination of fully connected layers to identify the driver case. Also, the stochastic gradient descent (SGD) is selected as an optimizer, and the categorical cross-entropy is the loss function through the training process. This architecture is performed on the State-Farm dataset after performing data augmentation by using shifting, rotation, and zooming. The architecture can achieve a validation accuracy of 89.63%, a validation recall of 88.8%, a validation precision of 90.7%, a validation f1-score of 89.8%, a validation loss of 0.3652, and a prediction time of about 0.01 seconds per image. The conclusion demonstrates the efficiency of the proposed architecture with respect to most of the related work.

ConCPDP: A Cross‐Project Defect Prediction Method Integrating Contrastive Pretraining and Category Boundary Adjustment

Software defect prediction (SDP) is a crucial phase preceding the launch of software products. Cross‐project defect prediction (CPDP) is introduced for the anticipation of defects in novel projects lacking defect labels. CPDP can use defect information of mature projects to speed up defect prediction for new projects. So that developers can quickly get the defect information of the new project, so that they can test the software project pertinently. At present, the predominant approaches in CPDP rely on deep learning, and the performance of the ultimate model is notably affected by the quality of the training dataset. However, the dataset of CPDP not only has few samples but also has almost no label information in new projects, which makes the general deep‐learning‐based CPDP model not ideal. In addition, most of the current CPDP models do not fully consider the enrichment of classification boundary samples after cross‐domain, leading to suboptimal predictive capabilities of the model. To overcome these obstacles, we present contrastive learning pretraining for CPDP (ConCPDP), a CPDP method integrating contrastive pretraining and category boundary adjustment. We first perform data augmentation on the source and target domain code files and then extract the enhanced data as an abstract syntax tree (AST). The AST is then transformed into an integer sequence using specific mapping rules, serving as input for the subsequent neural network. A neural network based on bidirectional long short‐term memory (Bi‐LSTM) will receive an integer sequence and output a feature vector. Then, the feature vectors are input into the contrastive module to optimise the feature extraction network. The pretrained feature extractor can be fine‐tuned by the maximum mean discrepancy (MMD) between the feature distribution of the source domain and the target domain and the binary classification loss on the source domain. This paper conducts a large number of experiments on the PROMISE dataset, which is commonly used for CPDP, to validate ConCPDP’s efficacy, achieving superior results in terms of F1 measure, area under curve (AUC), and Matthew’s correlation coefficient (MCC).

Deep Selective Fusion of Visible and Near-Infrared Images Using Unsupervised U-Net.

In low light conditions, visible (VIS) images are of a low dynamic range (low contrast) with severe noise and color, while near-infrared (NIR) images contain clear textures without noise and color. Multispectral fusion of VIS and NIR images produces color images of high quality, rich textures, and little noise by taking both advantages of VIS and NIR images. In this article, we propose the deep selective fusion of VIS and NIR images using unsupervised U-Net. Existing image fusion methods are afflicted with the low contrast in VIS images and flash-like effect in NIR images. Thus, we adopt unsupervised U-Net to achieve deep selective fusion of multiple scale features. Due to the absence of the ground truth, we use unsupervised learning by formulating an energy function as a loss function. To deal with insufficient training data, we perform data augmentation by rotating images and adjusting their intensity. We synthesize training data by degrading clean VIS images and masking clean NIR images using a circle. First, we utilize pretrained visual geometry group (VGG) to extract features from VIS images. Second, we build an encoding network to obtain edge information from NIR images. Finally, we combine all features and feed them into a decoding network for fusion. Experimental results demonstrate that the proposed fusion network produces visually pleasing results with fine details, little noise, and natural color and it is superior to state-of-the-art methods in terms of visual quality and quantitative measurements.

DMMG: Dual Min-Max Games for Self-Supervised Skeleton-Based Action Recognition.

In this work, we propose a new Dual Min-Max Games (DMMG) based self-supervised skeleton action recognition method by augmenting unlabeled data in a contrastive learning framework. Our DMMG consists of a viewpoint variation min-max game and an edge perturbation min-max game. These two min-max games adopt an adversarial paradigm to perform data augmentation on the skeleton sequences and graph-structured body joints, respectively. Our viewpoint variation min-max game focuses on constructing various hard contrastive pairs by generating skeleton sequences from various viewpoints. These hard contrastive pairs help our model learn representative action features, thus facilitating model transfer to downstream tasks. Moreover, our edge perturbation min-max game specializes in building diverse hard contrastive samples through perturbing connectivity strength among graph-based body joints. The connectivity-strength varying contrastive pairs enable the model to capture minimal sufficient information of different actions, such as representative gestures for an action while preventing the model from overfitting. By fully exploiting the proposed DMMG, we can generate sufficient challenging contrastive pairs and thus achieve discriminative action feature representations from unlabeled skeleton data in a self-supervised manner. Extensive experiments demonstrate that our method achieves superior results under various evaluation protocols on widely-used NTU-RGB+D, NTU120-RGB+D and PKU-MMD datasets.

Complex multiphase predicting of additive manufactured high entropy alloys based on data augmentation deep learning

Additive Manufacturing (AM) can rapidly fabricate and produce customized, large-scale components, thereby fully exploiting High-Entropy Alloys (HEAs)' performance and industrial value—their broad potential for application in extreme service environments such as nuclear energy and aerospace. Because HEAs have a vast composition space, researchers can accurately map their mechanical properties by HEAs phases to reduce time-consuming and intensive experimental work. However, it is still a considerable challenge to predict HEAs phases accurately. And machine learning (ML) can predict the phases to accelerate the study of HEAs.To accurately predict the HEAs phases, the study propose deep learning (DL) algorithms that rely on AM process parameters and data augmentation and do the following: (i) This study developed a more detailed phase classification strategy for HEAs to enable the model to predict the complex multiphase of HEAs; (ii) The process parameters of AM affect the formation of the HEAs phase. In this study, the process parameters of AM are added to the model features to improve the model's accuracy in predicting the HEAs phase; (iii) To alleviate the data scarcity dilemma of HEAs, this study use data augmentation to improve DL's performance further. The results show that after adding AM process parameters to the model and performing data augmentation, the DL model's accuracy to 91.23%. In addition, this study manufactured four new HEAs through AM experiments to verify the robustness and practicality of their algorithms.

Generative adversarial network based on Poincaré distance similarity constraint: Focusing on overfitting problem caused by finite training data

Generative adversarial networks face harsh opposition between training data and model performance. In facing insufficient training data, the training process is extremely unstable, and the modules are difficult to converge, which increases the risk of model collapse, and finally manifests itself as poor image quality generated by the model. The essence of this problem is that a single discriminator is prone to overfitting. However, the traditional methods are to perform data augmentation, optimize the loss function, or improve the training strategy. In this paper, starting from the structural characteristics of the model itself, a dual-ways model with internal interactive learning and synchronous training is designed. The dual-ways discriminator directly maximizes the Poincaré distance similarity, which is used to enhance the effectiveness of the error gradient and eliminate the risk of discriminator overfitting. Experimental results show that the dual-way model produces higher-quality images than the six well-known schemes in four benchmark datasets. The experimental results also show that thanks to the mechanism of internal interactive learning, the training process is more stable and the discriminator module converges in a timelier manner, which is the key to improving the performance of the model. Thus, we obtain a new stronger model that is different from the existing strategy.

Conditional generative adversarial network based data augmentation for fault diagnosis of diesel engines applied with infrared thermography and deep convolutional neural network

This paper tries to introduce a new intelligent method for the early fault diagnosis of diesel engines. Firstly, infrared thermography (IRT) is introduced into diesel engine condition monitoring, then infrared images of diesel engines in four health states, such as normal condition, single-cylinder misfire, multi-cylinder misfire and air filter blockage, are collected and the region of interest (ROI) of infrared images are extracted. Next, conditional generative adversarial network (CGAN) is deployed to perform data augmentation on infrared image datasets. Then, deep convolutional neural network (DCNN) and Softmax regression (SR) classifier are used for automatically extracting infrared image fault features and pattern recognition, respectively. Finally, a comparison with three deep learning (DL) models is performed. The validation results show that the data augmentation method proposed in the paper can significantly improve the early fault diagnosis accuracy, and DCNN has the best fault diagnosis effect and resistance to temperature fluctuation interference among the four DL models.