Deep Learning Network Architecture Research Articles

Summarization and Translation Using NLP

Abstract: Text summarization and translation are two critical tasks in natural language processing with significant applications in various domains such as news aggregation, document summarization, machine translation, and information retrieval. In recent years, there has been remarkable progress in the development of techniques and models for both tasks, leveraging advancements in deep learning and neural network architectures. This paper presents a comprehensive review and comparative analysis of state-of-the-art methods in text summarization and translation. First, we provide an overview of the different approaches to text summarization, including extractive, abstractive, and hybrid methods, highlighting their strengths and weaknesses. We discuss various evaluation metrics and datasets commonly used for benchmarking summarization systems, shedding light on the challenges and opportunities in this field.

LEET: stock market forecast with long-term emotional change enhanced temporal model.

The stock market serves as a macroeconomic indicator, and stock price forecasting aids investors in analysing market trends and industry dynamics. Several deep learning network models have been proposed and extensively applied for stock price prediction and trading scenarios in recent times. Although numerous studies have indicated a significant correlation between market sentiment and stock prices, the majority of stock price predictions rely solely on historical indicator data, with minimal effort to incorporate sentiment analysis into stock price forecasting. Additionally, many deep learning models struggle with handling the long-distance dependencies of large datasets. This can cause them to overlook unexpected stock price fluctuations that may arise from long-term market sentiment, making it challenging to effectively utilise long-term market sentiment information. To address the aforementioned issues, this investigation suggests implementing a new technique called Long-term Sentiment Change Enhanced Temporal Analysis (LEET) which effectively incorporates long-term market sentiment and enhances the precision of stock price forecasts. The LEET method proposes two market sentiment index estimation methods: Exponential Weighted Sentiment Analysis (EWSA) and Weighted Average Sentiment Analysis (WASA). These methods are utilized to extract the market sentiment index. Additionally, the study proposes a Transformer architecture based on ProbAttention with rotational position encoding for enhanced positional information capture of long-term emotions. The LEET methodology underwent validation using the Standard & Poor's 500 (SP500) and FTSE 100 indices. These indices accurately reflect the state of the US and UK equity markets, respectively. The experimental results obtained from a genuine dataset demonstrate that this method is superior to the majority of deep learning network architectures when it comes to predicting stock prices.

Cross-Modal Supervised Human Body Pose Recognition Techniques for Through-Wall Radar.

Through-wall radar human body pose recognition technology has broad applications in both military and civilian sectors. Identifying the current pose of targets behind walls and predicting subsequent pose changes are significant challenges. Conventional methods typically utilize radar information along with machine learning algorithms such as SVM and random forests to aid in recognition. However, these approaches have limitations, particularly in complex scenarios. In response to this challenge, this paper proposes a cross-modal supervised through-wall radar human body pose recognition method. By integrating information from both cameras and radar, a cross-modal dataset was constructed, and a corresponding deep learning network architecture was designed. During training, the network effectively learned the pose features of targets obscured by walls, enabling accurate pose recognition (e.g., standing, crouching) in scenarios with unknown wall obstructions. The experimental results demonstrated the superiority of the proposed method over traditional approaches, offering an effective and innovative solution for practical through-wall radar applications. The contribution of this study lies in the integration of deep learning with cross-modal supervision, providing new perspectives for enhancing the robustness and accuracy of target pose recognition.

Abstract 7379: Deep learning based risk stratification of pre-operative CT scans is prognostic of overall survival in kidney cancers

Abstract Introduction: Preoperative risk stratification of kidney cancer patients is critical for treatment selection and improving mortality rates. Current clinical approaches such as cancer grading are limited by inter-reviewer variability, require invasive biopsy procedures, and might not capture the full extent of patient risk. We sought to evaluate a deep learning (DL) model to quantify patient risk and prognosticate overall survival in kidney cancer patients via CT scans. Methods: This retrospective study used preoperative contrast CT scans from 4 public multi-institutional collections with date of death/censoring available. A deep learning (DL) network architecture was developed to analyze input CT scans via a convolutional feature extractor and a Cox proportional hazards-based cost function. The DL model assigned a risk score to each patient, with the median risk used as a cutoff for classifying patients as high-risk or low-risk. Model performance for prognosticating overall survival (OS) was evaluated separately in training, internal validation, and external validation cohorts via Kaplan-Meier analysis in terms of concordance index and hazard ratios, with risk groups compared via Wilcoxon log-rank tests (α = 0.05). Results: In the training/internal validation cohort (N=418, 8 institutions) and external validation cohort (N=102, 1 institution), the median age was 60 and 59 years, the male/female ratios were 62%/38% and 75%/25%, with a median follow up of 2.8 and 3.1 years, respectively. The DL model yielded a training c-index of 0.66 (95% CI: 0.63-0.68), internal validation c-index of 0.73 (95% CI: 0.70-0.76) and an external validation c-index of 0.67 (95% CI: 0.64-0.70). Kaplan-Meier analysis of associated survival curves based on DL-determined high- and low-risk patients demonstrated statistically significant separation in all cohorts: training (p = 0.003, HR = 8.63, 95% CI: 8.28-8.98), internal validation (p = 0.027, HR = 4.89, 95% CI: 4.52-5.26), and external validation (p = 0.01, HR = 6.54, 95% CI: 6.18-6.90). Baseline clinicopathological characteristics were not significantly different between DL high and low risk patients in external validation (chi-square testing; ISUP grade p=0.375, age p=0.824, race p=0.567, and sex p=0.440). ISUP grade was also not a significant predictor of OS in internal validation (HR=1.57, p=0.211, 95% CI: 1.28-1.85) or external validation (p = 0.465, HR=0.16, 95% CI: 0.33-0.74). Conclusion: A deep learning model based on pre-operative CT scans was found to accurately prognosticate overall survival in kidney cancers in a multi-institutional setting, independent of clinicopathological factors. This model is being validated on larger cohorts, and in the context of targeted therapies. Citation Format: Brennan Flannery, Mohsen Hariri, Thomas DeSilvio, Amir Sadri, Jane Nguyen, Erick M. Remer, Smitha Krishnamurthi, Satish E. Viswanath. Deep learning based risk stratification of pre-operative CT scans is prognostic of overall survival in kidney cancers [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 7379.

Resolution and Analysis of Transmission Line Fault Types Based on Recording Type Data and Deep Learning

The reliable identification of fault types in transmission lines is essential for restoring power supply swiftly and minimizing economic losses during outages, thereby ensuring the safe and efficient functioning of the power system. This paper addresses the challenge of low recognition accuracy in existing transmission line fault diagnosis methods and presents a novel approach based on fault recording data collected from both ends of the line. This method distinguishes between lightning-strike and non-lightning-strike faults, utilizing a deep learning network architecture to analyze time-domain information from recorded data, using the initial and terminal waveforms as inputs. The proposed fault identification model integrates fault current phase mode transformation, Local Mean Decomposition (LMD) decomposition, and spectral entropy analysis, applying deep learning principles to enhance fault detection precision. This comprehensive approach enables the effective identification of various fault types on transmission lines. Extensive simulation tests were conducted using a sophisticated fault simulation model developed within simulation software to validate the proposed algorithm’s efficacy. The results demonstrate the algorithm’s high accuracy and efficiency in recognizing various fault types on transmission lines.

ASFFuse: Infrared and visible image fusion model based on adaptive selection feature maps

Researchers continuously modify deep learning network architecture for improved fusion results. However, little attention is given to the influence of noise feature maps generated during the convolution process on the fusion outcomes. Here, we aim to minimize the influence of noisy feature maps on fusion results and propose a fusion model, the infrared and visible image fusion model based on adaptive selection feature maps (ASFFuse). We propose an adaptive selection feature maps module (ASFM). ASFM measures the amount of information contained in each feature map and filters out feature maps that contain more noise information. Additionally, we introduce a feature enhancement module (FEM) to enrich the fusion image with more source image information. For unsupervised training of the proposed model, we propose a texture loss function based on contrast learning. This loss function preserves the texture information of the image in a better way and makes the fusion image have a better visual effect. Our ASFFuse model has been shown to outperform state-of-the-art models in both quantitative and qualitative evaluations in extensive experiments on the TNO and RoadScene datasets. The code is available at https://github.com/LKZ1584905069/ASFFuse.

E2SCNet: Efficient Multiobjective Evolutionary Automatic Search for Remote Sensing Image Scene Classification Network Architecture.

Remote sensing image scene classification methods based on deep learning have been widely studied and discussed. However, most of the network architectures are directly reliant on natural image processing methods and are fixed. A few studies have focused on automatic search mechanisms, but they cannot weigh the interpretation accuracy and the parameter quantity for practical application. As a result, automatic global search methods based on multiobjective evolutionary computation have more advantages. However, in the ranking process, the network individuals with large parameter quantities are easy to eliminate, but a higher accuracy may be obtained after full training. In addition, evolutionary neural architecture search methods often take several days. In this article, in order to solve the above concerns, we propose an efficient multiobjective evolutionary automatic search framework for remote sensing image scene classification deep learning network architectures (E2SCNet). In E2SCNet, eight kinds of lightweight operators are used to build a diversified search space, and the coding connection mode is flexible. In the search process, a large model retention mechanism is implemented through two-step multiobjective modeling and evolutionary search, where one step involves the "parameter quantity and accuracy", and the other step involves the "parameter quantity and accuracy growth quantity." Moreover, a super network is constructed to share the weight in the process of individual network evaluation and promote the search speed. The effectiveness of E2SCNet is proven by comparison with several networks designed by human experts and networks obtained by gradient and evolutionary computing-based search methods.

Hierarchical Graph Convolutional Network Built by Multiscale Atlases for Brain Disorder Diagnosis Using Functional Connectivity.

Functional connectivity network (FCN) data from functional magnetic resonance imaging (fMRI) is increasingly used for the diagnosis of brain disorders. However, state-of-the-art studies used to build the FCN using a single brain parcellation atlas at a certain spatial scale, which largely neglected functional interactions across different spatial scales in hierarchical manners. In this study, we propose a novel framework to perform multiscale FCN analysis for brain disorder diagnosis. We first use a set of well-defined multiscale atlases to compute multiscale FCNs. Then, we utilize biologically meaningful brain hierarchical relationships among the regions in multiscale atlases to perform nodal pooling across multiple spatial scales, namely "Atlas-guided Pooling (AP)." Accordingly, we propose a multiscale-atlases-based hierarchical graph convolutional network (MAHGCN), built on the stacked layers of graph convolution and the AP, for a comprehensive extraction of diagnostic information from multiscale FCNs. Experiments on neuroimaging data from 1792 subjects demonstrate the effectiveness of our proposed method in the diagnoses of Alzheimer's disease (AD), the prodromal stage of AD i.e., mild cognitive impairment (MCI), as well as autism spectrum disorder (ASD), with the accuracy of 88.9%, 78.6%, and 72.7%, respectively. All results show significant advantages of our proposed method over other competing methods. This study not only demonstrates the feasibility of brain disorder diagnosis using resting-state fMRI empowered by deep learning but also highlights that the functional interactions in the multiscale brain hierarchy are worth being explored and integrated into deep learning network architectures for a better understanding of the neuropathology of brain disorders. The codes for MAHGCN are publicly available at "https://github.com/MianxinLiu/ MAHGCN-code."

Integrating PubMed Label Hierarchy Knowledge into a Complex Hierarchical Deep Neural Network

This paper proposes an innovative method that exploits a complex deep learning network architecture, called Hierarchical Deep Neural Network (HDNN), specifically developed for the eXtreme Multilabel Text Classification (XMTC) task, when the label set is hierarchically organized, such as the case of the PubMed article labeling task. In detail, the topology of the proposed HDNN architecture follows the exact hierarchical structure of the label set to integrate this knowledge directly into the DNN. We assumed that if a label set hierarchy is available, as in the case of the PubMed Dataset, forcing this information into the network topology could enhance the classification performances and the interpretability of the results, especially related to the hierarchy. We performed an experimental assessment of the PubMed article classification task, demonstrating that the proposed HDNN provides performance improvement for a baseline based on a classic flat Convolution Neural Network (CNN) deep learning architecture, in particular in terms of hierarchical measures. These results provide useful hints for integrating previous and innate knowledge in a deep neural network. The drawback of the HDNN is the high computational time required to train the neural network, which can be addressed with a parallel implementation planned as a future work.

Retracted: Bio-Optimization of Deep Learning Network Architectures

Retracted: Bio-Optimization of Deep Learning Network Architectures

SwinHCST: a deep learning network architecture for scene classification of remote sensing images based on improved CNN and Transformer

ABSTRACTRemote sensing image scene classification is a fundamental task in intelligent interpretation of remote sensing images. Although Transformers possess a powerful attention mechanism, they require lengthy training procedures to achieve good performance levels. To address this issue, this paper proposes a novel deep learning network model by combining CNN and Swin Transformer named SwinHCST. Firstly, the model uses Weighted Normalized CNN to quickly extract low-level features of the image. Secondly, the Receptive Field Block module facilitates multi-scale information fusion, Thirdly, the Information Fusion Transformer further excavates the deep-level features of the image. Furthermore, this paper has designed a plug-and-play Cross Spatial Information Fusion Block, which is used to encodes dimensional information and extracts global information to enhance information exchange. The scene classification experiments show that the proposed model outperforms other methods on the three selected datasets and can achieve excellent performance without requiring large amounts of data and training. Specifically, the classification accuracy of the proposed method on the three datasets is 93.76%, 93.60%, and 98.10%, which is 1.7% to 3.71% higher than ResNet50 and 3.7% to 5.7% higher than Swin Transformer.

Global-local attention with triplet loss and label smoothed crossentropy for person re-identification

Person re-identification (Person Re-ID) is a research direction on tracking and identifying people in surveillance camera systems with non-overlapping camera perspectives. Despite much research on this topic, there are still some practical problems that Person Re-ID has not yet solved, in reality, human objects can easily be obscured by obstructions such as other people, trees, luggage, umbrellas, signs, cars, motorbikes. In this paper, we propose a multibranch deep learning network architecture. In which one branch is for the representation of global features and two branches are for the representation of local features. Dividing the input image into small parts and changing the number of parts between the two branches helps the model to represent the features better. In addition, we add an attention module to the ResNet50 backbone that enhances important human characteristics and eliminates irrelevant information. To improve robustness, the model is trained by combining triplet loss and label smoothing cross-entropy loss (LSCE). Experiments are carried out on datasets Market1501, and duke multi-target multi-camera (DukeMTMC) datasets, our method achieved 96.04% rank-1, 88,11% mean average precision (mAP) on the Market1501 dataset, and 88.78% rank-1, 78,6% mAP on the DukeMTMC dataset. This method achieves performance better than some state-of-the-art methods.

Predicting Driver Takeover Decisions in Conditionally Automated Vehicles with a Gaze-based Deep Learning Model

This study proposes a novel methodology for modeling driver takeover behavior in conditionally automated vehicles (AVs) when exiting a freeway using deep learning (DL) network architectures. While previous research has focused on modeling takeover time in emergency scenarios, which require quick responses, these models may not be applicable to scheduled, non-time-critical takeovers. In such situations, drivers may employ varying strategies and take longer to resume control of the vehicle when there is no time pressure. To address this problem, a deep learning architecture based on a convolutional neural network (CNN) was implemented to predict drivers’ takeover behaviors in scheduled takeovers. The model was trained on drivers’ driving data and eye gaze with varying time windows, facilitating an analysis of drivers’ takeover decisions to various takeover request designs. The model achieved good performance metrics, with an F1 Score of 0.993, a recall of 0.996, and a precision of 0.991. The application of these models holds substantial potential for refining the design of the human-machine interface, specifically in calibrating the takeover request (ToR) lead time, thereby promoting safe freeway exiting takeovers in conditionally AVs.

Differential evolution-based neural architecture search for brain vessel segmentation

Brain vasculature analysis is critical in developing novel treatment targets for neurodegenerative diseases. Such an accurate analysis cannot be performed manually but requires a semi-automated or fully-automated approach. Deep learning methods have recently proven indispensable for the automated segmentation and analysis of medical images. However, optimizing a deep learning network architecture is another challenge. Manually selecting deep learning network architectures and tuning their hyper-parameters requires a lot of expertise and effort. To solve this problem, neural architecture search (NAS) approaches that explore more efficient network architectures with high segmentation performance have been proposed in the literature. This study introduces differential evolution-based NAS approaches in which a novel search space is proposed for brain vessel segmentation. We select two architectures that are frequently used for medical image segmentation, i.e. U-Net and Attention U-Net, as baselines for NAS optimizations. The conventional differential evolution and the opposition-based differential evolution with novel search space are employed as search methods in NAS. Furthermore, we perform ablation studies and evaluate the effects of specific loss functions, model pruning, threshold selection and generalization performance on the proposed models. The experiments are conducted on two datasets providing 335 single-channel 8-bit gray-scale images. These datasets are a public volumetric cerebrovascular system dataset (vesseINN) and our own dataset called KUVESG. The proposed NAS approaches, namely UNAS-Net and Attention UNAS-Net architectures, yield better segmentation performance in terms of different segmentation metrics. More specifically, UNAS-Net with differential evolution reveals high dice score/sensitivity values of 79.57/81.48, respectively. Moreover, they provide shorter inference times by a factor of 9.15 than the baseline methods.

A New Robust Weak Supervision Deep Learning Approach for Reservoir Properties Prediction in Malaysian Basin Field

The conventional seismic inversion approach is practical for operational work, as it only uses simple linearized algorithms and assumptions, but may be less applicable when dealing with a complex geological setting, especially in the Malay basin fields, as it may introduce non-linear noises and non-unique solutions. In the Malay basin, we also frequently struggle with a scarcity of reliable well data when performing seismic inversions. This makes finding an accurate prior model for inversion challenging and contributes to high uncertainty in properties’ estimation. Implementation of deep learning for seismic inversion has become routine and has shown increasing capability in addressing nonlinearity in inverse problems. In this work, we develop a robust approach to deep learning-based seismic inversion to predict elastic properties from seismic data. The approach incorporates synthetic well and seismic data generation from a set of rock physics knowledge called the rock physics library, which plays a significant role in dataset input for network training, validation, and testing to improve elastic properties in this field. The deep learning network architecture comprising UNET and RESNET-18 with weak supervision networks has proven to be useful to enhance computational work efficiency and prediction accuracy while handling the non-linearity of the data and the non-uniqueness of the solutions. We successfully validated the proposed method on actual field data from a clastic fluvial-dominated field in the Malay basin. Upon comparative analysis with the conventional method, both inversion results are comparable and capable of identifying the reservoir occurrence and distribution. The conventional method exposed the presence of scattered amplitude noises and prominent seismic imprints masking the reservoir. Meanwhile, the proposed method showed more stable, clearer definition and fewer noise inversion results but with a faster turn-around time and a more efficient workflow. There are substantial improvements of up to 31% in correlation accuracy achieved upon implementing the proposed method for elastic properties prediction compared to the conventional. The result implies that the proposed method can provide a good elastic properties prediction framework while addressing data limitations and sparsity issues in typical deep learning-based inversions.

Artificial neural networks algorithms for prediction of human hair loss related autoimmune disorder problem

<p>In this study, artificial neural networks (ANNs) are being used to diagnose hair loss in patients. An autoimmune condition known as Alopecia Areata (AA) results in hair loss in the affected area. The most recent figures from throughout the world show that AA affects 1 in 1000 persons and has a 2% incidence rate. Based on the look of photographs with healthy hair in the dataset, machine learning techniques were employed to classify the conditions. Before making predictions, each of these ANNs algorithms creates a prediction model using pictures of healthy hair. The aim of this study is to evaluate the accuracy of neural networks for alopecia detection in human subjects. The study presents a classification framework for distinguishing between healthy hairs (HHs) and Alopecia Areata (AA). The framework incorporates Contrast Limited Adaptive Histogram Equalization (CLAHE) enhancement and segmentation techniques to enhance the quality of the images. Additionally, Data Augmentation (DA) is employed to generate additional data and improve the precision of the proposed framework. To extract features from the images, two powerful techniques are utilized. The Visual Geometry Group (VGG), which consists of very deep convolutional networks designed for large-scale image recognition, is employed. VGG networks have proven to be effective in learning complex features directly from data, eliminating the need for manual feature extraction. Additionally, a Convolutional Neural Network (CNN), a deep learning network architecture specifically designed for image processing tasks, is employed. To create a machine learning model for classification, the Support Vector Machine (SVM) approach is utilized. SVM is a widely used algorithm in supervised learning, capable of solving both classification and regression problems. Its versatility and effectiveness make it a suitable choice for the classification task in this study. By combining the CLAHE enhancement, segmentation, data augmentation, feature extraction using VGG and CNN, and classification using SVM, the proposed framework aims to accurately classify HHs and AA cases.</p>

Crystal search - feasibility study of a real-time deep learning process for crystallization well images.

To avoid the time-consuming and often monotonous task of manual inspection of crystallization plates, a Python-based program to automatically detect crystals in crystallization wells employing deep learning techniques was developed. The program uses manually scored crystallization trials deposited in a database of an in-house crystallization robot as a training set. Since the success rate of such a system is able to catch up with manual inspection by trained persons, it will become an important tool for crystallographers working on biological samples. Four network architectures were compared and the SqueezeNet architecture performed best. In detecting crystals AlexNet accomplished a better result, but with a lower threshold the mean value for crystal detection was improved for SqueezeNet. Two assumptions were made about the imaging rate. With these two extremes it was found that an image processing rate of at least two times, but up to 58 times in the worst case, would be needed to reach the maximum imaging rate according to the deep learning network architecture employed for real-time classification. To avoid high workloads for the control computer of the CrystalMation system, the computing is distributed over several workstations, participating voluntarily, by the grid programming system from the Berkeley Open Infrastructure for Network Computing (BOINC). The outcome of the program is redistributed into the database as automatic real-time scores (ARTscore). These are immediately visible as colored frames around each crystallization well image of the inspection program. In addition, regions of droplets with the highest scoring probability found by the system are also available as images.

Research on the line spectrum denoising detection based on multi-scale feature autoencoder

Line spectrum detection is an important research direction in the field of underwater acoustic target detection. The relevant features of the target can be obtained by using the target line spectrum in the signal power spectrogram, but the radiated noise signal is susceptible to the influence of external noise, which degrades the quality of the target line spectrum and thus affects the complete extraction of the line spectrum. A denoising method is proposed in this paper based on deep learning network architecture and classical filter to extract the target line spectrum from the power spectrogram with strong background noise while maintaining the continuity of the line spectrum and improving the efficiency of line spectrum detection. The simulated signal power spectrogram is used to train the multi-scale feature autoencoder to remove different scale noise interference in the line spectrum part and then pass the morphological attribute filter to remove the background noise further and keep the line spectrum continuity. The experimental analysis of the measured fishing vessel noise signal and the simulated signal shows that the algorithm proposed in this paper can effectively remove the line spectrum noise interference, improve the line spectrum signal-to-noise ratio quality, and realize the effective detection of the line spectrum.

Apply deep learning to improve the question analysis model in the Vietnamese question answering system

Question answering (QA) system nowadays is quite popular for automated answering purposes, the meaning analysis of the question plays an important role, directly affecting the accuracy of the system. In this article, we propose an improvement for question-answering models by adding more specific question analysis steps, including contextual characteristic analysis, pos-tag analysis, and question-type analysis built on deep learning network architecture. Weights of extracted words through question analysis steps are combined with the best matching 25 (BM25) algorithm to find the best relevant paragraph of text and incorporated into the QA model to find the best and least noisy answer. The dataset for the question analysis step consists of 19,339 labeled questions covering a variety of topics. Results of the question analysis model are combined to train the question-answering model on the data set related to the learning regulations of Industrial University of Ho Chi Minh City. It includes 17,405 pairs of questions and answers for the training set and 1,600 pairs for the test set, where the robustly optimized BERT pre-training approach (RoBERTa) model has an F1-score accuracy of 74%. The model has improved significantly. For long and complex questions, the mode has extracted weights and correctly provided answers based on the question’s contents.

Automated Sagittal Skeletal Classification of Children Based on Deep Learning.

Malocclusions are a type of cranio-maxillofacial growth and developmental deformity that occur with high incidence in children. Therefore, a simple and rapid diagnosis of malocclusions would be of great benefit to our future generation. However, the application of deep learning algorithms to the automatic detection of malocclusions in children has not been reported. Therefore, the aim of this study was to develop a deep learning-based method for automatic classification of the sagittal skeletal pattern in children and to validate its performance. This would be the first step in establishing a decision support system for early orthodontic treatment. In this study, four different state-of-the-art (SOTA) models were trained and compared by using 1613 lateral cephalograms, and the best performance model, Densenet-121, was selected was further subsequent validation. Lateral cephalograms and profile photographs were used as the input for the Densenet-121 model, respectively. The models were optimized using transfer learning and data augmentation techniques, and label distribution learning was introduced during model training to address the inevitable label ambiguity between adjacent classes. Five-fold cross-validation was conducted for a comprehensive evaluation of our method. The sensitivity, specificity, and accuracy of the CNN model based on lateral cephalometric radiographs were 83.99, 92.44, and 90.33%, respectively. The accuracy of the model with profile photographs was 83.39%. The accuracy of both CNN models was improved to 91.28 and 83.98%, respectively, while the overfitting decreased after addition of label distribution learning. Previous studies have been based on adult lateral cephalograms. Therefore, our study is novel in using deep learning network architecture with lateral cephalograms and profile photographs obtained from children in order to obtain a high-precision automatic classification of the sagittal skeletal pattern in children.