Parameters Of Convolutional Neural Network Research Articles

Consistent Least-Squares Reverse Time Migration Using Convolutional Neural Networks

The data-consistency item is a necessary condition for a reliable solution to the inverse problem. However, the current supervised-based deep-learning reconstruction approaches generally lack the data-consistency item, which directly leads to unreliable subsurface images for field data. To resolve this problem, we have developed a consistent least-squares reverse time migration (CLSRTM) approach using convolutional neural networks (CNNs), which is referred to as CNN-CLSRTM. The key point is that we have enforced that the predicted recording via the inverted image from the CNN model is consistent with the observed recording in the least-squares sense. We utilize the standard reverse time migration (RTM) image of single-shot recording as the input of the constructed CNN model. As a result, the optimal reflection image can be obtained by iteratively updating the parameters of CNN by minimizing the data residuals. Benefiting from the similarity of RTM images of adjacent recordings and the representation ability of the well-trained CNN model, we can directly predict the optimal reflection image for the testing datasets in a very fast way, which can greatly improve computational efficiency. Through synthetic and field data sets, we have determined that the proposed CNN-CLSRTM approach can retrieve high-resolution images with balanced amplitudes and continuous events. At the same time, our approach has better antinoise ability inherited from the benefit of CNN model compared to the standard LSRTM approach. In addition, we analyze the generalization ability of the CNN model for synthetic and field datasets.

Frame Boundary Detection and Deep Learning-Based Doppler Shift Estimation for FBMC/OQAM Communication System in Underwater Acoustic Channels

Underwater acoustic (UWA) communication is the only option for underwater long-distance wireless communication. However, its data rate is very low, in tens of kbps, and the time-varying nature of the UWA channel exhibits a Doppler effect. The data rate can be increased using a multicarrier technique, such as Filter Bank Multicarrier (FBMC). The FBMC-OQAM system does not require a cyclic prefix (CP) to eliminate intersymbol interference (ISI). Therefore, compared to other multicarrier techniques, the FBMC-OQAM system achieves the maximum spectral efficiency. Despite the well-designed filter, synchronisation errors cause performance degradation in the FBMC-OQAM system. In this paper, we propose frame boundary and Doppler frequency estimation methods for the FBMC-OQAM system under UWA channels. In this study, we propose a preamble-based frame boundary. The conjugate symmetry property is used in the frame boundary estimation. The performance of the proposed technique was studied under different UWA channels with different Doppler scale. The proposed time synchronisation technique yielded multiple peaks under UWA channels. A method for identifying the peak that represents the frame boundary is proposed. We compared the performance of proposed timing metric with existing three reference timing metrics. The proposed timing metric exhibited superior over two metrics and almost identical performance with the Liming’s metric with less computational complexity. In addition, we propose a convolutional neural network (CNN)-based Doppler scale estimation method. The proposed technique was shown to estimate a wide range of Doppler scale factors. The performance of the proposed technique was studied by considering different UWA channels at different SNRs. The proposed CNN architecture is analysed by changing the CNN parameters. Stacked autoencoders were adopted for Doppler scale estimation and the performance of the proposed CNN-based Doppler scale estimation method was compared with the stacked autoencoders based Doppler scale estimation method. Simulation results show that the proposed CNN-based Doppler Scale estimator exhibits good performance in UWA channels at different SNRs and Doppler scale factors.

Detection of atrial fibrillation using variable length genetic algorithm and convolutional neural network

AbstractAtrial fibrillation (AF) is the most prevalent cardiac arrhythmia and it is considered as one of the most important risk factor for death, stroke, hospitalization, and heart failure. It is possible to detect AF by analyzing electrocardiogram (ECG) of patients. To work on clean signals and reduce errors resulted from noise, we have used Butterworth filter. The short‐term Fourier transform was used to analyze ECG segments to obtain ECG spectrogram images. Convolutional neural network (CNN) models have been proposed for improving automatic detection of AF. The number of convolutional layers varies in different CNN models, and as the model become deeper, more hyper parameters are added. So in this article, variable length genetic algorithm was used in order to optimize hyper parameters of CNN. The results of experiments that performed on the MIT‐BIH AF database showed that the proposed method achieved 100%, 98.90%, and 99.95% for the sensitivity, specificity, and accuracy, respectively, so the proposed method outperforms the deep CNNs. Hence, the proposed method is an accurate and efficient method for detection of AF.

Deep Learning to Detect and Classify the Purity Level of Luwak Coffee Green Beans

Luwak coffee (palm civet coffee) is known as one of the most expensive coffee in the world. In order to lower production costs, Indonesian producers and retailers often mix high-priced Luwak coffee with regular coffee green beans. However, the absence of tools and methods to classify Luwak coffee counterfeiting makes the sensing method’s development urgent. The research aimed to detect and classify Luwak coffee green beans purity into the following purity categories, very low (0-25%), low (25-50%), medium (50-75%), and high (75-100%). The classifying method relied on a low-cost commercial visible light camera and the deep learning model method. Then, the research also compared the performance of four pre-trained convolutional neural network (CNN) models consisting of SqueezeNet, GoogLeNet, ResNet-50, and AlexNet. At the same time, the sensitivity analysis was performed by setting the CNN parameters such as optimization technique (SGDm, Adam, RMSProp) and the initial learning rate (0.00005 and 0.0001). The training and validation result obtained the GoogLeNet as the best CNN model with optimizer type Adam and learning rate 0.0001, which resulted in 89.65% accuracy. Furthermore, the testing process using confusion matrix from different sample data obtained the best CNN model using ResNet-50 with optimizer type RMSProp and learning rate 0.0001, providing an accuracy average of up to 85.00%. Later, the CNN model can be used to establish a real-time, non-destructive, rapid, and precise purity detection system.

Lightweight end-to-end image steganalysis based on convolutional neural network

With the continuous improvement in the accuracy of steganalysis based on convolutional neural networks (CNNs), the network scale has shown explosive growth. Consequently, CNNs have a high demand for hardware resources and time-consuming training. To reduce the number of CNN parameters and improve the efficiency of steganalysis, we propose a lightweight steganalysis CNN called W-Net. The proposed W-Net first uses grouped convolution and channel shuffling units to extract noise residuals, strengthen the information exchange between groups, and improve feature extraction. In addition, the depth-wise separable convolution is applied to obtain different channels and spaces. The fusion of position information achieves the effect of conventional convolution while reducing the number of network parameters. We verified the effect of activation functions on steganalysis accuracy through experiments. In addition, the proposed W-Net can detect the steganographic data from the S-UNIWARAD spatial steganography algorithm with an embedding rate of 0.4 bpp. Compared with Xu-Net and Zhu-Net, the proposed W-Net improves the detection accuracy by 12.70% and 6.38%, respectively.

A High-Level Modeling Framework for Estimating Hardware Metrics of CNN Accelerators

GPUs became the reference platform for both training and inference phases of Convolutional Neural Networks (CNN) due to their tailored architecture to the CNN operators. However, GPUs are power-hungry architectures. A path to enable the deployment of CNNs in energy-constrained devices is adopting hardware accelerators for the inference phase. The design space exploration of CNNs using standard approaches, such as RTL, is limited due to their complexity. Thus, designers need frameworks enabling design space exploration that delivers accurate hardware estimation metrics to deploy CNNs. This work proposes a framework to explore CNNs design space, providing power, performance, and area (PPA) estimations. The heart of the framework is a system simulator. The system simulator front-end is TensorFlow, and the back-end is performance estimations obtained from the physical synthesis of hardware accelerators, not only from components like multipliers and adders. The first set of results evaluate the CNN accuracy using integer quantization, the accelerators PPA after physical synthesis, and the benefits of using a system simulator. These results allow a rich design space exploration, enabling selecting the best set of CNN parameters to meet the design constraints.

Semantic-Based Classification of Long Texts on Higher Education in China

The development level of higher education (HE) is an important indicator of the development level and development potential of a country. The HE-related document is the mirror to reflect the develop process of the HE. The research of high education (HE) has been developing rapidly in China, resulting in a huge number of texts, such as relevant policies, speech drafts, and yearbooks. The traditional manual classification of HE texts is inefficient and unable to deal with the huge number of HE texts. Besides, the effect of direct classification is rather poor because HE texts tend to be long and exist as an imbalanced dataset. To solve these problems, this paper improves the convolutional neural network (CNN) into the HE-CNN classification model for HE texts. Firstly, Chinese HE policies, speech drafts, and yearbooks (1979–2020) were downloaded from the official website of Chinese Ministry of Education. In total, 463 files were collected and divided into four classes, namely, definition, task, method, and effect evaluation. To handle the huge number of HE texts, the Twitter-latent Dirichlet allocation (LDA) topic model was employed to extract word frequency and critical information, such as age and author, enhancing the training effect of CNN. To address the dataset imbalance problem, CNN parameters were optimized repeatedly through comparative experiments, which further improve the training effect. Finally, the proposed HE-CNN model was found more effective and accurate than other classification models.

An automatic arrhythmia classification model based on improved Marine Predators Algorithm and Convolutions Neural Networks

Preparation of Convolutional Neural Networks (CNNs) for classification purposes depends heavily on the knowledge of hyper-parameters tuning. This study aims, in particular in task of automated electrocardiograms (ECG), to minimize the user variability in the CNN training by searching and optimizing the CNN parameters automatically. In the clinical practice, the task of ECG classification analysis is restricted by existing models’ configuration. The hyper-parameters of the CNN model should be adjusted for the ECG classification problem. The best configuration for hyper-parameters is selected to have an impact on the production of the model. Deep knowledge of deep learning algorithms and suitable optimization techniques are also needed. Although there are many strategies for automated optimization, different benefits and disadvantages occur as they are applied to ECG classification problem. Here we present a CNN model for classification of non-ectopic (N), ventricular ectopic (V), supraventricular ectopic (S), and fusion (F) ECG rhythmias by the hybrid models based on modified version of Marine Predators algorithm (MPA) and CNN, known as the IMPA-CNN. The proposed model summarizes the feature extraction techniques of major features and, thus, outperforms other current classification models through automatically select the best hyper-parameters configuration of the CNN model. To reduce the time and complication complexity, optimum characteristics have been extracted directly from the raw signal using 1D-local binary pattern, higher order statistics, central moment, Hermite basis function discrete wavelet transform, and R–R intervals. Then, a modified version of MPA algorithm is used to select appropriate hyper-parameters for the CNN model like initial learning rate for the CNN model that is one of the major hyper parameters effect output performance, optimizer type which can be set to stochastic gradient descent (SGD), adaptive moment estimation (Adam), root mean square propagation (RMSprop), the activation function form used for modeling non-linear functions, set to ‘Rectified Linear Unit (ReLU), or ‘sigmoid’ and some other hyper-parameters are related to the optimization and training process of CNN model. Many available optimization algorithms for hyper-parameters optimization problems are provided. In addition, experiments with well know data sets like MIT-BIH arrhythmia, European ST-T database, and St. Petersburg INCART database are carried out to compare the performance of various optimization approaches and to provide practical illustration of the optimization of hyper-parameters for the proposed CNN model.

A CSI-Based Indoor Positioning System Using Single UWB Ranging Correction.

A fingerprint-based localization system is an economic way to solve an indoor positioning problem. However, the traditional off-line fingerprint collection stage is a time-consuming and laborious process which limits the use of fingerprint-based localization systems. In this paper, based on ubiquitous Wireless Fidelity (Wi-Fi) equipment and a low-cost Ultra-Wideband (UWB) ranging system (with only one UWB anchor), a ready-to-use indoor localization system is proposed to realize long-term and high-accuracy indoor positioning. More specifically, in this system, it is divided into two stages: (1) an initial stage, and (2) a positioning stage. In the initial stage, an Inertial Measure Unit (IMU) is used to calculate the position using Pedestrian Dead Reckon (PDR) algorithm within a preset number of steps, and the location-related fingerprints are collected to train a Convolutional Neural Network (CNN) regression model; simultaneously, in order to make the UWB ranging system adapt to the Non-Line-of-Sight (NLoS) environment, the increments of acceleration and angular velocity in IMU and the increments of single UWB ranging measures are correlated to pre-train a Supported Vector Regression (SVR). After reaching the threshold of time or step number, the system is changed into a positioning stage, and the CNN predicts the position calibrated by corrected UWB ranging. At last, a series of practical experiments are conducted in the real environment; the experiment results show that, due to the corrected UWB ranging measures calibrating the CNN parameters in every positioning period, this system has stable localization results in a comparative long-term range. Additionally, it has the advantages of stability, low cost, anti-noise, etc.

A New Method to Improve the Performance of Deep Neural Networks in Detecting P300 Signals: Optimizing Curvature of Error Surface Using Genetic Algorithm.

Background: Deep neural networks have been widely used in detection of P300 signal in Brain Machine Interface (BMI) systems which are rely on Event-Related Potentials (ERPs) (i.e. P300 signals). Such networks have high curvature variation in their error surface hampering their favorable performance. Therefore, the variations in curvature of the error surface must be minimized to improve the performance of these networks in detecting P300 signals. Objective: The aim of this paper is to introduce a method for minimizing the curvature of the error surface during training Convolutional Neural Network (CNN). The curvature variation of the error surface is highly dependent on model parameters of deep neural network; therefore, we try to minimize this curvature by optimizing the model parameters.Material and Methods: In this experimental study an attempt is made to tune the CNN parameters affecting the curvature of its error surface in order to obtain the best possible learning. For achieving this goal, Genetic Algorithm is utilized to optimize the above parameters in order to minimize the curvature variations. Results: The performance of the proposed algorithm was evaluated on EPFL dataset. The obtained results demonstrated that the proposed method detected the P300 signals with maximally 98.91% classification accuracy and 98.54% True Positive Ratio (TPR). Conclusion: The obtained results showed that using genetic algorithm for minimizing curvature of the error surface in CNN increased its accuracy in parallel with decreasing the variance of the results. Consequently, it may be concluded that the proposed method has considerable potential to be used as P300 detection module in BMI applications.

Automatic segmentation of left and right ventricles in cardiac MRI using 3D-ASM and deep learning

Segmentation of the left and right ventricles in cardiac MRI (Magnetic Resonance Imaging) is a prerequisite step for evaluating global and regional cardiac function. This work presents a novel and robust schema for MRI segmentation by combining the advantages of deep learning localization and 3D-ASM (3D Active Shape Model) restriction without any user interaction. Three fundamental techniques are exploited: (1) manual 2D contours are used to build distance maps to get 3D ground truth shape, (2) derived right ventricle points are employed to rotate the coarse initial shape for a refined bi-ventricle initial estimation, (3) segmentation results from deep learning are utilised to build distance maps for the 3D-ASM matching process to help image intensity modelling. The datasets used for experimenting the cine MRI data are 1000 cases from UK Biobank, 500 subjects are selected to train CNN (Convolution Neural Network) parameters, and the remaining 500 cases are adopted for validation. Specifically, cases are used to rebuild point distribution and image intensity models, and also utilized to train CNN. In addition, the left 500 cases are used to perform the validation experiments. For the segmentation of the RV (Right Ventricle) endocardial contour, LV (Left Ventricle) endo- and epicardial contours, overlap, Jaccard similarity index, Point-to-surface errors and cardiac functional parameters are calculated. Experimental results show that the proposed method has advantages over the previous approaches.

EFFICIENT CNN-BASED PROFILED SIDE CHANNEL ATTACKS

Profiled side-channel attacks are now considered as powerful forms of attacks used to break the security of cryptographic devices. A recent line of research has investigated a new profiled attack based on deep learning and many of them have used convolution neural network (CNN) as deep learning architecture for the attack. The effectiveness of the attack is greatly influenced by the CNN architecture. However, the CNN architecture used for current profiled attacks have often been based on image recognition fields, and choosing the right CNN architectures and parameters for adaption to profiled attacks is still challenging. In this paper, we propose an efficient profiled attack for on unprotected and masking-protected cryptographic devices based on two CNN architectures, called CNNn, CNNd respectively. Both of CNN architecture parameters proposed in this paper are based on the property of points of interest on the power trace and further determined by the Grey Wolf Optimization (GWO) algorithm. To verify the proposed attacks, experiments were performed on a trace set collected from an Atmega8515 smart card when it performs AES-128 encryption, a DPA contest v4 dataset and the ASCAD public dataset

Convolutional neural networks based on fractional-order momentum for parameter training

This paper proposes a parameter training method via the fractional-order momentum for convolutional neural networks (CNNs). To update the parameters of CNNs more smoothly, the parameter training method via the fractional-order momentum is proposed based on the Grünwald-Letnikov (G-L) difference operation. The stochastic classical momentum (SCM) algorithm and adaptive moment (Adam) estimation algorithm are improved by replacing the integer-order difference with the fractional-order difference. Meanwhile, the linear and the nonlinear methods are discussed to adjust the fractional-order. Therefore, the proposed methods can improve the flexibility and the adaptive ability of CNN parameters. We analyze the validity of the methods by using MNIST dataset and CIFAR-10 dataset, and the experimental results show that the proposed methods can improve the recognition accuracy and the learning convergence speed of CNNs compared with the traditional SCM and Adam methods.

Research on Feature Extracted Method for Flutter Test Based on EMD and CNN

Due to the inevitable deviations between the results of theoretical calculations and physical experiments, flutter tests and flutter signal analysis often play significant roles in designing the aeroelasticity of a new aircraft. The measured structural response from aeroelastic models in both wind tunnel tests and real fight flutter tests contain an abundance of structural information, but traditional methods tend to have limited ability to extract features of concern. Inspired by deep learning concepts, a novel feature extraction method for flutter signal analysis was established in this study by combining the convolutional neural network (CNN) with empirical mode decomposition (EMD). It is widely hypothesized that when flutter occurs, the measured structural signals are harmonic or divergent in the time domain, and that the flutter modal (1) is singular and (2) its energy increases significantly in the frequency domain. A measured-signal feature extraction and flutter criterion framework was constructed accordingly. The measured signals from a wind tunnel test were manually labeled “flutter” and “no-flutter” as the foundational dataset for the deep learning algorithm. After the normalized preprocessing, the intrinsic mode functions (IMFs) of the flutter test signals are obtained by the EMD method. The IMFs are then reshaped to make them the suitable size to be input to the CNN. The CNN parameters are optimized though the training dataset, and the trained model is validated through the test dataset (i.e., cross-validation). The accuracy rate of the proposed method reached 100% on the test dataset. The training model appears to effectively distinguish whether or not the structural response signal contains flutter. The combination of EMD and CNN provides effective feature extraction of time series signals in flutter test data. This research explores the connection between structural response signals and flutter from the perspective of artificial intelligence. The method allows for real-time, online prediction with low computational complexity.

Investigation of convolutional neural networks using multiple computed tomography perfusion maps to identify infarct core in acute ischemic stroke patients.

Purpose: To assess acute ischemic stroke (AIS) severity, infarct is segmented using computed tomography perfusion (CTP) software, such as RAPID, Sphere, and Vitrea, relying on contralateral hemisphere thresholds. Since this approach is potentially patient dependent, we investigated whether convolutional neural networks (CNNs) could achieve better performances without the need for contralateral hemisphere thresholds. Approach: CTP and diffusion-weighted imaging (DWI) data were retrospectively collected for 63 AIS patients. Cerebral blood flow (CBF), cerebral blood volume (CBV), time-to-peak, mean-transit-time (MTT), and delay time maps were generated using Vitrea CTP software. U-net shaped CNNs were developed, trained, and tested for 26 different input CTP parameter combinations. Infarct labels were segmented from DWI volumes registered with CTP volumes. Infarct volumes were reconstructed from two-dimensional CTP infarct segmentations. To remove erroneous segmentations, conditional random field (CRF) postprocessing was applied and compared with prior results. Spatial and volumetric infarct agreement was assessed between DWI and CTP (CNNs and commercial software) using median infarct difference, median absolute error, dice coefficient, positive predictive value. Results: The most accurate combination of parameters for CNN segmenting infarct using CRF postprocessing was CBF, CBV, and MTT (4.83mL, 10.14mL, 0.66, 0.73). Commercial software results are: RAPID = (2.25mL, 21.48mL, 0.63, 0.70), Sphere = (7.57mL, 17.74mL, 0.64, 0.70), Vitrea = (6.79mL, 15.28mL, 0.63, 0.72). Conclusions: Use of CNNs with multiple input perfusion parameters has shown to be accurate in segmenting infarcts and has the ability to improve clinical workflow by eliminating the need for contralateral hemisphere comparisons.

Hyperspectral Image Classification via Discriminant Gabor Ensemble Filter.

For a broad range of applications, hyperspectral image (HSI) classification is a hot topic in remote sensing, and convolutional neural network (CNN)-based methods are drawing increasing attention. However, to train millions of parameters in CNN requires a large number of labeled training samples, which are difficult to collect. A conventional Gabor filter can effectively extract spatial information with different scales and orientations without training, but it may be missing some important discriminative information. In this article, we propose the Gabor ensemble filter (GEF), a new convolutional filter to extract deep features for HSI with fewer trainable parameters. GEF filters each input channel by some fixed Gabor filters and learnable filters simultaneously, then reduces the dimensions by some learnable 1×1 filters to generate the output channels. The fixed Gabor filters can extract common features with different scales and orientations, while the learnable filters can learn some complementary features that Gabor filters cannot extract. Based on GEF, we design a network architecture for HSI classification, which extracts deep features and can learn from limited training samples. In order to simultaneously learn more discriminative features and an end-to-end system, we propose to introduce the local discriminant structure for cross-entropy loss by combining the triplet hard loss. Results of experiments on three HSI datasets show that the proposed method has significantly higher classification accuracy than other state-of-the-art methods. Moreover, the proposed method is speedy for both training and testing.

Early-Stage Neural Network Hardware Performance Analysis

The demand for running NNs in embedded environments has increased significantly in recent years due to the significant success of convolutional neural network (CNN) approaches in various tasks, including image recognition and generation. The task of achieving high accuracy on resource-restricted devices, however, is still considered to be challenging, which is mainly due to the vast number of design parameters that need to be balanced. While the quantization of CNN parameters leads to a reduction of power and area, it can also generate unexpected changes in the balance between communication and computation. This change is hard to evaluate, and the lack of balance may lead to lower utilization of either memory bandwidth or computational resources, thereby reducing performance. This paper introduces a hardware performance analysis framework for identifying bottlenecks in the early stages of CNN hardware design. We demonstrate how the proposed method can help in evaluating different architecture alternatives of resource-restricted CNN accelerators (e.g., part of real-time embedded systems) early in design stages and, thus, prevent making design mistakes.

Analysis and best parameters selection for person recognition based on gait model using CNN algorithm and image augmentation

Person Recognition based on Gait Model (PRGM) and motion features is are indeed a challenging and novel task due to their usages and to the critical issues of human pose variation, human body occlusion, camera view variation, etc. In this project, a deep convolution neural network (CNN) was modified and adapted for person recognition with Image Augmentation (IA) technique depending on gait features. Adaptation aims to get best values for CNN parameters to get best CNN model. In Addition to the CNN parameters Adaptation, the design of CNN model itself was adapted to get best model structure; Adaptation in the design was affected the type, the number of layers in CNN and normalization between them. After choosing best parameters and best design, Image augmentation was used to increase the size of train dataset with many copies of the image to boost the number of different images that will be used to train Deep learning algorithms. The tests were achieved using known dataset (Market dataset). The dataset contains sequential pictures of people in different gait status. The image in CNN model as matrix is extracted to many images or matrices by the convolution, so dataset size may be bigger by hundred times to make the problem a big data issue. In this project, results show that adaptation has improved the accuracy of person recognition using gait model comparing to model without adaptation. In addition, dataset contains images of person carrying things. IA technique improved the model to be robust to some variations such as image dimensions (quality and resolution), rotations and carried things by persons. Results for 200 persons recognition, validation accuracy was about 82% without IA and 96.23 with IA. For 800 persons recognition, validation accuracy was 93.62% without IA.

Kalman Filter-Based Convolutional Neural Network for Robust Tracking of Froth-Middling Interface in a Primary Separation Vessel in Presence of Occlusions

Bitumen in the oil sands industry is separated from sand using a water-based gravity separation process in a primary separation vessel (PSV). The interface between the froth and the middlings layer is an important parameter to control for optimal operation of the PSV unit. In this paper, a method using computer vision based on convolutional neural networks (CNNs) and Kalman filter (KF) is designed for the detection of the interface in PSV, with the CNN estimating both the interface level and the image quality. The proposed method consists of two parts: an offline and an online stage, wherein the parameters of CNN and KF are trained in the offline stage using the available data. The algorithm is made robust to any new type of occlusions, not present in the training dataset, in the online stage. Experimental results demonstrate that the proposed method is accurate and robust to different abnormalities in the process.

Heterogeneous Defect Prediction Based on Federated Transfer Learning via Knowledge Distillation

Heterogeneous defect prediction (HDP) aims to predict defect-prone software modules in one project using heterogeneous data collected from other projects. There are two characteristics of defect data: data islands, and data privacy. In this article, we propose a novel Federated Transfer Learning via Knowledge Distillation (FTLKD) approach for HDP, which takes into consideration two characteristics of defect data. Firstly, Shamir sharing technology achieves homomorphic encryption for private data. During subsequent processing and operations, data remains encrypted all the time. Secondly, each participant uses public data to train convolutional neural networks(CNN), the parameters of the pre-trained CNN are transferred to a private model. A small amount of labeled private data fine-tunes the private model. Finally, knowledge distillation realizes the communication between the participants. The average of all softmax output (logits) is used for knowledge distillation to update the private models. Extensive experiments on 9 projects in 3 public databases (NASA, AEEEM and SOFTLAB) show that FTLKD outperforms the related competing methods.