Pre-trained Deep Network Research Articles

Automated Caries Screening Using Ensemble Deep Learning on Panoramic Radiographs

Caries prevention is essential for oral hygiene. A fully automated procedure that reduces human labor and human error is needed. This paper presents a fully automated method that segments tooth regions of interest from a panoramic radiograph to diagnose caries. A patient’s panoramic oral radiograph, which can be taken at any dental facility, is first segmented into several segments of individual teeth. Then, informative features are extracted from the teeth using a pre-trained deep learning network such as VGG, Resnet, or Xception. Each extracted feature is learned by a classification model such as random forest, k-nearest neighbor, or support vector machine. The prediction of each classifier model is considered as an individual opinion that contributes to the final diagnosis, which is decided by a majority voting method. The proposed method achieved an accuracy of 93.58%, a sensitivity of 93.91%, and a specificity of 93.33%, making it promising for widespread implementation. The proposed method, which outperforms existing methods in terms of reliability, and can facilitate dental diagnosis and reduce the need for tedious procedures.

All-star approach to a small medical imaging dataset: combined deep, transfer, and classical machine learning approaches for the determination of radial head fractures.

Radial head fractures are often evaluated in emergency departments and can easily be missed. Automated or semi-automated detection methods that help physicians may be valuable regarding the high miss rate. To evaluate the accuracy of combined deep, transfer, and classical machine learning approaches on a small dataset for determination of radial head fractures. A total of 48 patients with radial head fracture and 56 patients without fracture on elbow radiographs were retrospectively evaluated. The input images were obtained by cropping anteroposterior elbow radiographs around a center-point on the radial head. For fracture determination, an algorithm based on feature extraction using distinct prototypes of pretrained networks (VGG16, ResNet50, InceptionV3, MobileNetV2) representing four different approaches was developed. Reduction of feature space dimensions, feeding the most relevant features, and development of ensemble of classifiers were utilized. The algorithm with the best performance consisted of preprocessing the input, computation of global maximum and global mean outputs of four distinct pretrained networks, dimensionality reduction by applying univariate and ensemble feature selectors, and applying Support Vector Machines and Random Forest classifiers to the transformed and reduced dataset. A maximum accuracy of 90% with MobileNetV2 pretrained features was reached for fracture determination with a small sample size. Radial head fractures can be determined with a combined approach and limitations of the small sample size can be overcome by utilizing pretrained deep networks with classical machine learning methods.

Performance Investigation of Pre-Trained Convolutional Neural Networks in Olive Leaf Disease Classification

Zeytin ülkemizin belirli bölgelerinde yetişen oldukça önemli bir üründür. Gümrük ve Ticaret Bakanlığı’nın verilerine göre 2019 yılında yaklaşık 420 bin ton sofralık zeytin üretimi ile dünyadaki toplam üretimin %14’ten fazlası ülkemizde yapılmıştır. Böylece, zeytin yaprağındaki hastalıkların erken teşhisi ve tedavisi üretim kapasitesinin artmasına yol açabilir. Günümüzde birçok alanda olduğu gibi bitki hastalıklarının teşhisi için derin öğrenme algoritmaları yaygın olarak kullanılmaktadır. Bu çalışmada, AlexNet, SqueezeNet, ShuffleNet ve GoogleNet gibi sıklıkla tercih edilen ön eğitimli derin öğrenme ağları ile zeytin yaprağındaki hastalıkların sınıflandırılması gerçekleştirilmiştir. Ağ yapıları, zeytin yaprağındaki hastalıkların etiketlerine göre eğitim için yeniden düzenlenmiştir. Veri setinde, veri çoğaltma işlemi uygulanarak hem ham veri seti hem de çoğaltılmış veri seti için ayrı ayrı performans sonuçları alınmıştır. Elde edilen sonuçlar doğruluk, duyarlılık, özgüllük, kesinlik ve F1-Skor gibi performans ölçütleri ile değerlendirilmiştir. En iyi performans iyileştirmesi %7,56 ile AlexNet’in doğruluk değeri için elde edilirken, en düşük iyileştirme oranı %0,63 ile ShuffleNet’in özgüllük değerinden elde edilmiştir.

Rollback Ensemble With Multiple Local Minima in Fine-Tuning Deep Learning Networks.

Image retrieval is a challenging problem that requires learning generalized features enough to identify untrained classes, even with very few classwise training samples. In this article, to obtain generalized features further in learning retrieval data sets, we propose a novel fine-tuning method of pretrained deep networks. In the retrieval task, we discovered a phenomenon in which the loss reduction in fine-tuning deep networks is stagnated, even while weights are largely updated. To escape from the stagnated state, we propose a new fine-tuning strategy to roll back some of the weights to the pretrained values. The rollback scheme is observed to drive the learning path to a gentle basin that provides more generalized features than a sharp basin. In addition, we propose a multihead ensemble structure to create synergy among multiple local minima obtained by our rollback scheme. Experimental results show that the proposed learning method significantly improves generalization performance, achieving state-of-the-art performance on the Inshop and SOP data sets.

TDA-Net: A Novel Transfer Deep Attention Network for Rapid Response to Building Damage Discovery

The rapid and accurate discovery of damage information of the affected buildings is of great significance for postdisaster emergency rescue. In some related studies, the models involved can detect damaged buildings relatively accurately, but their time cost is high. Models that can guarantee both detection accuracy and high efficiency are urgently needed. In this paper, we propose a new transfer-learning deep attention network (TDA-Net). It can achieve a balance of accuracy and efficiency. The benchmarking network for TDA-Net uses a pair of deep residual networks and is pretrained on a large-scale dataset of disaster-damaged buildings. The pretrained deep residual networks have strong sensing properties on the damage information, which ensures the effectiveness of the network in prefeature grasping. In order to make the network have a more robust perception of changing features, a set of deep attention bidirectional encoding and decoding modules is connected after the TDA-Net benchmark network. When performing a new task, only a small number of samples are needed to train the network, and the damage information of buildings in the whole area can be extracted. The bidirectional encoding and decoding structure of the network allows two images to be input into the model independently, which can effectively capture the features of a single image, thereby improving the detection accuracy. Our experiments on the xView2 dataset and three datasets of disaster regions achieve high detection accuracy, which demonstrates the feasibility of our method.

Evolutionary deep feature selection for compact representation of gigapixel images in digital pathology

Despite the recent progress in Deep Neural Networks (DNNs) to characterize histopathology images, compactly representing a gigapixel whole-slide image (WSI) via salient features to enable computational pathology is still an urgent need and a significant challenge. In this paper, we propose a novel WSI characterization approach to represent, search and classify biopsy specimens using a compact feature vector (CFV) extracted from a multitude of deep feature vectors. Since the non-optimal design and training of deep networks may result in many irrelevant and redundant features and also cause computational bottlenecks, we proposed a low-cost stochastic method to optimize the output of pre-trained deep networks using evolutionary algorithms to generate a very small set of features to accurately represent each tissue/biopsy. The performance of the proposed method has been assessed using WSIs from the publicly available TCGA image data. In addition to acquiring a very compact representation (i.e., 11,000 times smaller than the initial set of features), the optimized features achieved 93% classification accuracy resulting in 11% improvement compared to the published benchmarks. The experimental results reveal that the proposed method can reliably select salient features of the biopsy sample. Furthermore, the proposed approach holds the potential to immensely facilitate the adoption of digital pathology by enabling a new generation of WSI representation for efficient storage and more user-friendly visualization.

An Intelligent Gender Classification System in the Era of Pandemic Chaos with Veiled Faces

In the world of chaos, the pandemic has driven individuals around the globe to wear face masks for preventing the virus’s transmission, however, this has made it difficult to determine the gender of the person wearing a mask. Gender information is part of soft biometrics, which provides extra information about a person’s identification, thus, identifying a gender based on a veiled face is among the urgent challenges that must be advocated for in the next decade. Therefore, this study exploited various pre-trained deep learning networks (DenseNet121, DenseNet169, ResNet50, ResNet101, Xception, InceptionV3, MobileNetV2, EfficientNetB0, and VGG16) to analyze the effect of the mask while identifying the gender using facial images of human beings. The study comprises two strategies. First, the experimental part involves the training of models using facial images with and without masks, while the second strategy considers images with masks only, to train the pre-trained models. Experimental results reveal that DenseNet121 and Xception networks performed well for both strategies. Besides this, the Inception network outperformed all others by attaining 98.75% accuracy for the first strategy, whereas EfficientNetB0 performed well for the second strategy by securing 97.27%. Moreover, results suggest that facemasks evidently impact the performance of state-of-the-art pre-trained networks for gender classification.

Driver drowsiness detection in video sequences using hybrid selection of deep features

Monitoring driver’s drowsiness is a complex problem that involves many indicators whether behavioral or physiological. Drowsiness is a challenging problem that can lead to road disasters. Sleeping driver is more dangerous on the road than a speeding driver. Many statistics showed that one-fifth of road accidents in the world were due to driver fatigue, hence safety modules that can alert drowsy drivers in the hopes of reducing the risk of accidents are very important. This paper proposes a framework for driver drowsiness detection based on a computer vision solution. The proposed framework’s first task is to detect the driver’s face. A transfer learning is then performed for extracting the deep features from the driver’s face image using a pre-trained deep convolutional network model trained on a facial recognition dataset. The previous tasks are applied in a sliding temporal window (less than a second) in which the frames are sampled. In this work, 9 frames were the best choice. The extracted features of these frames represent the observation matrix. Then temporal feature aggregation is applied to construct the raw feature vector. To obtain the final feature vector, a proposed feature selection is applied to omit possible irrelevant features. The final feature vector is finally fed to a binary classifier to decide whether there is drowsiness or not. Extensive experiments are applied to NTHU Drowsy Driver Detection (NTHU-DDD) video dataset. The outcomes show the outperformance of the proposed approach compared with the state-of-the-art approaches.

Design and implementation of transfer learned deep CNN with feature fusion for automated mammogram classification

Radiologists frequently struggle to define mammography mass lesions, resulting in unneeded breast biopsies to eliminate suspicions, which adds exorbitant costs to an already overburdened patient and medical system..Existing models have limited capability for feature extraction and representation, as well as cancer classification. Therefore, we built deep Convolution neural networks based Computer-aided Diagnosis system to assist radiologists in classifying mammography mass lesions. Here, Two-view LASSO regression feature fusion and fine-tuned transfer learning network model VGG16 were applied for identification of mammogram cancer. First, two independent CNN branches are utilized to extract mammography characteristics from two different perspectives. Feature Extraction is performed by fine-tuning pre-trained deep network models VGG16 which extracts deep convolutional features. Second, the features of the VGG16 models are serially fused using LASSO regression. Lastly, the fused features are entered into the Fully Connected Layer for mammogram classification. The high accuracy of 95.24, senstitivity of 96.11% and AUC score of 97.95% of the proposed approach revealed that it should be used to enhance clinical decision-making.

Deep neural network for diagnostics of railway track components

The problem of increasing the speed of railway transportation and ensuring reliability is associated with constant monitoring of the condition of the railway tracks. The modern track measuring cars are equipped with video cameras and computer equipment for processing the received information. However, manual processing of data by operators in real-time is not possible. The article proposes a deep convolutional neural network for automatically recognizing and classifying defects in rail joints on rail track images. The rail video observation forms the image array during the passage of the track recording car. The formation of classes of rail joints is described. Regular rail joints with connectors, insulating joints, and welded joints are considered. Additional classes are identified, corresponding to various anomalous configurations of rail joints in the images. A modified structure of a pre-trained deep convolutional network is constructed. When preparing training samples, the actual images of rail joints were supplemented with artificial images obtained by affine transformations. The process of training and testing the classifier based on the developed convolutional network is described. The Transfer Learning is used to train the neural network. As a result of the experiments, the accuracy of classifying rail joints and detecting defects was at least 96%.

Detection and classification of chilli leaf disease using a squeeze-and-excitation-based CNN model

Chilli leaf disease has a destructive effect on the chilli crop yield. Chilli leaf disease can result in a significant decrease in both the quantity and quality of the chilli crop. Early detection, perfect identification and accurately diagnosing the disease will aid in increasing the profit of the cultivator. However, after a comprehensive survey investigation, we discovered that no studies have been previously conducted to compare the classification performance of machine learning and deep learning for the chilli leaf disease classification problem. In this study, five main leaf diseases i.e. down curl of a leaf, Geminivirus, Cercospora leaf spot, yellow leaf disease, and up curl disease were identified, and images were captured using a digital camera and are labelled. These diseases were classified using 12 different pretrained deep learning networks (AlexNet, DarkNet53, DenseNet201, EfficientNetb0, InceptionV3, MobileNetV2, NasNetLarge, ResNet101, ShuffleNet, SqueezeNet, VGG19, and XceptionNet) using chilli leaf data with and without augmentation using deep learning transfer. Performance metrics such as accuracy, recall, precision, F1-score, specificity, and misclassification were calculated for each network. VGG19 had the best accuracy (83.54%) without augmentation, and DarkNet53 had the best result (98.82%) with augmentation among all pretrained deep learning networks in our self-built chilli leaf dataset. The result was enhanced by designing a squeeze-and-excitation-based convolutional neural network (SECNN) model. The model was tested on a chilli leaf dataset with different input sizes and mini-batch sizes. The proposed model produced the best accuracy of 98.63% and 99.12% without and with augmentation, respectively. The SECNN model was also tested on different datasets from the PlantVillage data, including apple, cherry, corn, grape, peach, pepper, potato, strawberry, and tomato leaves, separately and with the chilli dataset. The proposed model achieved an accuracy of 99.28% in classifying 43 different classes of plant leaf datasets.

PredPromoter-MF(2L): A Novel Approach of Promoter Prediction Based on Multi-source Feature Fusion and Deep Forest.

Promoters short DNA sequences play vital roles in initiating gene transcription. However, it remains a challenge to identify promoters using conventional experiment techniques in a high-throughput manner. To this end, several computational predictors based on machine learning models have been developed, while their performance is unsatisfactory. In this study, we proposed a novel two-layer predictor, called PredPromoter-MF(2L), based on multi-source feature fusion and ensemble learning. PredPromoter-MF(2L) was developed based on various deep features learned by a pre-trained deep learning network model and sequence-derived features. Feature selection based on XGBoost was applied to reduce fused features dimensions, and a cascade deep forest model was trained on the selected feature subset for promoter prediction. The results both fivefold cross-validation and independent test demonstrated that PredPromoter-MF(2L) outperformed state-of-the-art methods.

Feature Generalization for Breast Cancer Detection in Histopathological Images.

Recent period has witnessed benchmarked performance of transfer learning using deep architectures in computer-aided diagnosis (CAD) of breast cancer. In this perspective, the pre-trained neural network needs to be fine-tuned with relevant data to extract useful features from the dataset. However, in addition to the computational overhead, it suffers the curse of overfitting in case of feature extraction from smaller datasets. Handcrafted feature extraction techniques as well as feature extraction using pre-trained deep networks come into rescue in aforementioned situation and have proved to be much more efficient and lightweight compared to deep architecture-based transfer learning techniques. This research has identified the competence of classifying breast cancer images using feature engineering and representation learning over the established and contemporary notion of using transfer learning techniques. Moreover, it has revealed superior feature learning capacity with feature fusion in contrast to the conventional belief of understanding unknown feature patterns better with representation learning alone. Experiments have been conducted on two different and popular breast cancer image datasets, namely, KIMIA Path960 and BreakHis datasets. A comparison of image-level accuracy is performed on these datasets using the above-mentioned feature extraction techniques. Image level accuracy of 97.81% is achieved for KIMIA Path960 dataset using individual features extracted with handcrafted (color histogram) technique. Fusion of uniform Local Binary Pattern (uLBP) and color histogram features has resulted in 99.17% of highest accuracy for the same dataset. Experimentation with BreakHis dataset has resulted in highest classification accuracy of 88.41% with color histogram features for images with 200X magnification factor. Finally, the results are contrasted to that of state-of-the-art and superior performances are observed on many occasions with the proposed fusion-based techniques. In case of BreakHis dataset, the highest accuracies 87.60% (with least standard deviation) and 85.77% are recorded for 200X and 400X magnification factors, respectively, and the results for the aforesaid magnification factors of images have exceeded the state-of-the-art.

Fine-tuning-based Transfer Learning for Characterization of Adeno-Associated Virus.

Accurate and precise identification of adeno-associated virus (AAV) vectors play an important role in dose-dependent gene therapy. Although solid-state nanopore techniques can potentially be used to characterize AAV vectors by capturing ionic current, the existing data analysis techniques fall short of identifying them from their ionic current profiles. Recently introduced machine learning methods such as deep convolutional neural network (CNN), developed for image identification tasks, can be applied for such classification. However, with smaller data set for the problem in hand, it is not possible to train a deep neural network from scratch for accurate classification of AAV vectors. To circumvent this, we applied a pre-trained deep CNN (GoogleNet) model to capture the basic features from ionic current signals and subsequently used fine-tuning-based transfer learning to classify AAV vectors. The proposed method is very generic as it requires minimal preprocessing and does not require any handcrafted features. Our results indicate that fine-tuning-based transfer learning can achieve an average classification accuracy between 90 and 99% in three realizations with a very small standard deviation. Results also indicate that the classification accuracy depends on the applied electric field (across nanopore) and the time frame used for data segmentation. We also found that the fine-tuning of the deep network outperforms feature extraction-based classification for the resistive pulse dataset. To expand the usefulness of the fine-tuning-based transfer learning, we have tested two other pre-trained deep networks (ResNet50 and InceptionV3) for the classification of AAVs. Overall, the fine-tuning-based transfer learning from pre-trained deep networks is very effective for classification, though deep networks such as ResNet50 and InceptionV3 take significantly longer training time than GoogleNet.

On the performance evaluation of object classification models in low altitude aerial data.

This paper compares the classification performance of machine learning classifiers vs. deep learning-based handcrafted models and various pretrained deep networks. The proposed study performs a comprehensive analysis of object classification techniques implemented on low-altitude UAV datasets using various machine and deep learning models. Multiple UAV object classification is performed through widely deployed machine learning-based classifiers such as K nearest neighbor, decision trees, naïve Bayes, random forest, a deep handcrafted model based on convolutional layers, and pretrained deep models. The best result obtained using random forest classifiers on the UAV dataset is 90%. The handcrafted deep model's accuracy score suggests the efficacy of deep models over machine learning-based classifiers in low-altitude aerial images. This model attains 92.48% accuracy, which is a significant improvement over machine learning-based classifiers. Thereafter, we analyze several pretrained deep learning models, such as VGG-D, InceptionV3, DenseNet, Inception-ResNetV4, and Xception. The experimental assessment demonstrates nearly 100% accuracy values using pretrained VGG16- and VGG19-based deep networks. This paper provides a compilation of machine learning-based classifiers and pretrained deep learning models and a comprehensive classification report for the respective performance measures.

RAUM-VO: Rotational Adjusted Unsupervised Monocular Visual Odometry.

Unsupervised learning for monocular camera motion and 3D scene understanding has gained popularity over traditional methods, which rely on epipolar geometry or non-linear optimization. Notably, deep learning can overcome many issues of monocular vision, such as perceptual aliasing, low-textured areas, scale drift, and degenerate motions. In addition, concerning supervised learning, we can fully leverage video stream data without the need for depth or motion labels. However, in this work, we note that rotational motion can limit the accuracy of the unsupervised pose networks more than the translational component. Therefore, we present RAUM-VO, an approach based on a model-free epipolar constraint for frame-to-frame motion estimation (F2F) to adjust the rotation during training and online inference. To this end, we match 2D keypoints between consecutive frames using pre-trained deep networks, Superpoint and Superglue, while training a network for depth and pose estimation using an unsupervised training protocol. Then, we adjust the predicted rotation with the motion estimated by F2F using the 2D matches and initializing the solver with the pose network prediction. Ultimately, RAUM-VO shows a considerable accuracy improvement compared to other unsupervised pose networks on the KITTI dataset, while reducing the complexity of other hybrid or traditional approaches and achieving comparable state-of-the-art results.

Multiscale Feature Aggregation Capsule Neural Network for Hyperspectral Remote Sensing Image Classification

Models based on capsule neural network (CapsNet), a novel deep learning method, have recently made great achievements in hyperspectral remote sensing image (HSI) classification due to their excellent ability to implicitly model the spatial relationship knowledge embedded in HSIs. However, the number of labeled samples is a common bottleneck in HSI classification, limiting the performance of these deep learning models. To alleviate the problem of limited labeled samples and further explore the potential of CapsNet in the HSI classification field, this study proposes a multiscale feature aggregation capsule neural network (MS-CapsNet) based on CapsNet via the implementation of two branches that simultaneously extract spectral, local spatial, and global spatial features to integrate multiscale features and improve model robustness. Furthermore, because deep features are generally more discriminative than shallow features, two kinds of capsule residual (CapsRES) blocks based on 3D convolutional capsule (3D-ConvCaps) layers and residual connections are proposed to increase the depth of the network and solve the limited labeled sample problem in HSI classification. Moreover, a squeeze-and-excitation (SE) block is introduced in the shallow layers of MS-CapsNet to enhance its feature extraction ability. In addition, a reasonable initialization strategy that transfers parameters from two well-designed, pretrained deep convolutional capsule networks is introduced to help the model find a good set of initializing weight parameters and further improve the HSI classification accuracy of MS-CapsNet. Experimental results on four widely used HSI datasets demonstrate that the proposed method can provide results comparable to those of state-of-the-art methods.

Automated Detection of Nonmelanoma Skin Cancer Based on Deep Convolutional Neural Network.

One of the deadliest diseases is skin cancer, especially melanoma. The high resemblance between different skin lesions such as melanoma and nevus in the skin colour images increases the complexity of identification and diagnosis. An efficient automated early detection system for skin cancer detection is essential in order to save human lives, time, and effort. In this article, an automatic skin lesion classification system using a pretrained deep learning network and transfer learning was proposed. Here, diagnosing melanoma in premature stages, a detection system has been designed which contains the following digital image processing techniques. First, dermoscopy images of skin were taken and this is subjected to a preprocessing step for noise removal and postprocessing step for image enhancement. Then the processed image undergoes image segmentation using k-means and modified k-means clustering. Second, using feature extraction technology, Gray Level Co-occurrence Matrix, and first order statistics, characteristics are extracted. Features are selected on the basis of Harris Hawks optimization (HHO). Finally, various classifiers are used for predicting the stages and efficiency of the proposed work. Measures of well-known quantities, sensitivity, precision, accuracy, and specificity are used in assessing the efficiency of the suggested method, where higher values were obtained. Compared to the current methods, it is found that the classification rate exceeded the output of the current approaches in the performance of the proposed approach.

Computer-assisted grading of follicular lymphoma: a classification based on SVM, machine learning, and transfer learning approaches

ABSTRACT This proposed work implements various classification approaches to classify the H&E-stained Follicular Lymphoma tissue sample. As part of the process, k-means clustering is used to isolate cytological components like the nucleus extracellular and cytoplasmic regions from images. Then feature space vector is constructed by combining global feature extraction techniques like LBP, LDP, GLCM, and other local feature-extraction techniques. To classify FL images into their respective grades, feature space vectors obtained from feature extraction algorithms are input into a multiclass SVM. Moreover, classification accuracies were explicitly tested with different classifiers like CNN and other pre-trained deep learning networks that can directly operate on raw images without any preprocessing to classify the FL images into their respective grades. The efficacy of different classifiers is presented. This preliminary work provides a proof of concept for incorporating automated FL tissue diagnostic systems into future pathology workflows to supplement the pathologists'.

An Efficient In-Memory Computing Architecture for Image Enhancement in AI Applications

Random Spray Retinex (RSR) is an effective image enhancement algorithm due to its effectiveness in improving image quality. However, the algorithm computing complexity and the required hardware resources and memory accesses hampered its deployment in many applications scenarios, for instance in IoT systems with limited hardware resources. With the rise of Artificial Intelligence (AI), the use of image enhancement has become essential to improve the performance for many emerging applications. In this paper, we propose the use of the RSR as a pre-processing filter before the task of semantic segmentation of low-quality urban road scenes. Using the publicly available Cityscapes dataset, we compare the performance of a pre-trained deep semantic segmentation network on dark noisy images and on RSR pre-processed images. Our findings confirm the effectiveness of RSR in improving segmentation accuracy. In addition, to address the computation complexity and suitability to edge devices, we propose a novel efficient implementation of the RSR using resistive random access memory (RRAM) technology. The architecture provides highly parallel analog in-memory computing (IMC) capabilities. A detailed, efficient and low latency implementation of the RSR using RRAM-CMOS technology is described. The design is verified using SPICE simulations with measured data from fabricated RRAM and 65nm CMOS technologies. The approach provided here represents an important step towards low-complexity and real-time hardware-friendly architecture and design for Retinex algorithms for edge devices.