V3 Models Research Articles

A Novel Optimizer in Deep Neural Network for Diabetic Retinopathy Classification

In severe cases, diabetic retinopathy can lead to blindness. For decades, automatic classification of diabetic retinopathy images has been a challenge. Medical image processing has benefited from advances in deep learning systems. To enhance the accuracy of image classification driven by Convolutional Neural Network (CNN), balanced dataset is generated by data augmentation method followed by an optimized algorithm. Deep neural networks (DNN) are frequently optimized using gradient (GD) based techniques. Vanishing gradient is the main drawback of GD algorithms. In this paper, we suggest an innovative algorithm, to solve the above problem, Hypergradient Descent learning rate based Quasi hyperbolic (HDQH) gradient descent to optimize the weights and biases. The algorithms only use first order gradients, which reduces computation time and storage space requirements. The algorithms do not require more tuning of the learning rates as the learning rate tunes itself by means of gradients. We present empirical evaluation of our algorithm on two public retinal image datasets such as Messidor and DDR by using Resnet18 and Inception V3 architectures. The findings of the experiment show that the efficiency and accuracy of our algorithm outperforms the other cutting-edge algorithms. HDQHAdam shows the highest accuracy of 97.5 on Resnet18 and 95.7 on Inception V3 models respectively.

Rice Leaf Blast Classification Method Based on Fused Features and One-Dimensional Deep Convolutional Neural Network

Rice leaf blast, which is seriously affecting the yield and quality of rice around the world, is a fungal disease that easily develops under high temperature and humidity conditions. Therefore, the use of accurate and non-destructive diagnostic methods is important for rice production management. Hyperspectral imaging technology is a type of crop disease identification method with great potential. However, a large amount of redundant information mixed in hyperspectral data makes it more difficult to establish an efficient disease classification model. At the same time, the difficulty and small scale of agricultural hyperspectral imaging data acquisition has resulted in unrepresentative features being acquired. Therefore, the focus of this study was to determine the best classification features and classification models for the five disease classes of leaf blast in order to improve the accuracy of grading the disease. First, the hyperspectral imaging data were pre-processed in order to extract rice leaf samples of five disease classes, and the number of samples was increased by data augmentation methods. Secondly, spectral feature wavelengths, vegetation indices and texture features were obtained based on the amplified sample data. Thirdly, seven one-dimensional deep convolutional neural networks (DCNN) models were constructed based on spectral feature wavelengths, vegetation indices, texture features and their fusion features. Finally, the model in this paper was compared and analyzed with the Inception V3, ZF-Net, TextCNN and bidirectional gated recurrent unit (BiGRU); support vector machine (SVM); and extreme learning machine (ELM) models in order to determine the best classification features and classification models for different disease classes of leaf blast. The results showed that the classification model constructed using fused features was significantly better than the model constructed with a single feature in terms of accuracy in grading the degree of leaf blast disease. The best performance was achieved with the combination of the successive projections algorithm (SPA) selected feature wavelengths and texture features (TFs). The modeling results also show that the DCNN model provides better classification capability for disease classification than the Inception V3, ZF-Net, TextCNN, BiGRU, SVM and ELM classification models. The SPA + TFs-DCNN achieved the best classification accuracy with an overall accuracy (OA) and Kappa of 98.58% and 98.22%, respectively. In terms of the classification of the specific different disease classes, the F1-scores for diseases of classes 0, 1 and 2 were all 100%, while the F1-scores for diseases of classes 4 and 5 were 96.48% and 96.68%, respectively. This study provides a new method for the identification and classification of rice leaf blast and a research basis for assessing the extent of the disease in the field.

Performance Comparison of Different Pre-Trained Deep Learning Models in Classifying Brain MRI Images

A brain tumor is a collection of abnormal cells formed as a result of uncontrolled cell division. If tumors are not diagnosed in a timely and accurate manner, they can cause fatal consequences. One of the commonly used techniques to detect brain tumors is magnetic resonance imaging (MRI). MRI provides easy detection of abnormalities in the brain with its high resolution. MR images have traditionally been studied and interpreted by radiologists. However, with the development of technology, it becomes more difficult to interpret large amounts of data produced in reasonable periods. Therefore, the development of computerized semi-automatic or automatic methods has become an important research topic. Machine learning methods that can predict by learning from data are widely used in this field. However, the extraction of image features requires special engineering in the machine learning process. Deep learning, a sub-branch of machine learning, allows us to automatically discover the complex hierarchy in the data and eliminates the limitations of machine learning. Transfer learning is to transfer the knowledge of a pre-trained neural network to a similar model in case of limited training data or the goal of reducing the workload. In this study, the performance of the pre-trained Vgg-16, ResNet50, Inception v3 models in classifying 253 brain MR images were evaluated. The Vgg-16 model showed the highest success with 94.42% accuracy, 83.86% recall, 100% precision and 91.22% F1 score. This was followed by the ResNet50 model with an accuracy of 82.49%.The findings obtained in this study were compared with similar studies in the literature and it was found that it showed higher success than most studies.

Glaucoma detection in Latino population through OCT’s RNFL thickness map using transfer learning

Glaucoma is the leading cause of irreversible blindness worldwide. It is estimated that over 60 million people around the world have this disease, with only part of them knowing they have it. Timely and early diagnosis is vital to delay/prevent patient blindness. Deep learning (DL) could be a tool for ophthalmologists to give a more informed and objective diagnosis. However, there is a lack of studies that apply DL for glaucoma detection to Latino population. Our contribution is to use transfer learning to retrain MobileNet and Inception V3 models with images of the retinal nerve fiber layer thickness map of Mexican patients, obtained with optical coherence tomography (OCT) from the Instituto de la Visión, a clinic in the northern part of Mexico. The IBM Foundational Methodology for Data Science was used in this study. The MobileNet and Inception V3 topologies were chosen as the analytical approaches to classify OCT images in two classes, namely glaucomatous and non-glaucomatous. The OCT files were collected from a Zeiss OCT machine at the Instituto de la Visión, and classified by an expert into the two classes under study. These images conform a dataset of 333 files in total. Since this research work is focused on RNFL thickness map images, the OCT files were cropped to obtain only the RNFL thickness map images of the corresponding eye. This action was carried out for images in both classes, glaucomatous and non-glaucomatous. Since some images were damaged (with black spots in which data was missing), these images were cut-out and cut-off. After the preparation process, 50 images per class were used for training. Fifteen images per class, different than the ones used in the training stage, were used for running predictions. In total, 260 images were used in the experiments, 130 per eye. Four models were generated, two trained with MobileNet, one for the left eye and one for the right eye, and another two trained with Inception V3. TensorFlow was used for running transfer learning. The evaluation results of the MobileNet model for the left eye are, accuracy: 86%, precision: 87%, recall: 87%, and F1 score: 87%. The evaluation results of the MobileNet model for the right eye are, accuracy: 90%, precision: 90%, recall: 90%, and F1 score: 90%. The evaluation results of the Inception V3 model for the left eye are, accuracy: 90%, precision: 90%, recall: 90%, and F1 score: 90%. The evaluation results of the Inception V3 model for the right eye are, accuracy: 90%, precision: 90%, recall: 90%, and F1 score: 90%. In average, the evaluation results for right eye images were the same for both models. The Inception V3 model showed slight better average results than the MobileNet model in the case of classifying left eye images.

Rice yield potential as a function of sowing date in southern Brazil

AbstractMost studies about yield potential (Yp) of modern rice (Oryza sativa L.) varieties have been grown under tropical conditions of Asia, and little is known about the rice Yp in the subtropics of Brazil, the biggest rice producer outside Asia. Playing a key role in the global rice production, it is necessary to estimate the amount of rice that Brazil can potentially produce. The objective of this study is to provide estimations of Yp in southern Brazil by using the SimulArroz v1.1 and ORYZA v3 models. Models were calibrated and evaluated with data collected from five growing seasons across Rio Grande do Sul (RS) state in Brazil, where the cultivar IRGA 424 RI was sown from September to December. Both models presented similar performance in simulating phenology, with root mean square error (RMSE) of 9 d for ORYZA and 11 d for SimulArroz. For grain yield, the RMSE was 1.0 and 0.9 t ha−1 for ORYZA and SimulArroz, respectively. Using the SimulArroz model, yield potential maps were drawn, which ranged from <6 t ha−1 to >14 t ha−1, according to the region and sowing date. The penalty in Yp caused by the delay in sowing date is 0.03 t ha day−1 from 1 September to 13 October, 0.08 t ha day−1 from 14 October to 21 December, and 0.29 t ha day−1 after 21 December. SimulArroz model is a suitable model for studies on rice Yp in the Brazilian subtropics.

Performance Evaluation of Deep CNN-Based Crack Detection and Localization Techniques for Concrete Structures.

This paper proposes a customized convolutional neural network for crack detection in concrete structures. The proposed method is compared to four existing deep learning methods based on training data size, data heterogeneity, network complexity, and the number of epochs. The performance of the proposed convolutional neural network (CNN) model is evaluated and compared to pretrained networks, i.e., the VGG-16, VGG-19, ResNet-50, and Inception V3 models, on eight datasets of different sizes, created from two public datasets. For each model, the evaluation considered computational time, crack localization results, and classification measures, e.g., accuracy, precision, recall, and F1-score. Experimental results demonstrated that training data size and heterogeneity among data samples significantly affect model performance. All models demonstrated promising performance on a limited number of diverse training data; however, increasing the training data size and reducing diversity reduced generalization performance, and led to overfitting. The proposed customized CNN and VGG-16 models outperformed the other methods in terms of classification, localization, and computational time on a small amount of data, and the results indicate that these two models demonstrate superior crack detection and localization for concrete structures.

Activity recognition framework in sports videos

Recent advancements in activity recognition from sports videos have attracted wide scientific interest of the Computer Vision community. However, the activity recognition problem from cricket video sequences is largely under-represented in the literature. This paper aims to devise a convolutional neural network (CNN) based model for sports activity recognition. The model is trained on the pre-trained VGG16, VGG19, ResNet50, and Inception V3 Models and tested on the clustered cricket videos frames extracted from the data set especially prepared for this research. The clustering of the frames is done by using K-Mean clustering algorithm. K-Fold cross validation is done which gave an accuracy of 99% on clustered data and 91% on un-clustered data. The accuracy and time complexity of the proposed method is better as compared to the state of the art methods used for activity recognition from videos.

WITHDRAWN: Inception ResNet deep transfer learning model for human action recognition using LSTM

Abstract In recent days, variety of approaches using deep learning features have been proposed for human action recognition due to the worth of deep neural networks. In this work, we put forward a new deep neural network architecture based on transfer learning [TL] for human action recognition. We illustrate how to progress the recognition of human activities with a video data set of small size using transfer learning. The model constructed is based on Inception ResNet convolutional neural networks (CNN) and long short term model (LSTM).We train the model by extracting feature vectors from Inception_ResNet_v2 and then the output feature vectors from CNN are applied onto the RNN for learning the action sequence. Then we attempt to classify the input videos according to the model trained. He accuracy score of the model was compared against VGG16, ResNet152 and Inception_v3 models studied on. The results prove the LSTM architecture using Inception_ ResNet_v2 provide the best accuracy score of 92%, 91% on UCI 101 and HMDB 51 data sets.

A molecular dynamics study of the mechanical properties of the graphene/hexagonal boron nitride planar heterojunction for RRAM

Due to their high flexibility and precise interface modulation, resistive random access memory (RRAM) devices composed of graphene/hexagonal boron nitride (Gr/h-BN) planar heterojunctions are regarded as a possible solution and have gained a substantial amount of attention. Molecular dynamics simulations with a modified Tersoff potential are conducted to investigate the mechanical properties of three planar heterojunction RRAM molecular models (Gr/h-BN_V1/V2/V3), which are established according to the interface type and atomic position. Among them, the Gr/h-BN_V1 model exhibits the best mechanical properties, including the fracture strength, fracture strain, and Young's modulus, at all simulation temperatures. The extensive simulations reveal the effects of the temperature, aspect ratio, single-vacancy (SV) and Stone-Wales (SW) defects on the tensile strength of Gr/h-BN RRAM models. As the temperature increases from 10 K to 500 K, the fracture strengths of the Gr/h-BN_V1, Gr/h-BN_V2 and Gr/h-BN_V3 models decrease by 12.6%, 59.6% and 12.0%, respectively. It can be observed that when the aspect ratio is 2.83, the comprehensive mechanical properties of the Gr/ h-BN_V1 model are superior to those of the other models. A 6.7% concentration of single atomic defects in the SV_C, SV_B and SV_N models causes decreases in the fracture strength of 28.0 GPa, 33.9 GPa and 22.6 GPa, respectively, indicating that the mechanical properties are more sensitive to the introduction of boron atomic vacancies than carbon and nitrogen atomic vacancies. At all defect concentrations, the decrease in fracture strength and fracture strain caused by SW defects in the SW_CN model is minimized, while the demarcation point of the relative influence of the defect concentration on the tensile strength is 16% in the SW_CB and SW_CC models. Our present work provides useful insights into the application of highly flexible and stretchable Gr/h-BN planar heterojunctions for RRAM devices.

Deep learning based detection and analysis of COVID-19 on chest X-ray images.

Covid-19 is a rapidly spreading viral disease that infects not only humans, but animals are also infected because of this disease. The daily life of human beings, their health, and the economy of a country are affected due to this deadly viral disease. Covid-19 is a common spreading disease, and till now, not a single country can prepare a vaccine for COVID-19. A clinical study of COVID-19 infected patients has shown that these types of patients are mostly infected from a lung infection after coming in contact with this disease. Chest x-ray (i.e., radiography) and chest CT are a more effective imaging technique for diagnosing lunge related problems. Still, a substantial chest x-ray is a lower cost process in comparison to chest CT. Deep learning is the most successful technique of machine learning, which provides useful analysis to study a large amount of chest x-ray images that can critically impact on screening of Covid-19. In this work, we have taken the PA view of chest x-ray scans for covid-19 affected patients as well as healthy patients. After cleaning up the images and applying data augmentation, we have used deep learning-based CNN models and compared their performance. We have compared Inception V3, Xception, and ResNeXt models and examined their accuracy. To analyze the model performance, 6432 chest x-ray scans samples have been collected from the Kaggle repository, out of which 5467 were used for training and 965 for validation. In result analysis, the Xception model gives the highest accuracy (i.e., 97.97%) for detecting Chest X-rays images as compared to other models. This work only focuses on possible methods of classifying covid-19 infected patients and does not claim any medical accuracy.

Corrigendum to ‘Deep learning improves taphonomic resolution: high accuracy in differentiating tooth marks made by lions and jaguars'

Corrigendum to ‘Deep learning improves taphonomic resolution: high accuracy in differentiating tooth marks made by lions and jaguars'

Comparison of Convolutional Neural Network Models for Determination of Vocal Fold Normality in Laryngoscopic Images

Deep learning using convolutional neural networks (CNNs) is widely used in medical imaging research. This study was performed to investigate if vocal fold normality in laryngoscopic images can be determined by CNN-based deep learning and to compare accuracy of CNN models and explore the feasibility of application of deep learning on laryngoscopy. Laryngoscopy videos were screen-captured and each image was cropped to include abducted vocal fold regions. A total of 2216 image (899 normal, 1317 abnormal) were allocated to training, validation, and test sets. Augmentation of training sets was used to train a constructed CNN model with six layers (CNN6), VGG16, Inception V3, and Xception models. Trained models were applied to the test set; for each model, receiver operating characteristic curves and cutoff values were obtained. Sensitivity, specificity, positive predictive value, negative predictive value, and accuracy were calculated. The best model was employed in video-streams and localization of features was attempted using Grad-CAM. All of the trained models showed high area under the receiver operating characteristic curve and the most discriminative cutoff levels of probability of normality were determined to be 35.6%, 61.8%, 13.5%, 39.7% for CNN6, VGG16, Inception V3, and Xception models, respectively. Accuracy of the CNN models selecting normal and abnormal vocal folds in the test set was 82.3%, 99.7%, 99.1%, and 83.8%, respectively. All four models showed acceptable diagnostic accuracy. Performance of VGG16 and Inception V3 was better than the simple CNN6 model and the recently published Xception model. Real-time classification with a combination of the VGG16 model, OpenCV, and Grad-CAM on a video stream showed the potential clinical applications of the deep learning model in laryngoscopy.

Using Deep Learning for Image-Based Different Degrees of Ginkgo Leaf Disease Classification

Diseases from Ginkgo biloba have brought great losses to medicine and the economy. Therefore, if the degree of disease can be automatically identified in Ginkgo biloba leaves, people will take appropriate measures to avoid losses in advance. Deep learning has made great achievements in plant disease identification and classification. For this paper, the convolution neural network model was used to classify the different degrees of ginkgo leaf disease. This study used the VGGNet-16 and Inception V3 models. After preprocessing and training 1322 original images under laboratory conditions and 2408 original images under field conditions, 98.44% accuracy was achieved under laboratory conditions and 92.19% under field conditions with the VGG model. The Inception V3 model achieved 92.3% accuracy under laboratory conditions and 93.2% under field conditions. Thus, the Inception V3 model structure was more suitable for field conditions. To our knowledge, there is very little research on the classification of different degrees of the same plant disease. The success of this study will have a significant impact on the prediction and early prevention of ginkgo leaf blight.

Transfer Learning-Based DCE-MRI Method for Identifying Differentiation Between Benign and Malignant Breast Tumors

In this paper, we propose a transfer learning-based dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) method for classifying fibroadenoma and invasive ductal carcinoma (IDC) in breast tumors. A total of 207 breast tumors from patients were collected and identified by pathologic diagnosis within 15 days after enhanced DCE-MRI examination; 119 patients (average age 50.52±10.33 years) had fibroadenomas, and 88 patients (average age 42.20±10.10 years) had IDCs. Two lesion-level models were built based on the InceptionV3 and VGG19 models, which were pretrained with the ImageNet dataset. The effects of different depths of transfer learning were examined. The network's performance was evaluated through five-fold cross validation. In the lesion-level models, the model based on Inception V3 obtained better results (area under the receiver operating characteristic curve (AUC) = 0.89) when the weights were frozen before layer-276. The other model based on VGG19 obtained better results (AUC = 0.87) when the weights were frozen before layer-13. Compared with the image-level models, both lesion-level models displayed better discrimination (accuracy increased by 13% and 14%) based on the VGG19 and Inception V3 models, respectively. Our research confirms that transfer learning can be utilized to classify fibroadenomas and IDCs in DCE-MRI images. Different depths of transfer learning result in different performances, and our proposed lesion-level model notably improves the classification accuracy.

Improved analytical model for Rayleigh–Brillouin scattering spectrum in gases

ABSTRACTAn improved analytical model (G-factor model) for the Rayleigh–Brillouin scattering (RBS) spectrum of gases is proposed and discussed based on RBS spectra of nitrogen. Three analytical models (G3, V3, and G-factor models) were compared with the spontaneous RBS experimental data and coherent RBS simulation data based on Pan’s S7 model. The results show that the proposed model has a larger scope of application as compared to the G3 model, and offers less multivalued solutions compared to the V3 model. In addition, this analytical model provides a better representation of the profiles than the Tenti S6 model. This novel analytical model has fewer parameters and more stable fitting results. It maintains a high fitting accuracy and has a larger scope of application compared to the other analytical models.

Analysis of Rayleigh-Brillouin spectral profiles and Brillouin shifts in nitrogen gas and air

On the basis of experimental Rayleigh-Brillouin scattering data in gaseous nitrogen and air, simulations are performed to describe the observed frequency profiles in analytical form. The experimental data pertain to a λ = 366 nm scattering wavelength, a 90° scattering angle, pressures of 1 and 3 bar, and temperatures in the range 250 - 340 K. Two different models are used to represent the RB-profiles, to distinguish the RB-peaks, and to obtain the Brillouin shift associated with the acoustic waves generated in a gaseous medium. Calculations in the framework of V3 and G3 models, exhibiting composite profiles of three distinct peaks of Voigt or Gaussian functions, are compared to observation. Fitting results show that the V3 model yields an improvement over the G3 model. This mathematical model provides an even better representation of the observed profiles than the Tenti S6 model, which is considered to be the optimum representation in terms of physical parameters. For the derivation of Brillouin shifts, both models perform well at high gas pressure, while at lower pressures, the V3 model yields a higher accuracy than the G3 model.

Mutations in Gp41 are Correlated with Coreceptor Tropism but Do Not Improve Prediction Methods Substantially

The main determinants of HIV-1 coreceptor usage are located in the V3-loop of gp120, although mutations in V2 and gp41 are also known. Incorporation of V2 is known to improve prediction algorithms; however, this has not been confirmed for gp41 mutations. Samples with V3 and gp41 genotypes and Trofile assay (Monogram Biosciences, South San Francisco, CA, USA) results were taken from the HOMER cohort (n=444) and from patients screened for the MOTIVATE studies (n=1,916; 859 with maraviroc outcome data). Correlations of mutations with tropism were assessed using Fisher's exact test and prediction models trained using support vector machines. Models were validated by cross-validation, by testing models from one dataset on the other, and by analysing virological outcome. Several mutations within gp41 were highly significant for CXCR4 usage; most strikingly an insertion occurring in 7.7% of HOMER-R5 and 46.3% of HOMER-X4 samples (MOTIVATE 5.7% and 25.2%, respectively). Models trained on gp41 sequence alone achieved relatively high areas under the receiver-operating characteristic curve (AUCs; HOMER 0.713 and MOTIVATE 0.736) that were almost as good as V3 models (0.773 and 0.884, respectively). However, combining the two regions improved predictions only marginally (0.813 and 0.902, respectively). Similar results were found when models were trained on HOMER and validated on MOTIVATE or vice versa. The difference in median log viral load decrease at week 24 between patients with R5 and X4 virus was 1.65 (HOMER 2.45 and MOTIVATE 0.79) for V3 models, 1.59 for gp41-models (2.42 and 0.83, respectively) and 1.58 for the combined predictor (2.44 and 0.86, respectively). Several mutations within gp41 showed strong correlation with tropism in two independent datasets. However, incorporating gp41 mutations into prediction models is not mandatory because they do not improve substantially on models trained on V3 sequences alone.

Stability of Q-Balls and Catastrophe

We propose a practical method for analyzing stability of Q-balls for the whole parameter space, which includes the intermediate region between the thin-wall limit and thick-wall limit as well as Q-bubbles (Q-balls in false vacuum), using catastrophe theory. We apply our method to the two concrete models, V3 = m 2 φ 2 /2 − μφ 3 + λφ 4 and V4 = m 2 φ 2 /2 − λφ 4 + φ 6 /M 2 . We find that V3 and V4 Models fall into fold catastrophe and cusp catastrophe, respectively, and their stability structures are quite different from each other.