Basic Convolutional Neural Networks Model Research Articles

Efficient Brain Cancer Classification Based on Advanced CNN and Transformer Technologies

Abstract. This study addresses the challenge faced by primary medical staff in accurately distinguishing different types of brain cancer by comparing the training accuracy and iteration speed of various classification models. Specifically, thesis evaluated a basic Convolutional Neural Network (CNN) alongside two transfer learning models: EfficientNetV2 and Vision Transformer. The analysis is based on publicly available data from Kaggle, which provided a robust dataset for evaluation. The results indicate that both EfficientNetV2 and Vision Transformer models achieve significantly higher iteration convergence speeds and classification accuracy compared to the basic CNN model trained solely on the database. These advanced models demonstrate a capacity for efficient and rapid brain cancer differentiation, making them suitable for settings with limited computing resources. The study highlights that the transfer learning models can effectively support outpatient doctors by providing fast and accurate etiological screening without requiring extensive hardware. Overall, the findings suggest that EfficientNetV2 and Vision Transformer models offer a practical solution for improving brain cancer classification in primary healthcare environments, where rapid and precise diagnosis is crucial. This research underscores the potential of transfer learning techniques to enhance diagnostic capabilities while accommodating resource constraints.

A Deep Learning Model for Classifying Black Sigatoka Disease in Banana Leaves Based on Infection Stages

Objective: This research study aims to develop an efficient deep-learning model to detect and classify stages of Black Sigatoka disease in banana plants. Methods: In this study, deep learning techniques, specifically the basic Convolutional Neural Network (CNN) and VGG16 models, were used to address the challenge of identifying Black Sigatoka disease in banana leaves early on. The tests were conducted on a dataset containing labelled images of banana leaves, assessing their effectiveness based on criteria such as accuracy, precision, recall, and F1-score after adjusting hyperparameters for optimal outcomes. Findings: The results of the trials revealed that the basic CNN model attained a training accuracy of 96% and a validation accuracy of 89%, surpassing the performance of the VGG16 model. The VGG16 model, on the other hand, had a training accuracy of 92% and a validation accuracy of 89%. Across precision, recall, and F1 score measurements, the basic CNN model consistently outperformed the VGG16 model, with scores averaging 0.90 for all three metrics compared to VGG16's precision of 0.80, recall of 0.75, and F1 score of 0.75. The CNN model demonstrated its efficiency by stopping training at 26 epochs, whereas VGG16 completed training in 21 epochs. This demonstrates its effectiveness in detecting Black Sigatoka while utilising minimal resources. Novelty: A significant component of this study is its emphasis on identifying the stages of Black Sigatoka disease, which is commonly overlooked in research. By studying disease progression, this study provides insights for early intervention and disease management, aiding efforts to lessen the impact of Black Sigatoka on banana farming. Keywords: Black Sigatoka, Deep Learning, Disease Stages, Convolutional Neural Network, Classification, Identification

Decoding Innocence: Advancing Forensic Facial Discrimination through Comparative Analysis of Conventional CNN and Advanced Architectures

The field of Digital Image Forensics (DIF) faces a critical issue in accurately identifying children in digital images, notably in cases involving the proliferation of child sexual abuse content. Existing techniques face hurdles due to model architecture limitations, dataset suitability concerns, and classification imbalance, impeding their ability to recognize children to deter pornographic images. Addressing this challenge, this study introduces Implicit Feature Extraction (IFE), a specialized approach for distinguishing child and adult images in object detection. Leveraging Convolutional Neural Networks (CNNs), the IFE method automates the extraction of discriminative facial features, surpassing the constraints of Explicit Feature Extraction (EFE) methods, which achieve an accuracy of around 70%. The research focuses on three core objectives introducing IFE for detailed face feature detection in DIF's child and adult image identification, implementing IFE with CNNs to enhance image classification, and conducting a thorough evaluation of the proposed technique's performance using key metrics like accuracy and balanced classification results and comparing the result with a basic CNN model’s performance. This research's significance lies in its notable contributions to digital image forensics, particularly in combatting child exploitation. The fusion of IFE with CNNs showcases 92% accuracy in distinguishing child and adult images, promising advancements with practical implications in child protection and forensic investigations. The comprehensive evaluation using the UTKFace dataset underscores the proposed technique's efficacy, marking a substantial improvement in child image identification within digital image forensics.

Resolving data imbalance in alkaline battery defect detection: a voting-based deep learning approach

Alkaline battery defect detection is crucial for ensuring product quality and providing diagnostic feedback. Recently, high-performance deep learning algorithms have been introduced to recognise defects in alkaline batteries. However, the majority of deep learning-based methods overlook the significant data imbalance issues in alkaline battery electrode images, potentially resulting in performance degradation. Therefore, a voting-based recognition algorithm containing three parts is proposed in this study. Firstly, a resampling and training method is developed to provide richer information and stronger constraints. Secondly, a weak classification framework based on an improved convolutional neural network (CNN) is designed to provide fine-grained category representations. Finally, a voting-based prediction approach is proposed to improve accuracy and obtain the final results. Visualisation results demonstrate that the proposed algorithm has a stronger clustering ability and uses voting-based prediction to improve performance. Comparison research shows that the proposed method significantly improves the recall of minority classes and the precision of majority classes and reaches a state-of-the-art F1 score of 0.982 for alkaline battery defect recognition, which is 0.041 higher than the basic CNN model.

A Global Seawater Density Distribution Model Using a Convolutional Neural Network.

Seawater density is an important physical property in oceanography that affects the accuracy of calculations such as gravity fields and tidal potentials and the calibration of acoustic and optical oceanographic sensors. In related studies, constant density values are frequently used, which can introduce significant errors. Therefore, this study employs a basic convolutional neural network model to construct a comprehensive model showing the seawater density distribution across the globe. The model takes into account depth, latitude, longitude, and month as inputs. Numerous real seawater datasets were used to train the model, and it has been shown that the model has an absolute mean error and root mean square error of less than 1 kg/m3 in 99% of the test set samples. The model effectively demonstrates the influence of input parameters on the distribution of seawater density. In this paper, we present a newly developed global model for distributing seawater density which is both comprehensive and accurate, surpassing previous models. The utilization of the model presented in this paper for estimating seawater density can minimize errors in theoretical ocean models and serve as a foundation for designing and analyzing ocean exploration systems.

Image Forgery Detection Based on Parallel Convolutional Neural Networks

Abstract— Due to the availability of deep networks, progress has been made in the field of image recognition. Images and videos are spreading very conveniently and with the availability of strong editing tools the tampering of digital content become easy. To detect such scams, we proposed techniques. In our paper, we proposed two important aspects of employing deep convolutional neural networks to image forgery detection. We first explore and examine different preprocessing method along with convolutional neural networks (CNN) architecture. Later we evaluated the different transfer learning for pre-trained ImageNet(via-fine-tuning) and implement it over our dataset CASIA V2.0. So, it covers the pre-processing techniques with basic CNN model and later see the powerful effect of the transfer learning models. Keywords— image tampering, convolution neural network (CNN), error level analysis (ELA), transfer learning, sharpening filter, fine-tuning

IMCNN:Intelligent Malware Classification using Deep Convolution Neural Networks as Transfer learning and ensemble learning in honeypot enabled organizational network

Traditional malware detection systems based on signature-based detection methods cannot detect new and unseen malware. Moreover, conventional machine learning methods for malware detection have utilized features extracted through static program analysis or dynamic analysis, which requires code debugging and execution primarily through offline processing; hence not a scalable approach. This paper proposes a novel intelligent malware classification using a deep convolution neural network (IMCNN) in organizational networks enabled with Honeypots. Systematic customization of pre-trained convolutional neural networks(CNN) as a transfer learning and ensemble learning as a classification is presented to detect intelligent modern-day malware. Real-world malware samples are systematically labeled and visualized into grayscale images. Four cutting-edge deep CNN models - VGG16, VGG19, InceptionV3, and ResNet50, are trained on the ImageNet database (≥1 million) and fine-tuned as feature extractors along with a basic CNN model. Three strategies are designed for feature extraction and selection: Rectified linear unit (ReLU) fully connected layer embedded in a deep CNN model, principal component analysis (PCA), and singular value decomposition(SVD). Reduced sets of features are stacked and used to train k-nearest neighbor (k-NN), support vector machine (SVM), and random forest (RF) classifiers for predictions. Subsequently, the predictive probabilities of different machine-learned models are ensembled using a soft voting method for final classification. The proposed method is evaluated on MalImg datasets (9339 malware samples of 25 families) and real-world modern malware datasets (690 malware of 22 families). The experimental results reveal that despite using a reduced feature set, the IMCNN effectively detects malware with 99.36% test accuracy on unseen data for MalImg datasets and 92.11% for real-world malware. In addition, the proposed method is compared with several existing state-of-art malware detection models in terms of performance accuracy and found performing as the best. Experiments demonstrated that the proposed method is resilient to polymorphic code obfuscation used by the malware authors.

A Novel Method for Fashion Clothing Image Classification Based on Deep Learning

Image recognition and classification is a significant research topic in computational vision and widely used computer technology. Themethods often used in image classification and recognition tasks are based on deep learning, like Convolutional Neural Networks(CNNs), LeNet, and Long Short-Term Memory networks (LSTM). Unfortunately, the classification accuracy of these methods isunsatisfactory. In recent years, using large-scale deep learning networks to achieve image recognition and classification canimprove classification accuracy, such as VGG16 and Residual Network (ResNet). However, due to the deep network hierarchyand complex parameter settings, these models take more time in the training phase, especially when the sample number is small, which can easily lead to overfitting. This paper suggested a deep learning-based image classification technique based on a CNN model and improved convolutional and pooling layers. Furthermore, the study adopted the approximate dynamic learning rate update algorithm in the model training to realize the learning rate’s self-adaptation, ensure the model’s rapid convergence, and shorten the training time. Using the proposed model, an experiment was conducted on the Fashion-MNIST dataset, taking 6,000 images as the training dataset and 1,000 images as the testing dataset. In actual experiments, the classification accuracy of the suggested method was 93 percent, 4.6 percent higher than that of the basic CNN model. Simultaneously, the study compared the influence of the batch size of model training on classification accuracy. Experimental outcomes showed this model is very generalized in fashion clothing image classification tasks.

Hybrid Deep Learning Based Attack Detection for Imbalanced Data Classification

Internet of Things (IoT) is the most widespread and fastest growing technology today. Due to the increasing of IoT devices connected to the Internet, the IoT is the most technology under security attacks. The IoT devices are not designed with security because they are resource constrained devices. Therefore, having an accurate IoT security system to detect security attacks is challenging. Intrusion Detection Systems (IDSs) using machine learning and deep learning techniques can detect security attacks accurately. This paper develops an IDS architecture based on Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM) deep learning algorithms. We implement our model on the UNSW-NB15 dataset which is a new network intrusion dataset that categorizes the network traffic into normal and attacks traffic. In this work, interpolation data preprocessing is used to compute the missing values. Also, the imbalanced data problem is solved using a synthetic data generation method. Extensive experiments have been implemented to compare the performance results of the proposed model (CNN+LSTM) with a basic model (CNN only) using both balanced and imbalanced dataset. Also, with some state-of-the-art machine learning classifiers (Decision Tree (DT) and Random Forest (RF)) using both balanced and imbalanced dataset. The results proved the impact of the balancing technique. The proposed hybrid model with the balance technique can classify the traffic into normal class and attack class with reasonable accuracy (92.10%) compared with the basic CNN model (89.90%) and the machine learning (DT 88.57% and RF 90.85%) models. Moreover, comparing the proposed model results with the most related works shows that the proposed model gives good results compared with the related works that used the balance techniques.

Study on Tomato Disease Classification based on Leaf Image Recognition based on Deep Learning Technology

The utilization of computer vision technology is of the utmost significance in the examination of plant diseases. Research utilizing image processing to investigate plant diseases necessitates the analysis of discernible patterns on plants. Recently, numerous image processing and pattern classification techniques have been employed in the construction of a digital vision system capable of recognizing and categorizing the visual manifestations of plant diseases. Given the abundance of algorithms formulated for the purpose of plant leaf image classification for the detection of plant diseases, it is imperative to assess the accuracy of each algorithm, as well as its potential to identify diverse disease types. The main objective of this study is to explore accurate deep learning architectures that are more effective in deploying and detecting tomato diseases, thus eliminating human error when identifying tomato diseases through visual observation. and get more widespread use. An initial model was constructed from the ground up using a convolutional neural network (CNN), which was trained with 22930 tomato leaf images, and then compared to VGG16, Mobile Net, and Inceptionv3 architectures through a fine-tuning process. The basic CNN model achieved a training accuracy of 90%, whereas the training accuracies of VGG16, Mobile Net, and Inceptionv3 were respectively observed to be 89%, 91%, and 87%. The VGG16 model has a greater computational complexity than other approaches due to its considerable quantity of predefined parameters. Despite to be simpler, MobileNet proved to be the most efficient in terms of accuracy and thus is the most suitable for this research, due to its lightweight structure, fast functioning and adaptability for mobile devices. In contrast to other architectures, the suggested CNN architecture exhibits shallower characteristics, facilitating faster training on the same dataset. This research will provide a solid foundation for future scholars to easily improve the categorization of plant diseases, which is to develop algorithms that are lighter, faster, easier to run, and have higher accuracy.

Genetic Algorithm-Based Energy-Aware CNN Quantization for Processing-In-Memory Architecture

We present a genetic algorithm based energy-aware convolutional neural network (CNN) quantization framework (EGQ) for processing-in-memory (PIM) architectures. EGQ predicts layer-wise dynamic energy consumption based on the number of ADC access. Also, EGQ automatically optimizes layer-wise weight/activation bitwidth that can reduce total dynamic energy with negligible accuracy loss. As EGQ requires basic CNN model information such as weight/activation dimensions to predict the dynamic energy, various models can be compressed by EGQ. We analyse the effectiveness of EGQ on the area, dynamic energy, and energy efficiency of PIM architectures for VGG-19, ResNet-18, and ResNet-50 using NeuroSim. We observe EGQ is an effective approach for the CNN models to reduce the dynamic energy in various PIM designs with SRAM, RRAM, and FeFET technologies. EGQ achieves 6.1 bit of average weight bitwidth and 6.3 bit of average activation bitwidth in ResNet-18, that improves energy efficiency by <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$6.5\times $ </tex-math></inline-formula> than the 16-bit model. For ResNet-18 with CIFAR-10, 2.5 bit and 3.9 bit of average weight and activation bitwidth are achieved. Both results show the negligible accuracy loss of 2%.

Application of Artificial Intelligence Technology in Medical Imaging

Based on massive medical image data, through reasonable system design and parameter optimization, the AI-assisted diagnosis system can help doctors make more clinical decisions, reduce their daily work pressure, reduce unnecessary invasive tests, and thus improve the quality of life of patients. This paper mainly studies the application of artificial intelligence technology in the field of medical imaging. In this paper, deeplesion public data set is used as the training set to study and optimize the convolutional neural network structure and parameters of deep learning algorithm based on the basic convolutional neural network model, aiming at the pathological features of lung cancer in the data set, such as lobulation sign and burr sign. At the same time, combining with the characteristics of neural network recognition and data set, the data set is preprocessed to highlight the features of the image and improve the accuracy of deep learning algorithm. Moreover, an auxiliary recognition system is developed around the optimized algorithm, so that the deep learning algorithm can intuitively show the automatic recognition of lung cancer through the recognition system after a large amount of data training.

On a Vector towards a Novel Hearing Aid Feature: What Can We Learn from Modern Family, Voice Classification and Deep Learning Algorithms

(1) Background: The application of machine learning techniques in the speech recognition literature has become a large field of study. Here, we aim to (1) expand the available evidence for the use of machine learning techniques for voice classification and (2) discuss the implications of such approaches towards the development of novel hearing aid features (i.e., voice familiarity detection). To do this, we built and tested a Convolutional Neural Network (CNN) Model for the identification and classification of a series of voices, namely the 10 cast members of the popular television show “Modern Family”. (2) Methods: Representative voice samples were selected from Season 1 of Modern Family (N = 300; 30 samples for each of the classes of the classification in this model, namely Phil, Claire, Hailey, Alex, Luke, Gloria, Jay, Manny, Mitch, Cameron). The audio samples were then cleaned and normalized. Feature extraction was then implemented and used as the input to train a basic CNN model and an advanced CNN model. (3) Results: Accuracy of voice classification for the basic model was 89%. Accuracy of the voice classification for the advanced model was 99%. (4) Conclusions: Greater familiarity with a voice is known to be beneficial for speech recognition. If a hearing aid can eventually be programmed to recognize voices that are familiar or not, perhaps it can also apply familiar voice features to improve hearing performance. Here we discuss how such machine learning, when applied to voice recognition, is a potential technological solution in the coming years.

Identification of batik making method from images using convolutional neural network with limited amount of data

This study aims to apply the convolutional neural network (CNN) to classify batik based on its manufacturing method, namely Batik Tulis which are hand drawn, Batik Cap where stamps are used to create the pattern, and Batik Printing which are printed using textile printing machine. We collected 40 images for each type of batik with a total of 120 images. To speed up and simplify the model building process, we implemented transfer learning with 3 basic CNN model architectures, namely ResNet, DenseNet, and VGG with batch normalization. We also experimented with building a new dataset by breaking each image down into 30 smaller images. Image augmentation was also used to prevent overfitting as well as to provide variations in the training data. The experimental results with 5-fold cross validation show that densenet169 gives the best results on the original dataset with an accuracy of 79.17% while vgg13_bn shows the best performance on the modified dataset with an accuracy of 87.61%. All models showed an increase in performance when using the modified dataset, except densenet169 which did not show a significant difference in performance.

A Statistical Robust Glaucoma Detection Framework Combining Retinex, CNN, and DOE Using Fundus Images

Motivated by the challenge that manual glaucoma detection is costly and time consuming, and that existing automated glaucoma detection processes lack either good performance or any statistical robustness testing procedures, we proposed an effective, robust, and automated framework for glaucoma detection based on fundus images. The proposed framework using 1450 color fundus images provided by Kaohsiung Chang Gung (KCG) Memorial Hospital in Taiwan. The proposed framework combines the use of convolutional neural networks (CNN) with the proposed generalized loss function, robust design of experiment (DOE), and Retinex theory to improve the results of fundus photography flash by restoring the original colors via removing the light effect. The proposed framework outperformed most archival automatic glaucoma detection approaches in its effectiveness and simplicity. The effectiveness was demonstrated via the estimated sensitivity 0.95, specificity 0.98, and accuracy 0.97. The simplicity was shown via the adopted basic CNN model compared to deep CNNs such as GoogleLeNet and ResNet152. Further, the proposed framework outperformed all relevant archival work in terms of its robustness, illustrated in the associated standard errors (all less than 0.03). This paper demonstrated the proposed framework via intuitive graphs and clear mathematical notations to make it easy for others to reproduce our results. The proposed framework and demonstration have the potential to become the standard automated glaucoma detection approaches in practice.

Short-Term Travel Speed Prediction for Urban Expressways: Hybrid Convolutional Neural Network Models

Deep learning models for short-term travel speed prediction on urban expressways, such as the convolutional neural network (CNN), still present several limitations in multiscale spatiotemporal feature extraction. Hence, in this paper, three hybrid CNN models are proposed to improve the basic CNN model with regard to three target aspects for short-term (i.e., 5 min) travel speed prediction on urban expressways. More specifically, long short-term memory (LSTM), AutoEncoder (AE), and Inception module are incorporated into the basic CNN model to capture multiscale spatiotemporal features of travel speed data effectively and improve the accuracy and robustness of the basic CNN model. Based on loop detector data collected on the Yan&#x2019;an expressway in Shanghai, the proposed hybrid CNN models are trained and tuned. To validate the improvements on the target aspects, a comprehensive comparison is conducted using a classical statistical model (i.e., autoregressive integrated moving average), a typical shallow neural network model (i.e., artificial neural network), and two basic deep learning models (i.e., recurrent neural network and CNN). Results show that the prediction accuracies of all the proposed hybrid CNN models exceed 96&#x0025; and the mean absolute errors are less than 2.5 km/h, which are superior to other models. In terms of target improving aspects, two new metrics were introduced, and the proposed models, especially the AE&#x2013;CNN model, showed better robustness under various input data structures and traffic states. The LSTM&#x2013;CNN model outperformed the other models in learning time-series features, and the Inception&#x2013;CNN model is superior in reproducing the dynamics of traffic congestion patterns on urban expressways.

Evaluation of Transfer Learning with Deep Convolutional Neural Networks for Screening Osteoporosis in Dental Panoramic Radiographs

Dental panoramic radiographs (DPRs) provide information required to potentially evaluate bone density changes through a textural and morphological feature analysis on a mandible. This study aims to evaluate the discriminating performance of deep convolutional neural networks (CNNs), employed with various transfer learning strategies, on the classification of specific features of osteoporosis in DPRs. For objective labeling, we collected a dataset containing 680 images from different patients who underwent both skeletal bone mineral density and digital panoramic radiographic examinations at the Korea University Ansan Hospital between 2009 and 2018. Four study groups were used to evaluate the impact of various transfer learning strategies on deep CNN models as follows: a basic CNN model with three convolutional layers (CNN3), visual geometry group deep CNN model (VGG-16), transfer learning model from VGG-16 (VGG-16_TF), and fine-tuning with the transfer learning model (VGG-16_TF_FT). The best performing model achieved an overall area under the receiver operating characteristic of 0.858. In this study, transfer learning and fine-tuning improved the performance of a deep CNN for screening osteoporosis in DPR images. In addition, using the gradient-weighted class activation mapping technique, a visual interpretation of the best performing deep CNN model indicated that the model relied on image features in the lower left and right border of the mandibular. This result suggests that deep learning-based assessment of DPR images could be useful and reliable in the automated screening of osteoporosis patients.

A convolutional neural network based framework for classification of seizure types.

Epileptic seizures are caused by a disturbance in the electrical activity of the brain and classified as many different types of epileptic seizures based on the characteristics of EEG and other parameters. Till now research has been conducted to classify EEG as seizure and non-seizures, but the classification of seizure types has not been explored. Thus, in this paper, we have proposed the 8-class classification problem in order to classify different seizure types using convolutional neural networks (CNN). This research study suggests a CNN based framework for classification of epileptic seizure types that include simple partial, complex partial, focal non-specific, generalized non-specific, absence, tonic, and tonic-clonic, and non-seizures. EEG time series was converted into spectrogram stacks and used as input for CNN. To the best of authors knowledge, ours is the very first study that classified the seizures types using the computational algorithm. The four CNN models, namely AlexNet, VGG16, VGG19, and basic CNN model was applied to study the performance of 8-class classification problem. The proposed study showed a classification accuracy of 84.06%, 79.71%, 76.81%, and 82.14% using AlexNet, VGG16, VGG19 and basic CNN models respectively. The experimental results suggest that the proposed framework could be helpful to the neurology community for recognition of seizures types.

Learning fine-grained features via a CNN Tree for Large-scale Classification

We propose a novel approach to enhance the discriminability of Convolutional Neural Networks (CNN). The key idea is to build a tree structure that could progressively learn fine-grained features to distinguish a subset of classes, by learning features only among these classes. Such features are expected to be more discriminative, compared to features learned for all the classes. We develop a new algorithm to effectively learn the tree structure from a large number of classes. Experiments on large-scale image classification tasks demonstrate that our method could boost the performance of a given basic CNN model. Our method is quite general, hence it can potentially be used in combination with many other deep learning models.