Pre-trained CNN Model Research Articles

FERDCNN: an efficient method for facial expression recognition through deep convolutional neural networks

Facial expression recognition (FER) has caught the research community’s attention recently because it can affect many real-life applications. Multiple studies have focused on automatic FER, most of which use a machine learning methodology, FER has continued to be a difficult and exciting issue in computer vision. Deep learning has recently drawn increased attention as a solution to several practical issues, including facial expression recognition. This article introduces an efficient method for FER (FERDCNN) verified on five different pre-trained deep CNN (DCNN) models (AlexNet, GoogleNet, ResNet-18, ResNet-50, and ResNet-101). In the proposed method, firstly the input image has been pre-processed using face detection, resizing, gamma correction, and histogram equalization techniques. Secondly, the images go through DCNN to extract deep features. Finally, support vector machine (SVM) and transfer learning are used to classify generated features. Recent methods have been employed to evaluate and contrast the performance of the proposed approach on two publicly standard databases namely, CK+ and JAFFE on the seven classes of fundamental emotions, including anger, disgust, fear, happiness, sadness, and surprise beside neutrality for CK+ and contempt for JAFFE. The suggested method tested Four different traditional supervised classifiers with deep features, Experimental found that AlexNet excels as a feature extractor, while SVM demonstrates superiority as a classifier because of this combination achieving the highest accuracy rates of 99.0% and 95.16% for the CK+ database and the JAFFE datasets, respectively.

Face Recognition and Authentication using DLIB for Examination Attendance System

Facial recognition technology enables the identification and detection of individuals based on unique facial features. based on various labels such as name, ID number, etc. In this research, a face recognition system for examination attendance is implemented using the DLIB library. Since manual attendance is a tedious and time-consuming task, this research aims to automate the process, thereby increasing the integrity of examinations. The proposed solution reduces the workload of invigilators by eliminating the need to manually take attendance and fill out attendance sheets. The dataset for this research comprises a combination of student images captured by a camera and images from the database. The system utilizes the DLIB library to generate facial encodings, which are then used to identify students in real-time. Upon recognizing a student, the system logs their attendance by recording their name and the current timestamp in a CSV file. DLIB has pre-trained CNN models that can be used forface detection. This ensures accurate and efficient attendance tracking, providing a reliable record of student presence during examinations.

A novel hybrid CNN-GRU model for website-based user interface code generator

As software engineering techniques evolve, conventional methods become less efficient due to prolonged development cycles and repetitive tasks. There is a growing need for automated systems that can incorporate User Interface (UI) designs into code. This study proposes a novel Hybrid CNN-GRU model for automatically generate website-based UI codes from UI design images. The proposed method consists of a visual model and a language-decoder part. The visual model employs pretrained CNN models, including EfficientNetV2B0, Xception, and EfficientNetV2M, to extract features from images, whereas a GRU serves as the language model and decoder. An empirical study was conducted to evaluate the effectiveness of various hybrid CNN-GRU approaches for generating UI codes. The EfficientNetV2B0 GRU model demonstrated the fastest training time, whereas the Xception GRU model was the most robust for various data complexities. The results of this study will advance the field of automated code generation, simplify the front-end development process, and shorten the time and effort necessitated for UI code implementations.

Improving the industrial defect recognition in radiographic testing by pre-training on medical radiographs

Deep learning methodologies have gained substantial traction for defect recognition in industrial radiographic testing including welds, castings and other fields. Regardless of the deep learning utilized, it has emerged as a standard configuration to use a model pre-trained from ImageNet to accelerate convergence and enhance recognition accuracy. However, there is a significant gap between the domain of natural images and industrial radiographs, raising the question of whether there might be a superior pre-training method than relying on ImageNet pre-training. Fortunately, medical radiographs are more similar to industrial radiographs than natural images because of the same imaging method. In this paper, we initially utilize numerous medical radiographic images from CheXpert dataset to train a pre-trained CNN model. Then, we apply this model to four distinct tasks within two radiographic testing scenarios to validate its advantages and generalization capabilities. Finally, experiments on multiple datasets indicate that our method brings more benefits than ImageNet pre-training or training from scratch, with a F1 score improvement of 3.41 %–13.72 % for defect classification and a mIoU improvement of 1.05 %–6.58 % for defect segmentation. It demonstrates that pre-training from medical radiographs provides a cost-free improvement for all kinds of tasks in industrial defect recognition.

Surface quality prediction in-situ monitoring system: A deep transfer learning-based regression approach with audible signal

Surface roughness plays an indispensable and fundamental role as a leading indicator of the surface quality of machined parts in the manufacturing process. The precise and effective monitoring and prediction of surface roughness is crucial for surface quality control. In this regard, the development of an in-process surface quality monitoring system is necessary, which has the promising potential to achieve this goal. Such a system typically comprises data-driven models for decision-making and sensing techniques for detecting associated process information. However, some challenges still exist in building such systems. Firstly, the architecture design and deployment of data-driven models, specifically deep learning (DL)-based models, demand adequate domain knowledge. Secondly, most models trained on specific tasks with limited datasets are prone to suppressing their versatility and generalization across different machining conditions. Additionally, in most cases, reliance on handcrafted features to represent dynamic information on various signals during model training necessitates extensive expertise in selecting appropriate feature types. Furthermore, due to the nature of their low dimensionality, handcrafted features have difficulty in capturing of overall process-related underlying patterns from dynamics signatures, which is time-varying and often occurs in transient events. To address these challenges, this paper proposes the regression-based pre-trained convolutional neural network (pre-trained CNN) combined with Mel-spectrogram images based on the transfer learning method for surface roughness prediction. Within the context, the architecture of the transfer model is slightly adapted from already well-trained CNNs. Initial weights in each layer of the CNN model are directly inherited and then fine-tuned through the Bayesian optimization tuning method. Besides, the audible sound signals are captured and subsequently converted into 2D Mel-spectrogram images with variant time lengths, which are separately engaged to retrain and validate four existing pre-trained CNN models (VGG16, VGG19, ResNet50V2 and InceptionResNetV2). Eventually, the effectiveness of proposed models and comparison of their predictive capabilities are further validated through a case study in the turning process. The results demonstrate that each applied pre-trained CNN model is capable of effectively predicting surface quality with satisfactory prediction results. Therefore, the proposed method can facilitate the establishment of a machining monitoring system concerning its accuracy, reliability, and robustness.

Automated diagnosis of COVID-19 using chest X-ray image processing by a Convolutional Neural Network

The COVID-19 pandemic has severely impacted global healthcare and financial systems, Highlighting the need for an automatic computer-aided diagnosis system using image recognition for chest X-rays(CXR). This study aims to classify COVID-19, normal, and pneumonia patients from CXR images via a modified ResNet-50 pre-trained CNN model. Our experiments are based on Dataset-1, which contains CXR images of COVID-19 and normal, while Dataset-2 also includes pneumonia. Dataset-1 and Dataset-2 were collected from King Abdul-Aziz Medical City in National Guard, Jeddah, Saudi Arabia. Moreover, a sample from the Kaggle repository was added to Dataset-1 and 2 to make two more Datasets. Our results for diagnosis of raspatory diseases have shown reliability and high accuracy of 95.28% (97.66% Sensitivity and 93.12% Specificity), which will be beneficial in aiding physicians and healthcare centres in the global fight against harmful spreading viruses through employing AI and ML techniques in X-ray medical diagnosis.

Combining KAN with CNN: KonvNeXt’s Performance in Remote Sensing and Patent Insights

Rapid advancements in satellite technology have led to a significant increase in high-resolution remote sensing (RS) images, necessitating the use of advanced processing methods. Additionally, patent analysis revealed a substantial increase in deep learning and machine learning applications in remote sensing, highlighting the growing importance of these technologies. Therefore, this paper introduces the Kolmogorov-Arnold Network (KAN) model to remote sensing to enhance efficiency and performance in RS applications. We conducted several experiments to validate KAN’s applicability, starting with the EuroSAT dataset, where we combined the KAN layer with multiple pre-trained CNN models. Optimal performance was achieved using ConvNeXt, leading to the development of the KonvNeXt model. KonvNeXt was evaluated on the Optimal-31, AID, and Merced datasets for validation and achieved accuracies of 90.59%, 94.1%, and 98.1%, respectively. The model also showed fast processing speed, with the Optimal-31 and Merced datasets completed in 107.63 s each, while the bigger and more complicated AID dataset took 545.91 s. This result is meaningful since it achieved faster speeds and comparable accuracy compared to the existing study, which utilized VIT and proved KonvNeXt’s applicability for remote sensing classification tasks. Furthermore, we investigated the model’s interpretability by utilizing Occlusion Sensitivity, and by displaying the influential regions, we validated its potential use in a variety of domains, including medical imaging and weather forecasting. This paper is meaningful in that it is the first to use KAN in remote sensing classification, proving its adaptability and efficiency.

An End-to-End Scene Text Recognition for Bilingual Text

Text localization and recognition from natural scene images has gained a lot of attention recently due to its crucial role in various applications, such as autonomous driving and intelligent navigation. However, two significant gaps exist in this area: (1) prior research has primarily focused on recognizing English text, whereas Arabic text has been underrepresented, and (2) most prior research has adopted separate approaches for scene text localization and recognition, as opposed to one integrated framework. To address these gaps, we propose a novel bilingual end-to-end approach that localizes and recognizes both Arabic and English text within a single natural scene image. Specifically, our approach utilizes pre-trained CNN models (ResNet and EfficientNetV2) with kernel representation for localization text and RNN models (LSTM and BiLSTM) with an attention mechanism for text recognition. In addition, the AraElectra Arabic language model was incorporated to enhance Arabic text recognition. Experimental results on the EvArest, ICDAR2017, and ICDAR2019 datasets demonstrated that our model not only achieves superior performance in recognizing horizontally oriented text but also in recognizing multi-oriented and curved Arabic and English text in natural scene images.

Recognition of aggressive driving behavior under abnormal weather based on Convolutional Neural Network and transfer learning

Objectives Aggressive driving behavior can lead to potential traffic collision risks, and abnormal weather conditions can exacerbate this behavior. This study aims to develop recognition models for aggressive driving under various climate conditions, addressing the challenge of collecting sufficient data in abnormal weather. Methods Driving data was collected in a virtual environment using a driving simulator under both normal and abnormal weather conditions. A model was trained on data from normal weather (source domain) and then transferred to foggy and rainy weather conditions (target domains) for retraining and fine-tuning. The K-means algorithm clustered driving behavior instances into three styles: aggressive, normal, and cautious. These clusters were used as labels for each instance in training a CNN model. The pre-trained CNN model was then transferred and fine-tuned for abnormal weather conditions. Results The transferred models showed improved recognition performance, achieving an accuracy score of 0.81 in both foggy and rainy weather conditions. This surpassed the non-transferred models’ accuracy scores of 0.72 and 0.69, respectively. Conclusions The study demonstrates the significant application value of transfer learning in recognizing aggressive driving behaviors with limited data. It also highlights the feasibility of using this approach to address the challenges of driving behavior recognition under abnormal weather conditions.

A privacy-preserving image retrieval scheme with access control based on searchable encryption in media cloud

With the popularity of the media cloud computing industry, individuals and organizations outsource image computation and storage to the media cloud server to reduce the storage burden. Media images usually contain a large amount of private information. To prevent disclosure of privacy of the image owners, media images are encrypted before uploading to the server. However, this operation will greatly limit the utilization of the image for the user, such as content-based image retrieval. We propose an efficient similarity query algorithm with access control based on Bkd-tree in this paper, in which a searchable encryption scheme is designed for similarity image retrieval, and the encrypted image is used to extract image features by a pre-trained CNN model. The Bkd-tree is utilized to generate an index tree for the image features to speed up retrieval and make it faster than linear indexing. Finally, the security performances of the proposed scheme is analyzed and the performance of this scheme is evaluated by experiments. The results show that the security of the image content and image features can be ensured, and it has a shorter retrieval time and higher retrieval efficiency.

Human action recognition with transformer based on convolutional features

As one of the key research directions in the field of computer vision, human action recognition has a wide range of practical application values and prospects. In the fields of video surveillance, human-computer interaction, sports analysis, and healthcare, human action recognition technology shows a broad application prospect and potential. However, the diversity and complexity of human actions bring many challenges, such as handling complex actions, distinguishing similar actions, coping with changes in viewing angle, and overcoming occlusion problems. To address the challenges, this paper proposes an innovative framework for human action recognition. The framework combines the latest pose estimation algorithms, pre-trained CNN models, and a Vision Transformer to build an efficient system. The first step involves utilizing the latest pose estimation algorithm to accurately extract human pose information from real RGB image frames. Then, a pre-trained CNN model is used to perform feature extraction on the extracted pose information. Finally, the Vision Transformer model is applied for fusion and classification operations on the extracted features. Experimental validation is conducted on two benchmark datasets, UCF 50 and UCF 101, to demonstrate the effectiveness and efficiency of the proposed framework. The applicability and limitations of the framework in different scenarios are further explored through quantitative and qualitative experiments, providing valuable insights and inspiration for future research.

Unsupervised video summarization with adversarial graph-based attention network

Video summarization aims to select a subset of video segments that best capture the video storyline. Our study seeks to train an encoder to transform the raw frame features extracted from pre-trained CNN models into representations that embody importance and guide the selection of the video segment. Our main idea is to use graph modeling and attention mechanisms to train the encoder adversarially. The graph representation enables the model to learn the relationship among frames, revealing the intrinsic structure of a video. The attention mechanism allows the model to capture the magnitude of these relationships. In the proposed model, an attention-based encoder is trained using a graph-based generator that reconstructs videos using the encoded features and a discriminator that guides the generator, distinguishing the original and reconstructed video. Thus, by leveraging graph attention and refining mechanisms, the proposed model offers distinct advantages over existing methods, including enhanced summarization accuracy, improved preservation of temporal coherence, and the ability to capture complex semantic linkages within video content. These advancements are substantiated through a comprehensive ablation study, which demonstrates the efficacy of our model using various evaluation metrics — Kendall and Spearman coefficients. The proposed model is evaluated on TVSum and SumMe datasets and achieves results on par with supervised models that used similar encoders and achieved state-of-the-art results compared to other unsupervised models.

Deep Learning-Based Iris Recognition System Using Unprocessed Images

The iris image is a powerful and distinctive feature in biometrics that serves as a reliable instrument for human identification. Extracting significant features is crucial for developing iris-based recognition systems. While preprocessing for iris region detection is typically the first step in such systems, it can often fail in real acquisition conditions, leading to decreased performance. This study proposes a new iris-based recognition system that directly exploits original (noisy) images for feature extraction, avoiding the bottleneck of preprocessing. Additionally, a multimodal architecture is proposed for both right and left iris images to strengthen the decision step. Pretrained CNN models extract features classified by SoftMax and support vector machines (SVM). The performance of the proposed system is tested on four public datasets collected under different conditions. The results show that classification accuracy for the original iris dataset without pre-processing is higher than for the normalized database due to the rich CNN features that provide more cognitive information, resulting in greater discrimination power. The CNN multi-modal recognition system achieves the best accuracy compared to most state-of-the-art models, demonstrating the strength of the proposed fusion recognition system.

Advanced Dental Implant System Classification with Pre-trained CNN Models and Multi-branch Spectral Channel Attention Networks

Advanced Dental Implant System Classification with Pre-trained CNN Models and Multi-branch Spectral Channel Attention Networks

Enhancing Fashion Recommendations with Facial Recognition: A Gender and Age Prediction Perspective

This research paper presents a real-time system for age and gender estimation integrated into a fashion recommendation context. Leveraging deep learning models for facial detection and classification, alongside a Random Forest-based recommender system for fashion items, the system provides personalized recommendations based on user age, gender, and preferences. The methodology involves detecting faces in video streams, extracting facial regions, and preprocessing them for age and gender estimation using pre-trained CNN models. Fashion recommendations are then generated by assigning weighted scores to items based on various attributes and training a Random Forest regressor. Implementation details, including the use of Python, OpenCV, and machine learning libraries, are discussed. Evaluation metrics for age and gender detection models, as well as recommendation system performance, are outlined. The paper concludes with future directions and challenges, emphasizing enhancements in age and gender estimation accuracy, integration with e-commerce platforms, and the importance of data privacy and computational resources. Keywords — Age and gender estimation, Facial detection, Deep learning, Fashion recommendation, Random Forest.

Risky event classification leveraging transfer learning for very limited datasets in optical networks

Monitoring the state of polarization (SOP) is crucial for tracking vibrations or disturbances in the vicinity of optical fibers, such as precursors to fiber cuts. While SOP data are valuable for machine learning (ML) models in identifying vibrations, acquiring a sufficient amount of data presents a significant challenge. To overcome this hurdle, we introduce an innovative transfer learning framework designed for the identification of vibrations (events) when confronted with limited SOP data. Our methodology leverages the pre-trained convolutional neural network MobileNet as a feature extractor, incorporating the encoding of time series SOP measurements into images for MobileNet input. We explore different time series encoding techniques, including the Gramian Angular Difference Field (GADF) and the Gramian Angular Summation Field (GASF). Different architectures for building our transfer learning framework based on MobileNet are investigated. Validation of our proposed approaches is conducted using experimental data that simulates movements indicative of fiber break precursors. The experimental results clearly demonstrate the superior performance of our approaches compared to other ML algorithms, especially in scenarios with limited data. Furthermore, our framework surpasses pre-trained CNN models in terms of predictive power, affirming its effectiveness in enhancing the accuracy of vibration identification in the presence of constrained SOP data.

Performance Analysis of ResNet50 and Inception-V3 Image Classification for Defect Detection in 3D Food Printing

In the future, 3D food printing may be a fruitful area for the food industry. The presence of this technology opens the public's eyes to a revolution in the food sector, where food processing is limited to the conventional. The existence of this technology will also make food production much more effective and efficient. However, there is a problem when the 3D Food Printing process is carried out; when object defects occur, there will be material waste. This research proposes deep learning-based defect detection for defect and non-defect objects. This model mainly uses CNN as a deep learning method. The dataset is taken from the image of the print process performed at the time of object creation to be trained and validated to check the effectiveness of the proposed model. The architecture used uses pre-trained CNN models namely Inception-V3 and ResNet50 with the hope of classifying images that have a higher viscosity of the material. Where the model has been tested with previous datasets and applied with 3D Food Printing datasets with a dataset division ratio of 85% which is training data and 15% is validation data. After testing the two proposed scenarios, the accuracy result obtained in the test model scenario 1, Inception-V3, is 84.62% and for the test model scenario 2, ResNet50, the accuracy is 93.83%. The outcomes also demonstrate that improved accuracy, loss, and classification time may be obtained by applying CNN in conjunction with data augmentation.

Deep transfer CNNs models performance evaluation using unbalanced histopathological breast cancer dataset

Cancer is one of the top deadly diseases. Of this disease, around about 9.8 million death cause annually. It has been recorded by the American Cancer Society that every eight women die due to breast cancer in the USA. In this paper, we have identified eight different lesion categories: Benign Tumor: Adenosis-Adenoma, Fibro-Adenoma, Phyllodes-Tumor, Tubular-Adenoma, and Malignant Tumor; Ductal-Carcinoma, Lobular-Carcinoma, Mucinous-Carcinoma, Papillary-Carcinoma. The main contribution of this paper is to examine the performance of five pre-trained CNN models on an unbalanced cancer dataset for cancer prediction. The identification of different cancer tumors has been recognized by using transfer learning models namely ResNet50, ResNet101, ResNet152, VGG16, and VGG19. BreakHis dataset has four different magnifications (40x-100x-200x-400x), and used for experiments setup in this study. The total number of images for all magnification levels is 7909. The experimental results state that the pre-trained model Residual Net with different variations has worked well 91%~96% as compared to other pre-trained models. The ResNet101 architecture model has gained a multiclass identification higher than 95%. In this paper, the proposed methodology has different evaluation parameters such as accuracy, recall, and f1-score of all pre-trained models that will help to build optimal, and automated breast lesion multiclass identification.

Advancements in deep learning for accurate classification of grape leaves and diagnosis of grape diseases

AbstractPlant diseases cause significant agricultural losses, demanding accurate detection methods. Traditional approaches relying on expert knowledge may be biased, but advancements in computing, particularly deep learning, offer non-experts effective tools. This study focuses on fine-tuning cutting-edge pre-trained CNN and vision transformer models to classify grape leaves and diagnose grape leaf diseases through digital images. Our research examined a PlantVillage dataset, which comprises 4062 leaf images distributed across four categories. Additionally, we utilized the Grapevine dataset, consisting of 500 leaf images. This dataset is organized into five distinct groups, with each group containing 100 images corresponding to one of the five grape types. The PlantVillage dataset focuses on four classes related to grape diseases, namely Black Rot, Leaf Blight, Healthy, and Esca leaves. On the other hand, the Grapevine dataset includes five classes for leaf recognition, specifically Ak, Alaidris, Buzgulu, Dimnit, and Nazli. In experiments with 14 CNN and 17 vision transformer models, deep learning demonstrated high accuracy in distinguishing grape diseases and recognizing leaves. Notably, four models achieved 100% accuracy on PlantVillage and Grapevine datasets, with Swinv2-Base standing out. This approach holds promise for enhancing crop productivity through early disease detection and providing insights into grape variety characterization in agriculture.

Deep Learning Approaches for Sunflower Disease Classification: A Study of Convolutional Neural Networks with Squeeze and Excitation Attention Blocks

Diseases in agricultural plants are one of the most important problems of agricultural production. These diseases cause decreases in production and this poses a serious problem for food safety. One of the agricultural products is sunflower. Helianthus annuus, generally known as sunflower, is an agricultural plant with high economic value grown due to its drought-resistant and oil seeds. In this study, it is aimed to classify the diseases seen in sunflower leaves and flowers by applying deep learning models. First of all, it was classified with ResNet101 and ResNext101, which are pre-trained CNN models, and then it was classified by adding squeeze and excitation blocks to these networks and the results were compared. In the study, a data set containing gray mold, downy mildew, and leaf scars diseases affecting the sunflower crop was used. In our study, original Resnet101, SE-Resnet101, ResNext101, and SE-ResNext101 deep-learning models were used to classify sunflower diseases. For the original images, the classification accuracy of 91.48% with Resnet101, 92.55% with SE-Resnet101, 92.55% with ResNext101, and 94.68% with SE-ResNext101 was achieved. The same models were also suitable for augmented images and classification accuracies of Resnet101 99.20%, SE-Resnet101 99.47%, ResNext101 98.94%, and SE-ResNext101 99.84% were achieved. The study revealed a comparative analysis of deep learning models for the classification of some diseases in the Sunflower plant. In the analysis, it was seen that SE blocks increased the classification performance for this dataset. Application of these models to real-world agricultural scenarios holds promise for early disease detection and response and may help reduce potential crop losses.