Visual Geometry Group Research Articles

Deep Learning for Early Detection of Diabetic Retinopathy: A Colour Fundus Image-based Approach with VGG16 and VGG19

Diabetic retinopathy (DR) is a prevalent complication of diabetes, posing a significant threat to vision. Early detection and intervention are critical to prevent vision loss. However, manual screening of retinal images for DR is time-consuming and susceptible to human error. This study investigates the application of deep learning models, specifically VGG16 and VGG19, for automated DR diagnosis in Colour Fundus images. We evaluate their performance in classifying images as either Diabetic Retinopathy or No Diabetic Retinopathy. Our results demonstrate high testing accuracy with VGG16 achieving 93.96%, suggesting a promising approach for automated DR screening and aiding healthcare professionals in early and accurate diagnosis. Keywords : Convolutional Neural Network, Diabetic Retinopathy, Visual Geometry Group 16 (VGG16), Visual Geometry Group 19 (VGG19), Transfer Learning, Deep Learning.

An effective hybrid attention model for crop yield prediction using IoT-based three-phase prediction with an improved sailfish optimizer

Internet of Things (IoT)-based smart agriculture approaches have been adopted to achieve promising results in terms of food productivity. Moreover, high cost and technological complexity are major problems in existing agricultural management systems. Thus, a new IoT-based smart agriculture farming model is implemented in three phases namely crop detection, crop disease detection, and crop yield prediction. In the first phase, the plant leaf images are gathered from the Kaggle source PlantifyDr. The collected images are passed to the deep feature retrieval phase, where the Visual Geometry Group (VGG16) is utilized to calculate feature (f1). Then, a Hybrid Attention-based Crop Type Detection Network (HA-CTDecNet) is used for detecting the crop types, in which the parameters are optimized by Attack Power Improved Sailfish Optimizer (API-SFO). In the second step of crop disease detection phase, the input images are fed to Otsu Thresholding for Abnormality segmentation. The deep attributes from the segmented images are haul out using VGG16 and attain the feature (f2). The extracted features are forwarded to the Hybrid Attention-based Crop Disease Detection Network (HA-CDDecNet), where the constraints are optimized using API-SFO. In the third crop yield prediction phase, the data related to crop location are collected from the crop yield forecast dataset. Then, the collected data and the extracted features (f1) and (f2) are concatenated and fed to the Principle Component Analysis (PCA). Then, these features are given to the Hybrid Attention-based Crop Yield Prediction Network (HA-CYPreNet) with the parameter optimization of API-SFO. The investigation is accepted to verify the effectiveness of the suggested IoT-based smart agriculture model with the traditional approaches by considering distinct measures. The Mean Absolute Error (MAE) of the crop yield prediction model is improved with 35% than CNN, 27.5% than LSTM, 17.5% than GRU, and 7.5% than LSTM_GRU while considering the learning percentage value as 55.

Incremental-based YoloV3 model with hyper-parameter optimization for product image classification in E-commerce sector

Over the past few years, the E-commerce industry has grown tremendously for selling products to consumers. Here, the consumer can easily purchase the products from their residing seats and gets the products at the door step. Also, the image quality is suggested and also the image classification is intruded to meet the consumer prerequisite. Most of the image classification models are developed by machine learning approaches to improve the reliability, scalability, and accuracy level. Due to the heterogeneous nature of images, it restricts the features and lengthy dimensions for datasets which it degrades the performance whereas the classification of products becomes a cumbersome situation. In order to resolve the problem raised by existing methods, a novel product image classification model is proposed for the fashion E-commerce sector. The prime aspect of the proposed model is to build an effective E-commerce sector for a layman, where every individual can purchase the products appropriately. Initially, the dataset is constructed with diverse product images that are obtained from different E-commerce fields with various classes. Consequently, a novel Incremental-based Improved YoloV3 with Hyper-parameter Optimization (II-Yolov3-HO) model is introduced to enhance the classification performance. In the proposed model, some of the hyper parameters like convolution and pooling layers are replaced with the Visual Geometry Group16 (VGG16) model, the remaining convolutional layers are enhanced by the atrous convolution operation, and pooling layers are modified with Atrous Spatial Pyramid Pooling (ASPP). In entire architecture, the weight factor is optimized using Predator-based Squirrel Search Algorithm (PDP-SSA) algorithm. Thus, the new E-commerce sector becomes free to categorize the products, and if it deals with a new product, it is declared as unknown data, which is then trained again in the proposed model. Hence, the concept of classification score matching with the threshold value is accomplished to deal the untrained product and the training weight is updated optimally using the proposed PDP-SSA. Finally, the enhanced model is compared over the state-of-the-art classification, provides the expected result for significantly classifying the multiple classes of products. The developed PDP-SSA-II-YoloV3-HO model is also able to perform recommendations in E-commerce sites even when the input is given as images other than text for searching the items. These uploaded images are analyzed in the developed II-Yolov3-HO model for providing the relevant items to the customer based on their features. The developed PDP-SSA-II-YoloV3-HO model achieves 99.46 % regarding accuracy. Throughout the validation, the developed model outperforms enhanced and reliable performance than the existing approaches.

Classification of Ice Particle Shapes Using Machine Learning on Forward Light Scattering Images

Abstract Machine learning (ML) has rapidly transitioned from a niche activity to a mainstream tool for environmental research applications including atmospheric science cloud microphysics studies. Two recently developed cloud particle probes measure the light scattered in the near forward direction and save digital images of the scattering light. Scattering pattern images collected by the Particle Phase Discriminator mark 2, Karlsruhe edition (PPD-2K), and the Small Ice Detector, version 3 (SID-3) provide valuable information for particle shape and size characterization. Since different particle shapes have distinctly different light scattering characteristics, the images are ideally suited for ML. Here, results of a ML project to characterize ice particle shapes sampled by the PPD-2K in ice fog and diamond dust during a 3-yr project in Fairbanks, Alaska. About 2.15 million light-scattering pattern images were collected during 3 years of measurements with the PPD-2K. Visual Geometry Group (VGG) convolutional neural network (CNN) was trained to categorize light-scattering patterns into eight categories. Initial training images (120 each category) were selected by human visual examination of data, and the training dataset was augmented using an automated iterative method for image identification of further images which were all visually inspected by a human. Results were well correlated to similar categories identified from previously developed classification algorithms. ML identifies characteristics not included in automated analysis such as sublimation. Of the 2.15 million images analyzed, 1.3% were categorized as spherical (liquid), 43.5% were categorized as having rough surfaces, 15.3% were pristine, 16.3% were categorized as sublimating, and the remaining 23.6% did not fit into any of those categories (irregular or saturated). Significance Statement The shapes and sizes of cloud particles can be extremely important for understanding the conditions that exist in the cloud. In this study, we show that more information about cloud particle characteristics can be identified by using machine learning than by traditional means. To demonstrate this, data are analyzed from a 3-yr study of ice fog and diamond dust events in Fairbanks, Alaska. The Particle Phase Discriminator instrument collected 2.15 million light-scattering pattern images of cloud particles during ground-based measurements. Neither traditional techniques nor machine learning were able to identify all categories, but a combination of both techniques led to a more complete view of ice particle shapes.

Computer vision for eye diseases detection using pre‐trained deep learning techniques and raspberry Pi

AbstractEarly diagnosis of eye diseases is very important to prevent visual impairment and guide appropriate treatment methods. This paper presents a unique approach that can detect numerous eye diseases automatically. Initially, this approach used the pre‐trained ImageNet models that provides various pre‐trained models for training the acquired data. The existing data sets are composed of 645 data images acquired clinically, represented by two groups of subjects as healthy and others holding the proposed eye defect like cataracts, foreign bodies, glaucoma, subconjunctival haemorrhage, and viral conjunctivitis. Followed by comparisons of the pre‐trained model's coefficients and prediction performance. Later, the first‐class execution model is integrated within the Raspberry Pi staging and the real‐time digital camera detection. The evaluation process used the confusion matrix, model accuracy, precision factor, recall coefficient, F1 score, and the Matthews Correlation Coefficient (MCC). Resulting in the performance of these pre‐trained ImageNet models used in this study represented by 93% (InceptionResNetV2), 90% (MobileNet), 86% (Residual Network ResNet50), 85% (InceptionV3), 78% (Visual Geometry Group VGG19), and 72% (Neural Architecture Search Network NASNetMobile). The results show that the InceptionResNetV2 achieved the highest performance. This proposed approach shows its efficiency and strength by early detection of the subject's unhealthy eyes through real‐time monitoring in the field of ophthalmology.

Development and Validation of a Diagnosis System for Lung Infection Using Hybrid Deep-Learning Techniques

A fast and accurate test is necessary to detect COVID-19. A computed tomography (CT) scan has shown diagnostic accuracy. CT scan processing using a deep learning architecture may improve illness diagnosis and treatment. We proposed a deep learning system for COVID-19 detection using CT images, including using and comparing transfer-learning, fine-tuning, and the embedding process. This paper presents the development of a COVID-19 case identification model using deep learning techniques. The suggested model utilized a modified visual geometry group (VGG16) architecture as the deep learning framework. The model was trained and validated using a chest CT image dataset. The SARS-COV-2 dataset contains 2482 CT scans of 210 patients from publicly available sources. The modified model demonstrated encouraging outcomes by greatly enhancing the sensitivity measure (95.82±1.75)%, which is an essential criterion for accurately detecting instances of COVID-19 infection. In addition, the model achieved higher values for the accuracy metric (91.67±1.68)%, the specificity meter (88.08±3.72)%, the precision metric (87.51±3.27)%, the F1 score (91.43±1.55)%, and the area under the curve (91.98±1.55)%. Deep learning effectively detects COVID-19 in chest CT scan images. Clinical practitioners may employ the suggested approach to study, identify, and effectively mitigate a greater number of pandemics.

Augmenting Telepostpartum Care With Vision-Based Detection of Breastfeeding-Related Conditions: Algorithm Development and Validation.

Breastfeeding benefits both the mother and infant and is a topic of attention in public health. After childbirth, untreated medical conditions or lack of support lead many mothers to discontinue breastfeeding. For instance, nipple damage and mastitis affect 80% and 20% of US mothers, respectively. Lactation consultants (LCs) help mothers with breastfeeding, providing in-person, remote, and hybrid lactation support. LCs guide, encourage, and find ways for mothers to have a better experience breastfeeding. Current telehealth services help mothers seek LCs for breastfeeding support, where images help them identify and address many issues. Due to the disproportional ratio of LCs and mothers in need, these professionals are often overloaded and burned out. This study aims to investigate the effectiveness of 5 distinct convolutional neural networks in detecting healthy lactating breasts and 6 breastfeeding-related issues by only using red, green, and blue images. Our goal was to assess the applicability of this algorithm as an auxiliary resource for LCs to identify painful breast conditions quickly, better manage their patients through triage, respond promptly to patient needs, and enhance the overall experience and care for breastfeeding mothers. We evaluated the potential for 5 classification models to detect breastfeeding-related conditions using 1078 breast and nipple images gathered from web-based and physical educational resources. We used the convolutional neural networks Resnet50, Visual Geometry Group model with 16 layers (VGG16), InceptionV3, EfficientNetV2, and DenseNet169 to classify the images across 7 classes: healthy, abscess, mastitis, nipple blebs, dermatosis, engorgement, and nipple damage by improper feeding or misuse of breast pumps. We also evaluated the models' ability to distinguish between healthy and unhealthy images. We present an analysis of the classification challenges, identifying image traits that may confound the detection model. The best model achieves an average area under the receiver operating characteristic curve of 0.93 for all conditions after data augmentation for multiclass classification. For binary classification, we achieved, with the best model, an average area under the curve of 0.96 for all conditions after data augmentation. Several factors contributed to the misclassification of images, including similar visual features in the conditions that precede other conditions (such as the mastitis spectrum disorder), partially covered breasts or nipples, and images depicting multiple conditions in the same breast. This vision-based automated detection technique offers an opportunity to enhance postpartum care for mothers and can potentially help alleviate the workload of LCs by expediting decision-making processes.

Dual-source dual-energy CT and deep learning for equivocal lymph nodes on CT images for thyroid cancer.

This study investigated the diagnostic performance of dual-energy computed tomography (CT) and deep learning for the preoperative classification of equivocal lymph nodes (LNs) on CT images in thyroid cancer patients. In this prospective study, from October 2020 to March 2021, 375 patients with thyroid disease underwent thin-section dual-energy thyroid CT at a small field of view (FOV) and thyroid surgery. The data of 183 patients with 281 LNs were analyzed. The targeted LNs were negative or equivocal on small FOV CT images. Six deep-learning models were used to classify the LNs on conventional CT images. The performance of all models was compared with pathology reports. Of the 281 LNs, 65.5% had a short diameter of less than 4 mm. Multiple quantitative dual-energy CT parameters significantly differed between benign and malignant LNs. Multivariable logistic regression analyses showed that the best combination of parameters had an area under the curve (AUC) of 0.857, with excellent consistency and discrimination, and its diagnostic accuracy and sensitivity were 74.4% and 84.2%, respectively (p < 0.001). The visual geometry group 16 (VGG16) based model achieved the best accuracy (86%) and sensitivity (88%) in differentiating between benign and malignant LNs, with an AUC of 0.89. The VGG16 model based on small FOV CT images showed better diagnostic accuracy and sensitivity than the spectral parameter model. Our study presents a noninvasive and convenient imaging biomarker to predict malignant LNs without suspicious CT features in thyroid cancer patients. Our study presents a deep-learning-based model to predict malignant lymph nodes in thyroid cancer without suspicious features on conventional CT images, which shows better diagnostic accuracy and sensitivity than the regression model based on spectral parameters. Many cervical lymph nodes (LNs) do not express suspicious features on conventional computed tomography (CT). Dual-energy CT parameters can distinguish between benign and malignant LNs. Visual geometry group 16 model shows superior diagnostic accuracy and sensitivity for malignant LNs.

Enhancing natural disaster analysis and waste classification: a novel VGG-FL approach.

The study of natural disasters is a crucial field that involves analyzing the occurrence, impact, and aftermath of various natural hazards that can cause significant harm to communities and the environment. Efficient waste management and environmental protection require proper classification of waste. Analyzing natural disasters and categorizing waste can be a time-consuming task, and conventional methods often struggle with it. However, a new approach called Visual Geometry Group with Federated Learning (VGG-FL) has been introduced to address these challenges. This methodology uses the golden search optimization (GSO) algorithm for feature selection and leverages VGG with federated learning for feature extraction and classification. To test the effectiveness of this method, a disaster image dataset was used to train the VGG-FL model. The results showed that the VGG-FL model attained exceptional accuracy in discerning and categorizing various disaster scenarios. The waste classification dataset simultaneously trains the VGG-FL model to categorize waste based on its characteristics and potential hazards. To measure the model's performance, several evaluation metrics such as accuracy, specificity, precision, F1-score, and recall are utilized to assess the effectiveness of the proposed VGG-FL method. These results are then compared with existing methodologies. The VGG-FL method performs exceptionally well, achieving 98.52% accuracy, 97.48% precision, 97.83% recall, 97.58% F1-score, and 97.12% specificity. These experimental findings demonstrate the efficacy of the VGG-FL method in analyzing natural disasters and classifying waste materials.

Intelligent identification system of wild animals image based on deep learning in biodiversity conservation law

This study aims to overcome the impact of complex environmental backgrounds on the recognition of wildlife in monitoring images, thereby exploring the role of a deep learning-based intelligent wildlife recognition system in biodiversity conservation. The automatic identification of wildlife images is conducted based on convolutional neural networks (CNNs). Target detection technology, based on regression algorithms, is initially employed to extract Regions of Interest (ROI) containing wildlife from images. The wildlife regions in monitoring images are detected, segmented, and converted into ROI images. A dual-channel network model based on Visual Geometry Group 16 (VGG16) is implemented to extract features from sample images. Finally, these features are input into a classifier to achieve wildlife recognition. The proposed optimized model demonstrates superior recognition performance for five wildlife species, caribou, lynx, mule deer, badger, and antelope, compared to the dual-channel network model based on VGG16. The optimized model achieves a Mean Average Precision (MAP) of 0.714, with a maximum difference of 0.145 compared to the other three network structures, affirming its effectiveness in enhancing the accuracy of automatic wildlife recognition. The model effectively addresses the issue of low recognition accuracy caused by the complexity of background information in monitoring images, achieving high-precision recognition and holding significant implications for the implementation of biodiversity conservation laws.

Three-dimensional morphological characterization of blood droplets during the dynamic coagulation process.

In this study, we employed a method integrating optical coherence tomography (OCT) with the U-Net and Visual Geometry Group (VGG)-Net frameworks within a convolutional neural network for quantitative characterization of the three dimensional whole blood during the dynamic coagulation process. VGG-Net architecture for the identification of blood droplets across three distinct coagulation stages including drop, gelation, and coagulation achieves an accuracy of up to 99%. In addition, the U-Net architecture demonstrated proficiency in effectively segmenting uncoagulated and coagulated portions of whole blood, as well as the background. Notably, parameters such as volume of uncoagulated and coagulated segments of the whole blood were successfully employed for the precise quantification of the coagulation process, which indicates well for the potential of future clinical diagnostics and analyses.

Application value of artificial intelligence algorithm-based magnetic resonance multi-sequence imaging in staging diagnosis of cervical cancer.

The aim of this research is to explore the application value of Deep residual network model (DRN) for deep learning-based multi-sequence magnetic resonance imaging (MRI) in the staging diagnosis of cervical cancer (CC). This research included 90 patients diagnosed with CC between August 2019 and May 2021 at the hospital. After undergoing MRI examination, the clinical staging and surgical pathological staging of patients were conducted. The research then evaluated the results of clinical staging and MRI staging to assess their diagnostic accuracy and correlation. In the staging diagnosis of CC, the feature enhancement layer was added to the DRN model, and the MRI imaging features of CC were used to enhance the image information. The precision, specificity, and sensitivity of the constructed model were analyzed, and then the accuracy of clinical diagnosis staging and MRI staging were compared. As the model constructed DRN in this research was compared with convolutional neural network (CNN) and the classic deep neural network visual geometry group (VGG), the precision was 67.7, 84.9, and 93.6%, respectively. The sensitivity was 70.4, 82.5, and 91.2%, while the specificity was 68.5, 83.8, and 92.2%, respectively. The precision, sensitivity, and specificity of the model were remarkably higher than those of CNN and VGG models (P < 0.05). As the clinical staging and MRI staging of CC were compared, the diagnostic accuracy of MRI was 100%, while that of clinical diagnosis was 83.7%, showing a significant difference between them (P < 0.05). Multi-sequence MRI under intelligent algorithm had a high diagnostic rate for CC staging, deserving a good clinical application value.

Image Data Acquisition and Classification of Vannamei Shrimp Cultivation Results Based on Deep Learning

This research aimed to employ deep learning techniques to address the classification of Litopenaeus vannamei cultivation results in land ponds and tarpaulin ponds. Despite their similar appearance, distinguishable differences exist in various aspects such as color, shape, size, and market price between the two cultivation methods, often leading to consumer confusion and potential exploitation by irresponsible sellers. To mitigate this challenge, the research proposed a classification method utilizing two Convolutional Neural Network (CNN) architectures: Visual Geometry Group-16 (VGG-16) and Residual Network-50 (ResNet-50), renowned for their success in various image recognition applications. The dataset comprised 2,080 images per class of vannamei shrimp from both types of ponds. Augmentation techniques enhanced the dataset’s diversity and sample size, reinforcing the model’s ability to discern shrimp morphology variations. Experiments were conducted with learning rates of 0.001 and 0.0001 on the Stochastic Gradient Descent (SGD) and Adaptive Moment Estimation (ADAM) optimizers to evaluate their effectiveness in model training. The VGG-16 and ResNet-50 models were trained with a learning rate parameter of 0.0001, leveraging the flexibility and reasonable control provided by the SGD optimizer. Lower learning rate values were chosen to prevent overfitting and enhance training stability. The model evaluation demonstrated promising results, with both architectures achieving 100% accuracy in classifying vannamei shrimp from soil ponds and tarpaulin ponds. Furthermore, experimental findings highlight the superiority of using SGD with a learning rate of 0.0001 over 0.001 on both architectures, underscoring the significant impact of optimizer and learning rate selection on model training effectiveness in image classification tasks.

Star Generative Adversarial VGG Network-Based Sample Augmentation for Insulator Defect Detection

Deep learning-based automated detection of insulator defects in electric power systems is a critical technological challenge, pivotal for ensuring reliability and efficiency in the global energy infrastructure. However, the effectiveness of the deep learning model is severely compromised by the scarcity of defective insulator samples. To tackle this problem, the present study proposes a style transfer approach utilizing an improved Star Generative Adversarial Network 2 (StarGAN2) model to generate artificial samples of faulty insulators, which adeptly synthesizes artificial faulty insulator samples on a one-to-many basis, markedly diminishing the necessity for extensive empirical data collection. Through the integration of identity loss, the proposed model ensures the fidelity of content and the preservation of critical defect semantics. Additionally, the proposed model incorporates a pre-trained Visual Geometry Group (VGG) network and perceptual loss, thus improving the quality of generated samples without additional artificial labeling. Finally, various experiments are conducted to assess the quality and authenticity of the generated samples and their impact on the detection model. The results demonstrate that StarGAN2 could generate realistic insulator defect samples and improve the performance of defect detection models.

Improving the performance for automated brain tumor classification on magnetic resonance imaging deep learningbased

&lt;p&gt;Brain tumor is an uncontrolled growth of abnormal cell in the brain. Early diagnosis of brain tumor has a crucial step in this type of cancer, which is fatal. Magnetic resonance imaging (MRI) is one of the examination tools to examine brain anatomy in clinical practice. The high resolution and clear separation of the tissue enable medical experts to identify brain tumor. The earlier of brain tumor is detected, the wider of treatment options. However, manually analysed of brain anatomy on MRI images are time-consuming. Computer-aided diagnosis with automated way is helpful solution to help management with unreliable degrees of automation to trace various tissue boundaries. This study proposes convolutional neural network (CNN) with its excellences to automated features extraction in convolution layer. The popular architectures of CNN, i.e., visual geometry group16 (VGG16), residual network-50 (resNet-50), inceptionV3, mobileNet, and efficientNetB7 in medical image processing are compared to brain tumor classification task. As the results, VGG16 outperformed other architectures of CNN in this study. VGG16 yields 100% accuracy, precision, sensitivity, specificity, and F1-score for testing set data. The results show the excellent performance in classifying brain tumor and no tumor from MRI images that demonstrate the efficiency of system suggested.&lt;/p&gt;

Coffee bean graded based on deep net models

Coffee is a widely consumed beverage, and sorting coffee beans is a critical process that ensures high-quality graded coffee products. Coffee beans were graded into nine grades in robusta types. To automate the grading process, a deep learning-based approach was developed using a large dataset of high-resolution images and data augmentation techniques. In contrast to previous studies focusing on robusta type graded into six coffee bean grads, our research extends this framework by employing robusta type into nine grades with an outperformed accuracy. The proposed work uses four deep learning models, namely residual network 34(Resnet34), inception version 3 (Inception v3), efficient network bayesian optimization (EfficientNet-B0), and visual geometry group-16(VGG-16), where trained and evaluated for coffee bean classification into nine grades. The EfficientNet-B0 model exhibited outperformed accuracy, achieving 100% in distinguishing good and bad coffee beans, even in challenging lighting and background conditions.

Diversity-Learning Block: Conquer Feature Homogenization of Multibranch.

Visual Geometry Group (VGG)-style ConvNet is an neural-network process units (NPU)-friendly network; however, the accuracy of this architecture cannot keep up with other well-designed network structures. Although some reparameterization methods are proposed to remedy this weakness, their performance suffers from the homogenization issue of parallel branches, and the preset shape of convolution kernels also influences spatial perception. To address this problem, we propose a diversity-learning (DL) block to build the DLNet, which could adaptively learn various features to enrich the feature space. To balance floating point of operations (FLOPs) and accuracy, groupwise operation is introduced and finally, a lightweight DL ConvNet DLGNet is obtained. Extensive evaluations have been conducted on different computer vision tasks, e.g., image classification [Canadian Institute For Advanced Research (CIFAR) and ImageNet], object detection [PASCAL visual object classes (VOC) and Microsoft Common Objects in Context (MS COCO)], and semantic segmentation (Cityscapes). The experimental results show that our proposed DLGNet can achieve comparable performance with the state-of-the-art networks while the speed is 183% faster than GhostNet and even over 600% faster than MobileNetV3 with similar accuracy when running on NPU.

Defect analysis of 3D printed object using transfer learning approaches

Additive manufacturing (AM) is rapidly gaining traction across diverse industries, including healthcare, aerospace, and automotive, mainly due to its ability to enhance production efficiency. However, a critical challenge in AM is the early detection of defects, which can significantly reduce production costs and improve productivity. Addressing this need, our study explored the effectiveness of machine learning (ML) approaches, mainly focusing on transfer learning (TL) models, for defect detection in 3D-printed objects. We utilized a range of TL models such as visualgeometrygroup (VGG)-16, inception-residualnetwork (InceptionResNet)-V2, residual network (ResNet)-50, mobilenetwork (MobileNet)-V2, VGG19, ResNet101, extreme inception (Xception), efficientnetwork (EfficientNet)-B0, EfficientNetB7, efficientnetworkV2medium (EfficientNetV2M), and neural architecture search network large (NASNetLarge), analyzing images of 3D-printed objects across three datasets using various statistical measures. An ablation study was also conducted using adaptive moment estimation (Adam), stochastic gradient descent (Sgd), and root mean square propagation (Rmsprop) optimizers. We identified that TL models such as VGG16, MobileNetV2, InceptionResNetV2, and NASNetLarge significantly outperform others, especially when optimized with Adam and Rmsprop. Conversely, we found that models like EfficientNetB0, EfficientNetB7, and EfficientNetV2M exhibited lower performance when paired with the Sgd optimizer. Additionally, Local Interpretable Model-agnostic (LIME)-based approaches are used to provide explanations of the models’ predictions. These findings offer substantial insights into model optimization and integration, aiming to enhance AM product quality through advanced image-based monitoring and inspection.

A multimodal machine learning approach to generate news articles from geo-tagged images

Classical machine learning algorithms typically operate on unimodal data and hence it can analyze and make predictions based on data from a single source (modality). Whereas multimodal machine learning algorithm, learns from information across multiple modalities, such as text, images, audio, and sensor data. The paper leverages the functionalities of multimodal machine learning (ML) application for generating text from images. The proposed work presents an innovative multimodal algorithm that automates the creation of news articles from geo-tagged images by leveraging cutting-edge developments in machine learning, image captioning, and advanced text generation technologies. Employing a multimodal approach that integrates machine learning and transformer algorithms, such as visual geometry group network16 (VGGNet16), convolutional neural network (CNN) and a long short-term memory (LSTM) based system, the algorithm initiates by extracting the location from exchangeable image file format (Exif) data from the image. The features are extracted from the image and corresponding news headline is generated. The headlines are used for generating a comprehensive article with contemporary large language model (LLM). Further, the algorithm generates the news article big-science large open-science open-access multilingual language model (BLOOM). The algorithm was tested on real time photographs as well as images from the internet. In both the cases the news articles generated were validated with ROUGE and BULE score. The proposed work is found to be successful attempt in journalism field.

Deep Learning-Based Detection and Segmentation of Damage in Solar Panels

Renewable energy can lead to a sustainable future and solar energy is one the primary sources of renewable energy. Solar energy is harvested mainly by photovoltaic plants. Though there are a large number of solar panels, the economic efficiency of solar panels is not that high in comparison to energy production from coal or nuclear matter. The main risk involved in solar plants is the high maintenance cost involved in maintaining the plants. To help reduce this issue, automated solutions using Unmanned Aerial Vehicles (UAVs) and satellite imagery are proposed. In this research work, we propose a novel deep learning architecture for the segmentation of solar plant aerial images, which not only helps in automated solar plant maintenance, but can also be used for the area estimation and extraction of solar panels from an image. Along with this, we also propose a transfer learning-based model for the efficient classification of solar panel damage. Solar panel damage classification has a lot of applications. It can be integrated into monitoring systems, raising alerts when there is severe damage or damage of a certain type. The adaptive UNet model with Atrous Spatial Pyramid Pooling (ASPP) module that performed the dilated convolutions that we proposed achieved an overall accuracy of 98% with a Mean Intersection-Over-Union (IoU) Score of 95% and took under a second to process an image. Our classification model using Visual Geometry Group 19 (VGG19) as the backbone for feature extraction has achieved a classification accuracy of 98% with an F1 score of 99%, thus detecting the five classes of damage, including undamaged solar panels, in an efficient manner.