Different Classes Of Images Research Articles

A multi-weapon detection using ensembled learning

Abstract Recently, the level of criminals and terrorists using light weapons (such as knives and firearms) has increased rapidly around the world. Unfortunately, most current surveillance systems are still based mainly on human monitoring and intervention. For that reason, the requirement for a smart system for detecting different weapons becomes crucial in the field of security and computer vision. In this article, a novel technique for detecting various types of weapons has been proposed. This system is based mainly on deep learning techniques, namely, You Only Look Once, version 8 (YOLOv8), to detect a different class of light weapons. Furthermore, this study focuses on detecting two armed human poses based on ensemble learning techniques, which involve combining the outputs of different Yolov8 models to produce an accurate and robust detection system. The proposed system is evaluated on the self-created weapons dataset comprising thousands of images of different classes of weapons. The experiment results of this work show the effectiveness of ensemble learning for detecting various weapons with high accuracy, achieving 97.2% of mean average precision.

CropGCNN: color space-based crop disease classification using group convolutional neural network.

Classifying images is one of the most important tasks in computer vision. Recently, the best performance for image classification tasks has been shown by networks that are both deep and well-connected. These days, most datasets are made up of a fixed number of color images. The input images are taken in red green blue (RGB) format and classified without any changes being made to the original. It is observed that color spaces (basically changing original RGB images) have a major impact on classification accuracy, and we delve into the significance of color spaces. Moreover, datasets with a highly variable number of classes, such as the PlantVillage dataset utilizing a model that incorporates numerous color spaces inside the same model, achieve great levels of accuracy, and different classes of images are better represented in different color spaces. Furthermore, we demonstrate that this type of model, in which the input is preprocessed into many color spaces simultaneously, requires significantly fewer parameters to achieve high accuracy for classification. The proposed model basically takes an RGB image as input, turns it into seven separate color spaces at once, and then feeds each of those color spaces into its own Convolutional Neural Network (CNN) model. To lessen the load on the computer and the number of hyperparameters needed, we employ group convolutional layers in the proposed CNN model. We achieve substantial gains over the present state-of-the-art methods for the classification of crop disease.

Incremental semi-supervised graph learning NMF with block-diagonal

Non-negative matrix factorization (NMF), as a good data dimensionality reduction method, is widely used in the field of image recognition. Incremental non-negative matrix factorization (INMF) as an improvement solves the problem of inefficiency caused by repeated running of data samples during online learning. However, in the traditional incremental non-negative matrix factorization algorithm, the newly added training samples do not contain label information. Some samples may be marked in both the initial sample and the new training sample in the real application scenario. In order to make full use of the label information carried by the dataset, in this paper, we propose a semi-supervised non-negative matrix factorization model for batch incremental data, incremental semi-supervised graph learning NMF with block diagonal (ISGDNMF). The model is divided into three cases according to the degree of label-carrying for the new batch data: all label-carrying, no label-carrying, and partial label-carrying. The label information is also used to add a diagonal structure to the coefficient matrix, which makes it possible to have stronger discriminatory ability and to distinguish different classes of images more easily. And graph regularization information is added in order to maintain the spatial-geometric structure of the data. Experiments on six image datasets show that this algorithm has superior performance relative to the other seven NMF-based algorithms.

A new U-Net based system for multi-cultural wedding image classification

Use of social media for communication, sharing or expressing views, broadcasting news, threatening and blackmailing has become an integral part of society. One such activity is understanding multi-cultural wedding images uploaded on social media. This paper presents a novel method based on the combination of U-Net, Convolutional Neural Network and Random Forest for classification of multicultural wedding images. In the case of wedding images, bride and bridegroom draw the attention of the viewers. This observation led to propose a U-Net for segmenting the region of bride and bridegroom in a novel way. Similarly, it is noted that the costumes of bride and bridegroom are vital information for differentiating different cultures. This cue motivated us to extract features using CNN for classification. Since the extracted features using CNN are capable of discriminating images of different classes, we propose a simple and effective Random-Forest for Multicultural Wedding Image Classification. The efficiency of the proposed model is demonstrated by testing it on our own dataset of six multi-cultural wedding classes and standard dataset of wedding and non-wedding images classes. Experimental results on both the datasets show that the proposed model outperforms the state-of-the-art models in terms of average classification rate.

Deep metric learning framework combined with Gramian angular difference field image generation for Raman spectra classification based on a handheld Raman spectrometer

Rapid identification of unknown material samples using portable or handheld Raman spectroscopy detection equipment is becoming a common analytical tool. However, the design and implementation of a set of Raman spectroscopy-based devices for substance identification must include spectral sampling of standard reference substance samples, resolution matching between different devices, and the training process of the corresponding classification models. The process of selecting a suitable classification model is frequently time-consuming, and when the number of classes of substances to be recognised increases dramatically, recognition accuracy decreases dramatically. In this paper, we propose a fast classification method for Raman spectra based on deep metric learning networks combined with the Gramian angular difference field (GADF) image generation approach. First, we uniformly convert Raman spectra acquired at different resolutions into GADF images of the same resolution, addressing spectral dimension disparities induced by resolution differences in different Raman spectroscopy detection devices. Second, a network capable of implementing nonlinear distance measurements between GADF images of different classes of substances is designed based on a deep metric learning approach. The Raman spectra of 450 different mineral classes obtained from the RRUFF database were converted into GADF images and used to train this deep metric learning network. Finally, the trained network can be installed on an embedded computing platform and used in conjunction with portable or handheld Raman spectroscopic detection sensors to perform material identification tasks at various scales. A series of experiments demonstrate that our trained deep metric learning network outperforms existing mainstream machine learning models on classification tasks of different sizes. For the two tasks of Raman spectral classification of natural minerals of 260 classes and Raman spectral classification of pathogenic bacteria of 8 classes with significant noise, our suggested model achieved 98.05% and 90.13% classification accuracy, respectively. Finally, we also deployed the model in a handheld Raman spectrometer and conducted identification experiments on 350 samples of chemical substances attributed to 32 classes, achieving a classification accuracy of 99.14%. These results demonstrate that our method can greatly improve the efficiency of developing Raman spectroscopy-based substance detection devices and can be widely used in tasks of unknown substance identification.

Pseudo Label-Guided Model Inversion Attack via Conditional Generative Adversarial Network

Model inversion (MI) attacks have raised increasing concerns about privacy, which can reconstruct training data from public models. Indeed, MI attacks can be formalized as an optimization problem that seeks private data in a certain space. Recent MI attacks leverage a generative adversarial network (GAN) as an image prior to narrow the search space, and can successfully reconstruct even the high-dimensional data (e.g., face images). However, these generative MI attacks do not fully exploit the potential capabilities of the target model, still leading to a vague and coupled search space, i.e., different classes of images are coupled in the search space. Besides, the widely used cross-entropy loss in these attacks suffers from gradient vanishing. To address these problems, we propose Pseudo Label-Guided MI (PLG-MI) attack via conditional GAN (cGAN). At first, a top-n selection strategy is proposed to provide pseudo-labels for public data, and use pseudo-labels to guide the training of the cGAN. In this way, the search space is decoupled for different classes of images. Then a max-margin loss is introduced to improve the search process on the subspace of a target class. Extensive experiments demonstrate that our PLG-MI attack significantly improves the attack success rate and visual quality for various datasets and models, notably, 2 ∼ 3× better than state-of-the-art attacks under large distributional shifts. Our code is available at: https://github.com/LetheSec/PLG-MI-Attack.

A triplet graph convolutional network with attention and similarity-driven dictionary learning for remote sensing image retrieval

With the explosion in the volume of collected high-resolution aerial image data, the development of effective image retrieval methods for remote sensing (RS) has become a popular area of research. The problem of content-based image retrieval (CBIR) for high-resolution remote sensing (HRRS) requires robust feature extraction and representation to extract high-level semantics followed by a similarity measurement method to compute the similarity between images based on these semantics. In this paper, we propose a novel sparse graph learning-based triplet network for deep metric learning-based HRRS CBIR. First, the graph representations of the aerial images are constructed via local feature descriptors and unsupervised segmentation that provide an efficient graph-structured representation of the images. Then, a novel attention graph convolutional network is developed to extract high-level semantics from the spatial patterns of the images’ segmented regions by aggregating region-node features. The attention mechanism and the pyramid pooling structure of the graph network help the proposed model extract powerful spatial features by preserving the structural information of the images while reducing the data’s dimensionality. Finally, a novel task-driven dictionary learning (TDDL) method based on triplet loss constructs a sparse metric space for the similarity measurement of the images in which the computed sparse codes for the images promote intra-class similarity between same-class images and inter-class dissimilarity between images of different classes. The proposed sparse features reduce the dimensionality of model parameters which mitigates overfitting and improves the generalization of the overall framework for deep metric learning. Our TDDL method adopts a bi-level optimization strategy so that the dictionary can be trained alongside the parameters of the graph representation network in an end-to-end fashion. Finally, we conduct extensive evaluations of the performance of our proposed architecture on real-world datasets compared to state-of-the-art models and obtain superior results.

PGF-BIQA: Blind image quality assessment via probability multi-grained cascade forest

Blind image quality assessment (BIQA) aims to automatically predict the perceptual quality of a digital image without accessing its pristine reference and plays an important role in computer vision and image analysis. In fact, the difference in the subjective perception of image quality by evaluators greatly affected quality scores of the images. Aiming at the problem that the existing models cannot take into account the subjective perception differences in people, a blind image quality assessment algorithm via probability gcForest (PGF-BIQA) is proposed. Specifically, we use five qualitative labels to replace the specific quality score and extract image color and texture features to represent image quality. Unlike the widely used neural network, gcForest is ensemble by decision trees which randomly extract different features of various modes to learn the relationship between label and quality features. Therefore, a probability gcForest is designed, combining the frequency probability model and the gcForest, using probability confidence to represent the multiple decision modes results, and describing the subjective perception differences in evaluators. Next, through quality anchors and probability confidences of different classes, PGF-BIQA achieves image quality assessment. Moreover, this method proposes an image classification method that resolves the problem of an unbalanced number of images of different classes and the calculation process of the algorithm is simple and effective. Extensive experiments demonstrate that the proposed method is highly consistent with human perception and outperforms many state-of-the-art BIQA algorithms.

Fine-Grained Image Classification by Class and Image-Specific Decomposition With Multiple Views

Fine-grained image classification attempts to accurately classify images that are similar to each other. Multiview information is often used to improve the classification accuracy. Although great progress has been made, fine-grained image classification methods still have two drawbacks. On the one hand, they often treat each image independently without considering image correlations within the same class along with the distinctive characters of each image. On the other hand, multiview correlations are often used during classifier training, leaving the correlations of different views unconsidered. To solve these two problems, in this paper, we propose a novel fine-grained image classification method by class and image-specific decomposition with multiviews (CISD-MV). For each view, we treat images of the same class jointly by decomposing the class and image-specific information. Since images of different classes are similar and correlated, we linearly model class correlations of images using decomposed low-rank parts. In addition, for each image, the representations of different views are correlated, and we use linear transformation to model view correlations. We jointly optimise for the class and image-specific components along with the class correlation and view correlation transformation matrixes. A testing image is assigned to the class that has the minimum summed reconstruction error. We conduct fine-grained image classification experiments on several public fine-grained image datasets. Experimental results and analysis show the effectiveness of the proposed method.

Adversarial Sampling for Fairness Testing in Deep Neural Network

In this research, we focus on the usage of adversarial sampling to test for the fairness in the prediction of deep neural network model across different classes of image in a given dataset. While several framework had been proposed to ensure robustness of machine learning model against adversarial attack, some of which includes adversarial training algorithm. There is still the pitfall that adversarial training algorithm tends to cause disparity in accuracy and robustness among different group. Our research is aimed at using adversarial sampling to test for fairness in the prediction of deep neural network model across different classes or categories of image in a given dataset. We successfully demonstrated a new method of ensuring fairness across various group of input in deep neural network classifier. We trained our neural network model on the original image, and without training our model on the perturbed or attacked image. When we feed the adversarial samplings to our model, it was able to predict the original category/ class of the image the adversarial sample belongs to. We also introduced and used the separation of concern concept from software engineering whereby there is an additional standalone filter layer that filters perturbed image by heavily removing the noise or attack before automatically passing it to the network for classification, we were able to have accuracy of 93.3%. Cifar-10 dataset have ten categories of dataset, and so, in order to account for fairness, we applied our hypothesis across each categories of dataset and were able to get a consistent result and accuracy.

Blended multi-class text to image synthesis GANs with RoBerTa and Mask R-CNN

Generation of scenes from text description requires employing computer vision for processing image and natural language processing for decoding the text provided. The task here is difficult as it requires generating multiple images of different classes together. Existing methods construct images from captions using a single dataset class, utilising Generative Adversarial Networks (GANs) approaches. Training several classes with GANs is a cumbersome task that needs a large amount of data and huge computational power. With complex datasets the efficiency of the generated image according to text description may be poor. In this paper, we propose an application that generates images based on the Caltech-UCSD bird and Oxford 102 flowers dataset. It leverages the Attentional Generative Adversarial Network (AttnGANs) as the generative model and the RoBERTa neural language model for word embeddings to build an image from multiple classes trained in isolation. The image created by GANs is segmented using Mask R-CNN and blended together using Poisson blending. The application can create scenes based on the description provided by the user and produce them in less time compared to training multiple classes in a single generative network.

Performance Comparison for COVID-19 Chest X-ray Images Classification based on Different CNNs

Nowadays, the detection of the disease that is called Coronavirus or COVID-19 is essential for the whole world. Scientific researchers have spent significant efforts on better understanding the characteristics of the virus and possible means to prevent, diagnose and treat COVID-19. Convolutional neural networks (CNNs), have obtained remarkable results in numerous applications. One of these applications is image classification. Chest radiograph (X-ray) images can be requested for early COVID-19 classification of patients. Hence, this paper makes demonstrates different CNN architectures utilizing Chest radiograph database images for COVID-19: detection ( Kaggle’s X-ray chest images). It contains three different classes of images: 1) COVID-19, 2) normal, and 3) viral pneumonia Chest radiograph images. Therefore, three alternative CNN architectures like SqueezeNet, GoogleNet, and ResNet 50 have been realized using Matlab 2019a and numerical simulation has been performed. GoogleNet has achieved good performance based on the accuracy obtained with a value of 97.02% and it saves time-consuming. A performance comparison between different techniques has been carried out and this comparison shows that the detection is accurate enough for the non-uniform structure of the chest radiograph images.

One shot ancient character recognition with siamese similarity network

Ancient character recognition is not only important for the study and understanding of ancient history but also has a profound impact on the inheritance and development of national culture. In order to reduce the study of difficult professional knowledge of ancient characters, and meanwhile overcome the lack of data, class imbalance, diversification of glyphs, and open set recognition problems in ancient characters, we propose a Siamese similarity network based on a similarity learning method to directly learn input similarity and then apply the trained model to establish one shot classification task for recognition. Multi-scale fusion backbone structure and embedded structure are proposed in the network to improve the model's ability to extract features. We also propose the soft similarity contrast loss function for the first time, which ensures the optimization of similar images with higher similarity and different classes of images with greater differences while reducing the over-optimization of back-propagation leading to model overfitting. Specially, we propose a cumulative class prototype based on our network to solve the deviation problem of the mean class prototype and obtain a good class representation. Since new ancient characters can still be found in reality, our model has the ability to reject unknown categories while identifying new ones. A large number of experiments show that our proposed method has achieved high-efficiency discriminative performance and obtained the best performance over the methods of traditional deep learning and other classic one-shot learning.

An approach for characterization of infected area in tomato leaf disease based on deep learning and object detection technique

Tomato leaf infections are a common threat to long-term tomato production that affects many farmers worldwide. Early detection, treatment, and solution of tomato leaf specificity are critical for promoting healthy tomato plant growth and ensuring ample supply and health security for the world’s geometric growth (population). The detection of plant leaf disease using computer-assisted technologies is prevalent these days. In this work, use the 1610 tomato leaf images of different classes from PlantVillage standard repository for the localization of objects. An effective Deep Learning (DL) modified Mask Region Convolutional Neural Network (Mask R-CNN) is proposed for the autonomous segmentation and detection of tomato plant leaf disease in this research. Intending to conserve memory space and computational expense, the suggested model adds a light head “Region Convolutional Neural Network (R-CNN)”. By varying the proportions of anchor in the RPN network and also changing the feature extraction topology, which improves the detection accuracy and computing the metric performance. The proposed technique is compared to existing state-of-the-art models to check if it is viable and robust. The outcomes of the suggested model achieved the results in terms of Mean Average Precision (mAP), F1-score, and accuracy of 0.88, 0.912, and 0.98, respectively. Furthermore, as the model’s ability increases with some parameters, the detection time for lesion detection is reduced by two times than the existing models.

Deformable models for image segmentation: A critical review of achievements and future challenges

Image segmentation is a fundamental and tedious task of computer vision. Because of inherent noise and intensity inhomogeneity in real-world images, it remains a difficult problem in practical applications such as image analysis, scene understanding, object detection, and many others. Several mathematical models proposed for image segmentation in the past few decades with an effective policy. Among these models, deformable models earned more attention and are widely used techniques due to their accuracy, efficiency, and robust effectiveness. This study reviews and compares various deformable models for the segmentation of digital images available in the literature. First, for a comprehensive study, these deformable models are classified into two classes such as direct partial differential equation (PDE) based approaches and variational based approaches. Beside this, variational based approaches are further classified into parametric and geometric models. Their advantages as well as shortcomings are discussed in detail from an objective viewpoint. Then, to check the robustness of various discussed classes of deformable models, a set of synthetic, natural, and real medical images are considered along with inhomogeneity and noise. Also, to measure the segmentation accuracy, different quantitative metrics based on contour and region are utilized. Numerical experiments with different classes of images, reveal that both direct PDE based and variation based models perform well in clean and noisy images. Whereas, the variational based approaches are superior for images having intensity inhomogeneity with high computational time. The qualitative and quantitative investigations confirm that the subtle change in model assumptions can have a significant impact on segmentation.

Exemplar-Based, Semantic Guided Zero-Shot Visual Recognition

Zero-shot recognition has been a hot topic in recent years. Since no direct supervision is available, researchers use semantic information as the bridge instead. However, most zero-shot recognition methods jointly model images on the class level without considering the distinctive character of each image. To solve this problem, in this paper, we propose a novel exemplar-based, semantic guided zero-shot recognition method (EBSG). Both visual and semantic information of each image is used. We train visual sub-model to separate each image from the other images of different classes. We also train semantic sub-model to separate this image from the other images described with different semantics. We concatenate the outputs of visual and semantic sub-models to represent images. Image classification model is then learned by measuring visual similarity and semantic consistency of both source and target images. We conduct zero-shot recognition experiments on four widely used datasets. Experimental results show the effectiveness of the proposed EBSG method.

Automatic arteriosclerotic retinopathy grading using four-channel with image merging

Background and objectiveArteriosclerosis can reflect the severity of hypertension, which is one of the main diseases threatening human life safety. But Arteriosclerosis retinopathy detection involves costly and time-consuming manual assessment. To meet the urgent needs of automation, this paper developed a novel arteriosclerosis retinopathy grading method based on convolutional neural network. MethodsFirstly, we propose a good scheme for extracting features facing the fundus blood vessel background using image merging for contour enhancement. In this step, the original image is dealt with adaptive threshold processing to generate the new contour channel, which merge with the original three-channel image. Then, we employ the pre-trained convolutional neural network with transfer learning to speed up training and contour image channel parameter with Kaiming initialization. Moreover, ArcLoss is applied to increase inter-class differences and intra-class similarity aiming to the high similarity of images of different classes in the dataset. ResultsThe accuracy of arteriosclerosis retinopathy grading achieved by the proposed method is up to 65.354%, which is nearly 4% higher than those of the exiting methods. The Kappa of our method is 0.508 in arteriosclerosis retinopathy grading. ConclusionsAn experimental study on multiple metrics demonstrates the superiority of our method, which will be a useful to the toolbox for arteriosclerosis retinopathy grading.

Automated diagnosis of breast cancer using multi-modal datasets: A deep convolution neural network based approach

This paper proposes a deep convolutional neural network (CNN) model for automated breast cancer classification from a different class of images, namely, mammograms and ultrasound. The model contains only five learnable layers: four convolutional layers and a fully connected layer. The model facilitates extracting prominent features automatically from the images with a smaller number of tunable parameters. Exhaustive simulation results on mammograms dataset, namely, MIAS, DDSM, and INbreast, as well as ultrasound datasets, namely, BUS-1 and BUS-2, depict that the suggested model outperforms the recent state-of-the-art schemes. Data augmentation technique has been employed to reduce overfitting and provide good generalization. The proposed CNN model achieves an accuracy of 96.55%, 90.68%, and 91.28% on MIAS, DDSM, and INbreast datasets, respectively. Similarly, the accuracies obtained are 100% and 89.73% on BUS-1 and BUS-2 datasets, respectively.

Cross-evaluation of a parallel operating SVM – CNN classifier for reliable internal decision-making processes in composite inspection

In the aerospace industry, automated fibre laying processes are often applied for economical composite part fabrication. Unfortunately, the current mandatory visual quality assurance process takes up to 50% of the entire manufacturing time. An automised classification of manufacturing deviations using Neural Networks potentially improves the inspection's effectiveness. Unfortunately, the automated decision-making procedures of machine learning approaches are challenging to trace. Therefore, we introduce an approach for evaluating the classifiers response for this use case.For this purpose, we present a parallel classification approach of Convolutional Neural Network (CNN) and Support Vector Machine (SVM) with suitable intermediate checking stages between both classification processes. The particular novelty of this study is this intermediate comparison to trace the behaviour of the two classifiers along their image processing chains and to project the results back to the input image.With respect to the SVM, we analyse their extracted input features via t-Distributed Stochastic Neighbor Embedding calculations and parallel coordinates plots. Moreover, the classification score of the SVM as well as the feature vector distances within the SVM are investigated. For the CNN, the outputs of its first joined convolutional layer are correlated with the raw input images of different classes using Structural Similarity Index Measure metrics. Additionally, also the CNN's classification rates are analysed. Accordingly, a suitable uncertainty confidence interval for the CNN is determined on the bases of its neural activations. Finally, the relevance of individual pixels for the CNN decision is determined through Smooth Integrated Gradients and linked to the manually extracted image features for the SVM Classifier.The results of this paper are particularly valuable for developers and users of visual inspection systems in safety-critical domains.

An effective hybrid framework for content based image retrieval (CBIR)

In recent years, we have witnessed a massive growth in the generation of images on the cyberspace which demands to develop automated solutions for effective content management. Content-based image retrieval (CBIR) systems have been proposed to reduce the dependency on textual annotations-based image retrieval systems. There exists a variety of features-classifier combinations based CBIR methods to analyze the content of query image for relevant images retrieval. Although, these methods provide better retrieval performance in single-class scenario, however, we experience a significant performance drop in multi-class search environments due to semantics similarity among the images of different classes. CBIR methods based on the hybrid classification model offer better retrieval accuracy, however, we experience a biased classification towards the negative class due to the class imbalance problem when we experience an increase in the number of negative samples due to highly correlated semantic classes. Thus, multiple classifiers based CBIR models become unstable especially in one-against-all classification settings. To address the aforementioned problem, we proposed a CBIR method based on a hybrid features descriptor with the genetic algorithm (GA) and SVM classifier for image retrieval in multi-class scenario. More specifically, we employed the first three color moments, Haar Wavelet, Daubechies Wavelet and Bi-Orthogonal wavelets for features extraction, refine the features using GA and then train the multi-class SVM using one-against-all approach. L2 Norm is used as a similarity measurement function between the query image and retrieved images against the query image from the image repository. The proposed technique successfully addresses the class imbalance problem in CBIR. Performance of the proposed method is evaluated on four standard datasets i.e. WANG, Oxford Flower, CIFAR-10, and kvasir and compared with 25 different CBIR methods. Experimental results illustrate that our method outperforms the existing state-of-the-art CBIR methods in terms of image retrieval.