Visual Classification Problems Research Articles

PSXII-15 Automatic Estimation of Body Condition Score in Finished Cattle Using Deep Learning Computational Architectures in Two-Dimensional Images

Abstract The objective of this research was to investigate the possibility of using computational architectures of deep learning neural networks (deep learning) in two-dimensional images to automatically estimate the BCS (body condition score) in beef cattle. Data collection lasted from March 2021 to December 2022 and was carried out on a commercial farm in Mato Grosso do Sul, Brazil. Data from 1,570 animals of different genetic compositions, sex classes, ages and body weights were used. The animals were randomly selected in the confinement stalls, identified and concomitantly. To obtain the RGB digital images, the 3DBeef @Tech device was used, which was implemented in Python (based on OpenCV, 2015). The sensor was positioned in the passageway at a distance of 2.50 meters, allo wing the collection of images of only one animal at a time. The bBCS was evaluated remotely by trained professionals during the data collection period, ranging from 1 to 9, through the visualization of fat deposition and muscle coverage in certain anatomical points of the animal. The collected images and evaluations were used as input for a segmentation convolutional neural network (CNN; Figure 1). The labeled data set was divided into training, validation and testing, respectively in the proportions of 65%, 25% and 10%. A deep neural network was constructed to classify BCS using Python and the deep learning library Tensorflow and run through a Pro+ instance of Google Collaboratory. The results obtained show that, despite the small amount of data currently available for training, the neural network had a performance designed to classify BCS, with training accuracy of 80.5% and validation accuracy of 65.9% (Figure 2 and 3). when compared with professional raters. The validation still exhibited accuracy, recall, F1-score and AUC of 0.6676, 0.6562, 0.6562 and 0.9519 (P < 0.05), respectively. Despite the fact that the BCS assessment is a problem of difficult visual classification, both due to the variability of possible scenarios and the subtle difference between different BCS scores. It is believed that with a greater number of evaluated images, an accuracy greater than 90% will be achieved for the BCS. Therefore, it is necessary to continue with studies to obtain a greater number of images collected in search of greater accuracy.

A Multi-Layer Feature Fusion Method for Few-Shot Image Classification.

In image classification, few-shot learning deals with recognizing visual categories from a few tagged examples. The degree of expressiveness of the encoded features in this scenario is a crucial question that needs to be addressed in the models being trained. Recent approaches have achieved encouraging results in improving few-shot models in deep learning, but designing a competitive and simple architecture is challenging, especially considering its requirement in many practical applications. This work proposes an improved few-shot model based on a multi-layer feature fusion (FMLF) method. The presented approach includes extended feature extraction and fusion mechanisms in the Convolutional Neural Network (CNN) backbone, as well as an effective metric to compute the divergences in the end. In order to evaluate the proposed method, a challenging visual classification problem, maize crop insect classification with specific pests and beneficial categories, is addressed, serving both as a test of our model and as a means to propose a novel dataset. Experiments were carried out to compare the results with ResNet50, VGG16, and MobileNetv2, used as feature extraction backbones, and the FMLF method demonstrated higher accuracy with fewer parameters. The proposed FMLF method improved accuracy scores by up to 3.62% in one-shot and 2.82% in five-shot classification tasks compared to a traditional backbone, which uses only global image features.

Intelligent classification of coal structure using multinomial logistic regression, random forest and fully connected neural network with multisource geophysical logging data

The structure of coal indicates the degree of its fragmentation after tectonic movement, which affects the exploration and development of coalbed methane (CBM). Although coal core observations are the most convenient and intuitive way of identifying the coal structure, they are not applicable for use in unexplored coal seams without CBM wells, and they are also very time-consuming. In comparison, geophysical-logging interpretation of the coal structure is more efficient and economical. However, although qualitative methods, such as principal component analysis (PCA), can be used to identify the coal structure with geophysical logging, the interpretation is limited by the calculation ability, and improvements are required based on the structure of an empirical model. Multinomial logistic regression (MLR), random forest (RF), and deep fully connected neural network (DNN) are effective machine learning methods more accurate than the traditional method that with model-aided identification. In this respect, the MLR method is a classical method based on mathematical linear regression, and it has a low construction cost; RF is an ensemble learning algorithm based on a decision tree and use of a bagging algorithm; and DNN is a deep learning model based on self-built feature engineering that has high classification accuracy under a large amount of data training and provides obvious advantages in visual coal classification problems. In this work, the three machine learning methods, MLR, RF, and DNN, were used to identify the coal structure. Two sets of logging data comprising different quantities from the Anze Block of the southern Qinshui Basin, North China, were selected to quantitatively compare the accuracy of coal structure identification with partial coal core observation. The results showed that for 210 and 840 samples, respectively, the accuracy was 76% and 77% for MLR, 83% and 86% for RF, and 82% and 86% for DNN. These results show that the MLR and DNN methods are superior for use with minimal and maximum amounts of data, respectively, and the RF method provides overall accuracy. Furthermore, an algorithmic classification of the coal structure was established, and the geological factors controlling the predicted structure, such as geostress, coal seam thickness, and burial depth, were distinguished.

An Assessment of the Effectiveness of Pretrained Neural Networks for Malware Detection

The speed at which malware develops is much faster than analysts are able to analyze it, and it is now a dangerous threat to businesses, critical infrastructure and individuals, forcing anti-virus companies to develop fast and reliable methods for detecting malware. Transfer learning, a deep learning technique, has special abilities such as training speed, lower training dataset size requirements and reduced demand for domain expert knowledge. In this paper, we compared the prediction and model generation performances by using modern convolutional neural networks (Resnet, DenseNet, VGG and AlexNet), which have proven to be successful in the visual classification problem for malware detection. In addition, a dataset containing 14,972 malware samples and 14,500 samples of benign files with some function similar to that possessed by malwares was proposed and used in the study. The best accuracy obtained was 96.35% corresponding to the DenseNet-201 architecture.

Fine Traffic Congestion Detection With Hierarchical Description

Is video congestion detection a visual classification problem? Most studies say yes. They classify road states into 1–3 congestion levels to determine whether congestion occurs. However, in real traffic scenes, congestion is a dynamic process, including generation, evolution, and dissipation. Congestion detection should not only be a classification problem but also the detection of the entire congestion process. This paper proposes a novel method to realize fine congestion detection using road surveillance video. The first step is to analyze road scene and detect vehicle information. The road area is modeled as a grid to construct three region levels: segment, lane, and cell. Vehicle position sequences and speeds are detected by using object detection, multi-object tracking, and sparse optical flow. The second step is to discriminate congestion state. Traffic flow, average speed, and visual impact parameters of each region level are counted. And a three-dimensional model is formed to obtain the short-time congestion state. The third step is to describe the congestion spatiotemporal changes by accumulating multiple short-term congestion states. Experiments are organized from three aspects: whether congestion occurs, continuous congestion degree, and spatiotemporal changes of congestion. The results indicate that the proposed method has higher detection accuracy while reflecting the whole spatiotemporal changes of congestion degree.

Automated identification of diverse Neotropical pollen samples using convolutional neural networks

Abstract Pollen is used to investigate a diverse range of ecological problems, from identifying plant–pollinator relationships to tracking flowering phenology. Pollen types are identified according to a set of distinctive morphological characters which are understood to capture taxonomic differences and phylogenetic relationships among taxa. However, categorizing morphological variation among hyperdiverse pollen samples represents a challenge even for an expert analyst. We present an automated workflow for pollen analysis, from the automated scanning of pollen sample slides to the automated detection and identification of pollen taxa using convolutional neural networks (CNNs). We analysed aerial pollen samples from lowland Panama and used a microscope slide scanner to capture three‐dimensional representations of 150 sample slides. These pollen sample images were annotated by an expert using a virtual microscope. Metadata were digitally recorded for ~100 pollen grains per slide, including location, identification and the analyst's confidence of the given identification. We used these annotated images to train and test our detection and classification CNN models. Our approach is two‐part. We first compared three methods for training CNN models to detect pollen grains on a palynological slide. We next investigated approaches to training CNN models for pollen identification. Because the diversity of pollen taxa in environmental and palaeontological samples follows a long‐tailed distribution, we experimented with methods for addressing imbalanced representation using our most abundant 46 taxa. We found that properly weighting pollen taxa in our training objective functions yielded improved accuracy for individual taxa. Our average accuracy for the 46‐way classification problem was 82.3%. We achieved 89.5% accuracy for our 25 most abundant taxa. Pollen represents a challenging visual classification problem that can serve as a model for other areas of biology that rely on visual identification. Our results add to the body of research demonstrating the potential for a fully automated pollen classification system for environmental and palaeontological samples. Slide imaging, pollen detection and specimen identification can be automated to produce a streamlined workflow.

A Graph-Related High-Order Neural Network Architecture via Feature Aggregation Enhancement for Identification Application of Diseases and Pests.

Diseases and pests are essential threat factors that affect agricultural production, food security supply, and ecological plant diversity. However, the accurate recognition of various diseases and pests is still challenging for existing advanced information and intelligence technologies. Disease and pest recognition is typically a fine-grained visual classification problem, which is easy to confuse the traditional coarse-grained methods due to the external similarity between different categories and the significant differences among each subsample of the same category. Toward this end, this paper proposes an effective graph-related high-order network with feature aggregation enhancement (GHA-Net) to handle the fine-grained image recognition of plant pests and diseases. In our approach, an improved CSP-stage backbone network is first formed to offer massive channel-shuffled features in multiple granularities. Secondly, relying on the multilevel attention mechanism, the feature aggregation enhancement module is designed to exploit distinguishable fine-grained features representing different discriminating parts. Meanwhile, the graphic convolution module is constructed to analyse the graph-correlated representation of part-specific interrelationships by regularizing semantic features into the high-order tensor space. With the collaborative learning of three modules, our approach can grasp the robust contextual details of diseases and pests for better fine-grained identification. Extensive experiments on several public fine-grained disease and pest datasets demonstrate that the proposed GHA-Net achieves better performances in accuracy and efficiency surpassing several other existing models and is more suitable for fine-grained identification applications in complex scenes.

Deep Discriminative Feature Models (DDFMs) for Set Based Face Recognition and Distance Metric Learning.

This article introduces two methods that find compact deep feature models for approximating images in set based face recognition problems. The proposed method treats each image set as a nonlinear face manifold that is composed of linear components. To find linear components of the face manifold, we first split image sets into subsets containing face images which share similar appearances. Then, our first proposed method approximates each subset by using the center of the deep feature representations of images in those subsets. Centers modeling the subsets are learned by using distance metric learning. The second proposed method uses discriminative common vectors to represent image features in the subsets, and entire subset is approximated with an affine hull in this approach. Discriminative common vectors are subset centers that are projected onto a new feature space where the combined within-class variances coming from all subsets are removed. Our proposed methods can also be considered as distance metric learning methods using triplet loss function where the learned subcluster centers are the selected anchors. This procedure yields to applying distance metric learning to quantized data and brings many advantages over using classical distance metric learning methods. We tested proposed methods on various face recognition problems using image sets and some visual object classification problems. Experimental results show that the proposed methods achieve the state-of-the-art accuracies on the most of the tested image datasets.

Deep compact polyhedral conic classifier for open and closed set recognition

In this paper, we propose a new deep neural network classifier that simultaneously maximizes the inter-class separation and minimizes the intra-class variation by using the polyhedral conic classification function. The proposed method has one loss term that allows the margin maximization to maximize the inter-class separation and another loss term that controls the compactness of the class acceptance regions. Our proposed method has a nice geometric interpretation using polyhedral conic function geometry. We tested the proposed method on various visual classification problems including closed/open set recognition and anomaly detection. The experimental results show that the proposed method typically outperforms other state-of-the-art methods, and becomes a better choice compared to other tested methods especially for open set recognition type problems. The source code of the proposed method is available at https://github.com/bdrhn9/dc-epcc.

Multi-stream hybrid architecture based on cross-level fusion strategy for fine-grained crop species recognition in precision agriculture

Precision farming aims to optimizing the crop production process and managing sustainable supply chain practices as more efficient and reasonable as possible. Recently, various advanced technologies, such as deep-learning and internet of things (IoT), have achieved remarkable intelligence progress in realistic agricultural conditions. However, crops species recognition can be considered as fine-grained visual classification (FGVC) problem, suffering the low inter-class discrepancy and high intra-class variances from the subordinate categories, which is more challenging than common basic-level category classification depended on traditional deep neural networks (DNNs). This paper presents a fine-grained visual recognition model named as MCF-Net to classifying different crop species in practical farmland scenes. Proposed MCF-Net is consisted of cross stage partial network (CSPNet) as backbone module, three parallel sub-networks, and cross-level fusion module. With multi-stream hybrid architecture utilizing massive fine-granulometric information, MCF-Net obtains preferable representation ability for distinguishing interclass discrepancy and tolerating intra-class variances. In addition, the end-to-end implementation of MCF-Net is optimized by cross-level fusion strategy to accurately identify different crop categories. Several experiments on in-field CropDeepv2 datasets demonstrate that our method favorably against the state-of-the-art methods. The recognition accuracy and F1-score of MCF-Net achieves very competitive results up to 90.6% and 0.962 separately, both of which outperforming contrasted models indicate better recognition accuracy and model stability of our method. Moreover, the overall parameters of MCF-Net are only 807 MByte with achieving a good balance between model’s performance and complexity. It is acceptable and suitable to the implementation of IoT platforms in precision agricultural practices.

Class mean vector component and discriminant analysis

• We propose a component analysis method for kernel-based subspace learning. • We give extensive analysis of the method and its connection with KPCA and KDA. • A discriminant analysis method is proposed based on properties of KDA projections. The kernel matrix used in kernel methods encodes all the information required for solving complex nonlinear problems defined on data representations in the input space using simple, but implicitly defined, solutions. Spectral analysis on the kernel matrix defines an explicit nonlinear mapping of the input data representations to a subspace of the kernel space, which can be used for directly applying linear methods. However, the selection of the kernel subspace is crucial for the performance of the proceeding processing steps. We propose a new optimization criterion , leading to a new component analysis method for kernel-based dimensionality reduction that optimally preserves the pair-wise distances of the class means in the feature space. This leads to efficient kernel subspace learning, which is crucial for kernel-based machine learning solutions. We provide extensive analysis on the connections and differences between the proposed criterion and the criteria used in kernel Principal Component Analysis, kernel Entropy Component Analysis and Kernel Discriminant Analysis, leading to a discriminant analysis version of the proposed method. Our theoretical analysis also provides more insights on the properties of the feature spaces obtained by applying these methods. Results on a variety of visual classification problems illustrate the properties of the proposed methods.

Multi-stream aggregation network for fine-grained crop pests and diseases image recognition

Pests and disease recognition can be considered as Fine-Grained Visual Classification (FGVC) problems, suffering low inter-class discrepancy and high intra-class variances from the sub-categories, ...

Fine-Grained Age Estimation With Multi-Attention Network

Human age estimation from a single image is a quite challenging task due to the subtle appearance change in the slow aging process. In this article, we propose a compact multi-attention deep network for age estimation based on the idea of fine-grained learning and visual attention mechanism. Concerning the problem that age estimation is a fine-grained visual classification problem, it relies on not only the global features of the face image, but also the fine-grained feature representations from age-sensitive local regions. Therefore, accurate age estimation benefits from multi-scale features and their fusion. Therefore, in this article, a multi-attention model built on a complementary two-stream compact network is proposed for age estimation. For a given intermediate feature map from the network, spatial attentions and channel attentions can be inferred in both self-attention and mutual-attention way. To emphasize crucial features from age-sensitive regions, the multi-attention maps are then multiplied to the input feature map for adaptive feature refinement. Finally, the refined feature maps at multiple layers are aggregated as the fine-grained feature for age estimation. Compared to bulky models, our model is compact and end-to-end. However, the performance of our model is competitive compared with those state-of-the-art methods.

Incremental learning with a homeostatic self-organizing neural model

We present a new self-organized neural model that we term resilient self-organizing tissue (ReST), which can be run as a convolutional neural network, possesses a $$c^\infty $$ energy function as well as a probabilistic interpretation of neural activities. The latter arises from the constraint of lognormal activity distribution over time that is enforced during ReST learning. The principal message of this article is that self-organized models in general are, due to their localized learning rule that updates only those units close to the best-matching unit, ideal representation learners for incremental learning architectures. We present such an architecture that uses ReST layers as a building block, benchmark its performance w.r.t. incremental learning in three real-world visual classification problems, and justify the mechanisms implemented in the architecture by dedicated experiments.

Exploiting multiplex data relationships in Support Vector Machines

In this paper, a novel method for introducing multiplex data relationships to the SVM optimization process is presented. Different properties about the training data are encoded in graph structures, in the form of pairwise data relationships. Then, they are incorporated to the SVM optimization problem, as modified graph-regularized basekernels, each highlighting a different property about the training data. The contribution of each graph-regularized kernel to the SVM classification problem, is estimated automatically. Thereby, the solution of the proposed modified SVM optimization problem lies in a regularized space, where data similarity is expressed by a linear combination of multiple single-graph regularized kernels. The proposed method exploits and extends the findings of Multiple Kernel Learning and graph-based SVM method families. It is shown that the available kernel options for the former can be broadened, and the exhaustive parameter tuning for the latter can be eliminated. Moreover, both method families can be considered as special cases of the proposed formulation, hereafter. Our experimental evaluation in visual data classification problems denote the superiority of the proposed method. The obtained classification performance gains can be explained by the exploitation of multiplex data relationships, during the classifier optimization process.

Long-Term Ensemble Learning for Cross-Season Visual Place Classification

This paper addresses the problem of cross-season visual place classification (VPC) from the novel perspective of long-term map learning. Our goal is to enable transfer learning efficiently from one season to the next, at a small constant cost, and without wasting the robot’s available long-term-memory by memorizing very large amounts of training data. To achieve a good tradeoff between generalization and specialization abilities, we employ an ensemble of deep convolutional neural network (DCN) classifiers and consider the task of scheduling (when and which classifiers to retrain), given a previous season’s DCN classifiers as the sole prior knowledge. We present a unified framework for retraining scheduling and we discuss practical implementation strategies. Furthermore, we address the task of partitioning a robot’s workspace into places to define place classes in an unsupervised manner, as opposed to using uniform partitioning, so as to maximize VPC performance. Experiments using the publicly available NCLT dataset revealed that retraining scheduling of a DCN classifier ensemble is crucial in achieving a good balance between generalization and specialization. Additionally, it was found that the performance is significantly improved when using planned scheduling.

Optimization of CNN through Novel Training Strategy for Visual Classification Problems.

The convolution neural network (CNN) has achieved state-of-the-art performance in many computer vision applications e.g., classification, recognition, detection, etc. However, the global optimization of CNN training is still a problem. Fast classification and training play a key role in the development of the CNN. We hypothesize that the smoother and optimized the training of a CNN goes, the more efficient the end result becomes. Therefore, in this paper, we implement a modified resilient backpropagation (MRPROP) algorithm to improve the convergence and efficiency of CNN training. Particularly, a tolerant band is introduced to avoid network overtraining, which is incorporated with the global best concept for weight updating criteria to allow the training algorithm of the CNN to optimize its weights more swiftly and precisely. For comparison, we present and analyze four different training algorithms for CNN along with MRPROP, i.e., resilient backpropagation (RPROP), Levenberg–Marquardt (LM), conjugate gradient (CG), and gradient descent with momentum (GDM). Experimental results showcase the merit of the proposed approach on a public face and skin dataset.

Robust multi-source adaptation visual classification using supervised low-rank representation

How to guarantee the robustness of multi-source adaptation visual classification is an important challenge in current visual learning community. To this end, we address in this paper the problem of robust visual classification with few labeled samples from the target domain of interest by leveraging multiple prior source models. Motivated by the recent success of low rank representation, we formulate this problem as a robust multi-source adaptation visual classification (RMAVC) model with supervised low rank representation by combining the strength of discriminative information from the target domain and the prior models from multiple source domains. Specifically, we propose a joint supervised low rank representation and multi-source adaptation visual classification framework, which achieves dual goals of finding the most discriminative low rank representation and multi-source adaptation classifier parameters for the target domain. While it is showed in this paper that the proposed RMAVC framework is effective and can produce high accuracy on several tasks of multi-source adaptation visual classification, this framework fails to consider the local geometrical structure of the target data and the heterogeneousness among multiple source domains. Hence, under this framework, we further present two effective extensions or variants, i.e., RMAVCK and RMAVC_FM, by exploiting multiple kernel trick and flexible manifold regularization, respectively. The proposed framework and its variants are robust for classifying visual objects accurately and the experimental results demonstrate the effectiveness of our methods on several types of image and video datasets.

Multi-source adaptation joint kernel sparse representation for visual classification

Most of the existing domain adaptation learning (DAL) methods relies on a single source domain to learn a classifier with well-generalized performance for the target domain of interest, which may lead to the so-called negative transfer problem. To this end, many multi-source adaptation methods have been proposed. While the advantages of using multi-source domains of information for establishing an adaptation model have been widely recognized, how to boost the robustness of the computational model for multi-source adaptation learning has only recently received attention. To address this issue for achieving enhanced performance, we propose in this paper a novel algorithm called multi-source Adaptation Regularization Joint Kernel Sparse Representation (ARJKSR) for robust visual classification problems. Specifically, ARJKSR jointly represents target dataset by a sparse linear combination of training data of each source domain in some optimal Reproduced Kernel Hilbert Space (RKHS), recovered by simultaneously minimizing the inter-domain distribution discrepancy and maximizing the local consistency, whilst constraining the observations from both target and source domains to share their sparse representations. The optimization problem of ARJKSR can be solved using an efficient alternative direction method. Under the framework ARJKSR, we further learn a robust label prediction matrix for the unlabeled instances of target domain based on the classical graph-based semi-supervised learning (GSSL) diagram, into which multiple Laplacian graphs constructed with the ARJKSR are incorporated. The validity of our method is examined by several visual classification problems. Results demonstrate the superiority of our method in comparison to several state-of-the-arts.

Where Are We After Five Editions?: Robot Vision Challenge, a Competition that Evaluates Solutions for the Visual Place Classification Problem

This article describes the Robot Vision challenge, a competition that evaluates solutions for the visual place classification problem. Since its origin, this challenge has been proposed as a common benchmark where worldwide proposals are measured using a common overall score. Each new edition of the competition introduced novelties, both for the type of input data and subobjectives of the challenge. All the techniques used by the participants have been gathered up and published to make it accessible for future developments. The legacy of the Robot Vision challenge includes data sets, benchmarking techniques, and a wide experience in the place classification research that is reflected in this article.