ImageNet Images Research Articles

Beyond transfer learning: Leveraging ancillary images in automated classification of plankton

AbstractWe assess whether a supervised machine learning algorithm, specifically a convolutional neural network (CNN), achieves higher accuracy on planktonic image classification when including non‐plankton and ancillary plankton during the training procedure. We focus on the case of optimizing the CNN for a single planktonic image source, while considering ancillary images to be plankton images from other instruments. We conducted two sets of experiments with three different types of plankton images (from a Zooglider, Underwater Vision Profiler 5, and Zooscan), and our results held across all three image types. First, we considered whether single‐stage transfer learning using non‐plankton images was beneficial. For this assessment, we used ImageNet images and the 2015 ImageNet contest‐winning model, ResNet‐152. We found increased accuracy using a ResNet‐152 model pretrained on ImageNet, provided the entire network was retrained rather than retraining only the fully connected layers. Next, we combined all three plankton image types into a single dataset with 3.3 million images (despite their differences in contrast, resolution, and pixel pitch) and conducted a multistage transfer learning assessment. We executed a transfer learning stage from ImageNet to the merged ancillary plankton dataset, then a second transfer learning stage from that merged plankton model to a single instrument dataset. We found that multistage transfer learning resulted in additional accuracy gains. These results should have generality for other image classification tasks.

Positional normalization-based mixed-image data augmentation and ensemble self-distillation algorithm

The prior approach to data augmentation using positional normalization necessitated manual determination of weight parameters for blending two images. However, theoretically, these parameters cannot cater to all samples. We propose a novel data augmentation approach called pnMix (Positional normalization-based data mixing) based on data blending and positional normalization to address these issues. Moreover, we have improved the teacher model of an existing network structure based on self-distillation technology using an ensembling approach called ESKD. Our method enhances performance across multiple datasets and tasks. We outperform previous methods in complex real-world datasets such as imagenet-a and imagenet-o, which we believe are highly meaningful for practical applications. Specifically, we obtain good results on the CIFAR100 and Imagenet image classification datasets, the Pascal VOC and COCO object detection datasets, and the COCO and Cityscapes image segmentation datasets. Furthermore, we analyze the pnMix method's advantages compared to other data augmentation methods through visualization techniques, demonstrating its effectiveness and interpretability. Our source code and pre-trained models are available at https://github.com/sydney72380/pnMix.

Deep Learning Transfer Methods for Biomedical Classification of Images

Research using computers has been carried out on the effectiveness of applying deep learning transfer methods to solve the problem of identifying human brain tumors using MRI imaging. Various deep learning and fine-tuning methodologies of models have been proposed and implemented. The deep convolution networks MobileNetV2, VGG-16, Xception and ResNet-50, trained on the ImageNet image set, were used as basic models. A deep convolutional neural network 2D-CNN has also been developed and trained. A computer study of the performance indicators revealed that the fine-tuning method was effective On an enlarged data set, the Xception model outperformed other deep learning models in terms of accuracy: the clarity with which brain tumors are classified using MRI images was 94%, precision 97.7%, recall 94.01%, f1 score 96%, AUC 96.90%.

Out-of-distribution detection with in-distribution voting using the medical example of chest x-ray classification.

Deep learning models are being applied to more and more use cases with astonishing success stories, but how do they perform in the real world? Models are typically tested on specific cleaned data sets, but when deployed in the real world, the model will encounter unexpected, out-of-distribution (OOD)data. To investigate the impact of OOD radiographs on existing chest x-ray classification models and to increase their robustness against OODdata. The study employed the commonly used chest x-ray classification model, CheXnet, trained on the chest x-ray 14 data set, and tested its robustness against OOD data using three public radiography data sets: IRMA, Bone Age, and MURA, and the ImageNet data set. To detect OOD data for multi-label classification, we proposed in-distribution voting (IDV). The OOD detection performance is measured across data sets using the area under the receiver operating characteristic curve (AUC) analysis and compared with Mahalanobis-based OOD detection, MaxLogit, MaxEnergy, self-supervised OOD detection (SS OOD), andCutMix. Without additional OOD detection, the chest x-ray classifier failed to discard any OOD images, with an AUC of 0.5. The proposed IDV approach trained on ID (chest x-ray 14) and OOD data (IRMA and ImageNet) achieved, on average, 0.999 OOD AUC across the three data sets, surpassing all other OOD detection methods. Mahalanobis-based OOD detection achieved an average OOD detection AUC of 0.982. IDV trained solely with a few thousand ImageNet images had an AUC 0.913, which was considerably higher than MaxLogit (0.726), MaxEnergy (0.724), SS OOD (0.476), and CutMix (0.376). The performance of all tested OOD detection methods did not translate well to radiography data sets, except Mahalanobis-based OOD detection and the proposed IDV method. Consequently, training solely on ID data led to incorrect classification of OOD images as ID, resulting in increased false positive rates. IDV substantially improved the model's ID classification performance, even when trained with data that will not occur in the intended use case or test set (ImageNet), without additional inference overhead or performance decrease in the target classification. The corresponding code is available at https://gitlab.lrz.de/IP/a-knee-cannot-have-lung-disease.

Adopting Two Supervisors for Efficient Use of Large-Scale Remote Deep Neural Networks

Recent decades have seen the rise of large-scale Deep Neural Networks (DNNs) to achieve human-competitive performance in a variety of artificial intelligence tasks. Often consisting of hundreds of millions, if not hundreds of billion parameters, these DNNs are too large to be deployed to, or efficiently run on resource-constrained devices such as mobile phones or IoT microcontrollers. Systems relying on large-scale DNNs thus have to call the corresponding model over the network, leading to substantial costs for hosting and running the large-scale remote model, costs which are often charged on a per-use basis. In this paper, we propose BiSupervised , a novel architecture, where, before relying on a large remote DNN, a system attempts to make a prediction on a small-scale local model. A DNN supervisor monitors said prediction process and identifies easy inputs for which the local prediction can be trusted. For these inputs, the remote model does not have to be invoked, thus saving costs, while only marginally impacting the overall system accuracy. Our architecture furthermore foresees a second supervisor to monitor the remote predictions and identify inputs for which not even these can be trusted, allowing to raise an exception or run a fallback strategy instead. We evaluate the cost savings, and the ability to detect incorrectly predicted inputs on four diverse case studies: IMDB movie review sentiment classification, Github issue triaging, Imagenet image classification, and SQuADv2 free-text question answering. In all four case studies, we find that BiSupervised allows to reduce cost by at least 30% while maintaining a similar system-level prediction performance. In two case studies (IMDB and SQuADv2), we find that BiSupervised even achieves a higher system-level accuracy, at reduced cost, compared to a remote-only model. Furthermore, measurements taken on our setup indicate a large potential of BiSupervised to reduce average prediction latency.

Classification of Appearance Quality of Red Grape Based on Transfer Learning of Convolution Neural Network

Grapes are a globally popular fruit, with grape cultivation worldwide being second only to citrus. This article focuses on the low efficiency and accuracy of traditional manual grading of red grape external appearance and proposes a small-sample red grape external appearance grading model based on transfer learning with convolutional neural networks (CNNs). In the naturally growing vineyards, 195,120,135 samples of Grade I, Grade II, and Grade III red grapes were collected using a Canon EOS 550D camera, and a data set of 1800 samples was obtained using data enhancement technology. Then, the CNN transfer learning method was used to transfer the pre-trained AlexNet, VGG16, GoogleNet, InceptionV3, and ResNet50 network models on the ImageNet image dataset to the red grape image grading task. By comparing the classification performance of the CNN models of these five different network depths with fine-tuning, ResNet50 with a learning rate of 0.001 and a loop number of 10 was determined to be the best feature extractor for red grape images. Moreover, given the small number of red grape image samples in this study, different convolutional layer features output by the ResNet50 feature extractor were analyzed layer by layer to determine the effect of deep features extracted by each convolutional layer on Support Vector Machine (SVM) classification performance. This analysis helped to obtain a ResNet50 + SVM red grape external appearance grading model based on the optimal ResNet50 feature extraction strategy. Experimental data showed that the classification model constructed using the feature parameters extracted from the 10th node of the ResNet50 network achieved an accuracy rate of 95.08% for red grape grading. These research results provide a reference for the online grading of red grape clusters based on external appearance quality and have certain guiding significance for the quality and efficiency of grape industry circulation and production.

Few-shot and meta-learning methods for image understanding: a survey

State-of-the-art deep learning systems (e.g., ImageNet image classification) typically require very large training sets to achieve high accuracies. Therefore, one of the grand challenges is called few-shot learning where only a few training samples are required for good performance. In this survey, we illuminate one of the key paradigms in few-shot learning called meta-learning. These meta-learning methods, by simulating the tasks which will be presented at inference through episodic training, can effectively employ previous prior knowledge to guide the learning of new tasks. In this paper, we provide a comprehensive overview and key insights into the meta-learning approaches and categorize them into three branches according to their technical characteristics, namely metric-based, model-based and optimization-based meta-learning. Due to the major importance of the evaluation process, we also present an overview of current widely used benchmarks, as well as performances of recent meta-learning methods on these datasets. Based on over 200 papers in this survey, we conclude with the major challenges and future directions of few-shot learning and meta-learning.

О сегментации опухолей головного мозга по МРТ-изображениям с применением методов глубокого обучения

Segmentation of a brain tumor is one of the most difficult tasks in the analysis of medical images. The purpose of brain tumor segmentation is to create an accurate outline of brain tumor areas. Gliomas are the most common type of brain tumors. Diagnosis of patients with this disease is based on the analysis of the results of magnetic resonance imaging and segmentation of the tumor boundaries manually. However, due to the time-consuming nature of the manual segmentation process and errors, there is a need for a fast and reliable automatic segmentation algorithm. In recent years, deep learning methods have shown promising effectiveness in solving various computer vision problems, such as image classification, object detection and semantic segmentation. A number of methods based on deep learning have been applied to segmentation of brain tumors, and promising results have been achieved. The article proposes a hybrid method for solving the problem of segmentation of brain tumors based on its MRI images based on the U-Net architecture, the encoder of which uses a model of a deep convolutional neural network pre-trained on a set of ImageNet images. Among such models were used VGG16, VGG19, MobileNetV2, Inception, ResNet50, EfficientNetb7, InceptionResnetV2, DenseNet201, DenseNet121. Based on the hybrid method, the TL-U-Net model was implemented, and numerical experiments were carried out to train it with different encoder models for segmentation of brain tumors based on its MRI images. Computer experiments on a set of MRI images of the brain showed the effectiveness of the proposed approach, the best encoder model turned out to be the neural network Densenet121, which provided indicators of segmentation accuracy MeanIoU=90.34%, MeanDice=94.33%, accuracy=94.17%. The obtained estimates of segmentation accuracy are comparable or exceed similar estimates obtained by other researchers.

A deep learning framework based on improved self‐supervised learning for ground‐penetrating radar tunnel lining inspection

AbstractIt is not practical to obtain a large number of labeled data to train a supervised learning network in tunnel lining nondestructive testing with ground‐penetrating radar (GPR). To decrease the dependence of supervised learning on the number of labeled data, an improved self‐supervised learning algorithm—self‐attention dense contrastive learning (SA‐DenseCL)—is proposed and incorporated with a mask region‐convolution neural network (Mask R‐CNN), which is trained by unlabeled and labeled GPR data. The proposed SA‐DenseCL adds a self‐attention‐based relevant projection head to the DenseCL architecture of self‐supervised learning, capturing the spatially continuing information between adjacent GPR traces. In the workflow, some unlabeled GPR images are used to pre‐train the SA‐DenseCL network for feature extraction and obtaining the backbone weights, which is superior to the conventional pre‐training methods of supervised learning pre‐trained by ImageNet images. The weights of the pre‐trained backbone are then used to initialize the Mask R‐CNN through transfer learning. Subsequently, a limited number of labeled GPR images are used to fine‐tune the Mask R‐CNN for automatically identifying the locations of the reinforcement bars and voids and estimating the secondary lining thickness. The experimental results show that the average precision reaches 96.70%, 81.04%, and 94.67% in identifying reinforcement bar locations, detecting void defects, and estimating secondary lining thickness, respectively, which outperform the conventional methods that use ImageNet‐based supervised learning or GPR image‐based DenseCL for initializing the Mask R‐CNN backbone weights. It is observed that the improved self‐supervised learning‐based framework can improve the detection and estimation accuracy in GPR tunnel lining inspection.

SOYBEAN PEST IDENTIFICATION BASED ON CONVOLUTIONAL NEURAL NETWORK AND TRANSFER LEARNING

Despite the fact that the ensemble classifier improved classification precision by integrating texture and colour. However, the significant image preparation process is a laborious and time-consuming. Manually depicting constrained feature extraction can result in a semantic void in a picture. These limits cause inaccuracy of agricultural disease identification. Thus, this study proposes a soybean pest detection method based on a hybrid of Transfer learning and pyramidal convolutional neural networks that can identify soybean pests quickly and accurately on small sample sets. Bean borer, soybean poison moth, mite, stink bug, and pod borer photographs are first pre-processed using standard data improvement methods, and then manually categorized into six groups based on pest characteristics. The weight parameters from the VGG16 model trained on the ImageNet image dataset were then transferred to the recognition of soybean pests using the transfer learning method, and the VGG16 convolutional and pooling layers were used as feature extraction layers, while the top layer was redesigned as a pyramidal convolutional layer, an average pooling layer, and a SoftMax output layer, with some of the convolutional layers frozen during training. According to the testing statistics, the model's average test accuracy is 98.23%, and the model size is only 95.4 M. For bean borer, soybean poison moth, mite, skewed night moth, stink bug, and bean pod moth, the model's recognition accuracy is 96.4, 97.78, 98.12, 98.4, 99.56, and 99.16, respectively. The results of the experiments show that the method has a high identification efficiency and a good recognition effect.

Adaptive Edge Offloading for Image Classification Under Rate Limit

This article considers a setting where embedded devices are used to acquire and classify images. Because of limited computing capacity, embedded devices rely on a parsimonious classification model with uneven accuracy. When local classification is deemed inaccurate, devices can decide to offload the image to an edge server with a more accurate but resource-intensive model. Resource constraints, e.g., network bandwidth, however, require regulating such transmissions to avoid congestion and high latency. This article investigates this offloading problem when transmissions regulation is through a token bucket, a mechanism commonly used for such purposes. The goal is to devise a lightweight, online offloading policy that optimizes an application-specific metric (e.g., classification accuracy) under the constraints of the token bucket. This article develops a policy based on a deep <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$Q$ </tex-math></inline-formula> -network (DQN), and demonstrates both its efficacy and the feasibility of its deployment on embedded devices. Of note is the fact that the policy can handle complex input patterns, including correlation in image arrivals and classification accuracy. The evaluation is carried out by performing image classification over a local testbed using synthetic traces generated from the ImageNet image classification benchmark. Implementation of this work is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/qiujiaming315/edgeml-dqn</uri> .

Understanding transformation tolerant visual object representations in the human brain and convolutional neural networks

Forming transformation-tolerant object representations is critical to high-level primate vision. Despite its significance, many details of tolerance in the human brain remain unknown. Likewise, despite the ability of convolutional neural networks (CNNs) to exhibit human-like object categorization performance, whether CNNs form tolerance similar to that of the human brain is unknown. Here we provide the first comprehensive documentation and comparison of three tolerance measures in the human brain and CNNs. We measured fMRI responses from human ventral visual areas to real-world objects across both Euclidean and non-Euclidean feature changes. In single fMRI voxels in higher visual areas, we observed robust object response rank-order preservation across feature changes. This is indicative of functional smoothness in tolerance at the fMRI meso-scale level that has never been reported before. At the voxel population level, we found highly consistent object representational structure across feature changes towards the end of ventral processing. Rank-order preservation, consistency, and a third tolerance measure, cross-decoding success (i.e., a linear classifier's ability to generalize performance across feature changes) showed an overall tight coupling. These tolerance measures were in general lower for Euclidean than non-Euclidean feature changes in lower visual areas, but increased over the course of ventral processing for all feature changes. These characteristics of tolerance, however, were absent in eight CNNs pretrained with ImageNet images with varying network architecture, depth, the presence/absence of recurrent processing, or whether a network was pretrained with the original or stylized ImageNet images that encouraged shape processing. CNNs do not appear to develop the same kind of tolerance as the human brain over the course of visual processing.

Recognition of esophagitis in endoscopic images using transfer learning.

BACKGROUNDEsophagitis is an inflammatory and damaging process of the esophageal mucosa, which is confirmed by endoscopic visualization and may, in extreme cases, result in stenosis, fistulization and esophageal perforation. The use of deep learning (a field of artificial intelligence) techniques can be considered to determine the presence of esophageal lesions compatible with esophagitis.AIMTo develop, using transfer learning, a deep neural network model to recognize the presence of esophagitis in endoscopic images.METHODSEndoscopic images of 1932 patients with a diagnosis of esophagitis and 1663 patients without any pathological diagnosis provenient from the KSAVIR and HyperKSAVIR datasets were splitted in training (80%) and test (20%) and used to develop and evaluate a binary deep learning classifier built using the DenseNet-201 architecture, a densely connected convolutional network, with weights pretrained on the ImageNet image set and fine-tuned during training. The classifier model performance was evaluated in the test set according to accuracy, sensitivity, specificity and area under the receiver operating characteristic curve (AUC).RESULTSThe model was trained using Adam optimizer with a learning rate of 0.0001 and applying binary cross entropy loss function. In the test set (n = 719), the classifier achieved 93.32% accuracy, 93.18% sensitivity, 93.46% specificity and a 0.96 AUC. Heatmaps for spatial predictive relevance in esophagitis endoscopic images from the test set were also plotted. In face of the obtained results, the use of dense convolutional neural networks with pretrained and fine-tuned weights proves to be a good strategy for predictive modeling for esophagitis recognition in endoscopic images. In addition, adopting the classification approach combined with the subsequent plotting of heat maps associated with the classificatory decision gives greater explainability to the model.CONCLUSIONIt is opportune to raise new studies involving transfer learning for the analysis of endoscopic images, aiming to improve, validate and disseminate its use for clinical practice.

Texture Image Recognition Based on Deep Convolutional Neural Network and Transfer Learning

The traditional texture image recognition methods have a complex design process, and the existing methods based on deep learning can’t effectively solve the problem of insufficient texture image samples which lead to unsatisfying recognition accuracy. To solve the above problems, a texture image recognition method based on deep convolutional neural network and transfer learning is proposed. Firstly, a new transfer learning model is constructed by using the deep learning model pretrained on the large-scale ImageNet image dataset. Secondly, the reasonable model super-parameters are set, and the weighted sum of the training loss, the validation loss, and the deep feature distance between the training set and the validation set is taken as the cost function of training process. Finally, the best transfer learning model is determined by layer-by-layer training and validation. The experimental results show that the proposed method achieves 99.76%, 99.87%, 99.80%, 100.00% and 94.01% recognition accuracies on the CUReT, KTH-TIPS, UIUC, UMD and NewBarkTex texture datasets respectively, and has good robustness and recognition ability.

IDF21-0299 An automatic deep learning-based system for screening and management of DME

Background: Diabetic macular oedema (DME) is the most common complication form of sight-threatening retinopathy in people with a diabetic condition. Previous studies demonstrate specifically that poor glycaemic and blood pressure control are associated with the presence and development of DME condition. The aim of this study is to validate the implementation of deep learning systems as a tool into routinely practiced diabetic retinopathy screening to improve diagnosis and management of DME. Method: 62,383 color images of the eye fundus were used. These images were acquired between 2016 and 2020 with a non mydriatic Centervue DRS camera. Images were graded as DME and no DME by retinal specialists according to the classification of American Association of Ophthalmology. The dataset was randomly divided into train-validation and test sets in proportions of 89.5% and 10.5%, respectively. We trained 4 CNNs based on the ResNeXt architecture with standardised convolutions as in Kolesnikov et al (ECCV 2020). Model parameters were initialized from a solution to the ImageNet image classification task and further optimized by minimizing the cross-entropy loss in its Sharpness-Aware extension, which attempts to find a parameter configuration lying in neighborhoods with uniformly low loss values, leading to improved generalization ability. The training was carried out by error back-propagation with weight updates given by the Stochastic Gradient Descent algorithm, and a learning rate of 0.01 that was cyclically annealed towards 0. The optimization ran for 20 cycles but was stopped as soon as the AUC computed on the validation set did not improve for two cycles. Final predictions were computed from the average ensemble of the four models. Results: The resulting model obtained a high performance on the test dataset. The area under the curve (AUC) was 0.9591(0.9536-0.9645). In addition, at an optimal cut-off given by the Youden index, the accuracy, the sensitivity and the specificity achieved were 0.9033(0.8959-0.9104), 0.8942 and 0.9054 respectively. An F1 score of 0.7759 and a Kappa score of 0.7156 indicate a high agreement of the model’s predictions with ophthalmologists’ opinions. Conclusion: The proposed system achieved high sensitivity and specificity in DME screening from retinal images of the eye fundus. The performance of the proposed approach is comparable to that of human experts and could be a valuable tool to help primary care triage.

Improvement of the model of object recognition in aero photographs using deep convolutional neural networks

Detection and recognition of objects in images is the main problem to be solved by computer vision systems. As part of solving this problem, the model of object recognition in aerial photographs taken from unmanned aerial vehicles has been improved. A study of object recognition in aerial photographs using deep convolutional neural networks has been carried out. Analysis of possible implementations showed that the AlexNet 2012 model (Canada) trained on the ImageNet image set (China) is most suitable for this problem solution. This model was used as a basic one. The object recognition error for this model with the use of the ImageNet test set of images amounted to 15 %. To solve the problem of improving the effectiveness of object recognition in aerial photographs for 10 classes of images, the final fully connected layer was modified by rejection from 1,000 to 10 neurons and additional two-stage training of the resulting model. Additional training was carried out with a set of images prepared from aerial photographs at stage 1 and with a set of VisDrone 2021 (China) images at stage 2. Optimal training parameters were selected: speed (step) (0.0001), number of epochs (100). As a result, a new model under the proposed name of AlexVisDrone was obtained. The effectiveness of the proposed model was checked with a test set of 100 images for each class (the total number of classes was 10). Accuracy and sensitivity were chosen as the main indicators of the model effectiveness. As a result, an increase in recognition accuracy from 7 % (for images from aerial photographs) to 9 % (for the VisDrone 2021 set) was obtained which has indicated that the choice of neural network architecture and training parameters was correct. The use of the proposed model makes it possible to automate the process of object recognition in aerial photographs. In the future, it is advisable to use this model at ground stations of unmanned aerial vehicle complex control when processing aerial photographs taken from unmanned aerial vehicles, in robotic systems, in video surveillance complexes and when designing unmanned vehicle systems

Compression Helps Deep Learning in Image Classification.

The impact of JPEG compression on deep learning (DL) in image classification is revisited. Given an underlying deep neural network (DNN) pre-trained with pristine ImageNet images, it is demonstrated that, if, for any original image, one can select, among its many JPEG compressed versions including its original version, a suitable version as an input to the underlying DNN, then the classification accuracy of the underlying DNN can be improved significantly while the size in bits of the selected input is, on average, reduced dramatically in comparison with the original image. This is in contrast to the conventional understanding that JPEG compression generally degrades the classification accuracy of DL. Specifically, for each original image, consider its 10 JPEG compressed versions with their quality factor (QF) values from . Under the assumption that the ground truth label of the original image is known at the time of selecting an input, but unknown to the underlying DNN, we present a selector called Highest Rank Selector (HRS). It is shown that HRS is optimal in the sense of achieving the highest Top k accuracy on any set of images for any k among all possible selectors. When the underlying DNN is Inception V3 or ResNet-50 V2, HRS improves, on average, the Top 1 classification accuracy and Top 5 classification accuracy on the whole ImageNet validation dataset by 5.6% and 1.9%, respectively, while reducing the input size in bits dramatically—the compression ratio (CR) between the size of the original images and the size of the selected input images by HRS is 8 for the whole ImageNet validation dataset. When the ground truth label of the original image is unknown at the time of selection, we further propose a new convolutional neural network (CNN) topology which is based on the underlying DNN and takes the original image and its 10 JPEG compressed versions as 11 parallel inputs. It is demonstrated that the proposed new CNN topology, even when partially trained, can consistently improve the Top 1 accuracy of Inception V3 and ResNet-50 V2 by approximately 0.4% and the Top 5 accuracy of Inception V3 and ResNet-50 V2 by 0.32% and 0.2%, respectively. Other selectors without the knowledge of the ground truth label of the original image are also presented. They maintain the Top 1 accuracy, the Top 5 accuracy, or the Top 1 and Top 5 accuracy of the underlying DNN, while achieving CRs of 8.8, 3.3, and 3.1, respectively.

Using convolutional neural networks for tick image recognition - a preliminary exploration.

Smartphone cameras and digital devices are increasingly used in the capture of tick images by the public as citizen scientists, and rapid advances in deep learning and computer vision has enabled brand new image recognition models to be trained. However, there is currently no web-based or mobile application that supports automated classification of tick images. The purpose of this study was to compare the accuracy of a deep learning model pre-trained with millions of annotated images in Imagenet, against a shallow custom-build convolutional neural network (CNN) model for the classification of common hard ticks present in anthropic areas from northeastern USA. We created a dataset of approximately 2000 images of four tick species (Ixodes scapularis, Dermacentor variabilis, Amblyomma americanum and Haemaphysalis sp.), two sexes (male, female) and two life stages (adult, nymph). We used these tick images to train two separate CNN models - ResNet-50 and a simple shallow custom-built. We evaluated our models' performance on an independent subset of tick images not seen during training. Compared to the ResNet-50 model, the small shallow custom-built model had higher training (99.7%) and validation (99.1%) accuracies. When tested with new tick image data, the shallow custom-built model yielded higher mean prediction accuracy (80%), greater confidence of true detection (88.7%) and lower mean response time (3.64s). These results demonstrate that, with limited data size for model training, a simple shallow custom-built CNN model has great prospects for use in the classification of common hard ticks present in anthropic areas from northeastern USA.

Submodular Span, with Applications to Conditional Data Summarization

As an extension to the matroid span problem, we propose the submodular span problem that involves finding a large set of elements with small gain relative to a given query set. We then propose a two-stage Submodular Span Summarization (S3) framework to achieve a form of conditional or query-focused data summarization. The first stage encourages the summary to be relevant to a given query set, and the second stage encourages the final summary to be diverse, thus achieving two important necessities for a good query-focused summary. Unlike previous methods, our framework uses only a single submodular function defined over both data and query. We analyze theoretical properties in the context of both matroids and polymatroids that elucidate when our methods should work well. We find that a scalable approximation algorithm to the polymatroid submodular span problem has good theoretical and empirical properties. We provide empirical and qualitative results on three real-world tasks: conditional multi-document summarization on the DUC 2005-2007 datasets, conditional video summarization on the UT-Egocentric dataset, and conditional image corpus summarization on the ImageNet dataset. We use deep neural networks, specifically a BERT model for text, AlexNet for video frames, and Bi-directional Generative Adversarial Networks (BiGAN) for ImageNet images to help instantiate the submodular functions. The result is a minimally supervised form of conditional summarization that matches or improves over the previous state-of-the-art.

Classification of jujube defects in small data sets based on transfer learning

Although convolutional neural networks have achieved success in the field of image classification, there are still challenges in the field of agricultural product quality sorting such as machine vision-based jujube defects detection. The performance of jujube defect detection mainly depends on the feature extraction and the classifier used. Due to the diversity of the jujube materials and the variability of the testing environment, the traditional method of manually extracting the features often fails to meet the requirements of practical application. In this paper, a jujube sorting model in small data sets based on convolutional neural network and transfer learning is proposed to meet the actual demand of jujube defects detection. Firstly, the original images collected from the actual jujube sorting production line were pre-processed, and the data were augmented to establish a data set of five categories of jujube defects. The original CNN model is then improved by embedding the SE module and using the triplet loss function and the center loss function to replace the softmax loss function. Finally, the depth pre-training model on the ImageNet image data set was used to conduct training on the jujube defects data set, so that the parameters of the pre-training model could fit the parameter distribution of the jujube defects image, and the parameter distribution was transferred to the jujube defects data set to complete the transfer of the model and realize the detection and classification of the jujube defects. The classification results are visualized by heatmap through the analysis of classification accuracy and confusion matrix compared with the comparison models. The experimental results show that the SE-ResNet50-CL model optimizes the fine-grained classification problem of jujube defect recognition, and the test accuracy reaches 94.15%. The model has good stability and high recognition accuracy in complex environments.