Image Recognition Performance Research Articles

Parameterization before Meta-Analysis: Cross-Modal Embedding Clustering for Forest Ecology Question-Answering

In the field of forestry ecology, image data capture factual information, while literature is rich with expert knowledge. The corpus within the literature can provide expert-level annotations for images, and the visual information within images naturally serves as a clustering center for the textual corpus. However, both image data and literature represent large and rapidly growing, unstructured datasets of heterogeneous modalities. To address this challenge, we propose cross-modal embedding clustering, a method that parameterizes these datasets using a deep learning model with relatively few annotated samples. This approach offers a means to retrieve relevant factual information and expert knowledge from the database of images and literature through a question-answering mechanism. Specifically, we align images and literature across modalities using a pair of encoders, followed by cross-modal information fusion, and feed these data into an autoregressive generative language model for question-answering with user feedback. Experiments demonstrate that this cross-modal clustering method enhances the performance of image recognition, cross-modal retrieval, and cross-modal question-answering models. Our method achieves superior performance on standardized tasks in public datasets for image recognition, cross-modal retrieval, and cross-modal question-answering, notably achieving a 21.94% improvement in performance on the cross-modal question-answering task of the ScienceQA dataset, thereby validating the efficacy of our approach. Essentially, our method targets cross-modal information fusion, combining perspectives from multiple tasks and utilizing cross-modal representation clustering of images and text. This approach effectively addresses the interdisciplinary complexity of forestry ecology literature and the parameterization of unstructured heterogeneous data encapsulating species diversity in conservation images. Building on this foundation, intelligent methods are employed to leverage large-scale data, providing an intelligent research assistant tool for conducting forestry ecological studies on larger temporal and spatial scales.

The effect of sports expertise on the performance of orienteering athletes’ real scene image recognition and their visual search characteristics

As a sport conducted in dynamically changing natural environments, orienteering places high demands on athletes' cognitive processing abilities and visual search efficiency. However, previous studies on orienteering have been primarily limited by the use of fixed stimulus materials on computer screens, which are unable to fully simulate authentic sports scenarios. To better understand the sports expertise of orienteering athletes in terms of their real scene image recognition performance and visual search characteristics, this study recruited 40 orienteering athletes, both experts and novices, as participants. By utilizing eye-tracking technology and setting observation points in real-world scenarios to conduct image recognition task tests, the ecological validity of the experiment was further enhanced. The results showed that the experts demonstrated a high level of accuracy and a short response time, with visual search characteristics including few saccade counts, low fixation frequency, concentrated fixation points, simple and clear fixation paths, and higher visual search efficiency. This study further reveals that long-term specialized training will lead to the formation of a unique cognitive structure related to the specific knowledge and long-term memory required by expert orienteering athletes, thereby promoting the development of expert advantage.

A two-stream decision fusion network for cervical pap-smear image classification tasks

Deep learning, especially Convolution Neural Networks (CNNs), has demonstrated superior performance in image recognition and classification tasks. They make complex pattern recognition possible by extracting image features through layers of abstraction. However, despite the excellent performance of deep learning in general image classification, its limitations are becoming apparent in specific domains such as cervical cell medical image classification. This is because although the morphology of cervical cells varies between normal, diseased and cancerous, these differences are sometimes very small and difficult to capture. To solve this problem, we propose a two-stream feature fusion model comprising a manual feature branch, a deep feature branch, and a decision fusion module. Specifically, We process cervical cells through a modified DarkNet backbone network to extract deep features. In order to enhance the learning of deep features, we have devised scale convolution blocks to substitute the original convolution, termed Basic convolution blocks. The manual feature branch comprises a range of traditional features and is linked to a multilayer perceptron. Additionally, we design three decision feature channels trained from both manual and deep features to enhance the model performance in cervical cell classification. Our proposed model demonstrates superior performance when compared to state-of-the-art cervical cell classification models. We establish a 15-category 148762 cervical cytopathology image dataset (CCID). In addition, we additionally conducted experiments on the SIPaKMeD dataset. Numerous experiments show that our proposed model performs excellently compared to state-of-the-art classification models. The outcomes illustrate that our approach can significantly aid pathologists in accurately evaluating cervical smears.

A Study of Multi-Pose Effects On a Face Recognition System

Interpersonal and intrapersonal face variation interference caused by multiple poses is challenging for distance-based face recognition systems. In this paper, we investigate the face-feature distance distribution for Chinese multi-pose faces. The simulation shows that the number of individuals in the gallery database will greatly affect the recognition performance for near-profile face images. It also provides a prediction of the Top-N occurrence rates in different gallery-size environments.

Advances in Brain-Inspired Deep Neural Networks for Adversarial Defense

Deep convolutional neural networks (DCNNs) have achieved impressive performance in image recognition, object detection, etc. Nevertheless, they are susceptible to adversarial attacks and interferential noise. Adversarial attacks can mislead DCNN models by manipulating input data with small perturbations, causing security risks to intelligent system applications. Comparatively, these small perturbations have very limited perceptual impact on humans. Therefore, the research on brain-inspired adversarial robust models has gained increasing attention. Beginning from the adversarial attack concepts and schemes, we present a review of the conventional adversarial attack and defense methods and compare the advantages and differences between brain-inspired robust neural networks and the conventional adversarial defense methods. We further review the existing adversarial robust DCNN models, including methods inspired by the early visual systems and supervised by neural signals. Representative examples have validated the efficacy of brain-inspired methods for designing adversarial robust models, which may benefit the further research and development of brain-inspired robust deep convolutional neural networks and the intelligent system applications.

Artificial intelligence in hepatocellular carcinoma diagnosis: a comprehensive review of current literature.

Hepatocellular carcinoma (HCC) diagnosis mainly relies on its pathognomonic radiological profile, obviating the need for biopsy. The project of incorporating artificial intelligence (AI) techniques in HCC aims to improve the performance of image recognition. Herein, we thoroughly analyze and evaluate proposed AI models in the field of HCC diagnosis. A comprehensive review of the literature was performed utilizing MEDLINE/PubMed and Web of Science databases with the end of search date being the 30th of September 2023. The MESH terms "Artificial Intelligence," "Liver Cancer," "Hepatocellular Carcinoma," "Machine Learning," and "Deep Learning" were searched in the title and/or abstract. All references of the obtained articles were also evaluated for any additional information. Our search resulted in 183 studies meeting our inclusion criteria. Across all diagnostic modalities, reported area under the curve (AUC) of most developed models surpassed 0.900. A B-mode US and a contrast-enhanced US model achieved AUCs of 0.947 and 0.957, respectively. Regarding the more challenging task of HCC diagnosis, a 2021 deep learning model, trained with CT scans, classified hepatic malignant lesions with an AUC of 0.986. Finally, a MRI machine learning model developed in 2021 displayed an AUC of 0.975 when differentiating small HCCs from benign lesions, while another MRI-based model achieved HCC diagnosis with an AUC of 0.970. AI tools may lead to significant improvement in diagnostic management of HCC. Many models fared better or comparable to experienced radiologists while proving capable of elevating radiologists' accuracy, demonstrating promising results for AI implementation in HCC-related diagnostic tasks.

Intelligent image recognition using lightweight convolutional neural networks model in edge computing environment

In order to enhance the performance of intelligent image recognition, this study optimizes the image recognition model through lightweight convolutional neural networks (CNNs) and cloud computing technology. The study begins by introducing the relevant theories and models of edge computing (EC) and lightweight CNNs models. Next, this study focuses on optimizing traditional image recognition models. Finally, the effectiveness and reliability of the proposed model are experimentally validated. The experimental results indicate that, when recognizing 1000 images, the average recognition times per image on cloud servers and edge servers are 13.33 ms and 50.11 ms, respectively. Despite the faster speed of cloud servers, the performance of edge servers can be improved by stacking servers. When the number of edge servers reaches 4, their recognition speed surpasses that of the cloud server model. Additionally, comparing the latency and processing time between EC and cloud computing architectures, it is observed that, with an increase in the number of processed images, the average processing time per image in the EC architecture remains relatively stable and consistent. In contrast, the average processing time gradually increases in the cloud computing architecture. This indicates a significant impact of the number of images on the processing rate of the cloud computing architecture. Therefore, as the time gap in processing between cloud computing and EC increases, the advantages of the EC architecture become more apparent. This study’s significance lies in advancing the development of deep learning technology and providing possibilities for its widespread practical application. The contribution of this study lies in promoting the development of EC and lightweight neural network models, offering valuable references and guidance for practical applications in related fields.

HyperFace: A Deep Fusion Model for Hyperspectral Face Recognition.

Face recognition has been well studied under visible light and infrared (IR) in both intra-spectral and cross-spectral cases. However, how to fuse different light bands for face recognition, i.e., hyperspectral face recognition, is still an open research problem, which has the advantages of richer information retention and all-weather functionality over single-band face recognition. Thus, in this research, we revisit the hyperspectral recognition problem and provide a deep learning-based approach. A new fusion model (named HyperFace) is proposed to address this problem. The proposed model features a pre-fusion scheme, a Siamese encoder with bi-scope residual dense learning, a feedback-style decoder, and a recognition-oriented composite loss function. Experiments demonstrate that our method yields a much higher recognition rate than face recognition using only visible light or IR data. Moreover, our fusion model is shown to be superior to other general-purpose image fusion methods that are either traditional or deep learning-based, including state-of-the-art methods, in terms of both image quality and recognition performance.

Image clustering using generated text centroids

In recent years, deep neural networks pretrained on large-scale datasets have been used to address data deficiency and achieve better performance through prior knowledge. Contrastive language–image pretraining (CLIP), a vision-language model pretrained on an extensive dataset, achieves better performance in image recognition. In this study, we harness the power of multimodality in image clustering tasks, shifting from a single modality to a multimodal framework using the describability property of image encoder of the CLIP model. The importance of this shift lies in the ability of multimodality to provide richer feature representations. By generating text centroids corresponding to image features, we effectively create a common descriptive language for each cluster. It generates text centroids assigned by the image features and improves the clustering performance. The text centroids use the results generated by using the standard clustering algorithm as a pseudo-label and learn a common description of each cluster. Finally, only text centroids were added when the image features on the same space were assigned to the text centroids, but the clustering performance improved significantly compared to the standard clustering algorithm, especially on complex datasets. When the proposed method is applied, the normalized mutual information score rises by 32% on the Stanford40 dataset and 64% on ImageNet-Dog compared to the k-means clustering algorithm.

Multi-scale spatial pyramid attention mechanism for image recognition: An effective approach

Attention mechanisms have gradually become necessary to enhance the representational power of convolutional neural networks (CNNs). Despite recent progress in attention mechanism research, some open problems still exist. Most existing methods ignore modeling multi-scale feature representations, structural information, and long-range channel dependencies, which are essential for delivering more discriminative attention maps. This study proposes a novel, low-overhead, high-performance attention mechanism with strong generalization ability for various networks and datasets. This mechanism is called Multi-Scale Spatial Pyramid Attention (MSPA) and can be used to solve the limitations of other attention methods. For the critical components of MSPA, we not only develop the Hierarchical-Phantom Convolution (HPC) module, which can extract multi-scale spatial information at a more granular level utilizing hierarchical residual-like connections, but also design the Spatial Pyramid Recalibration (SPR) module, which can integrate structural regularization and structural information in an adaptive combination mechanism, while employing the Softmax operation to build long-range channel dependencies. The proposed MSPA is a powerful tool that can be conveniently embedded into various CNNs as a plug-and-play component. Correspondingly, using MSPA to replace the 3 × 3 convolution in the bottleneck residual blocks of ResNets, we created a series of simple and efficient backbones named MSPANet, which naturally inherit the advantages of MSPA. Without bells and whistles, our method substantially outperforms other state-of-the-art counterparts in all evaluation metrics based on extensive experimental results from CIFAR-100 and ImageNet-1K image recognition. When applying MSPA to ResNet-50, our model achieves top-1 classification accuracy of 81.74% and 78.40% on the CIFAR-100 and ImageNet-1K benchmarks, exceeding the corresponding baselines by 3.95% and 2.27%, respectively. We also obtained promising performance improvements of 1.15% and 0.91% compared to the competitive EPSANet-50. In addition, empirical research results in autonomous driving engineering applications also demonstrate that our method can significantly improve the accuracy and real-time performance of image recognition with cheaper overhead. Our code is publicly available at https://github.com/ndsclark/MSPANet.

Improving the Machine Learning Performance for Image Recognition Using a New Set of Mountain Fourier Moments

The orthogonal moments are giving relevant results of these last years within the framework of object detection, pattern recognition and image reconstruction. This article is based on orthogonal functions called "Orthogonal Mountain functions (OMFs)" and we introduce a new set of moments called the multichannel Mountain Fourier moments (MMFMs), their performance is in reconstruction, noise invariants, rotation, scale and translation for image color. To validate these proposed techniques, we made several experimental tests to analyse images. We compare the results obtained from invariant moments and other current orthogonal invariant moments; the experiments show the power of the proposed moments.

Simulated multimodal deep facial diagnosis

Facial phenotypes are extensively studied in medical and biological research, serving as critical markers that potentially indicate underlying genetic traits or medical conditions. With the recent advancements in big data, algorithms, and hardware, deep facial diagnosis, which employs deep learning techniques to systematically examine facial phenotypes and identify signs of certain diseases or medical conditions, has attracted significant attention and research, gradually emerging as a promising tool in precision medicine. Primarily limited by the scarcity of data for training facial diagnosis models, the accuracy of facial diagnosis for various conditions remains low up to now. In the past decade, RGB-D cameras, measuring depth information along with standard RGB capabilities, have proven superior in processing spatial details with more stability and accuracy. Motivated by the facts mentioned above, in this paper, we propose a Simulated Multimodal Framework, which effectively improves the computer-aided facial diagnosis performance of state-of-the-art models in experiments under different conditions. The underlying principle is to leverage the simulated depth by generative models to improve the performance of RGB image recognition. Furthermore, as a rapid and non-invasive tool for disease screening and detection, our proposal demonstrated an average accuracy improvement of over 20% compared to practicing physicians in the study.

Evasion Attacks on Deep Learning-Based Helicopter Recognition Systems

Identifying objects in surveillance and reconnaissance systems with the human eye can be challenging, underscoring the growing importance of employing deep learning models for the recognition of enemy weapon systems. These systems, leveraging deep neural networks known for their strong performance in image recognition and classification, are currently under extensive research. However, it is crucial to acknowledge that surveillance and reconnaissance systems utilizing deep neural networks are susceptible to vulnerabilities posed by adversarial examples. While prior adversarial example research has mainly utilized publicly available internet data, there has been a significant absence of studies concerning adversarial attacks on data and models specific to real military scenarios. In this paper, we introduce an adversarial example designed for a binary classifier tasked with recognizing helicopters. Our approach generates an adversarial example that is misclassified by the model, despite appearing unproblematic to the human eye. To conduct our experiments, we gathered real attack and transport helicopters and employed TensorFlow as the machine learning library of choice. Our experimental findings demonstrate that the average attack success rate of the proposed method is 81.9%. Additionally, when epsilon is 0.4, the attack success rate is 90.1%. Before epsilon reaches 0.4, the attack success rate increases rapidly, and then we can see that epsilon increases little by little thereafter.

Multi-tailed vision transformer for efficient inference

Recently, Vision Transformer (ViT) has achieved promising performance in image recognition and gradually serves as a powerful backbone in various vision tasks. To satisfy the sequential input of Transformer, the tail of ViT first splits each image into a sequence of visual tokens with a fixed length. Then, the following self-attention layers construct the global relationship between tokens to produce useful representation for the downstream tasks. Empirically, representing the image with more tokens leads to better performance, yet the quadratic computational complexity of self-attention layer to the number of tokens could seriously influence the efficiency of ViT’s inference. For computational reduction, a few pruning methods progressively prune uninformative tokens in the Transformer encoder, while leaving the number of tokens before the Transformer untouched. In fact, fewer tokens as the input for the Transformer encoder can directly reduce the following computational cost. In this spirit, we propose a Multi-Tailed Vision Transformer (MT-ViT) in the paper. MT-ViT adopts multiple tails to produce visual sequences of different lengths for the following Transformer encoder. A tail predictor is introduced to decide which tail is the most efficient for the image to produce accurate prediction. Both modules are optimized in an end-to-end fashion, with the Gumbel-Softmax trick. Experiments on ImageNet-1K demonstrate that MT-ViT can achieve a significant reduction on FLOPs with no degradation of the accuracy and outperform compared methods in both accuracy and FLOPs.

Dynamic Band‐Alignment Modulation in MoTe2/SnSe2 Heterostructure for High Performance Photodetector

AbstractFor two‐dimensional (2D) layered material heterojunctions, dynamic modulation of band alignments allows for the design of devices with flexible multi‐functional applications. In this paper, a device structure is presented based on a MoTe2/SnSe2 field‐effect transistor. By applying a bias voltage to the electrostatic gate, the gate voltage is adjusted from negative to positive, causing the heterojunction to transition from type‐III band alignment to type‐II band alignment. The working mechanism and device performance of the heterojunctions with different band alignments are investigated. The device exhibited outstanding detection range (from ultraviolet to infrared), detectivity (2.42 × 109 Jones), and speed (1.3 ms). Under a positive gate voltage, a higher ratio of light current to dark current (2×103) is achieved. To further demonstrate the potential of the high‐performance devices, their reliability is confirmed through their performance in image recognition using deep learning.

Improving crop image recognition performance using pseudolabels

In crop image recognition, when faced with a large quantity of unlabeled data, the traditional manual labeling method requires a large amount of human and material resources. To solve this problem, this study proposes an image recognition method based on a pseudolabeling technique. First, the data are divided into labeled and unlabeled data. The initial network model is trained on labeled data. Then, pseudolabeling of the unlabeled data is predicted, and only the data that satisfy the confidence threshold are regarded as valid pseudolabeling. To convert the unlabeled data into supervised training data, the two types of data are mixed. The training is terminated when the number of remaining unlabeled data satisfies the end condition and when the fivefold cross-validation method is used to evaluate model performance. Compared with the traditional semisupervised method, the experimental method is simpler and more applicable. Experiments were conducted on rice growth stage recognition and crop weed seedling recognition tasks. The results showed that the proposed method achieved 99.17% accuracy in rice growth stage recognition and a high AUC value of 99.93% in crop weed seedling recognition, which demonstrated excellent performance. Compared with the traditional model, this method not only improves in accuracy but also has better stability and wider applicability and is expected to provide an efficient, accurate and scalable solution for crop image recognition.

Sentiment analysis of art and design works using deep learning

Abstract Deep learning technology has the characteristics of automatic learning, nonlinear modeling, etc., which makes the machine algorithm better adapt to the data so as to better complete the task of sentiment analysis. The purpose of this paper is to explore the application of deep learning technology in the emotional analysis of art design works. After combining the emotion construction mechanism of art design works, based on the superior performance of convolutional neural network image recognition, it is introduced into the emotion analysis of art design works, combined with the attention mechanism model, to construct the emotion analysis model of art design works based on deep learning. Art design works are collected as the experimental dataset. The emotion analysis effect of this paper’s model for art design works is evaluated through the comparison of emotion classification indexes. The comparison of ROC curves of different models, and the emotion instance analysis of art design works is carried out. The experiments show that the experimental accuracy, precision, recall, and F1 value of this paper’s model range from 81% to 86%, and the F1 and ROC values are improved by 2.3% to 7.0% and 1.7% to 4.2% compared with the comparison model, which reflects the high accuracy and precision of this paper’s model for the sentiment analysis of art design works. Seven emotional patterns were found in tea art design works, and the model can be used to analyze the emotions of art design works.

Multiscale Feature Extraction by Using Convolutional Neural Network: Extraction of Objects from Multiresolution Images of Urban Areas

The emergence of deep learning-based classification methods has led to considerable advancements and remarkable performance in image recognition. This study introduces the Multiscale Feature Convolutional Neural Network (MSFCNN) for the extraction of complex urban land cover data, with a specific emphasis on buildings and roads. MSFCNN is employed to extract multiscale features from three distinct image types—Unmanned Aerial Vehicle (UAV) images, high-resolution satellite images (HR), and low-resolution satellite images (LR)—all collected within the Fengshan District of Kaohsiung, Taiwan. The model in this study demonstrated remarkable accuracy in classifying two key land cover categories. Its success in extracting multiscale features from different image resolutions. In the case of UAV images, MSFCNN achieved an accuracy rate of 91.67%, with a Producer’s Accuracy (PA) of 93.33% and a User’s Accuracy (UA) of 90.0%. Similarly, the model exhibited strong performance with HR images, yielding accuracy, PA, and UA values of 92.5%, 93.33%, and 91.67%, respectively. These results closely align with those obtained for LR imagery, which achieved respective accuracy rates of 93.33%, 95.0%, and 91.67%. Overall, the MSFCNN excels in the classification of both UAV and satellite images, showcasing its versatility and robustness across various data sources. The model is well suited for the task of updating cartographic data related to urban buildings and roads.

A Research Paper on Recent Lung Cancer Detection Techniques Using Deep Learning

Lung cellular breakdown is a devastating and elusive malignant development. It’s crucial to concentrate on quickly and accurately evaluating knobs. As a result, several methods have been created to identify lung cellular deterioration at an early stage. These research papers' distinct division calculations are coupled to a range of classifier approaches to perform image recognition to identify cellular breakdown in the lung knobs. The inception V3 model is an image recognition model that has been shown to attain greater accuracy on the Image Net dataset. The results of this analysis demonstrate that CT check images are more reliable for getting better results. , an With the proposed model with marker-controlled watershed division yields, accuracy of 90.38% and data validation of 95%.

FTG: Score-based black-box watermarking by fragile trigger generation for deep model integrity verification

Deep neural networks (DNNs) are widely used in real-world applications, thanks to their exceptional performance in image recognition. However, their vulnerability to attacks, such as Trojan and data poison, can compromise the integrity and stability of DNN applications. Therefore, it is crucial to verify the integrity of DNN models to ensure their security. Previous research on model watermarking for integrity detection has encountered the issue of overexposure of model parameters during embedding and extraction of the watermark. To address this problem, we propose a novel score-based black-box DNN fragile watermarking framework called fragile trigger generation (FTG). The FTG framework only requires the prediction probability distribution of the final output of the classifier during the watermarking process. It generates different fragile samples as the trigger, based on the classification prediction probability of the target classifier and a specified prediction probability mask to watermark it. Different prediction probability masks can promote the generation of fragile samples in corresponding distribution types. The whole watermarking process does not affect the performance of the target classifier. When verifying the watermarking information, the FTG only needs to compare the prediction results of the model on the samples with the previous label. As a result, the required model parameter information is reduced, and the FTG only needs a few samples to detect slight modifications in the model. Experimental results demonstrate the effectiveness of our proposed method and show its superiority over related work. The FTG framework provides a robust solution for verifying the integrity of DNN models, and its effectiveness in detecting slight modifications makes it a valuable tool for ensuring the security and stability of DNN applications.