Image Description Research Articles

Retracted: Attention Feature Network Extraction Combined with the Generation Algorithm of Multimedia Image Description

Retracted: Attention Feature Network Extraction Combined with the Generation Algorithm of Multimedia Image Description

The symbolic meaning of the sculpture "Faceless Buddha" in the work of Tsiu Tsitsing

History of development of Chinese jade carving art is dating back some 8000 years into Neolithic period The development of the ideas of Chinese jade carving, the change of their external forms, reflects the close relationships between reality (the objective world) and the inner world of man. The difference between contemporary and traditional jade carvings lies in the artist’s conscious desire to convey and disseminate his own worldview in society. The purpose of contemporary jade carving does not stop at the description of the traditional image and form by the artist, but also, through embodiment in form, tries to reveal an individual aesthetic perception. Jade carved works, possessing symbolic meaning, have a significant impact on the complex interaction between material and spiritual world of the artist; having a deep, symbolic meaning, these works affect the feelings and spirit of people as well. Tsiu Tsitsing (born in 1979) is the most famous and influential jade sculptor in contemporary China from the end of the 20th century and at the start of the 21st century, his jade articles have a distinct personal style, are original and modern. His works not only contain the thoughts and concepts of Buddhism (Zen-Buddhism), Taoism (Lao-Zhuang philosophy), and Confucianism (literary view) in traditional Chinese culture, but also integrate the concepts and formal language of Western modern art. As the most representative type of his (Tsiu Tsitsing) extensive oeuvre of sculptures, the series of jade carvings of " faceless Buddha" contain profound cultural information. The creative subject by inventing new system of symbolic language creates a new world of jade carving. At the same time, the creative subject re-examines the possibility of the current integration of Chinese traditional culture and Western modern art with the help of this symbol system.

A New Finding on Magnetic Resonance Imaging for Diagnosis of Hemifacial Spasm with High Accuracy and Interobserver Correlation.

Among patients with clinical hemifacial spasm (HFS), imaging exams aim to identify the neurovascular conflict (NVC) location. It has been proven that the identification in the preoperative exam increases the rate of surgical success. Despite the description of specific magnetic resonance image (MRI) acquisitions, the site of neurovascular compression is not always visualized. The authors describe a new MRI finding that helps in the diagnosis of HFS, and evaluate the sensitivity, specificity, and interobserver correlation of the described sign. A cross-sectional study including cases of hemifacial spasm treated surgically from 1 August 2011 to 31 July 2021 was performed. The MRIs of the cases were independently evaluated by two experienced neuroradiologists, who were blinded regarding the side of the symptom. The neuroradiologists were assigned to evaluate the MRIs in two separate moments. Primarily, they evaluated whether there was a neurovascular conflict based on the standard technique. Following this initial analysis, the neuroradiologists received a file with the description of the novel sign, named Prevedello Sign (PS). In a second moment, the same neuroradiologists were asked to identify the presence of the PS and, if it was present, to report on which side. A total of 35 patients were included, mostly females (65.7%) with a mean age of 59.02 (+0.48). Since the 35 cases were independently evaluated by two neuroradiologists, a total of 70 reports were included in the analysis. The PS was present in 66 patients (sensitivity of 94.2%, specificity of 91.4% and positive predictive value of 90.9%). When both analyses were performed in parallel (standard plus PS), the sensitivity increased to 99.2%. Based on the findings of this study, the authors conclude that PS is helpful in determining the neurovascular conflict location in patients with HFS. Its presence, combined with the standard evaluation, increases the sensitivity of the MRI to over 99%, without increasing risks of harm to patients or resulting in additional costs.

A Review of Transformer-Based Approaches for Image Captioning

Visual understanding is a research area that bridges the gap between computer vision and natural language processing. Image captioning is a visual understanding task in which natural language descriptions of images are automatically generated using vision-language models. The transformer architecture was initially developed in the context of natural language processing and quickly found application in the domain of computer vision. Its recent application to the task of image captioning has resulted in markedly improved performance. In this paper, we briefly look at the transformer architecture and its genesis in attention mechanisms. We more extensively review a number of transformer-based image captioning models, including those employing vision-language pre-training, which has resulted in several state-of-the-art models. We give a brief presentation of the commonly used datasets for image captioning and also carry out an analysis and comparison of the transformer-based captioning models. We conclude by giving some insights into challenges as well as future directions for research in this area.

Признаки прогрессирования заболевания сетчатки глаза, используемые для повышения валидности системы поддержки принятия врачебных решений на основе искусственного интеллекта

Purpose is to determine the most significant diagnostic signs of retinal disease progression, which are used to increase the validity of the medical decision support system (mdSS) for describing oct images using the method of an expert survey. Material and methods. An expert analysis of the binary signs used in mdSS when describing the oct image of the macular area was carried out: each sign was assessed in terms of the development of the pathological process, the need for telemedicine consultation and (or) referral of patients to a medical organization. The expert method included a survey of 15 ophthalmologists with the necessary qualifications. the experts were asked to rank on a ten-point scale 123 binary signs used by mdSS to form a conclusion about the need for referral to an ophthalmological department for specialized medical care. Results and discussion. The conducted expert survey showed a similar understanding of the signs obtained in the description of retina oct image, which can be used as predictors of the deterioration of the clinical course of the disease in patients with a previously diagnosed diagnosis. If there are signs identified as predictors of the pathological process, mdSS offers to go to the portal «telemedical system of remote consultations of the federal and regional levels» (https://fcmk.minzdrav.gov.ru/), upload the received documents to receive advice on patient routing. Conclusion. The established list of diagnostic features allowed to increase the validity of mdSS in terms of substantiating the conclusion about the need to refer the patient for a consultation with an ophthalmologist to resolve the issue of providing timely specialized medical care. this mdSS can be used as an information service in the work of specialists with higher medical education, a certificate of a medical specialist in the field of «ophthalmology», helps to structure data when uploading to an existing portal of telemedicine consultations. Keywords: retinal diseases, OCT, medical decision support system, signs of disease progression

Diffuse Leptomeningeal Glioneuronal Tumors: A Case Series of Five Patients with Parenchymal Forms and an Analysis of the Diagnostic Challenges, Treatment Options and Outcomes.

Diffuse leptomeningeal glioneuronal tumors (DL-GNT) are rare glioneuronal neoplasms with oligodendroglioma-like cells. These tumors can present as a dominant intracranial mass or as a solitary spinal cord mass without leptomeningeal involvement. In this study, we aimed to determine the magnetic resonance imaging and histopathological features, treatment modalities, and clinical outcomes of the parenchymal forms of DL-GNTs. This is a retrospective three-center case series study of 5 patients with a confirmed parenchymal form of DLGTs, out of which 4 patients were adults. Brain and spinal cord MR imaging were performed in all patients at either 1.5 or 3T. The patients' age ranged from 5 years to 50 years with a mean age of 27.6 years at presentation. Four of the tumors were located in the frontal lobe, and one in the tectum. They were usually solid-cystic enhancing tumors as the other mixed neuronal-glial tumors. All of the tumors had an extension to the superficial surface of a cerebral hemisphere. One had systemic bone metastases. The clinical signs and symptoms of the parenchymal form varied based on the location of the mass, in contrast to the leptomeningeal form associated with hydrocephalus. In one case, the tumor's initial grade was defined as intermediate. The initial histopathology of the two cases was low-grade and no upgrade occurred in the follow-up period. In two cases, although the tumors were low grade initially, they progressed to an anaplastic form in the follow-up period. The parenchymal form of DL-GNTs is common in adults. Extension to the superficial surface of a cerebral hemisphere is a distinctive imaging feature. Systemic osseous metastasis may occur. Due to the presence of common histopathological features, including the biphasic composition of glial and neuronal cell elements and oligodendroglioma-like cells, a proposed classification approach might be more beneficial for the histopathological and imaging description, and management of the glioneuronal tumors with oligodendroglioma-like features.

Generative Image Steganography based on Guidance Feature Distribution

Without modification, generative steganography is more secure than modification-based steganography. However, existing generative steganography methods still have the limitations, such as low embedding capacity and poor quality. To solve these issues, a synthesis-based generative steganographic model is proposed in this paper. In the image synthesis task, guidance features are utilized to synthesize images with specific styles and attributes. Due to the consistency of the guidance features before and after image synthesis, the features can be used as cover for steganography. The proposed model adopts the mean and standard deviation to quantify the distribution of guidance features, enabling the secret hiding within different trends of the feature distribution. By controlling the statistical dispersion of the embedded guidance features through the mean and standard deviation, the original feature distribution is preserved, and the synthesized image maintains good generation quality. The space of guidance features contains styles and attribute descriptions of various images, and offering a large space for information hiding. According to the experimental results, compared with existing steganography, the proposed steganographic model achieves better quality and hidden capacity with strong robustness.

'Eye-Tracking' with Words for Alzheimer's Disease Detection: Time Alignment of Words Enunciation with Image Regions During Image Description Tasks.

Alzheimer's disease (AD) is a progressive neurodegenerative disease that results in cognitive decline, dementia, and eventually death. Diagnosing early signs of AD can help clinicians to improve the quality of life. We developed a non-invasive approach to help neurologists and clinicians to distinguish probable AD patients and healthy controls (HC). The patients' gaze points were followed based on the words they used to describe the Cookie Theft (CT) picture description task. We hypothesized that the timing of words enunciation aligns with the participant's eye movements. The moments that each word was spoken were then aligned with specific regions of the image. We then applied machine learning algorithms to classify probable AD and HC. We randomly selected 60 participants (30 AD and 30 HC) from the Dementia Bank (Pitt Corpus). Five main classifiers were applied to different features extracted from the recorded audio and participants' transcripts (AD and HC). Support vector machine and logistic regression had the highest accuracy (up to 80% and 78.33%, respectively) in three different experiments. In conclusion, point-of-gaze can be applied as a non-invasive and less expensive approach compared to other available methods (e.g., eye tracker devices) for early-stage AD diagnosis.

Linguistic and Cultural Code in Blazing of National Heraldry

The article is devoted to the study of the linguocultural coding of national heraldry specifics. The purpose of the article is to establish the semantic and linguocultural characteristics of the concept “culture code” and to identify the functions of its units – symbols in verbal descriptions of national heraldry. The relevance of the study is determined by the need to understand the essence and lingocultural features of the verbalization of national heraldry symbols. The study was conducted on the material of blazoning of the coats of arms of the former USSR republics, the states of the former colonies of Great Britain, France, Spain and Portugal, which once had a common joint past and a common language of interethnic communication. The novelty of this study lies in the fact that for the first time blazonization is the object of a linguistic description. It is established that a linguocultural code differs from a simple code by the unconsciousness and unintentionality of its creation; it “inherited” the password property from the symbol: possession of this code opens access to a certain culture for its “users”. The linguocultural code is defined in the most general form as a system of images referring to the verbalized concepts of spiritual culture, it is always a set of symbols of a particular ethnic or national culture that can refer to different semantic content. The study showed that the in the creation of the coat of arms the verbal description of the heraldic image is primary – the blazon obtained as a result of blazonig, which consists in explaining the symbols present on the coats of arms. The author came to the conclusion that the figurative embodiment of the basic linguocultural concepts of the national idea in the heraldry of countries united by the common language of blazonization indicates the use of units of the most diverse cultural codes, distinguished by thematic feature. In the heraldic symbolization of power, sovereignty, unity of the nation, the struggle for independence and the positive personal qualities of the inhabitants of the country, units of subject, animalistic, plant, astronomical, “weaponry” and chromatic cultural codes are used. The study uses methods of semantic, semiotic and linguocultural analysis of images symbolizing the ideas of national heraldry.

A New Generative Model for Textual Descriptions of Medical Images Using Transformers Enhanced with Convolutional Neural Networks.

The automatic generation of descriptions for medical images has sparked increasing interest in the healthcare field due to its potential to assist professionals in the interpretation and analysis of clinical exams. This study explores the development and evaluation of a generalist generative model for medical images. Gaps were identified in the literature, such as the lack of studies that explore the performance of specific models for medical description generation and the need for objective evaluation of the quality of generated descriptions. Additionally, there is a lack of model generalization to different image modalities and medical conditions. To address these issues, a methodological strategy was adopted, combining natural language processing and features extraction from medical images and feeding them into a generative model based on neural networks. The goal was to achieve model generalization across various image modalities and medical conditions. The results showed promising outcomes in the generation of descriptions, with an accuracy of 0.7628 and a BLEU-1 score of 0.5387. However, the quality of the generated descriptions may still be limited, exhibiting semantic errors or lacking relevant details. These limitations could be attributed to the availability and representativeness of the data, as well as the techniques used.

A Comprehensive Analysis of Real-World Image Captioning and Scene Identification

Image captioning is a computer vision task that involves generating natural language descriptions for images. This method has numerous applications in various domains, including image retrieval systems, medicine, and various industries. However, while there has been significant research in image captioning, most studies have focused on high quality images or controlled environments, without exploring the challenges of real-world image captioning. Real-world image captioning involves complex and dynamic environments with numerous points of attention, with images which are often very poor in quality, making it a challenging task, even for humans. This paper evaluates the performance of various models that are built on top of different encoding mechanisms, language decoders and training procedures using a newly created real-world dataset that consists of over 800+ images of over 65 different scene classes, built using MIT Indoor scenes dataset. This dataset is captioned using the IC3 approach that generates more descriptive captions by summarizing the details that are covered by standard image captioning models from unique view-points of the image.

FDG-guided excision of peripheral non-palpable lymph nodes in the axilla with suspected malignancy

ObjectiveTo define the feasibility of 18F-fluorodeoxyglucose (FDG) uptake tracing in the excision of potentially malignant non-palpable lymph nodes (NLN), especially in the axilla. Materials and methodsFindings of 493 peripheral lymph node (LN) excisions were examined in the study. Three hundred seventy-eight LNs were palpable and the remaining 115 LNs were non-palpable. The excised NLNs were analyzed in two groups, NLNs excised based on the anatomical descriptions of imaging methods (n=62), image-guided NLN excisions (n=53). Among the image-guided excisions, 27 NLNs were excised using real-time ultrasound (US) guidance, and 26 NLNs were excised using FDG uptake tracing. ResultsImage-guided excisions provided better diagnostic accuracy for potentially malignant NLNs compared with excisions made located based on imaging descriptions (P=0.029). In these image-guided NLN excisions, the FDG uptake-trace method combined with positron emission tomography-computed tomography (PET-CT) was more successful in detecting malignant NLNs compared with US guidance (P=0.04). NLN excisions based on FDG uptake were completed in a shorter time than both excisions with US guidance (P=0.006) and excisions made locating based on imaging definitions (P=0.001). ConclusionPotentially malignant NLN excisions can be challenging for surgeons, and can result in prolonged surgical time. It may be extrapolated from our study that anatomy and metabolism-based LN excision guidance using information obtained in PET-CT and following FDG uptake tracing may improve the accuracy of histologic examinations, and thus increase the probability of achieving a correct diagnosis.

A performance analysis of transformer-based deep learning models for Arabic image captioning

Image captioning has become a fundamental operation that allows the automatic generation of text descriptions of images. However, most existing work focused on performing the image captioning task in English, and only a few proposals exist that address the image captioning task in Arabic. This paper focuses on understanding the factors that affect the performance of machine learning models performing Arabic image captioning (AIC). In particular, we focus on transformer-based models for AIC and study the impact of various text-preprocessing methods: CAMeL Tools, ArabertPreprocessor, and Stanza. Our study shows that using CAMeL Tools to preprocess text labels improves the AIC performance by up to 34–92% in the BLEU-4 score. In addition, we study the impact of image recognition models. Our results show that ResNet152 is better than EfficientNet-B0 and can improve BLEU scores performance by 9–11%. Furthermore, we investigate the impact of different datasets on the overall AIC performance and build an extended version of the Arabic Flickr8k dataset. Using the extended version improves the BLEU-4 score of the AIC model by up to 148%. Finally, utilizing our results, we build a model that significantly outperforms the state-of-the-art proposals in AIC by up to 196–379% in the BLUE-4 score.

The Curious Layperson: Fine-Grained Image Recognition Without Expert Labels

Most of us are not experts in specific fields, such as ornithology. Nonetheless, we do have general image and language understanding capabilities that we use to match what we see to expert resources. This allows us to expand our knowledge and perform novel tasks without ad-hoc external supervision. On the contrary, machines have a much harder time consulting expert-curated knowledge bases unless trained specifically with that knowledge in mind. Thus, in this paper we consider a new problem: fine-grained image recognition without expert annotations, which we address by leveraging the vast knowledge available in web encyclopedias. First, we learn a model to describe the visual appearance of objects using non-expert image descriptions. We then train a fine-grained textual similarity model that matches image descriptions with documents on a sentence-level basis. We evaluate the method on two datasets (CUB-200 and Oxford-102 Flowers) and compare with several strong baselines and the state of the art in cross-modal retrieval. Code is available at: https://github.com/subhc/clever.

Crowdsourcing Thumbnail Captions: Data Collection and Validation

Speech interfaces, such as personal assistants and screen readers, read image captions to users. Typically, however, only one caption is available per image, which may not be adequate for all situations (e.g., browsing large quantities of images). Long captions provide a deeper understanding of an image but require more time to listen to, whereas shorter captions may not allow for such thorough comprehension yet have the advantage of being faster to consume. We explore how to effectively collect both thumbnail captions—succinct image descriptions meant to be consumed quickly—and comprehensive captions—which allow individuals to understand visual content in greater detail. We consider text-based instructions and time-constrained methods to collect descriptions at these two levels of detail and find that a time-constrained method is the most effective for collecting thumbnail captions while preserving caption accuracy. Additionally, we verify that caption authors using this time-constrained method are still able to focus on the most important regions of an image by tracking their eye gaze. We evaluate our collected captions along human-rated axes—correctness, fluency, amount of detail, and mentions of important concepts—and discuss the potential for model-based metrics to perform large-scale automatic evaluations in the future.

RNNtcs: A test case selection method for Recurrent Neural Networks

Recurrent Neural Network (RNN) is a typical feedback neural network, which is particularly effective in processing time-series data tasks such as image description, text generation and classification, etc. However, its safety and quality issues have aroused strong concern in academia and industry. One of the most commonly used methods to ensure RNN-driven software quality is testing and optimizing RNN. Its core idea is to introduce human efforts to label test cases so that many labeled test cases can be used to trigger faults in RNNs. But this brings a huge overhead. Therefore, it is of great importance to study the test case selection method for RNN. Current research on test case selection for RNN has achieved some results from the stateful view of RNN, but there are still some problems, such as a low bug detection rate, poor inclusiveness, and lack of diversity.In this paper, according to the structural characteristics of RNN, we propose a method of RNN test case selection, RNNtcs, which combines clustering and uncertainty, in order to select test cases that can touch RNN bugs as much as possible, thus reducing the cost of labeling. In order to evaluate RNNtcs, we conduct experiments on popular image datasets, text datasets and RNN models. The experimental results show that RNNtcs is superior to the state-of-the-art approach DeepState in bug detection effectiveness, bug detection inclusiveness and bug detection diversity. The test cases selected by RNNtcs can effectively improve the robustness of the RNN model.

Key Issues Affecting the Inclusion of Alt Text in Scholarly PDF Publications

Abstract Alternative (alt) text descriptions for images in digital publications provide comparable information for people who cannot effectively see the visuals. They are relied upon by people who are blind or who have a moderate to severe visual impairment and who use assistive technologies. However, consistent provision of alt text is proving challenging for publishers because of the required changes in workflow, budgeting, and resourcing; in particular, scholarly publications contain high numbers of images requiring alt text. Most industry knowledge-sharing and studies on alt text in scholarly PDFs focus on large and medium-sized publishers and high JIF titles. To explore key issues affecting the consistent inclusion of alt text in publications produced by small and non-profit publishers this study uses the case of a UK-based independent research organization. Drawing on PDF analyses and a focus group discussion with publishing staff, the article makes recommendations for similar organizations seeking to publish alt text.

Generating Image Descriptions of Rice Diseases and Pests Based on DeiT Feature Encoder

We propose a DeiT (Data-Efficient Image Transformer) feature encoder-based algorithm for identifying disease types and generating relevant descriptions of diseased crops. It solves the scarcity problem of the image description algorithm applied in agriculture. We divided the original image into a sequence of image patches to fit the input form of the DeiT encoder, which was distilled by RegNet. Then, we used the Transformer decoder to generate descriptions. Compared to “CNN + LSTM” models, our proposed model is entirely convolution-free and has high training efficiency. On the Rice2k dataset created by us, the model achieved a 47.3 BLEU-4 score, 65.0 ROUGE_L score, and 177.1 CIDEr score. The extensive experiments demonstrate the effectiveness and the strong robustness of our model. It can be better applied to automatically generate descriptions of similar crop disease characteristics.

Sequence based local-global information fusion framework for vertebrae detection under pathological and FOV variation challenges.

Automated vertebrae detection (identification and localization) aims to identify vertebrae and locate their centroids in medical images, which is a critical step of spinal computer-aided systems. However, due to unpredictable field-of-view and various pathology cases, the image content is diverse and the vertebral morphology can be abnormal in a variety of ways, which challenges the designed systems. In this paper, we propose an effective sequence-based framework robust to various tough cases for accurate vertebrae identification and localization. Our method consists of three sub-modules: (1) Local Feature Extraction (LFE) module designs a shape-compatible category-balanced sampler to collect patches to train a convolution neural network, which extracts representative local features and generates score maps. (2) Discriminative Sequential Image Description (DSID) module proposes a node screening strategy for reliable vertebral feature sequence construction based on feature maps and score maps. This effectively prevents false positives and false negatives in light-weighted dense prediction schemes and fuses local features into a hierarchical discriminative description of given images. (3) Spinal Pattern Exploitation (SPE) module designs an end-balanced relative position learning scheme to fuse hierarchical local-global information for comprehensively exploiting spinal patterns to overcome the FOV and pathological variation challenges in vertebrae detection. Extensive experiments on a challenging dataset consisting of 450 spinal MRIs show that the identification rate of FSDF reaches 0.974 ±0.025 and the localization error is only 4.742 ±2.928 pixels, which demonstrates the effectiveness of our method with pathological and field-of-view variations and its superiority over other state-of-the-art methods.

Word self-update contrastive adversarial networks for text-to-image synthesis

Synthesizing realistic fine-grained images from text descriptions is a significant computer vision task. Although many GANs-based methods have been proposed to solve this task, generating high-quality images consistent with text information remains a difficult problem. These existing GANs-based methods ignore important words due to the use of fixed initial word features in generator, and neglect to learn semantic consistency between images and texts for discriminators. In this article, we propose a novel attentional generation and contrastive adversarial framework for fine-grained text-to-image synthesis, termed as Word Self-Update Contrastive Adversarial Networks (WSC-GAN). Specifically, we introduce a dual attention module for modeling color details and semantic information. With a new designed word self-update module, the generator can leverage visually important words to compute attention maps in the feature synthesis module. Furthermore, we contrive multi-branch contrastive discriminators to maintain better consistency between the generated image and text description. Two novel contrastive losses are proposed for our discriminators to impose image-sentence and image-word consistency constraints. Extensive experiments on CUB and MS-COCO datasets demonstrate that our method achieves better performance compared with state-of-the-art methods.