Image Annotation System Research Articles

A Privacy Study on How Social Media Users Create Paradoxical Collective Behavioral Portraits

This study investigates the group characteristics of WeChat users’ paradoxical privacy behaviors by constructing user portraits to fully understand the differences between user group characteristics and paradoxical privacy behaviors on social media platforms, and to provide users with personalized privacy protection and targeted services, strategy and specific instructions. This study takes WeChat users as the research object and uses the Vals2 attitude scale as a reference to construct a group image annotation system for WeChat users' paradoxical privacy behavior from the perspective of psychological preferences. By issuing questionnaires, the paradoxical privacy behavior of WeChat users was analyzed using methods such as factor analysis, cluster analysis, and discriminant analysis. Clustering users' paradoxical behaviors in privacy to obtain data and characterize different user groups. Research results show that the group image of WeChat users’ paradoxical privacy behavior can be divided into three types: experience-oriented, mature and rational, and egocentric. This study proposes an ambivalent group behavior model of social media user privacy. In the future, we will modify the group portrait model and consider using it in other scenarios.

Night Lighting Fault Detection Based on Improved YOLOv5

Night lighting is essential for urban life, and the occurrence of faults can significantly affect the presentation of lighting effects. Many reasons can cause lighting faults, including the damage of lamps and circuits, and the typical manifestation of the faults is that the lights do not light up. The current troubleshooting mainly relies on artificial visual inspection, making detecting faults difficult and time-consuming. Therefore, it is necessary to introduce technical means to detect lighting faults. However, current research on lighting fault detection mainly focuses on using non-visual methods such as sensor data analysis, which has the disadvantages of having a high cost and difficulty adapting to large-scale fault detection. Therefore, this study mainly focuses on solving the problem of the automatic detection of night lighting faults using machine vision methods, especially object detection methods. Based on the YOLOv5 model, two data fusion models have been developed based on the characteristics of lighting fault detection inverse problems: YOLOv5 Channel Concatenation and YOLOv5 Image Fusion. Based on the dataset obtained from the developed automatic image collection and annotation system, the training and evaluation of these three models, including the original YOLOv5, YOLOv5 Channel Concatenation, and YOLOv5 Image Fusion, have been completed. Research has found that applying complete lighting images is essential for the problem of lighting fault detection. The developed Image Fusion model can effectively fuse information and accurately detect the occurrence and area of faults, with a mAP value of 0.984. This study is expected to play an essential role in the intelligent development of urban night lighting.

Efficient Image Annotation and Caption System Using Deep Convolutional Neural Networks

This study aims to develop an annotation and image annotation system using the Fashion MNIST dataset, which consists of 70,000 grayscale images of ten clothing categories. The system uses a long short-term memory (LSTM) network to generate captions and a convolutional neural network (CNN) to extract image features. Performance evaluation metrics such as Precision, Recall, F1 score, BLEU score, METEOR score, CIDEr score, and ROUGE-L score are used where the accuracy of each clothing category is calculated to evaluate the performance of the model across different categories. Visual analysis of the generated captions is performed to gain insight into the effectiveness of the model and potential areas for improvement. The results indicate the model's success in classifying clothing items, as evidenced by its high accuracy on the test set. The qualitative study reveals the model's ability to identify different types of clothing by providing relevant captions, where the feature representation layer (normalization) plays a crucial role in transforming the detected features. to a flattened row which is then passed to a fully connected layer to learn the relationships and make final decisions with the output layer using a softmax activation function to assign probabilities to each image class, with the class with the highest probability selected as the predicted image class.

Assessment of the ecological status of large lowland rivers (Sava and Tisa, Serbia) based on digital microscopic investigation of the diatom communities

Large lowland rivers represent an important type of ecosystem that has been under a constant anthropogenic influence (industry, thermal power plants, untreated communal water, and agriculture) for centuries, which has led to most of them no longer meeting natural reference conditions. These pressures affect diatom communities which represent a reliable bioindicator of the overall degradation of water bodies. The current study provides detailed information about benthic diatoms from the large international lowland rivers Sava and Tisa, analyzed using digital microscopy. Using the digital method and the BIIGLE 2.0 image annotation system, a total of 205 and 206 diatom taxa in the Sava and Tisa rivers were identified, belonging to 65 and 63 genera, respectively. Some of the dominant genera in the Sava River were Achnanthidium, Cocconeis, Gyrosigma, and Navicula, while in the Tisa River, mostly centric diatoms (Discostella, Cyclotella, Stephanodiscus) were dominant. Most of the recorded species are alkaliphilic and characteristic of running waters with high trophic conditions. The presence of halophilic taxa was also noted (e.g. Bacillaria paxilifera, Tryblionella species, Nitzschia clausii, Navicula salinarum). These taxa are sensitive indicators of even short-term changes in lowland lotic ecosystems characterized by elevated conductivity. The diatom indices showed very consistent ecological status, mainly moderate in terms of the general pollution index IPS, and poor to bad ecological status according to trophic diatom indices (Rott TI and TDI).

Automatic image annotation system using deep learning method to analyse ambiguous images

Automatic image annotation system using deep learning method to analyse ambiguous images

Automated Image Annotation With Novel Features Based on Deep ResNet50-SLT

Due to their vast size, the growing number of digital images found in personal archives and on websites has become unmanageable, making it challenging to accurately retrieve images from these large databases. While these collections are popular due to their convenience, they are often not equipped with proper indexing information, making it difficult for users to find what they need. One of the most significant challenges in the field of computer vision and multimedia is image annotation, which involves labeling images with descriptive keywords. However, computers do not possess the capability to understand the essence of images in the same way that humans do, and people can only identify images based on their visual attributes, not their deeper semantic meaning. Therefore, image annotation requires the use of keywords to effectively communicate the contents of an image to a computer system. However, raw pixels in an image do not provide enough information to generate semantic concepts, making image annotation a complex task. Unlike text annotation, where the dictionary linking words to semantics is well established, image annotation lacks a clear definition of "words" or "sentences" that can be associated with the meaning of the image, known as the semantic gap. To address this challenge, this study aimed to characterize image content meaningfully to make information retrieval easier. An improved automatic image annotation (AIA) system was proposed to bridge the semantic gap between low-level computer features and human interpretation of images by assigning one or multiple labels to images. The proposed AIA system can convert raw image pixels into semantic-level concepts, providing a clearer representation of the image content. The study combined the ResNet50 and slantlet transform with word2vec and principal component analysis with t-distributed stochastic neighbor embedding to balance precision and recall. This allowed the researchers to determine the optimal model for the proposed ResNet50-SLT AIA framework. A Word2vec model with ResNet50-SLT was used with principal component analysis and t-distributed stochastic neighbor embedding to improve IA prediction accuracy. The distributed representation approach involved encoding and storing information about image features. The proposed AIA system utilized seq2seq to generate sentences depending on feature vectors. The system was implemented on the most popular datasets (Flickr8k, Corel-5k, ESP-Game). The results showed that the newly developed AIA scheme overcame the computational time complexity associated with most existing image annotation models during the training phase for large datasets. The performance evaluation of the AIA scheme showed its excellent flexibility of annotation, improved accuracy, and reduced computational costs, thus outperforming the existing state-of-the-art methods. In conclusion, this AIA framework can provide immense benefits in accurately selecting and extracting image features and easily retrieving images from large databases. The extracted features can effectively be used to represent the image, thus accelerating the annotation process and minimizing the computational complexity.

Suggesting an Integration System for Image Annotation

Suggesting an Integration System for Image Annotation

EyeHealer: A large-scale anterior eye segment dataset with eye structure and lesion annotations.

ABSTRACTAnterior segment eye diseases account for a significant proportion of presentations to eye clinics worldwide, including diseases associated with corneal pathologies, anterior chamber abnormalities (e.g. blood or inflammation), and lens diseases. The construction of an automatic tool for segmentation of anterior segment eye lesions would greatly improve the efficiency of clinical care. With research on artificial intelligence progressing in recent years, deep learning models have shown their superiority in image classification and segmentation. The training and evaluation of deep learning models should be based on a large amount of data annotated with expertise; however, such data are relatively scarce in the domain of medicine. Herein, the authors developed a new medical image annotation system, called EyeHealer. It is a large-scale anterior eye segment dataset with both eye structures and lesions annotated at the pixel level. Comprehensive experiments were conducted to verify its performance in disease classification and eye lesion segmentation. The results showed that semantic segmentation models outperformed medical segmentation models. This paper describes the establishment of the system for automated classification and segmentation tasks. The dataset will be made publicly available to encourage future research in this area.

An Auto-Labeling based Smart Image Annotation System

An Auto-Labeling based Smart Image Annotation System

Racing Bib Number Recognition Method using Deep Learning

Mass running event has gained popularity ever since recreational running becomes more common as they often held annually by various organizers. As image documentation took a huge part to showcase the event, many thousands of images were generated during the event. Along with thousands of images that were generated, the participant is unlikely to found an image of themselves. To solve this problem, image annotation could be performed to address image with specific tags such as participant attribute like racing bib number (RBN). Manually annotate thousands of images would result in inefficiency of time and hard-labor. As a work to tackle this problem, this paper proposed an automatic image annotation system using the YOLOv3 algorithm based RBN recognition method. The experiment result shows 83.0% precision, 81.5% recall, and 82.2% F1 score as a result of our proposed method on running event dataset. Therefore, this implemented method will promote efficiency to solve the image annotation problem because it doesn't require manual annotation over thousand of running event images

Architecture to improve the accuracy of automatic image annotation systems

Automatic image annotation (AIA) is an image retrieval mechanism to extract relative semantic tags from visual content. So far, the improvement of accuracy in newly developed such methods have been about 1 or 2% in the F1‐score and the architectures seem to have room for improvement. Therefore, the authors designed a more detailed architecture for AIA and suggested new algorithms for its main parts. The proposed architecture has three main parts: feature extraction, learning, and annotation. They designed a novel learning method using machine learning and probability bases. In the annotation part, they suggest a novel method that gains the maximum benefit from the learning part. The combination of the proposed architecture, algorithms, and novel ideas resulted in new accuracy milestones in F1‐score on most commonly used datasets. In their architecture, N+ measure which shows the number of tags with non‐zero recalls showed that they could recall all tags for IAPRTC‐12 and ESP‐Games datasets.

Detection Method of Data Integrity in Network Storage Based on Symmetrical Difference

In order to enhance the recall and the precision performance of data integrity detection, a method to detect the network storage data integrity based on symmetric difference was proposed. Through the complete automatic image annotation system, the crawler technology was used to capture the image and related text information. According to the automatic word segmentation, pos tagging and Chinese word segmentation, the feature analysis of text data was achieved. Based on the symmetrical difference algorithm and the background subtraction, the feature extraction of image data was realized. On the basis of data collection and feature extraction, the sentry data segment was introduced, and then the sentry data segment was randomly selected to detect the data integrity. Combined with the accountability scheme of data security of the trusted third party, the trusted third party was taken as the core. The online state judgment was made for each user operation. Meanwhile, credentials that cannot be denied by both parties were generated, and thus to prevent the verifier from providing false validation results. Experimental results prove that the proposed method has high precision rate, high recall rate, and strong reliability.

Filtering Instagram Hashtags Through Crowdtagging and the HITS Algorithm

Instagram is a rich source for mining descriptive tags for images and multimedia in general. The tags–image pairs can be used to train automatic image annotation (AIA) systems in accordance with the learning by example paradigm. In previous studies, we had concluded that, on average, 20% of the Instagram hashtags are related to the actual visual content of the image they accompany, i.e., they are descriptive hashtags, while there are many irrelevant hashtags, i.e., stop-hashtags, that are used across totally different images just for gathering clicks and for searchability enhancement. In this paper, we present a novel methodology, based on the principles of collective intelligence that helps in locating those hashtags. In particular, we show that the application of a modified version of the well-known hyperlink-induced topic search (HITS) algorithm, in a crowdtagging context, provides an effective and consistent way for finding pairs of Instagram images and hashtags, which lead to representative and noise-free training sets for content-based image retrieval. As a proof of concept, we used the crowdsourcing platform Figure-eight to allow collective intelligence to be gathered in the form of tag selection (crowdtagging) for Instagram hashtags. The crowdtagging data of Figure-eight are used to form bipartite graphs in which the first type of nodes corresponds to the annotators and the second type to the hashtags they selected. The HITS algorithm is first used to rank the annotators in terms of their effectiveness in the crowdtagging task and then to identify the right hashtags per image.

Inductive Zero-Shot Image Annotation via Embedding Graph

Conventional image annotation systems can only handle those images having labels within the exist library, but cannot recognize those novel labels. In order to learn new concepts, one has to gather large amount of labeled images and train the model from scratch. More importantly, it can come with a high price to collect those labeled images. For these reasons, we put forward a zero-shot image annotation model, to reduce the demand for the images with novel labels. In this paper, we focus on the two big challenges of zero-shot image annotation: polysemous words and a strong bias in the generalized zero-shot setting. For the first problem, instead of training on large corpus datasets as previous methods, we propose to adopt Node2Vec to obtain contextualized word embeddings, which can easily produce word vectors of the polysemous words. For the second problem, we alleviate the strong bias in two ways: on one hand, we utilize a model based on graph convolutional network (GCN) to make target images involved in the training process; on the other hand, we put forward a novel semantic coherent (SC) loss to capture the semantic relations of the source and target labels. The extensive experiments on NUSWIDE, COCO, IAPR TC-12, and Core15k datasets show the superiority of the proposed model and the annotation performance get improved by 4%-6% comparing with state-of-the-art methods.

An Architecture for Retrieval and Annotation of Images from Big Image Datasets

Rapid advancements in mobile devices and communication technologies have resulted in the daily production of enormous quantities of visual data at both personal and business settings. These developments have stoked a need for improved methods of storing, annotating, and retrieving digital photographs. Images need to be well annotated and indexed for precise retrieval from large-scale picture classes. As a result, there is a growing need for sophisticated strategies for indexing, annotating, and retrieving images. In response to problems discovered during research into existing Image annotation and retrieval systems, the suggested system was developed. The Autoencoder is built with three levels of complexity: an encoder, a decoder, and a single layer of code. Dimensionality reduction is accomplished by extracting the Micro-Structure Descriptor (MSD) of the training picture, which has 72 dimensions. Dimensionality reduction is the focus of a new Autoencoder technique. To organise the wide variety of picture types, a unique Autoencoder Hashing method has been developed.

Annotation of enhanced radiographs for medical image retrieval with deep convolutional neural networks.

The number of images taken per patient scan has rapidly increased due to advances in software, hardware and digital imaging in the medical domain. There is the need for medical image annotation systems that are accurate as manual annotation is impractical, time-consuming and prone to errors. This paper presents modeling approaches performed to automatically classify and annotate radiographs using several classification schemes, which can be further applied for automatic content-based image retrieval (CBIR) and computer-aided diagnosis (CAD). Different image preprocessing and enhancement techniques were applied to augment grayscale radiographs by virtually adding two extra layers. The Image Retrieval in Medical Applications (IRMA) Code, a mono-hierarchical multi-axial code, served as a basis for this work. To extensively evaluate the image enhancement techniques, five classification schemes including the complete IRMA code were adopted. The deep convolutional neural network systems Inception-v3 and Inception-ResNet-v2, and Random Forest models with 1000 trees were trained using extracted Bag-of-Keypoints visual representations. The classification model performances were evaluated using the ImageCLEF 2009 Medical Annotation Task test set. The applied visual enhancement techniques proved to achieve better annotation accuracy in all classification schemes.

Image annotation tactics: transitions, strategies and efficiency

Human interpretation of images during image annotation is complicated, but most existing interactive image annotation systems are generally operated based on social tagging, while ignoring that tags are insufficient to convey image semantics. Hence, it is critical to study the nature of image annotation behaviors and process. This study investigated annotation tactics, transitions, strategies and their efficiency during the image annotation process. A total of 90 participants were recruited to annotate nine pictures in three emotional dimensions with three interactive annotation methods. Data collected from annotation logs and verbal protocols were analyzed by applying both qualitative and quantitative methods. The findings of this study show that the cognitive process of human interpretation of images is rather complex, which reveals a probable bias in research involving image relevance feedback. Participants preferred applying scroll bar (Scr) and image comparison (Cim) tactics comparing with rating tactic (Val), and they did fewer fine tuning activities, which reflects the influence of perceptual level and users’ cognitive load during image annotation. Annotation tactic transition analysis showed that Cim was more likely to be adopted at the beginning of each phase, and the most remarkable transition was from Cim to Scr. By applying sequence analysis, the authors found 10 most commonly used sequences representing four types of annotation strategies, including Single tactic strategy, Tactic combination strategy, Fix mode strategy and Shift mode strategy. Furthermore, two patterns, “quarter decreasing” and “transition cost,” were identified based on time data, and both multiple tactics (e.g., the combination of Cim and Scr) and fine tuning activities were recognized as efficient tactic applications. Annotation patterns found in this study suggest more research needs to be done considering the need for multi-interactive methods and their influence. The findings of this study generated detailed and useful guidance for the interactive design in image annotation systems, including recommending efficient tactic applications in different phases, highlighting the most frequently applied tactics and transitions, and avoiding unnecessary transitions.

Meeting Report: Tissue-based Image Analysis

Quantitative image analysis (IA) is a rapidly evolving area of digital pathology. Although not a new concept, the quantification of histological features on photomicrographs used to be cumbersome, resource-intensive, and limited to specialists and specialized laboratories. Recent technological advances like highly efficient automated whole slide digitizer (scanner) systems, innovative IA platforms, and the emergence of pathologist-friendly image annotation and analysis systems mean that quantification of features on histological digital images will become increasingly prominent in pathologists' daily professional lives. The added value of quantitative IA in pathology includes confirmation of equivocal findings noted by a pathologist, increasing the sensitivity of feature detection, quantification of signal intensity, and improving efficiency. There is no denying that quantitative IA is part of the future of pathology; however, there are also several potential pitfalls when trying to estimate volumetric features from limited 2-dimensional sections. This continuing education session on quantitative IA offered a broad overview of the field; a hands-on toxicologic pathologist experience with IA principles, tools, and workflows; a discussion on how to apply basic stereology principles in order to minimize bias in IA; and finally, a reflection on the future of IA in the toxicologic pathology field.

ConceptRank for search-based image annotation

Multimedia information is becoming an ubiquitous part of our lives, which brings an equally ubiquitous need for efficient multimedia retrieval. One of the possible solutions to this problem is to attach text descriptions to multimedia data objects, thus allowing users to utilize traditional text search mechanisms. Search-based annotation techniques attempt to determine the descriptive keywords by analyzing the descriptions of similar, already annotated multimedia objects, which are detected by content-based retrieval techniques. One of the main challenges of this approach is the extraction of semantically connected keywords from the possibly noisy descriptions of similar objects. In this paper, we address this challenge by proposing the ConceptRank, a new keyword ranking algorithm that exploits semantic relationships between candidate keywords and utilizes the random walk mechanism to compute the probability of individual candidates. The effectiveness of the ConceptRank algorithm is evaluated in context of web image annotation. We present a complex image annotation system that includes the ConceptRank component, and compare it to other state-of-the–art annotation techniques.

Image Retrieval using Fused Features and Annotation Techniques

Due to the development of digital technology large amount of images are stored and shared in internet and personal database. Lot of research has been done in this area. Traditional image retrieval focus on text and content based. It has some drawback while receiving the required informations. AIA is the new interesting research area. It automatically assigns the keywords to an image. In this work fusion of texture feature and shape features are used. Proposed technique used to learns the relationship between image features and keywords. This proposed work focus automatic image retrieval and annotation system with multi class SVM