Image Tagging Systems Research Articles

Taxonomizing and Measuring Representational Harms: A Look at Image Tagging

In this paper, we examine computational approaches for measuring the "fairness" of image tagging systems, finding that they cluster into five distinct categories, each with its own analytic foundation. We also identify a range of normative concerns that are often collapsed under the terms "unfairness," "bias," or even "discrimination" when discussing problematic cases of image tagging. Specifically, we identify four types of representational harms that can be caused by image tagging systems, providing concrete examples of each. We then consider how different computational measurement approaches map to each of these types, demonstrating that there is not a one-to-one mapping. Our findings emphasize that no single measurement approach will be definitive and that it is not possible to infer from the use of a particular measurement approach which type of harm was intended to be measured. Lastly, equipped with this more granular understanding of the types of representational harms that can be caused by image tagging systems, we show that attempts to mitigate some of these types of harms may be in tension with one another.

Model and Method for Contributor’s Quality Assessment in Community Image Tagging Systems

Introduction: Large-scale human-computer systems involving people of various skills and motivation into the information processing process are currently used in a wide spectrum of applications. An acute problem in such systems is assessing the expected quality of each contributor; for example, in order to penalize incompetent or inaccurate ones and to promote diligent ones.Purpose: To develop a method of assessing the expected contributor’s quality in community tagging systems. This method should only use generally unreliable and incomplete information provided by contributors (with ground truth tags unknown).Results:A mathematical model is proposed for community image tagging (including the model of a contributor), along with a method of assessing the expected contributor’s quality. The method is based on comparing tag sets provided by different contributors for the same images, being a modification of pairwise comparison method with preference relation replaced by a special domination characteristic. Expected contributors’ quality is evaluated as a positive eigenvector of a pairwise domination characteristic matrix. Community tagging simulation has confirmed that the proposed method allows you to adequately estimate the expected quality of community tagging system contributors (provided that the contributors' behavior fits the proposed model).Practical relevance: The obtained results can be used in the development of systems based on coordinated efforts of community (primarily, community tagging systems).

Tag Refinement for User-Contributed Images via Graph Learning and Nonnegative Tensor Factorization

Social image tagging systems mostly suffer from poor performance for image retrieval due to the noisy and incomplete correspondences between user-contributed images and their associated tags. In this letter, we aim to refine tag allocations in the social tagging data provided by these systems. In particular, we propose to harness the tagged and untagged data with a two-stage strategy according to different types of data relations, i.e. item similarity defined by prior knowledge and item co-occurrence learned from data statistics. To solve the sparsity problem, we first introduce a new graph learning (GL) method for enriching the tagging data according to item similarities. Then, we develop a method of nonnegative tensor factorization (NTF) for learning more coherent ternary relations among users, images and tags coupled by the manifold constraints learned from item co-occurrences. Experimental results with the tagging data from the NUS-WIDE dataset have been reported to validate the effectiveness of the proposed method.

Linked tag: image annotation using semantic relationships between image tags

State of the art image tagging systems are limited because they allow users to annotate image tags in noun form, which cannot fully express the semantics of image content. In this paper, we propose Linked Tag, a semi-automatic image annotation system that inserts semantic relationships between tags. The proposed annotation method connects image tags using predicate words that can capture the contexts in which the image tags are used. In particular, we exploit Linked Data such as DBPedia in order to connect the image tags with a property value. Compared with tag-based annotation and ontology-based annotation systems, Linked Tag eliminates a large amount of manual labor and enhances the semantic expression of image content. We also introduce two annotation-based applications on Linked Tag. First, we propose SPARQL query processing for image retrieval, which enables us to express visual appearance as well as semantic information. Second, we propose a novel tag-ranking algorithm based on the link analysis in the RDF annotation graph. Finally, we demonstrate the operation of our proposed system and analyze its efficacy.

Automatic Abstract Tag Detection for Social Image Tag Refinement and Enrichment

Collaborative image tagging systems, such as Flickr, are very attractive for supporting keyword-based image retrieval, but some user-provided tags of collaboratively-tagged social images might be imprecise. Some people may use general or high-level words (i.e., abstract tags) to tag their images for saving time and effort, but such general or high-level tags are too abstract to describe the visual content of social images precisely. As a result, users may not be able to find what they need when they use the specific keywords for query specification. To tackle the problem of abstract tags, an ontology with three-level semantics is constructed for detecting the candidates of abstract tags from large-scale social images. Then the image context (nearest neighbors) and tag context (most relevant tags) of social images with abstract tags are used to ultimately confirm whether these candidates are abstract or not and identify the specific tags which can further depict the images with abstract tags. In addition, all the relevant tags, which correspond with intermediate nodes between the abstract tags and specific tags on our concept ontology, are added to enrich the tags of social images so that users can have more choices to select various keywords for query specification. We have tested our proposed algorithms on two types of data sets (revised standard datasets and self-constructed dataset) and compared our approach with other approaches.