Games With A Purpose Research Articles

Juegos con propósito para la anotación delCorpus Oral Sonoro del Español rural

AbstractThe study of dialectal microvariation in spoken Spanish faces challenges due to the absence of an adequate morpho-syntactically annotated and parsed corpus. Therefore, this article introduces a novel technique, a game-based approach, for creating resources for non-standard Spanish language varieties. The article provides an overview of the progress in designing three Games With A Purpose (GWAPs) prototypes, to wit, Agentes, Tesoros, andAnotatlón. These games aim to facilitate the confirmation and correction of the morpho-syntactic tagging task of theCOSER-AP(Corpus Oral y Sonoro del Español Rural-Anotado y Parseado, ‘Annotated and Parsed Audible Corpus of Spoken Rural Spanish’). First, the article presents the methodology used to build the games. Second, it offers a detailed description of the implemented Game Design Elements (GDEs). Finally, the article discusses the results of a pilot evaluation that assesses player enjoyment and the linguistic accuracy. Findings are promising, withTesorosandAnotatlóndemonstrating high levels of enjoyment. Additionally,Agentesproves to be effective in collecting a large number of annotations. The linguistic accuracy also shows potential benefits of gamified approaches in linguistic annotation tasks. However, it also emphasizes the importance of considering regional in player assessment and training them in multidialectal contexts.

Measuring Orthogonal Mechanics in Linguistic Annotation Games

Gamification has been recently growing in popularity among researchers investigating Information and Communication Technologies. Scholars have been trying to take advantage of this approach in the field of natural language processing (NLP), developing Games With A Purpose (GWAPs) for corpus annotation that have obtained encouraging results both in annotation quality and overall cost. However, GWAPs implement gamification in different ways and to different degrees. We propose a new framework to investigate the mechanics employed in the gamification process and their magnitude in terms of complexity. This framework is based on an analysis of some of the most important contributions in the field of NLP-related gamified applications and GWAP theory. Its primary purpose is to provide a first step towards classifying mechanics that mimic mainstream video games and may require skills that are not relevant to the annotation task, defined as orthogonal mechanics. In order to test our framework, we develop and evaluate Spacewords, a linguistic space game for synonymy annotation.

Progression in a Language Annotation Game with a Purpose

Within traditional games design, incorporating progressive difficulty is considered of fundamental importance. But despite the widespread intuition that progression could have clear benefits in Games-With-A-Purpose (GWAPs)–e.g., for training non-expert annotators to produce more complex judgements– progression is not in fact a prominent feature of GWAPs; and there is even less evidence on its effects. In this work we present an approach to progression in GWAPs that generalizes to different annotation tasks with minimal, if any, dependency on gold annotated data. Using this method we observe a statistically significant increase in accuracy over randomly showing items to annotators.

A Survey on the Design of Gamified Systems for Energy and Water Sustainability

In a world affected by the constant growth and concentration of the population in urban areas, the problem of preserving natural resources has become a priority. A promising approach to resource conservation is demand management, i.e., the ability to positively influence the behaviour of the population towards more sustainable consumption. Information and Communication Technology (ICT) tools have shown a great potential in influencing consumers’ behaviour, which could be exploited for the common good. However, the design of so-called persuasive systems for environmental purposes is a challenging task, because it cannot be based solely on the utilitarian motivation of users, but must be able to trigger a broader range of engagement factors deeply rooted in the human psychology. In this paper, we review the main design principles and models at the base of a class of persuasive system that exploits gamification and Games with a Purpose (GWAPs) to engage users towards sustainability; we identify the most commonly used incentive mechanisms for inducing behavioural changes; and present a selection of gamified systems for energy and water conservation. From such a survey, we distill design guidelines to be applied in the design of demand management socio-technical systems.

Simulating gamified crowdsourcing of knowledge base refinement: effects of game rule design

ABSTRACTThis paper discusses a gamification design for knowledge base refinement. We present a simulation model for game playing and examine the possibility of estimating the effects of game rule design. The maintenance of a knowledge base involves human intervention and is one of the application domains of human computation. Using the concept of crowdsourcing, refinement tasks can be delegated to many casual users over a network. In addition, gamification, such as games with a purpose (GWAP), is a useful idea to motivate workers in crowdsourcing by making tasks enjoyable. For effective gamification, designing the game rules is critical. In this paper, we present a model for simulating the gamified knowledge base refinement process to estimate the effects of different game rule designs beforehand. We use a rental apartment FAQ system as an example knowledge base and show that by invoking game playing when a problem is found in a knowledge base, it can be properly updated. In addition, we discuss the effectiveness of using a calibration quiz to estimate the reliability of users.

Web information monitoring and crowdsourcing for promoting and enhancing the Algerian geoheritage

The documentation and management of information concerning geological sites are important activities in all the steps of the geoheritage conservation and its day-to-day management. This documentation is mostly available on the web; it includes varied types of data such as pictures, scientific publications, news, and blog articles. In order to ensure the efficient management of the geoheritage resources, scientists and people involved in the promotion of the geoheritage and geotourism should have access to this documentation and kept updated of the most recent and relevant information. In this study, we propose the development of a web information monitoring system that automates the search and the collection of different types of documents related to the Algerian geosites available on the web. It is designed as a tool that continuously browses the web using the most popular search engines and then disseminates the results to the users through periodic newsletters. This proposed system will also serve to chronologically organize all the pictures of geosites gathered from the web and thus in order to study the evolution of Algerian geosites over time. For this purpose, we use one of the crowdsourcing methods in the field of knowledge management that is games with a purpose (GWAP). The idea here is to involve people in sorting pictures of geosites through a playful interface. The results obtained show that this system is a valuable tool that can assist geoscientists and decision makers in enhancing the geoheritage.

Topical Video Search: Analysing Video Concept Annotation through Crowdsourcing Games

Games with a purpose (GWAPs) are increasingly used in audio-visual collections as a mechanism for annotating videos through tagging. One such GWAP is Waisda?, a video labeling game where players tag streaming video and win points by reaching consensus on tags with other players. The open-ended and unconstrained manner of tagging in the fast-paced setting of the game has fundamental impact on the resulting tags. We find that Waisda? tags predominately describe visual objects and rarely refer to the topics of the videos. In this study we evaluate to what extent the tags entered by players can be regarded as topical descriptors of the video material. Moreover, we characterize the quality of the user tags as topical descriptors with the aim to detect and filter out the bad ones. Our results show that after filtering, game tags perform equally well compared to the manually crafted metadata when it comes to accessing the videos based on topic. An important consequence of this finding is that tagging games can provide a cost-effective alternative in situations when manual annotation by professionals is too costly.

Topical Video Search: Analysing Video Concept Annotation through Crowdsourcing Games

Games with a purpose (GWAPs) are increasingly used in audio-visual collections as a mechanism for annotating videos through tagging. One such GWAP is Waisda?, a video labeling game where players tag streaming video and win points by reaching consensus on tags with other players. The open-ended and unconstrained manner of tagging in the fast-paced setting of the game has fundamental impact on the resulting tags. We find that Waisda? tags predominately describe visual objects and rarely refer to the topics of the videos. In this study we evaluate to what extent the tags entered by players can be regarded as topical descriptors of the video material. Moreover, we characterize the quality of the user tags as topical descriptors with the aim to detect and filter out the bad ones. Our results show that after filtering, game tags perform equally well compared to the manually crafted metadata when it comes to accessing the videos based on topic. An important consequence of this finding is that tagging games can provide a cost-effective alternative in situations when manual annotation by professionals is too costly.

Gamification in theory and action: A survey

Gamification has drawn the attention of academics, practitioners and business professionals in domains as diverse as education, information studies, human–computer interaction, and health. As yet, the term remains mired in diverse meanings and contradictory uses, while the concept faces division on its academic worth, underdeveloped theoretical foundations, and a dearth of standardized guidelines for application. Despite widespread commentary on its merits and shortcomings, little empirical work has sought to validate gamification as a meaningful concept and provide evidence of its effectiveness as a tool for motivating and engaging users in non-entertainment contexts. Moreover, no work to date has surveyed gamification as a field of study from a human–computer studies perspective. In this paper, we present a systematic survey on the use of gamification in published theoretical reviews and research papers involving interactive systems and human participants. We outline current theoretical understandings of gamification and draw comparisons to related approaches, including alternate reality games (ARGs), games with a purpose (GWAPs), and gameful design. We present a multidisciplinary review of gamification in action, focusing on empirical findings related to purpose and context, design of systems, approaches and techniques, and user impact. Findings from the survey show that a standard conceptualization of gamification is emerging against a growing backdrop of empirical participants-based research. However, definitional subjectivity, diverse or unstated theoretical foundations, incongruities among empirical findings, and inadequate experimental design remain matters of concern. We discuss how gamification may to be more usefully presented as a subset of a larger effort to improve the user experience of interactive systems through gameful design. We end by suggesting points of departure for continued empirical investigations of gamified practice and its effects.

Human Computation: Object Recognition for Mobile Games Based on Single Player

Smart phones and its applications gain a lot of popularity nowadays. Many people depend on them to finish their tasks banking, social networking, fun and a lot other things. Games with a purpose (GWAP) and microtask crowdsourcing are considered two techniques of the human-computation. GWAPs depend on humans to accomplish their tasks. Porting GWAPs to smart phones will be great in increasing the number of humans in it. One of the systems of human-computation is ESP Game. ESP Game is a type of games with a purpose. ESP game will be good candidate to be ported to smart phones. This paper presents a new mobile game called MemoryLabel. It is a single player mobile game. It helps in labeling images and gives description for them. In addition, the game gives description for objects in the image not the whole image. We deploy our algorithm at the University of Menoufia for evaluation. In addition, the game is published on Google play market for android applications. In this trial, we first focused on measuring the total number of labels generated by our game and also the number of objects that have been labeled. The results reveal that the proposed game has promising results in describing images and objects.

ARTigo: Building an Artwork Search Engine With Games and Higher-Order Latent Semantic Analysis

This article describes how a semantic search engine has been build from, and still is continuously improved by, a semantic analysis of the “footprints” left by players on the gaming Web platform ARTigo. The Web platform offers several Games With a Purpose (GWAPs) some of which have been specifically designed to collect the data needed for building the artwork search engine. ARTigo is a “tagging ecosystem” of games that cooperate so as to gather a wide range of information on artworks. The ARTigo ecosystem generates a folksonomy saved as 3rd-order tensor, that is a generalization of a matrix, the three orders or dimensions of which represent (1) who (2) tagged an (3) an artwork. The semantic search engine is build using a non-trivial generalization of the well-known, matrix-based, Latent Semantic Analysis (LSA) methods and algorithms. ARTigo is in service for five years and is subject to an active research constantly resulting in new developments, some of which are reported about for the first time in this article.

Human computation: Image metadata acquisition based on a single-player annotation game

Effective acquisition of descriptive semantics for images is still an open issue today. Crowd-based human computation represents a family of approaches able to provide large scale metadata with decent quality. Within this field, games with a purpose (GWAP) have become increasingly important, as they have the potential to motivate contributors to the process through entertainment. However, the existing solutions are weak, when specific metadata are needed. In this work, we present a game with a purpose called PexAce, which utilizes manpower to collect tags characterizing a set of given images. Using novel game mechanics, the game is a single-player, less prone to cold-start problems and suitable for deployment in the domain of personal imagery. As our experiments show, the game delivers tags that characterize images with high precision (using a posteriori expert evaluation and evaluation against the gold standard: the extended Corel 5k dataset). We also employ the game in the domain of personal images, where very specific metadata are needed for their proper organization (person names, places, events) and show, that the game is able to collect even these kinds of metadata. We show that the key to higher quality metadata lies in combining the fun factor of the game with motivation for personal gain.

A Novel Algorithm for Improving the ESP Game

one of the human-computation techniques is games with a purpose (GWAP) and microtask crowdsourcing. These techniques can help in making the image retrieval (IR) be more accurate and helpful. It provides the IR system’s database with a rich of information by adding more descriptions and annotations to images. One of the systems of human-computation is ESP Game. ESP Game is a type of games with a purpose. In the ESP game there has been a lot of work was proposed to solve many of the problems in it and make the most benefit of the game. One of these problems is that the ESP game neglects player's answers for the same image that don't match. This paper presents a new algorithm to use neglected data to generate new labels for the images. We deploy our algorithm at the University of Menoufia for evaluation. In this trial, we first focused on measuring the total number of labels generated by our Recycle Unused Answers For Images algorithm (RUAI). In our evaluation of the RUAI algorithm we present a new evaluation measure we called it quality of labels measure. This measure identifies the quality of the labels in compared to the pre-qualified labels. The results reveal that the proposed algorithm improved the results in compared to the ESP game in all cases.

LinkIT: A Ludic Elicitation Game for Eliciting Risk Perceptions

The mental models approach, a leading strategy to develop risk communications, involves a time- and labor-intensive interview process and a lengthy questionnaire to elicit group-level risk perceptions. We propose that a similarity ratings approach for structural knowledge elicitation can be adopted to assist the risk mental models approach. The LinkIT game, inspired by games with a purpose (GWAP) technology, is a ludic elicitation tool designed to elicit group understanding of the relations between risk factors in a more enjoyable and productive manner when compared to traditional approaches. That is, consistent with the idea of ludic elicitation, LinkIT was designed to make the elicitation process fun and enjoyable in the hopes of increasing participation and data quality in risk studies. Like the mental models approach, the group mental model obtained via the LinkIT game can hence be generated and represented in a form of influence diagrams. In order to examine the external validity of LinkIT, we conducted a study to compare its performance with respect to a more conventional questionnaire-driven approach. Data analysis results conclude that the two group mental models elicited from the two approaches are similar to an extent. Yet, LinkIT was more productive and enjoyable than the questionnaire. However, participants commented that the current game has some usability concerns. This presentation summarizes the design and evaluation of the LinkIT game and suggests areas for future work.

Purposive Hidden-Object-Game: Embedding Human Computation in Popular Game

Having sufficient training images with fully annotated object locations is undoubtedly critical for modern learning-based image annotation, retrieval, and object detection methods. Typically, collecting such annotations for large-scale datasets is notoriously tedious because the process involves amount of manual cropping and hand labeling operations. In this work, following the principle of games with a purpose (GWAP), we design a so-called purposive hidden-object-game (P-HOG), which imperceptibly embeds localizing objects into enjoyable playing game process and thus attracts many people to make voluntary contribution to annotating images. In particular, besides preserving the interestingness as popular HOG games, P-HOG is able to automatically generate satisfactory game images (i.e., “hide” certain items into target images) by integrating several semantic and visual processing techniques. P-HOG is also built in an effective mechanism to prevent the players from cheating. The mechanism inherits the merit of Recaptcha and identifies potential cheating behavior based on the annotation accuracy of some known items. Moreover, P-HOG will filter noisy annotations effectively based on a weighted majority method and improve the accuracy of the raw annotations from the players. Most importantly, players only play P-HOG for entertainment purpose and they are unaware of the background data collection procedure. The collected data are used towards constructing a large database, which may benefit general learning-based algorithms for multimedia tasks. To the best of our knowledge, this is the first work dedicated to such a specific and important task under the GWAP framework. We conduct a pilot study of the game prototype and the comprehensive experiments show that the P-HOG appeals to general players, and is effective for collecting massive object locations with satisfactory accuracy, which further boosts the algorithmic performances for both tag refinement and image annotation tasks.

An Experiment in Comparing Human-Computation Techniques

Human computation can address complex computational problems by tapping into large resource pools for relatively little cost. Two prominent human-computation techniques — games with a purpose (GWAP) and microtask crowdsourcing — can help resolve semantic-technology-related tasks, including knowledge representation, ontology alignment, and semantic annotation. To evaluate which approach is better with respect to costs and benefits, the authors employ categorization challenges in Wikipedia to ultimately create a large, general-purpose ontology. They first use the OntoPronto GWAP, then replicate its problem-solving setting in Amazon Mechanical Turk, using a similar task-design structure, evaluation mechanisms, and input data.

Applying human computation mechanisms to information retrieval

AbstractCrowdsourcing and Games with a Purpose (GWAP) have each received considerable attention in recent years. These two human computation mechanisms assist with tasks that cannot be solved by computers alone. Despite this increased attention, much of this transformation has been limited to a few aspects of Information Retrieval (IR). In this paper, we examine these two mechanisms' applicability to IR. Using an IR model, we apply criteria to determine the suitability of these crowdsourcing and GWAP mechanisms to each step of the model. Our analysis illustrates that these mechanisms can apply to several of these steps with good returns.

Constructing the Semantic Information Model using A Collective Intelligence Approach

Knowledge is often represented as a set of rules or a semantic network in intelligent systems. Recently, ontology has been widely used to represent semantic knowledge, because it organizes thesaurus and hierarchal information between concepts in a particular domain. However, it is not easy to collect semantic relationships among concepts. Much time and expense are incurred in ontology construction. Collective intelligence can be a good alternative approach to solve these problems. In this paper, we propose a collective intelligence approach of Games With A Purpose (GWAP) to collect various semantic resources, such as words and word-senses. We detail how to construct the semantic information model or ontology from the collected semantic resources, constructing a system named FunWords. FunWords is a Korean lexical-based semantic resource collection tool. Experiments demonstrated the resources were grouped as common nouns, abstract nouns, adjective and neologism. Finally, we analyzed their characteristics, acquiring the semantic relationships noted above. Common nouns, with structural semantic relationships, such as hypernym and hyponym, are highlighted. Abstract nouns, with descriptive and characteristic semantic relationships, such as synonym and antonym are underlined. Adjectives, with such semantic relationships, as description and status, illustration - for example, color and sound - are expressed more. Last, neologism, with the semantic relationships, such as description and characteristics, are emphasized. Weighting the semantic relationships with these characteristics can help reduce time and cost, because it need not consider unnecessary or slightly related factors. This can improve the expressive power, such as readability, concentrating on the weighted characteristics. Our proposal to collect semantic resources from the collective intelligence approach of GWAP (our FunWords) and to weight their semantic relationship can help construct the semantic information model or ontology would be a more effective and expressive alternative.

Intelligence in Human Computation Games

Human computation is a field that studies the employment of humans' computing ability to solve complex problems that computers cannot easily solve in time. Games present one plat form for human computation in an entertaining manner. Pioneered by Luis Von Ahn and his colleagues, many games "with a purpose (GWAP) systems have been developed in recent years, such as ESP, GWAP for the Semantic Web, and CityExplorer. Computational tasks, including image tagging, building the Semantic Web, and collecting geo spatial data for cities, are "crowd sourced" to the players and solved by human power.

An analytical model for generalized ESP games

The ESP game belongs to the genre called Games With A Purpose (GWAP), which leverage people’s desire to be entertained and also outsource certain steps of the computational process to humans. The games have shown promise in solving a variety of problems, which computer computation has been unable to resolve completely thus far. In this study, we consider generalized ESP games with two objectives. First, we propose an analytical model for computing the utility of generalized ESP games, where the number of players, the consensus threshold, and the stopping condition are variable. We show that our model can accurately predict the stopping condition that will yield the optimal utility of a generalized ESP game under a specific game setting. A service provider can therefore utilize the model to ensure that the hosted generalized ESP games produce high-quality labels efficiently. Second, we propose a metric, called system gain, for evaluating the performance of ESP-like GWAP systems, and also use analysis to study the properties of generalized ESP games. We believe that GWAP systems should be designed and played with strategies. To this end, we implement an optimal puzzle selection strategy (OPSA) based on our analysis. Using a comprehensive set of simulations, we demonstrate that the proposed OPSA approach can effectively improve the system gain of generalized ESP games, as long as the number of puzzles in the system is sufficiently large.