Wizard Of Oz Research Articles

Acquisition and evaluation of a human-robot elderly spoken dialog corpus for developing computerized cognitive assessment systems

Personal communication robots are expected to assist daily living of elderly people. Aiming at developing computerized cognitive assessment systems, we collected human-robot spoken dialog of a cognitive impairment test scenario based on TICS (Telephone Interview for Cognitive Status) and COGNISTAT (Cognitive Status Examination) in Japanese. For the efficient acquisition of the spoken dialog corpus of this scenario, we implemented a WOz (Wizard of Oz) style spoken dialog system on a commercial personal robot, PaPeRo by NEC. By using this system, we collected 147 dialogs spoken by 48 elderlies whose ages varied about 75-85 and MMSE (Mini Mental State Examination) scores varied around 26.5. Each dialog took about 30 min, and contains around 100 human utterances. In order to evaluate feasibility of automatic assessment, we conducted speech recognition experiments with the speech corpus. In the recognition experiments of the elderly speech of answering to the temporal orientation test of asking today's month and date, the accuracies of discriminating correctness of the answers exceeds 80% by using speech dictation engines. This preliminary results show potential feasibilities for computerized cognitive assessment systems.

Using Wizard of Oz technology for telemedicine

Traditional computer simulations do not incorporate human intuition and human-like engagement that acknowledges complex and abstract questions. This paper presents WOTS (Wizard of Oz Telemedicine Simulator), a generic cognitive training platform that supports transfer of medical knowledge through remote training and consultations (telemedicine training). Medical training via the WOTS provides remote learners a risk-free environment in which they can practice and make mistakes, while receiving online feedback about their decisions. WOTS leverages the Wizard of Oz (WOZ) concept, where human intuition is applied through corrective feedback to complex and abstract misconceptions of learners’ thinking (flawed mental models). The contribution of WOTS to enhanced learning is demonstrated by a number of pilot runs and a case study.

WozARd: A Wizard of Oz Method for Wearable Augmented Reality Interaction—A Pilot Study

Head-mounted displays and other wearable devices open up for innovative types of interaction for wearable augmented reality (AR). However, to design and evaluate these new types of AR user interfaces, it is essential to quickly simulate undeveloped components of the system and collect feedback from potential users early in the design process. One way of doing this is the wizard of Oz (WOZ) method. The basic idea behind WOZ is to create the illusion of a working system by having a human operator, performing some or all of the system’s functions. WozARd is a WOZ method developed for wearable AR interaction. The presented pilot study was an initial investigation of the capability of the WozARd method to simulate an AR city tour. Qualitative and quantitative data were collected from 21 participants performing a simulated AR city tour. The data analysis focused on seven categories that can have an impact on how the WozARd method is perceived by participants: precision, relevance, responsiveness, technical stability, visual fidelity, general user-experience, and human-operator performance. Overall, the results indicate that the participants perceived the simulated AR city tour as a relatively realistic experience despite a certain degree of technical instability and human-operator mistakes.

Wizard of Oz Studies in HRI: A Systematic Review and New Reporting Guidelines

Many researchers use Wizard of Oz (WoZ) as an experimental technique, but there are methodological concerns over its use, and no comprehensive criteria on how to best employ it. We systematically review 54 WoZ experiments published in the primary HRI publication venues from 2001 -- 2011. Using criteria proposed by Fraser and Gilbert (1991), Green et al. (2004), Steinfeld et al. (2009), and Kelley (1984), we analyzed how researchers conducted HRI WoZ experiments. Researchers mainly used WoZ for verbal (72.2%) and non-verbal (48.1%) processing. Most constrained wizard production (90.7%), but few constrained wizard recognition (11%). Few reported measuring wizard error (3.7%), and few reported pre-experiment wizard training (5.4%). Few reported using WoZ in an iterative manner (24.1%). Based on these results we propose new reporting guidelines to aid future research.

Supporting older adults in using technology for lifelong learning

In the past, many technological initiatives have failed to engage adult learners in any significant way in part due to the lack of technological initiatives that build on people's existing interests but also because of an insufficient focus on designing co-operative technologies that can be used effortlessly. In this paper we provide our experiences of the use of Wizard of Oz (WOz) studies in order to develop a technological tool to support older adults' use of the Internet for learning, where we aimed to achieve a more learner centred approach to design. Wizard of Oz simulation is a research method common in Human-Computer Interface (HCI), Natural Language Processing (NLP) and other human factors/computer usability research where researchers conceal themselves from research participants and use communications technology to pretend that a prototype or incomplete computer-based conversational system is fully functioning. The purpose of a WOz simulation is usually to investigate and inform the development of a technology that has yet to be developed, or to learn more about how people interact with the system in order to improve the design. In this paper we outline the understandings we developed using this technique, highlighting the methodological and conceptual value of using this approach. Based on conducting Wizard of Oz simulations with 20 older adults with a diverse range of educational backgrounds and technological expertise we ask: what are the methodological benefits and challenges of using Wizard of Oz simulations in studies of learning and the Internet? In this paper, we argue that the use of Wizard of Oz approach could be a valuable method to employ for a wider range of research that examines networked learning in formal or informal settings. This is particularly the case as one (unanticipated) effect of the study was how the use of this technique developed our understandings of the everyday experiences older adults have with computers and the Internet when trying to learn new things. In addition we argue that the use of this method is important in the current context where a common criticism of technologies that are designed to support learning tend not to be fit for purpose, as they have either been developed initially for commercial uses or developed without a strong understanding of the learner in mind.

Context-aware web search in ubiquitous sensor environments

This article proposes a new concept for a context-aware Web search method that automatically retrieves a webpage related to the daily activity that a user currently is engaged in and displays the page on nearby Internet-connected home appliances such as televisions. For example, when a user is washing a coffeemaker, a webpage is retrieved that includes tips such as “cleaning a coffee maker with vinegar removes stains well,” and the page is displayed on a nearby appliance. In this article, we design and implement a Web search method that employs ubiquitous sensors to monitor a user's daily life. Our proposed method automatically searches for a webpage related to a daily activity by using a query constructed from the use of daily objects employed in the activity that is detected with object-attached sensors. We evaluate the search method with real datasets collected from vast numbers of sensors and achieve very accurate webpage retrieval. We then investigate the usefulness and effectiveness of a daily life Web search with Wizard-of-Oz (WOz)-like experiments. We confirm that the presentation of webpages related to daily activities improves participants' future daily lives and triggers communication among the participants in the experiment.

Learning and Evaluation of Dialogue Strategies for New Applications: Empirical Methods for Optimization from Small Data Sets

We present a new data-driven methodology for simulation-based dialogue strategy learning, which allows us to address several problems in the field of automatic optimization of dialogue strategies: learning effective dialogue strategies when no initial data or system exists, and determining a data-driven reward function. In addition, we evaluate the result with real users, and explore how results transfer between simulated and real interactions. We use Reinforcement Learning (RL) to learn multimodal dialogue strategies by interaction with a simulated environment which is “bootstrapped” from small amounts of Wizard-of-Oz (WOZ) data. This use of WOZ data allows data-driven development of optimal strategies for domains where no working prototype is available. Using simulation-based RL allows us to find optimal policies which are not (necessarily) present in the original data. Our results show that simulation-based RL significantly outperforms the average (human wizard) strategy as learned from the data by using Supervised Learning. The bootstrapped RL-based policy gains on average 50 times more reward when tested in simulation, and almost 18 times more reward when interacting with real users. Users also subjectively rate the RL-based policy on average 10% higher. We also show that results from simulated interaction do transfer to interaction with real users, and we explicitly evaluate the stability of the data-driven reward function.

Active adaptation in human-agent collaborative interaction

When a human user interacts with an adaptive agent to achieve human-agent collaboration, active adaptation is considered to be one of the critical characteristics of the agent. In order to investigate the principal features of active adaptation, we developed a human-agent collaborative experimental environment called WAITER (waiter agent interactive training experimental restaurant) and conducted two types of experiments, a Wizard of OZ (WOZ) agent experiment and an autonomous agent experiment. The objective of these experiments is to observe how human users perceive the agents and change their instructions when interacting with adaptive agents. The results indicate that humans can recognize changes in the agent's actions and change their instruction methods accordingly. This implies that active adaptation of the agents may encourage the adaptation of the human users and help to build an adaptation loop between them. The experimental results also suggest that active adaptation may play an important role in a human-agent collaborative task.

Inducing Genuine Emotions in Simulated Speech-Based Human-Machine Interaction: The NIMITEK Corpus

Emotional corpora provide an important empirical foundation for investigation when researchers aim at implementing emotion-aware spoken dialog systems. One of the fundamental research questions is how to acquire an appropriate, realistic emotion corpus. The primary aim of this paper is to address the methodological desiderata in producing emotion corpora in human-machine interaction (HMI). It proposes a substantial refinement of the Wizard-of-Oz (WOZ) technique in order that a scenario designed to elicit affected speech in HMI could result in realistic and useful data. In addition, the paper reports about the NIMITEK corpus of affected behavior in HMI produced during a refined WOZ simulation. The evaluation of the corpus with respect to the perception of its emotional content demonstrated that the corpus contains recordings of emotions that were overtly signaled. The range of emotional reactions is indicative of the kind of emotional reactions than can be expected to occur in the interaction with the sort of spoken dialog systems considered in this study. Since the subjects were not restricted by given predetermined linguistic constraints on the language to use, their utterances are indicative of the way in which nontrained, nontechnical users probably like to converse with conversational agents as well.

Mobile user profiles for Personal Networks: The MAGNET Beyond case

AbstractIn this paper, a methodology for the definition and application of profiles for mobile devices is presented. The methodology, following the recommendations of ETSI, is used to present the definition of profiles capable of supporting Personal Networking capabilities. Furthermore, a case study for the development of a socializing service (‘Icebreaker’) is also presented. The service has just passed the early evaluation phase and proceeds to final implementation. To conclude with, this paper showcases several user interfaces already developed, as also some of their preliminary evaluation results obtained through the Wizard‐of‐Oz (WoZ) technique. The ideas described here constitute part of the work performed in the context of the European IST FP6 project MAGNET Beyond. Copyright © 2010 John Wiley & Sons, Ltd.

WOZ acoustic data collection for interactive TV

This paper describes a multichannel acoustic data collection recorded under the European DICIT project, during Wizard of Oz (WOZ) experiments carried out at FAU and FBK-irst laboratories. The application of interest in DICIT is a distant-talking interface for control of interactive TV working in a typical living room, with many interfering devices. The objective of the experiments was to collect a database supporting efficient development and tuning of acoustic processing algorithms for signal enhancement. In DICIT, techniques for sound source localization, multichannel acoustic echo cancellation, blind source separation, speech activity detection, speaker identification and verification as well as beamforming are combined to achieve a maximum possible reduction of the user speech impairments typical of distant-talking interfaces. The collected database permitted to simulate at preliminary stage a realistic scenario and to tailor the involved algorithms to the observed user behaviors. In order to match the project requirements, the WOZ experiments were recorded in three languages: English, German and Italian. Besides the user inputs, the database also contains non-speech related acoustic events, room impulse response measurements and video data, the latter used to compute three-dimensional positions of each subject. Sessions were manually transcribed and segmented at word level, introducing also specific labels for acoustic events.

A wizard of oz component-based approach for rapidly prototyping and testing input multimodal interfaces

In this paper we present a novel approach for prototyping, testing and evaluating multimodal interfaces, OpenWizard. OpenWizard allows the designer and the developer to rapidly evaluate a non-fully functional multimodal prototype by replacing one modality or a composition of modalities that are not yet available by wizard of oz techniques. OpenWizard is based on a conceptual component-based approach for rapidly developing multimodal interfaces, an approach first implemented in the ICARE software tool and more recently in the OpenInterface tool. We present a set of wizard of oz components that are implemented in OpenInterface. While some wizard of oz (WoZ) components are generic to be reused for different multimodal applications, our approach allows the integration of tailored WoZ components. We illustrate OpenWizard using a multimodal map navigator.

Corporate governance and corporate social responsibility: lessons from the land of OZ

Australia corporate boards and senior management have been spirited from the Land of Milk and Honey (profit) to the Land of OZ. They are to embark on a journey, following the “yellow-brick road”, a proverbial path to a promised land of one’s hopes and dreams, in order to find brains, a heart and courage. The effect of new regulations introduced in Australia to curtail corporate misbehaviour is detailed by storytelling. The Wizard of OZ is the title of a story written by L. Frank Baum and published in 1899. In 1939, Metro Goldwyn Mayer Studios made a movie of the story. I have used The Wizard of Oz to argue that corporate boards and senior management need to make decisions using a balance of intellect (brains), emotionality (heart), and a sense of purpose (courage). The inspiration for using Baum’s story, as an analogy for the transformation needed in corporate boards and senior management, comes from Biberman and Whitty (Journal of Organisational Change Management 10(2):130–188, 1997). This research is based on interviews with Board members and non-executive directors from five companies listed in the Business Review Weekly (BRW) Top Twenty-five Companies in Australia (2007) and a range of secondary data sources. The financial and reputational success of the organisation and its members is out of balance with the human and social costs and benefits. Respondents confirmed that board members and senior management should willingly provide information about the corporation and its activities to its stakeholders, that information and data should be transparent, the true extent of director remuneration should be revealed and that financial reporting should be true and accurate. Board members and senior management can be assisted to operate in a way that observes socially responsible values and balances the obligation for profit maximisation with corporate social responsibilities (CSR). This study provides steps that organisations can take to achieve a balance of intellect, emotionality and sense of purpose and therefore realise their corporate social responsibility. The results of this empirical and secondary research suggest a method that may be used to make board members and senior managers more aware of their corporate social responsibilities and curtail corporate misbehaviour where the introduction of a range of new regulations has had little effect.

Communicating with multiple users for embodied conversational agents in quiz game context

This paper presents two approaches to achieve attentiveness of a virtual quiz agent in the interactions with multiple users at the same time. One attempts to improve the agent with an utterance strategy to determine whether, when, and whom to suggest the users the availability of hints or to urge the users to answer. In the other one, a transition state model presenting the agent's attitude that allows the agent to perform meaningful idle motions is introduced. The transition model is obtained in a Wizard-of-Oz (WOZ) experiment and is learned with support vector machine (SVM). These two prototype systems are evaluated with an automatic attitude test, go/no-go task (GNAT) and regular questionnaires. From the joint results of the evaluation subject experiments, the direction of finding an appropriate triggering timing of the agent's behaviours is proved to be effective.

Benedikte Nauberts 'Neue Volksmärchen der Deutschen': Strukturen des Wandels by Laura Martin

580 Reviews Sich weilB Rechenschaft zu geben, I Bleib imDunkeln unerfahren, I Mag von Tag zu Tage leben.' ST JOHN'S COLLEGE, OXFORD RITCHIE ROBERTSON Benedikte Nauberts 'Neue Volksmdrchen der Deutschen': Strukturen desWandels. By LAURAMARTIN. Wiirzburg: Konigshausen & Neumann. 2oo6. 379 PP. ?49.80. ISBN 978-3-8260-34I6-9. Laura Martin's monograph is the first book-length study ofBenedikte Naubert's four volume fairy-tale collection Neue Volksmdrchen der Deutschen ( 789-92). Naubert who was rediscovered by feminist critics in the I98os-is gradually attractingmore scholarly attention as awriter of fairy tales: theNeue Volksmdrchenwere reprinted in 2001 (Neue Volksmdrchen derDeutschen, ed. byMarianne Henn and others, 4 vols (G6ttingen: Wallstein, 2001), reviewed in MLR, 97 (2002), IO20), andMartin's book is awelcome addition to the secondary literature.Martin states at theoutset that she is fascinated byNaubert's stories as a type ofwriting unfamiliar to us from canoni cal literature.Her central idea is that the author draws on a wide range of different sources and reworks her material inways which surprise us as readers brought up on the likes of theGrimms: here we findheroes who turn theirback on violence, for example, or heroines who help to shape their own fate. Martin hopes ultimately to show thatNaubert deserves to be read and studied more widely. The book is divided into three parts. The firstgives some general background information. It startswith a short introduction toNaubert and gives an overview of research. It then surveys the tradition of fairy-talewriting inGermany and briefly compares Naubert with her contemporary Johann Karl August Musaus. The most substantial section comes next:Martin takes us through the collection and examines each of the fifteenstories in turn. For each she provides a detailed plot summary (sixteen pages in the case of 'Erdmann und Marie: Ein Nachtrag zu den Legenden von Ruibezahl'!) and an interpretation of the textwhich looks, for instance, at themes, characterization, and style.The finalpart of thebook isentitled 'Helden und Heldin nen'. It includes a chapter on curiosity and disobedience and a chapter on violence, and hereMartin fleshes out her argument about Naubert's unconventional portrayal ofmale and female characters. The book concludes by listing again the reasons why Naubert should be granted a place in literaryhistory. Overall, Martin certainlymanages to communicate her enthusiasm forNaubert's writing. In the second and thirdparts she places the stories themselves verymuch at centre stage. She focuses on close readings of the textsand consciously avoids bring ing in toomuch discussion of thewider literaryand historical context. She skirtsover the question of different sources, forexample, when considering Naubert's 'Ottilie' and theGrimms' 'Marienkind' side by side. She does not always engage fullywith the secondary literature, and the relatively few footnotes are just as likely to be com paring the tales to The Grapes ofWrath, The Scarlet Letter, or The Wizard ofOz. The book iswritten in a colourful, informal stylewith many an ironic comment and passing reference toBilbo Baggins orHarry Potter. One does wonder ifit would have brought Naubert to awider audience if ithad been published inEnglish instead of being translated intoGerman. The long plot summaries seem a little superfluous for German readers, whereas at present just one of these stories is available inEnglish translation. But as it is, this is a lively and interesting studywhich certainly deserves towin some new readers for this still largelyunknown eighteenth-century author. SWANSEAUNIVERSITY HILARY BROWN ...

Adventures in the West: Stories for Young Readers ed. by Susanne George Bloomfield, Eric Melvin Reed

BO O K REVIEW S Adventures in the West: Stories for Young Readers. Edited by Susanne George Bloomfield and Eric Melvin Reed. Lincoln: University of Nebraska Press, 2007. 280 pages, $19.95. Reviewed by Richard D. Jensen Foley, Alabama Adventures in the West: Stories for Young Readers is a compilation of short sto­ ries from a bygone era, a time when youth magazines existed in profusion and young people read for entertainment. Editors Susanne George Bloomfield, an English professor, and Eric Melvin Reed, a graduate student, deserve high praise for assembling stories that were originally published between 1890 and 1917. These tales are well written and entertaining and provide a window into the values and culture of the times. These stories served many purposes when first printed. They were entertainment, examples of fine writing, encourage­ ment for, and a reflection of, the expansiveness of youthful imagination, and reinforcers of the unique values that made America great. The stories in this volume are by writers whose names have, sadly, drifted away on the winds of time. Only The Wizard ofOz (1900) author L. Frank Baum, whose Aunt ’Phroney’s Boy (1912) is featured here, is known widely today. Each story is well crafted, and the prose is simply terrific. For example, May Roberts Clark, in Her Neighbor’s Claim (1897), writes, “The wagon trail of hot July dust bent to the harmony of nature, and wavered to and fro like a river. A bird rising from the sear buffalo grass wheeled a wide orbit up and up, and out of sight. In one vast curve God had drawn the mighty sweep of the far-reaching horizon, and had arched above it in the limitless vault of azure” (48). Some of the stories feature young girls as heroines, providing girls of that time (and now today) with protagonists to whom they could relate. The female reader will especially enjoy Sister Anne and the Cowboy (1904), Her Neighbor’s Claim, and Kit (1914), which feature heroines of considerable pluck and re­ sourcefulness. As the reader enjoys each tale, a sense of nostalgia for that bygone era takes hold. One can’t help but notice how degraded the English language has become and how children dreamed big dreams in a time when the world was free of today’s electronic marvels and pervasive cynicism. Adventures in the West: Stories for Young Readers is laudable for its restora­ tion of these literary gems and for the scholarship of its editors, who bring to us stories that both entertain us and remind us of a time when American children were taught the resourcefulness and self-reliance of the American spirit. ...

WOZ experiments for understanding mutual adaptation

A robot that is easy to teach not only has to be able to adapt to humans but also has to be easily adaptable to. In order to develop a robot with mutual adaptation ability, we believe that it will be beneficial to first observe the mutual adaptation behaviors that occur in human–human communication. In this paper, we propose a human–human WOZ (Wizard-of-Oz) experiment setting that can help us to observe and understand how the mutual adaptation procedure occurs between human beings in nonverbal communication. By analyzing the experimental results, we obtained three important findings: alignment-based action, symbol-emergent learning, and environmental learning.

The Diffusion of WOZ: Expanding the Topology of IS Innovations

The growth and diffusion of self-service technology (SST) over the last decade has resulted in an increasing number of business and government transactions being completed without human assistance. One innovation in this area, the speech-enabled business system, is characterised by complex implementations that bring together language-processing technologies, applications development, and end-user psychology. A resulting secondary innovation, the Wizard-of-OZ (WOZ) experiment is a valuable technique for simulating and building human–machine prototypes to ensure successful deployment of the completed service. The objective of the paper is to examine these innovations in relation to the changing business landscape; the technology and innovation literature, and the population of likely adopters. The review is carried out by placing the authors’ former experience as practitioners within current theoretical research frameworks. The result is a number of suggestions relating to both IT technology research and IT innovation research. Firstly, it proposes the simplification and diffusion of the WOZ methodology to support the growth in demand for automated e-business transactions that is mindful of human and ethical challenges. Secondly, the paper argues that because SST and business extends the traditional boundaries of the customer service function, it now needs to be incorporated into Swanson's tri-core innovation typology. The paper concludes by presenting the suggested reorientation of information systems research that incorporates an outward facing perceptive as a conceptual model.

Unique collaboration finds the wizard of OZ

How academic and industry researchers joined forces to develop a much-needed antimalarial drug.

Multimedia Corpus of In-Car Speech Communication

An ongoing project for constructing a multimedia corpus of dialogues under the driving condition is reported. More than 500 subjects have been enrolled in this corpus development and more than 2 gigabytes of signals have been collected during approximately 60 minutes of driving per subject. Twelve microphones and three video cameras are installed in a car to obtain audio and video data. In addition, five signals regarding car control and the location of the car provided by the Global Positioning System (GPS) are recorded. All signals are simultaneously recorded directly onto the hard disk of the PCs onboard the specially designed data collection vehicle (DCV). The in-car dialogues are initiated by a human operator, an automatic speech recognition (ASR) system and a wizard of OZ (WOZ) system so as to collect as many speech disfluencies as possible. In addition to the details of data collection, in this paper, preliminary results on intermedia signal conversion are described as an example of the corpus-based in-car speech signal processing research.