End-user Programming of Ambient Narratives for Smart Retail Environments

Abstract

Ambient Intelligence is a vision on the future of the consumer electronics, telecommunications and computer industry that refers to electronic environments that respond to the presence and activity of people and objects. The goal of these intelligent environments is to support the performance of our everyday activities using technology and media in way that puts users in control. Many research prototypes and demonstrations of ambient intelligence systems and applications have been developed since the introduction of this vision, but many of these examples focus on a relatively small application domain and set of functionality. The downside of this reductionist approach is that it surpasses the open-ended dynamic nature and complexity that is inherent to social environments. This thesis aims to find a generic interaction concept to capture the way we form experiences in our everyday life and integrates technology and media into that process. It proposes the design of an end-user programming environment that supports retail designers without special programming skills to create, simulate and deploy smart retail environments within this interaction concept. To derive such a generic interaction concept it is necessary to look at the social, cultural and economical factors that shape the ambient intelligence landscape to better understand how ambient intelligence helps people in performing their everyday life activities and rituals. This analysis shows that notions like play and performance are not just seen on stage in the theatre: Everywhere around us people perform culturally defined social scripts, e.g. in court, in a restaurant or on the street. Social interaction itself in fact can be seen as an improvised performance that takes shape through the interaction of people with their surroundings. This implies technology can be applied to detect these social scripts and in turn affect the environment to improve the performance. This can be found back in shopping malls and shops for example: The shop employees perform a service for the customer in which the shop itself is seen as the stage and the increasingly interactive, dynamic lighting, audio, video the special effects to enhance the shopping experience for customers. In this experience economy, next to the multi-sensory trend also a development towards co-creation environments can be seen in which the consumer becomes an active producer in the construction of the experience that is offered to him. By looking at such co-creation environment from a literary, semiotic perspective they can be considered as interactive narratives consisting of interrelated social scripts with associated device actions. Through interaction with this possibly dynamically changing ambient narrative, i.e. by performing social scripts, players or readers construct their own story or ambient intelligence experience. To apply this ambient narrative concept in practice in the domain of retail, user research was conducted through interviews and workshops with professional users, i.e. retailers, designers and consultants to discover the types of interactive multi-sensory retail experiences these people would want to see and the requirements placed by this group on an end-user programmable ambient narrative system. From this research we learned that designers had a preference for a 3D simulation on a PC in their office to design ambient narratives, while retailers preferred a PDA version to adjust the retail experience on location. Furthermore, a list of thirty requirements was derived that can be grouped into four categories: ambient narrative concept implementation, runtime system performance, functionality end-user programming environment and system extensibility. On the basis of these findings, a formal model was defined to describe the problem of reconstruction ambient intelligence from the (dynamically changing) modular fragments of which a (dynamic) ambient narrative consists. Each fragment consists of a description of the social script and the actions on devices that are coupled to this script. The author of the ambient narrative can specify which fragments may be activated when by setting event triggers in the action section of another fragment. This model can be represented in a hypertext model in which every fragment is a node and each event trigger a link that connects nodes. An ambient narrative engine continuously sequences these fragments based on contextual information, session state and user feedback into a coherent story that is rendered by the devices that surround people. Next to the design and implementation of the ambient narrative engine, the functional user requirements were used to compose a system architecture for an intelligent shop window that supports the entire lifecycle of ambient narratives, from the initial design in a 3D simulation until the modification of fragments in-situ with the PDA. The intelligent shop window prototype with authoring environment was realized and evaluated in ShopLab at the High Tech Campus in Eindhoven on three main criteria: usability, run-time performance and extensibility. To test the usability a user study was conducted in which the participants were asked to perform four different programming tasks with this prototype tool and fill in a questionnaire afterwards with both questions on the intuitiveness, expressive power and efficiency of both authoring tools. From the results of this study we conclude the target user group is able to design intelligent shop window ambient narratives with this prototype system. An observation-in-use experiment revealed the response time of the system was satisfactory in the majority of cases but in some special cases could lead to long response times as a result of the current implementation of the ambient narrative engine. In terms of system architecture and extensibility towards other applications and domains, the strongest restrictions were placed by the assumed fixed sensor infrastructure and partial implementation of the formal ambient narrative model in the prototype.