Multimedia Representation Research Articles

The development and evaluation of Chinese digital science and technology museum

Abstract There has been a considerable increase in the needs for implementation of multimedia representation technology into the museum field to develop online exhibitions and enhance museum educational functions. China Digital Science and Technology Museum (CDSTM) is such an emerging effort in this field. The purpose of CDSTM is to provide solutions to issues of unbalanced museum distribution, exhibition space limitation in China and seeking more ways to offer diverse and multimedia-rich exhibitions. This paper addresses the application of incorporating innovative and rapid digital technology in digital science and technology oriented museum development in China. Three modes of online exhibitions have been developed to emphasize user experiences through extensive application of animation and virtual reality technologies. CDSTM's e-learning functionality in promotion of scientific literacy in the public and enhancement of formal school science curriculum is also discussed. In addition, evaluation on users’ expectation and satisfaction is conducted to identify perception based on questionnaire and web logs. The feedbacks indicate that CDSTM has positive impacts on mitigating traditional science and technology museum challenges, and has the potential to promote science education and science literacy in public as well.

A Multimedia Data Management Approach with GeM-Tree

In this paper, we propose a multimedia data management framework using GeM-Tree. GeM-Tree is a multidimensional tree-based index structure which provides a generalized framework to organize and retrieve images and videos seamlessly. In addition to supporting different multimedia data types and diverse representations, the proposed data management framework supports varied multimedia retrieval strategies like content-based image and video retrieval, mixed multimedia data retrieval where cross-similarity between images and videos is considered, region-based retrieval, etc. The framework embeds a high-level semantic relationship between multimedia data objects with a construct called Hierarchical Markov Model Mediator via a novel affinity promotion technique to improve query result relevance manifold. Extensive experiments were conducted with different multimedia data types possessing varied representations, different retrieval approaches, and different data sizes. The encouraging results demonstrate a potential solution to the important and complex issue of managing a large volume of multimedia data.

An online multimedia language assistant for people with aphasia and other language barriers

Individuals with aphasia, a language disorder, are seldom able to utilize the Internet as a source of information, because their disability makes many written words incomprehensible. A similar situation exists for individuals who read and speak a second language but do not possess a vocabulary as extensive as in their native language. Pictures have been demonstrated to help in both cases. However, few existing paper or electronic picture dictionaries provide efficient, scalable, and adaptable support for looking up unknown terms encountered when browsing the web. For my thesis research, I am designing and developing an Online Multimedia Language Assistant, which allows a reader to click on an unknown word and receive interpretations of the word in as many as five different multimedia representations. The system provides a popup dictionary, images, animations and audio, where applicable. It is also adaptable (by user) and adaptive (system).

Making Practice Public: Teacher Learning in the 21st Century

We propose that the advent and ubiquity of new media tools and social networking resources provide a means for professional, networked learning to “scale up.” We preface our discussion with a review of research that has led us to argue for professional learning communities, document the policies and practices of professional development in high-achieving countries internationally that have transformed the way teachers learn, and discuss online social networking as it is being used for teacher learning. Our 10-year experience at the Carnegie Foundation for the Advancement of Teaching gave us an opportunity to learn to make multimedia representations of practice for use in both preservice and professional development. We believe that making practice public in this way can be transformative.

Multimedia Representation of Music Pieces Encoded in Symbolic Format: an Approach Based on Csound, MPEG-4, SuperCollider, CML, Chuck and IEEE 1599

The IEEE 1599 standard offers new potentialities to the most common and effective synthesis systems for the manipulation of timbres. This articles describes how sounds produced in a particular synthesis system can be managed in a complete and intuitive environment thanks by interacting with files designed according to the IEEE 1599 standard and with their contents. It is shown how easy it is to obtain the notes expressed in the syntax of a particular synthesis language starting from IEEE 1599 symbolic information. Therefore, it is possible to let IEEE 1599 files sound with any synthesis algorithm.

Learning to Look Beyond the Boundaries of Representation

[FIGURE 1 OMITTED] At first glance, the video tape of one day in Yvonne Hutchinson's classroom showed work that was effortless. Hutchinson calmly moved around among the ninth-grade students while they went over their homework and discussed their reading of A Call to Assembly, an autobiography of the jazz musician and professor Willie Ruff. There was some commotion in the classroom as Hutchinson organized a group discussion, but soon the students began talking, calling on one another, responding to one another, referring to the text they were reading, and making connections to their own lives. Hutchinson casually interjected a comment or a question here or there, but for the most part, the students seemed to be talking about the book among themselves. From the perspective of the preservice teachers from the Stanford Teacher Education Program who were watching the video, Hutchinson's classroom provided a vision of the possible--an image of what a group discussion could look like. In some ways, however, the video also provided a vision of the impossible: How could these preservice teachers, many of whom had never seen or led a group discussion in their own teaching placements, produce or even approximate the teaching moves that Yvonne Hutchinson had cultivated over a career of more than 30 years? For teacher educators and the novice teachers they seek to support, these kinds of representations of teaching provide a dual challenge: These viewers need to be able to what is there and to what is not; they need to be able to analyze the many elements of teaching and learning that are captured in video and other media, but they also need to have a sense of what those representations fail to capture---crucial details that might be obscured, larger contexts in which work may be situated, overarching purposes, histories, and long-term relationships invisible in daily interactions (Ball & Lampert, 1999). Part of this challenge involves the difficulty of analyzing the highly complex practice of leading a rich discussion. Leading a classroom discussion involves multiple components, including establishing norms for participation, assisting students in engaging in careful readings of text ahead of time, and modeling features of academic discourse. In other work, Grossman and her colleagues (Grossman et. al., 2009) refer to this as the decomposition of practice--breaking down complex practice into its constituent parts for the purposes of teaching and learning. If decomposing practice enables novices to see and supports them in enacting practice, how can multimedia records of practice illustrate both the fluid performance and the individual parts that contribute to such fluidity without making teaching seem rote or simplistic? [FIGURE 2 OMITTED] This challenge--to make teaching accessible for analysis while still capturing its complexity--serves as the focus of a digital exhibition that brings together four Web sites that represent teaching using group discussions in four different ways and contexts (A list of Web sites referred to is included at the end of this article). This overview of the exhibition describes the background of the work on these Web sites, the conceptual framework that guides the development of the Web sites and this digital exhibition, and a discussion of the exhibition and the implications for the development and exchange of these kinds of multimedia representations of teaching, and their use in teacher education, in the future. Background This exhibition grows out of work begun at the Carnegie Foundation for the Advancement of Teaching, where Hutchinson was a member of the Carnegie Academy for the Scholarship of Teaching and Learning (CASTL). (1) CASTL provided fellowships to faculty in both K-12 and higher education who had been nominated for both their excellence in teaching and their involvement in efforts to study and document their practice. …

Daisy 3: A Standard for Accessible Multimedia Books

The Daisy standard for multimedia representation of books and other material is designed to facilitate technologies that foster easy navigation and synchronized multimodal presentation for people with print-reading-related disabilities.

A Study of the Impact of Dress Model Technology on Intention to Buy on Evolving e-CRM European Union Web Sites

Introduction Electronic business (e-Business) continues to benefit from the growth in the number of consumers on the Internet, which is currently estimated to be 750 + million globally, a 10% increase from 2006 (Worldwide internet audience, 2007). The number of consumers on the Internet in America in contrast is estimated to be 150 + million, 20% of the 750 + million (Worldwide internet audience, 2007). From a 2006 global information technology index of 122 countries of the World Economic Forum, countries in Europe are considered to be applying the benefits of information and communication technology (ICT) and of the Web at competitively high levels (Scott, 2007). Though America improved to be first from sixth in the index since 2006 (O'Connor, 2007), the bulk of the countries in the highest tenth of the index is European (Scott). Emerging is an ecosystem of innovation in Europe (World's most innovative countries, 2007). Enabling benefits of e-Business and technology is the connectivity of the Web, as displayed in Figure 1, from a study of the European Commission on countries in the European Union. Networked readiness is enabling a foundation for competitiveness in business-to-consumer (B2C) and customer relationship management (CRM) in countries of the European Union. Finland, France, Germany, Poland, and the United Kingdom exceed other European Union countries in indicators of B2C of the European Commission, as displayed in Figure 2. Countries in the European Union having high networked readiness furnish benefits for consumers and firms to flourish in a B2C global economy (Carlin, 2007). Estimates indicate increased competitiveness in B2C, in electronic customer relationship management (e-CRM) interaction with customers on the Web, and in ICT by firms in the European Union in 2007-2011 (Information communication technology, 2007). This growth is contributing to creative design of Web sites that differentiate experiences for consumers in the European Union to buy on the Web, which is evidence of technology as a continued enabler in differentiation (Fitzsimmons & Fitzsimmons, 1997). Experiences have to be perceived to be fully functional, engaging, and compelling in order to convert consumers into customers (Pullman & Gross, 2004). Such experiences form a foundation for one-to-one relationship marketing and service on the Web that impact the intention of customers to continue to buy on the differentiated sites (Chen, Chen, & Kazman, 2007, pp. 2-3). Innovation in the design of buying experiences and relationships and in the integration of technology may give an edge in competitiveness to especially apparel dress model European Union Web sites, if the design and integration on the sites furnish discernable differential for the customers, which is the focus of this study. Literature Review Firms in Europe and in America continue to differentiate apparel Web sites by enhancing dress models of their products with media richness (Lawler & Joseph, 2006; Lawler, Vandepeutte, & Joseph, 2007). Literature distinguishes richness as an enabling factor in a mediated environment that affects consumer decisions to buy the products (Bezjian-Avery & Calder, 1998; Coyle & Thorson, 2001; Klein, 2003) and be loyal to the sites (Gharbi & Soltani, 2007). Richness is defined as multimedia representations of products for enhancing not only the pleasure of shopping but also the process of shopping on these sites and stimulating the senses of shoppers (Bezjian-Avery & Calder, 1998; Coyle & Thorson, 2001; Klein, 2003). Richness is evident in consumer customized 3D dynamic imaging of products on models (Olson, 2007) and in graphical representations of shoppers (Wolfendale, 2007), improved with multimedia technology. Such models and representations evolved from impersonal default 2D mannequins (Greenemeier, 2005) on apparel Web sites. …

Time-Varying Mesh Compression Using an Extended Block Matching Algorithm

Time-varying mesh, which is attracting a lot of attention as a new multimedia representation method, is a sequence of 3-D models that are composed of vertices, edges, and some attribute components such as color. Among these components, vertices require large storage space. In conventional 2-D video compression algorithms, motion compensation (MC) using a block matching algorithm is frequently employed to reduce temporal redundancy between consecutive frames. However, there has been no such technology for 3-D time-varying mesh so far. Therefore, in this paper, we have developed an extended block matching algorithm (EBMA) to reduce the temporal redundancy of the geometry information in the time-varying mesh by extending the idea of the 2-D block matching algorithm to 3-D space. In our EBMA, a cubic block is used as a matching unit. MC in the 3-D space is achieved efficiently by matching the mean normal vectors calculated from partial surfaces in cubic blocks, which our experiments showed to be a suboptimal matching criterion. After MC, residuals are transformed by the discrete cosine transform, uniformly quantized, and then encoded. The extracted motion vectors are also entropy coded after differential pulse code modulation. As a result of our experiments, 10%-18% compression has been achieved.

Multimedia representation of experiments in physics

In most physics courses using multimedia, real experiments are represented as digital video demonstrations. These time-based media have the disadvantage that students are often in the state of passive learners. Also, traditional multimedia learning environments only allow for the selection of different digitized media, but the learning process is not supported through interactions within these media. More suitable representations of real experiments in multimedia systems force students to manipulate the experimental setup directly and authentically within a digital photo. We refer to this new type of representation as an ‘interactive screen experiment’ (ISE). The question concerning the didactic ‘increase in value’ through ISEs in the teaching and learning of physics has been a central issue in didactic research for an extended time. One of our research goals is the development of multimedia components like ISEs for the purpose of creating a learning environment that relates to a practical, realistic experimental context. This paper will review our research on ISEs since 1996 and will discuss the experience gained with respect to new instructional approaches with multimedia applications in physics education.

Factors that influence pupil engagement with science simulations: the role of distraction, vividness, logic, instruction and prior knowledge

Constructivist perspectives advocate high quality visual and auditory multimedia to simulate complex and authentic situations. However, the influence of symbolic or representational learning materials on pupil engagement or learning outcomes is not clear. This paper reports on pupil engagement with two types of simulation commonly found in school science (to illustrate practical experiments or depict microscopic chemical interactions). The project pilot phase involved three 15-16 year old male pupils and a main phase involved twenty one 14-15 year old pupils. They were presented with a digital record of their ‘think aloud’ behaviour with the simulation and they were asked for retrospective comment. Pre and post surveys were also used. Distraction, vividness, logic, instruction and prior knowledge played a significant role in determining the nature of engagement and the outcome of engagement. E-assessment involving multimedia or symbolic representation in science education must take great care if it is to ensure that what it is assessing is the pupil’s science capability and not information processing skills that rely on shared symbol identification or on the ability to follow the designers’ logic of instructions. [Chem. Educ. Res. Pract., 2007, 8 (1), 1-12]

Alternate methods of conjoint analysis for estimating housing preference functions: Effects of presentation style

This paper reports the results of analysis conducted to test whether a verbal description of attribute profiles and a multimedia (virtual reality) representation in conjoint experiments generate differences in estimated utilities. Using possible extensions of a base housing design as an example, no evidence was found of significance differences in the overall internal and external validity of these alternate approaches in the present study. However, results suggest that the reliability of the measurement was better for the virtual reality experiment. In contrast, the face validity of the price attribute was better for the verbal- description-only style of presentation.

ON KNOWLEDGE GRID AND GRID INTELLIGENCE: A SURVEY

The next generation Web Intelligence (WI) aims at enabling users to go beyond the existing online information search and knowledge queries functionalities and to gain, from the Web, practical wisdom for problem solving. To support such a Wisdom Web, we envision that a grid‐like computing infrastructure with intelligent service agencies is needed, where these agencies can interact, self‐organize, learn, and evolve their course of actions, identities, and interrelationships for new knowledge creation, as well as scientific and social evolution. In this paper, we first provide an overview of recent development in WI and Semantic/Knowledge Grid. Then, the fundamental capabilities of the Wisdom Web as well as the conceptual architecture of an intelligent Grid for supporting it are described. Technical challenges for realizing Grid Intelligence are highlighted and the recent advancements in related research areas are reviewed.

Merging multimedia presentations and semistructured temporal data: a graph-based model and its application to clinical information

In this paper, we focus on the issue of providing physicians with the capability of representing in a seamless way both temporal aspects of multimedia semistructured data and their temporal presentation requirements. Semistructured data are data having some structure, that may be irregular or incomplete and does not necessarily conform to a fixed schema. Semistructured data often contain the description of histories of the considered real world. The eXtensible Markup Language (XML) is becoming a cross compatible and standardized means for representing semistructured clinical data. In the field of medical informatics, there are many ongoing activities concerning XML. In the field of multimedia database systems, the topic related to the integration of several media objects (with their temporal aspects) have been considered both for data modeling and querying issues and for modeling multimedia presentations. We first propose the Multimedia Temporal Graphical Model (MTGM), by representing a clinical database for cardiology patients undergoing cardiac angiographies and then describe it in a formal way. We deal with the problem of expressing MTGM data by XML and of managing MTGM clinical data through an XML-based system. We provide both a technique for translating (a part of) an MTGM database into an XML document and some techniques allowing us to obtain presentations defined by means of the Synchronized Multimedia Integration Language (SMIL) from MTGM presentations. MTGM allows one to represent and store clinical information in a semistructured, temporal, and multimedia database. The physician can define multimedia presentations based on the stored data. Multimedia presentations are then stored in the same MTGM database together with temporal clinical information and are thus represented according to the same data model. A prototype based on an XML native database system has been designed and implemented. In this work we have considered the theoretical and methodological issues concerning the definition of a general data model for describing temporal and multimedia features of semistructured clinical information. Other research and application oriented features, which have not been considered in MTGM, could be investigated for completing MTGM with regard to its applicability to clinical domains: MTGM does not allow one to express times at different levels of granularities, i.e. with different time units, or with indeterminacy; besides the considered valid time, it could be interesting to manage also other temporal dimensions such as the transaction and availability times. Besides being useful for managing multimedia data stored according to widely accepted standards as MPEG and DICOM, nowadays semistructured data, and XML in particular, are becoming the most important way for expressing and exchanging medical knowledge and data: MTGM can be considered as a data model allowing the seamless representation of both (multimedia and temporal) clinical data and knowledge.

Designing Dynamic Online Lessons with Multimedia Representations

The focus of this paper is on the use of multimedia representations in the online lesson. The information presented here may be of interest to online instructors with varied levels of experience, yet it is geared more toward those who are new to online teaching. A central feature of this paper is the importance of the lesson plan, which can include identification of how multimedia will be used to create instructional messages for the lesson. The potential exists to increase the quality of instruction through the careful use of static and dynamic multimedia representations. Support for the use of multiple representations is documented in the research literature. The Journal of Educators Online, Volume 2, Number 1, January 2005

An Interactive Visual Interface for Online Product Catalogs

One of the key elements of e-commerce systems is the online product catalog. It provides sellers with a content management system that assembles, aggregates, normalizes, and distributes product information. It also provides potential buyers with an interactive interface that offers a multimedia representation of the product information as well as retrieval, classification and ordering services. In this paper, we discuss the interface aspect of online product catalogs, focusing on its ability to help shoppers navigate and analyze product information. Specifically, we present a new interactive interface for online product catalogs that effectively enables the shopper to navigate through the product information space and analytically select suitable products.

MPEG-7 based description schemes for multi-level video content classification

MPEG-7 has emerged as the standard for multimedia data content description for efficiently describing multimedia content. In this context, its primary goal is to provide flexible and effective searching and retrieval of multimedia resources. Most of the earlier work on MPEG-7 description schemes (DSs) and descriptors (Ds) focuses on the description of a single multimedia document, whereas MPEG-7 can be further exploited to support more advances implementations under multimedia database systems. Therefore, it is important to reconsider issues related to high level multimedia modeling and representation, in the light of the MPEG-7 perspective. In this paper, we propose a high level multimedia representation and description scheme based on multi-level video modeling and semantic video classification. The proposed multi-level multimedia representation and DSs are expected to support more effective video content indexing and accessing operations. The presented DSs and Ds are further described by using the XML Schema language, which has been adopted as the basis of the Description Definition Language (DDL) of the MPEG-7 standard.

Spatial Multimedia Representation of Chicago Transit Authority Rail Infrastructure

When key decisions are made on capital infrastructure programs at the Chicago Transit Authority (CTA), it is often necessary to relate maps to mental or physical images of the built environment. Multimedia tools have evolved so that such queries can be addressed using off-the-shelf software combined with innovative development techniques to achieve powerful results. A spatial multimedia system was developed for CTA. Such spatial multimedia technologies can integrate spatial representations of existing or proposed urban environments with qualitative descriptions by linking maps to static and dynamic images.

The virtual customer

Communication and information technologies are adding new capabilities for rapid and inexpensive customer input to all stages of the product development (PD) process. In this article we review six web-based methods of customer input as examples of the improved Internet capabilities of communication, conceptualization, and computation. For each method we give examples of user-interfaces, initial applications, and validity tests. We critique the applicability of the methods for use in the various stages of PD and discuss how they complement existing methods. For example, during the fuzzy front end of PD the information pump enables customers to interact with each other in a web-based game that provides incentives for truth-telling and thinking hard, thus providing new ways for customers to verbalize the product features that are important to them. Fast polyhedral adaptive conjoint estimation enables PD teams to screen larger numbers of product features inexpensively to identify and measure the importance of the most promising features for further development. Meanwhile, interactive web-based conjoint analysis interfaces are moving this proven set of methods to the web while exploiting new capabilities to present products, features, product use, and marketing elements in streaming multimedia representations. User design exploits the interactivity of the web to enable users to design their own virtual products thus enabling the PD team to understand complex feature interactions and enabling customers to learn their own preferences for new products. These methods can be valuable for identifying opportunities, improving the design and engineering of products, and testing ideas and concepts much earlier in the process when less time and money is at risk. As products move toward pretesting and testing, virtual concept testing on the web enables PD teams to test concepts without actually building the product. Further, by combining virtual concepts and the ability of customers to interact with one another in a stock-market-like game, securities trading of concepts provides a novel way to identify winning concepts. Prototypes of all six methods are available and have been tested with real products and real customers. These tests demonstrate reliability for web-based conjoint analysis, polyhedral methods, virtual concept testing, and stock-market-like trading; external validity for web-based conjoint analysis and polyhedral methods; and consistency for web-based conjoint analysis versus user design. We report on these tests, commercial applications, and other evaluations.

The virtual customer

Communication and information technologies are adding new capabilities for rapid and inexpensive customer input to all stages of the product development (PD) process. In this article we review six web‐based methods of customer input as examples of the improved Internet capabilities of communication, conceptualization, and computation. For each method we give examples of user‐interfaces, initial applications, and validity tests. We critique the applicability of the methods for use in the various stages of PD and discuss how they complement existing methods.For example, during the fuzzy front end of PD the information pump enables customers to interact with each other in a web‐based game that provides incentives for truth‐telling and thinking hard, thus providing new ways for customers to verbalize the product features that are important to them. Fast polyhedral adaptive conjoint estimation enables PD teams to screen larger numbers of product features inexpensively to identify and measure the importance of the most promising features for further development. Meanwhile, interactive web‐based conjoint analysis interfaces are moving this proven set of methods to the web while exploiting new capabilities to present products, features, product use, and marketing elements in streaming multimedia representations. User design exploits the interactivity of the web to enable users to design their own virtual products thus enabling the PD team to understand complex feature interactions and enabling customers to learn their own preferences for new products. These methods can be valuable for identifying opportunities, improving the design and engineering of products, and testing ideas and concepts much earlier in the process when less time and money is at risk. As products move toward pretesting and testing, virtual concept testing on the web enables PD teams to test concepts without actually building the product. Further, by combining virtual concepts and the ability of customers to interact with one another in a stock‐market‐like game, securities trading of concepts provides a novel way to identify winning concepts.Prototypes of all six methods are available and have been tested with real products and real customers. These tests demonstrate reliability for web‐based conjoint analysis, polyhedral methods, virtual concept testing, and stock‐market‐like trading; external validity for web‐based conjoint analysis and polyhedral methods; and consistency for web‐based conjoint analysis versus user design. We report on these tests, commercial applications, and other evaluations. © 2002 Elsevier Science Inc. All rights reserved.