Playback Environment Research Articles

Combining online aural experimentation and physical analysis

The physical phenomenon of sound can be visualized with a high degree of freedom in terms of sound pressure level and loudness with respect to time and frequency. On the other hand, to obtain a sensory evaluation of the sound, it is a labor-intensive and costly task to prepare a certain number of subjects, set up a playback environment and playback equipment, tabulate the results, and perform subsequent analysis. The system introduced here can easily create simple listening experiments using sound source data managed on the cloud, and distribute the experiments on the Web. Subjects can participate in experiments without time and travel constraints, and experiments with an unprecedented number of subjects can be performed in a short period of time. In addition, the correlation between the rating scale and the physical quantity is automatically calculated, reducing the man-hours required for analysis. In this paper, we present the advantages and disadvantages of this system over conventional controlled subjective evaluation experiments, and mention the value of using this system based on actual examples.

Personalized and Immersive Sound Experiences Based on an Interoperable End-to-End Next Generation Audio (NGA) Chain Using the Audio Definition Model (ADM)

<italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Next-generation audio (NGA) delivers the best possible listening experience in varying situations (e.g., improved intelligibility and understanding, adapting to reproduction set-up and listening context, audio content tailored to individual preferences and needs), saving bandwidth and production efforts. These scenarios are enabled by the so-called renderer, the purpose of which is to convert a set of audio signals with associated metadata to a different configuration of audio signals (e.g., loudspeaker feeds) based on the metadata and control inputs from the playback environment and the user’s preferences. The approach of defining a specific renderer with its own metadata is extremely viable in clearly defined vertical businesses such as cinema, packaged media, and so on, but for broadcast, which is by its nature a transversal business, this is definitely not the case, so standards that describe the metadata and the behavior of the renderer become beneficial. The audio definition model (ADM) standard defined in recommendation International Telecommunications Union – Radiocommunication (ITU-R) BS.2076 is particularly relevant in this context to ensure interoperability and reproducibility along the chain. The aim of this tutorial is to describe ADM-based use cases and workflows and the ongoing efforts to promote a wide adoption and integration of the ADM</i> .

Partially Matching Projection Decoding Method Evaluation Under Different Playback Conditions

The Ambisonic technique has been widely used in fields such as sound-field reproduction, sound source localization, and acoustic noise control. In this paper, a recently proposed Ambisonic decoding method, the partially matching projection method (PMPD), was described in detail and extended for more practical use with near-field sources and reverberant playback environments. Several experiments were conducted to evaluate the performance of the method. In the experiments, we compared performances of PMPD method with three other Ambisonic decoding methods, “All-round Ambisonic Decoding” (AllRAD) method, mode matching decoding (MMD) method and regularized mode matching decoding (RMMD) method. Different playback conditions were considered to investigate the generalization of the method, including unevenly and evenly distributed loudspeakers, as well as free-field and reverberant playback environments. In terms of reproduction errors near sweep-spot and energy of loudspeaker gains, PMPD method worked well for different signals and playback environments. Whereas the other three methods encontered ill-posed problems or large reproduction errors in certain conditions. This means that PMPD method achieves a balence between precise reproductioin near sweet-spot and energy of loudspeaker gains with ill-posed problems. PMPD method has a more robust performance than the other three compared methods for different signals and playback environments. And we conclude that PMPD method is effective to be used as an Ambisonic decoding method, especially for different playback environments.

Influences of low-frequency energy and testing environment on annoyance responses to supersonic aircraft noise when heard indoors.

The research reported is part of a larger effort to develop models to predict community response to transient sounds, including sonic booms. Such models can be used along with aircraft sound predictions to guide the design of supersonic aircraft to produce generally acceptable sounds. A test was conducted to examine the influence of low frequencies on people's responses to recorded and simulated booms and other environmental transients, heard indoors over earphones. The results of this test and a companion test conducted in a sonic boom simulator were compared to see if the playback environment affected responses. Annoyance models were also examined. E-weighted Sound Exposure level (ESEL) was the sound metric most highly correlated to mean annoyance with B-weighted Sound Exposure Level (BSEL) and Perceived Level performing similarly. Predictions were improved by including Heaviness, Duration, and rate of change of Loudness in models with a loudness metric. Models were also estimated by using the average responses from both tests and metrics generated from outdoor versions of the sounds. These models also produced accurate annoyance predictions. BSEL was the best single-metric predictor, with ESEL close behind. Including Heaviness, Duration, and rate of change of Loudness resulted in R2 values as high as 0.90.

An integrated 6DoF video camera and system design

Designing a fully integrated 360° video camera supporting 6DoF head motion parallax requires overcoming many technical hurdles, including camera placement, optical design, sensor resolution, system calibration, real-time video capture, depth reconstruction, and real-time novel view synthesis. While there is a large body of work describing various system components, such as multi-view depth estimation, our paper is the first to describe a complete, reproducible system that considers the challenges arising when designing, building, and deploying a full end-to-end 6DoF video camera and playback environment. Our system includes a computational imaging software pipeline supporting online markerless calibration, high-quality reconstruction, and real-time streaming and rendering. Most of our exposition is based on a professional 16-camera configuration, which will be commercially available to film producers. However, our software pipeline is generic and can handle a variety of camera geometries and configurations. The entire calibration and reconstruction software pipeline along with example datasets is open sourced to encourage follow-up research in high-quality 6DoF video reconstruction and rendering 1 .

Automatic collection of user behavior in 360° multimedia

The arrival of 360° video to the everyday life creates the necessity of assessing both the audiovisual production and the playback environment offered to the final user. Leveraging the standard Experience API (xAPI), that considers collecting micro-interactions with e-learning content, we propose a platform to automatically collect the users’ interaction with applications based on interactive 360° multimedia. To validate the platform, we introduce an example of educational activities based on interactive 360° videos and the tools used to first, annotate these videos and convert them into interactive activities; second, to perform said activity and collect the users’ behavior via xAPI statements; and finally, to convert these statements to meaningful information in the form of user metrics and charts, both at individual level and also aggregated by activity, creating the possibility of finding singular and group behavior. This work concludes that the presented platform helps to analyze how users behave with omnidirectional interactive productions, with the aim of validating and improving its usability, ending with the discussion of future work ideas.

Automatic Sound Scene Control Using Image Sensor Network

We proposed the automatic sound scene control system using the image sensor network for preserving the constant sound scene without respect to the users' movement. In the proposed system, the image sensor network detects the human location in the multichannel playback environment and the SSC (sound scene control) module automatically controls the sound scene of the multichannel audio signals according to the estimated human location which is the angle information. To estimate the direction of the human face, we used the normalized RGB (red, green, and blue) and the HSV (hue, saturation, and value) calculated from the images obtained by the image sensor network. The direction of the human face can be easily decided as the image sensor to capture the image with the highest number of pixels to satisfy the thresholds of the normalized RGB and the HSV. The estimated direction of the human face is directly fed to the SSC module, and the controlled sound scene can be simply generated. Experimental results show that the image sensor network successfully detected the human location with the accuracy of about 98% and the controlled sound scene by the SSC according to the detected human location was perceived as the original sound scene with the accuracy of 95%.

실감형 교과서를 위한 멀티모달 콘텐츠 저작 및 재생 프레임워크 설계

가상교육 환경에 있어서 보다 효과적인 지식 전달을 위해서는 시청각적 정보에만 의존하는 기존의 학습 매체에서 탈피하여 상황에 맞는 촉각 피드백이 포함된 '실감형 교과서'의 도입이 필요하다. 그러나 저작 및 재생 환경상의 제약으로 인해 실감형 교과서를 위한 학습 콘텐츠의 확보와 활용은 아직 요원한 실정이다. 우리는 이러한 문제점에 착안하여 실감형 교과서를 위한 접근성 높은 멀티모달 학습 콘텐츠 저작 및 재생 프레임워크를 제안하였다. 본 프레임워크는 직관적인 콘텐츠 저작을 위한 스크립트 포맷과 이를 재생하기 위한 콘텐츠 재생부로 구성되어 있다. 스크립트 규격 정의 단계에서는 학습 콘텐츠에 요구되는 요소들을 규명하고 이를 반영한 XML 기반의 메타언어를 정의하였다. 그리고 콘텐츠 재생부는 작성된 콘텐츠를 해석하고 사용자로부터의 입력에 대응하여 시각 및 촉각 렌더링 루프를 통해 사용자에게 멀티모달피드백을 제공하도록 설계되었다. 이렇게 제안된 내용을 바탕으로 프로토타입을 구현하고 사용자 평가를 수행하여 본 프레임워크의 효용성을 검증하는 한편 앞으로의 개선 방향에 대해 논의하였다. For virtual education, the multimodal learning environment with haptic feedback, termed 'immersive textbook', is necessary to enhance the learning effectiveness. However, the learning contents for immersive textbook are not widely available due to the constraints in creation and playback environments. To address this problem, we propose a framework for producing and displaying the multimodal contents for immersive textbook. Our framework provides an XML-based meta-language to produce the multimodal learning contents in the form of intuitive script. Thus it can help the user, without any prior knowledge of multimodal interactions, produce his or her own learning contents. The contents are then interpreted by script engine and delivered to the user by visual and haptic rendering loops. Also we implemented a prototype based on the aforementioned proposals and performed user evaluation to verify the validity of our framework.

Efficient Low-Cost Sharing Design of Fast 1-D Inverse Integer Transform Algorithms for H.264/AVC and VC-1

In this letter, the fast one-dimensional (1-D) algorithms and their sharing design for 1-D inverse integer transforms of H.264/AVC and VC-1 are proposed by using the matrix decompositions with the sparse matrices and the matrix offset computations. The computational complexities of the proposed fast 1-D 4 times 4 and 8 times 8 inverse integer transforms for H.264/AVC are the same as those of the previous fast methods. Then the shift operations of the proposed fast 1-D inverse integer transform for VC-1 are equivalent to those of the previous fast method. For playback environments, the video decoder can support the multiple modes, which include H.264/AVC and VC-1 video standards. The proposed hardware sharing architecture requires lower hardware cost than the individual and separate design for the VLSI realization.

Solid-state coherent laser radar wind field measurement systems

Diode-pumped solid-state pulsed coherent laser radar systems have recently been developed at Coherent Technologies, Inc., for the remote measurement of atmospheric wind fields. Flashlamp-pumped systems have been utilized since 1990 for obtaining wind field measurements. These flashlamp-pumped lidar systems have been applied to wind profiling, aircraft-wake vortex measurements, airport wind shear and gust front monitoring, military cargo air drops and many other applications. The diode-pumped coherent lidar systems currently available are capable of near turnkey operation. The Tm:YAG laser transceivers operate at with output pulse energies of 1-10 mJ with PRFs of 1000 to 100 Hz, respectively. Range resolutions of 30-75 m are typical. A real-time lidar signal processor has also been developed for collecting and analysing laser radar (lidar) data. The signal processor is based on a commercial PC architecture and offers a real-time data acquisition, analysis, display, recording and playback environment. Wind measurements and overall system performance results will be presented. Wind measurement performance, for a variety of applications, is presented using the flashlamp- and diode-pumped coherent lidars including measured wind profiles from ground and on aircraft, wake vortex tracking results, and example flows over mountain terrain.

Playback of Crows of Male Japanese Quail Elicits Female Phonotaxis

We examined the response of male and female Japanese Quail (Coturnix japonica) to playback of crows recorded from unmated males and reversed crows (crows played backwards as a control noise), which were broadcast from opposite corners of a playback environment. Females showed a significant preference for the speaker broadcasting crows vs. the speaker broadcasting reversed crows, as measured by the number of time samples females were located within 1 m of the speakers and based upon the mean distance females maintained from the two speakers. Males showed no differential response to crows and reversed crows. Thus, female Japanese Quail, but not males, exhibit positive phonotaxis to the crows of males. Our data also suggest that the temporal pattern of crow syllables is an important factor eliciting female response.

Adaptive feedback techniques for synchronized multimedia retrieval over integrated networks

The authors have developed intermedia synchronization techniques for multimedia on-demand retrieval over integrated networks in the absence of global clocks. In these techniques, multimedia servers use lightweight messages called feedback units transmitted by media display sites (such as audiophones and videophones, generically referred to as mediaphones) to detect asynchronies among those sites. They present strategies by which the multimedia server can adaptively control the feedback transmission rate from that mediaphone, so as to minimize the associated overheads without permitting the asynchrony to exceed tolerable limits. They compare the performance of various resynchronization policies such as conservative, aggressive, and probabilistic. Performance evaluation of the feedback techniques indicates that their overheads are negligible; for a typical audio/video playback environment, the feedback frequency was about one in a hundred. >

Three‐dimensional audio image reconstruction

An imaging technique is developed for the three‐dimensional reconstruction of a desired sound field within a playback environment. Through proper coupling of room design and speaker placement, and deduction of the needed inputs to the system to obtain the desired outputs (for a finite length time window of data), a discrete‐sliding window algorithm is implemented to obtain the necessary time‐domain inputs. As a result of these procedures, the feedback/reverberatory effects of the playback environment are annihilated, leaving any desired sound field, while maintaining the three‐dimensionality of the original field.

Bioman—an improved occupant-crew station compliance modeling system : G. D. Frisch and L. A. D'Aulerio 1980 Aviation, Space and Environmental Medicine51, 160–167. (8 pages, 10 figures, 13 references)

Abstract : The need to visualize and interpret human body movement data from experiments and simulations has led to the development of a computerized, three-dimensional representation of tbe human body and crew station. While conventional charts and graphs can be used to follow movements of individual body parts, it has been our experience that only by observing the entire movement of the various body segments can experimental results be integrated with simulation studies. Such a process requires that program output be used to animate a realistically formed and jointed human body model incorporated within an existing or projected crew station. Animations are essential whenever the volume of data collected or generated is too great to assimilate piecemeal, or when the complexity of the motion under study leads to visualization difficulties in a two-dimensional graph. Dissatisfaction with existing body models and stick figure displays led to the development of a new human and crew station model for the computer with distinct advantages in display realism, movement definition, collision or interaction detection, and cost-effectiveness in a real-time animation play-back environment. Development of this program was meant to provide an improved method for evaluating the physical compatibility of crew members with crew stations under all types of G environ- ments.