Audio Interface Research Articles

Building technology platform aimed to develop service robot with embedded personality and enhanced communication with social environment

The paper is addressed to prototyping of technology platform aimed to develop of ambient-aware human-centric indoor service robot with attributes of emotional intelligence to enhance interaction with social environment. The robot consists of a wheel-based mobile platform with spinal (segmented) torso, bi-manual manipulation system with multi-finger robot hands and robot head. Robot prototype was designed to see, hear, speak and use its multimodal interface for enhanced communication with humans. Robot is capable of demonstrating its affective and social behavior by using audio and video interface as well as body gestures. Robot is equipped with advanced perceptive system based on heterogeneous sensorial system, including laser range finder, ultrasonic distance sensors and proximity detectors, 3-axis inertial sensor (accelerometer and gyroscope), stereo vision system, 2 wide-range microphones, and 2 loudspeakers. The device is foreseen to operate autonomously but it may be also operated remotely from a host computer through wireless communication link as well as by use of a smart-phone based on advanced client-server architecture. Robot prototype has embedded attributes of artificial intelligence and utilizes advanced cognitive capabilities such as spatial reasoning, obstacle and collision avoidance, simultaneous localization and mapping, etc. Robot is designed in a manner to enable uploading of new or changing existing algorithms of emotional intelligence that should provide to robot human-like affective and social behavior. The key objective of the project presented in the paper regards to building advanced technology platform for research and development of personal robots aimed to use for different purpose, e.g. robot-entertainer, battler, robot for medical care, security robot, etc. In a word, the designed technology platform is expected to help in development human-centered service robots to be used at home, in the office, public institutions, etc.

What Are the Causes of Noncompliance Behaviors in Bar Code Medication Administration System Processes?

Healthcare information systems (e.g., Bar Code Medication Administration [BCMA] system) have been adopted to deliver efficient healthcare services recently. However, though it is seemingly simple to use (scanning barcodes before medication), users of the BCMA system (e.g., nurses and pharmacists) often show noncompliance behaviors. Therefore, the goal of this study is to comprehensively understand why such noncompliance behaviors occur with BCMA system. Through comprehensive literature review, 128 instances of causes were identified, which were categorized into five categories: Poor Visual and Audio Interface, Poor Physical Ergonomic Design, Poor Information Integrity, Abnormal Situation for System Use, and User Reluctance and Negligence. The results show that successful use of a BCMA system requires supportive systems and environments, so it is more like an issue of the system rather than that of an individual user or a device. It is believed that the proposed categories could be applicable in investigating noncompliance behaviors in other healthcare information systems as well.

Designing an Interactive Audio Interface for Climate Science

This article presents a user-centered design approach for creating an audio interface in the context of climate science. The author's team used contextual inquiry to gather information about scientists' workflows and focus groups to assess data about the scientists' specific use of language. The goal was to realize a domain-specific sonification platform and to identify climate metaphors to build a metaphoric sound identity for the sonification. In a separate set of experiments, participants were asked to pair sound stimuli with climate terms extracted from the initial interviews and to evaluate the sound samples aesthetically. They were also asked to choose sound textures (from a given set of sounds) that best express the specific climate parameter and to rate the relevance of the sound to the metaphor. The author's team assessed correlations between climate terminology and sound stimuli for the sonification tool to improve the sound design. Results show a tendency toward natural sounds by climate scientists.

Developing Mathematics Skills through Audio Interfaces for Blind Children

Developing Mathematics Skills through Audio Interfaces for Blind Children

Assisting Visually Impaired People to Acquire Targets on a Large Wall-Mounted Display

Large displays have become ubiquitous in our everyday lives, but these displays are designed for sighted people. This paper addresses the need for visually impaired people to access targets on large wall-mounted displays. We developed an assistive interface which exploits mid-air gesture input and haptic feedback, and examined its potential for pointing and steering tasks in human computer interaction (HCI). In two experiments, blind and blindfolded users performed target acquisition tasks using mid-air gestures and two different kinds of feedback (i.e., haptic feedback and audio feedback). Our results show that participants perform faster in Fitts’ law pointing tasks using the haptic feedback interface rather than the audio feedback interface. Furthermore, a regression analysis between movement time (MT) and the index of difficulty (ID) demonstrates that the Fitts’ law model and the steering law model are both effective for the evaluation of assistive interfaces for the blind. Our work and findings will serve as an initial step to assist visually impaired people to easily access required information on large public displays using haptic interfaces.

Drumkit simulator from everyday desktop objects

In this paper, an augmented reality application for drumkit simulation is presented. The system is capable of classifying any percussive sounds produced by the user from an everyday desktop environment, e.g. clapping, snapping, stroking different objects with a pencil, etc., recognizing up to six different classes of drum hits. These different types of user-generated sounds will subsequently be associated to predefined drumkit sounds, resulting in a natural and intuitive audio interface for drummers and percussionists, which only requires a computer with a built-in microphone. A set of audio features and classification techniques are evaluated for the implementation of the aforementioned system.

Development of a novel marking system for laparoscopic gastrectomy using endoclips with radio frequency identification tags: feasibility study in a canine model

Intraoperative identification of early gastric cancer is difficult to conduct during laparoscopic procedures. In this study, we investigated the feasibility and accuracy of a newly developed marking system using endoclips with radio frequency identification (RFID) tags in a canine model. RFID is a wireless near field communication technology. Among the open frequency bands available for medical use, 13.56 MHz is suitable for a surgical marking system because of the similar and linear signal decay both in air and in biological tissues. The proposed system consists of four parts: (a) endoclips with RFID tags, (b) endo-clip applier equipment, (c) laparoscopic locating probe, and (d) signal processing units with audio interface. In the experimental setting using canine models, RFID-tagged endoclips were applied to the mucosa of each dog's stomach. During the subsequent operation, the clips with RFID tags placed in five dogs were located by the detection of the RFID signal from the tag (RFID group), and the conventional clips in the other six dogs were located by finger palpation (FP group). The detected sites were marked by ablation on the serosal surface. Distance between the clips and the metal pin needles indicating ablated sites were measured with X-ray radiographs of the resected specimen. All clips were successfully detected by the marking system in the RFID group (10/10) and by finger palpation in the FP group (17/17). The medians of detection times were 31.5 and 25.0 s, respectively; the distances were 5.63 and 7.62 mm, respectively. The differences were not statistically significant. No adverse event related to the procedures was observed. Endoclips with RFID tags were located by our novel marking system in an experimental laparoscopic setting using canine stomachs with substantial accuracy comparable to conventional endoclips located by finger palpation through an open approach.

Design and evaluation of a low-cost smartphone pulse oximeter.

Infectious diseases such as pneumonia take the lives of millions of children in low- and middle-income countries every year. Many of these deaths could be prevented with the availability of robust and low-cost diagnostic tools using integrated sensor technology. Pulse oximetry in particular, offers a unique non-invasive and specific test for an increase in the severity of many infectious diseases such as pneumonia. If pulse oximetry could be delivered on widely available mobile phones, it could become a compelling solution to global health challenges. Many lives could be saved if this technology was disseminated effectively in the affected regions of the world to rescue patients from the fatal consequences of these infectious diseases. We describe the implementation of such an oximeter that interfaces a conventional clinical oximeter finger sensor with a smartphone through the headset jack audio interface, and present a simulator-based systematic verification system to be used for automated validation of the sensor interface on different smartphones and media players. An excellent agreement was found between the simulator and the audio oximeter for both oxygen saturation and heart rate over a wide range of optical transmission levels on 4th and 5th generations of the iPod Touch™ and iPhone™ devices.

Design of a Digital Audio Player Based on μc/Os and μc/GUI

At present, with the rapid development of digital audio technology, high speed and high performance digital audio processing technology faces an important issue. This paper proposes and designs a digital audio player combined with digital audio and embedded systems two cutting-edge technology. Hardware circuitry takes STM32F103VE as the main controller, configurated with VS1003 audio decoder chip and the corresponding SD card storage module, LCD module, power supply module and so on; software design transplants embedded operating system μC / OS. This paper takes advantage of μC/GUI to design audio player graphical user interface, improves the human-machine interface friendliness and aesthetics greatly.

FPGA를 이용한 SMART TV용 내장형 카메라 불량 검출 장비 개발

Recently, as the market for SMART TV expands, the camera is embedded for providing various user experience. However, this leads to occurrence of camera failure due to TV power up sequence problem, which are usually not detectable in conventional test equipments. Although the failure-detection can be possible by re-generating control signals for audio interface with new equipment, it is expensive and also requires much time to test. In this paper, for SMART TV, FPGA(Field Programmable Gate Array)-based failure-detection system is proposed which can lead to reduction of both cost and time for test.

A novel surgical marking system for small peripheral lung nodules based on radio frequency identification technology: Feasibility study in a canine model

We investigated the feasibility and accuracy of a novel surgical marking system based on radiofrequency identification (RFID) technology for the localization of small peripheral lung nodules (SPLNs) in a canine model. The system consists of 4 components: (1) micro RFID tags (13.56 MHz, 1.0×1.0×0.8 mm), (2) a tag delivery system with a bronchoscope, (3) a wand-shaped locating probe (10-mm diameter), and (4) a signal processing unit with audio interface. Before the operation, pseudolesions mimicking SPLNs were prepared in 7dogs by injecting colored collagen. By use of a computed tomographic (CT) guide, an RFID tag was placed via a bronchoscope close to each target lesion. This was then followed by scanning with the locating probe, and wedge resection was performed when possible. Operators can locate the tag by following the sound emitted by the system, which exhibits tone changes according to the tag-probe distance. The primary outcome measure was the rate of wedge resection with good margins. A total of 10 pseudolesions imitating SPLNs were selected as targets. During thoracoscopic procedures, 9 of 10 tags were detected by the system within a median of 27 seconds. Wedge resections were performed for these 9 lesions with a median margin of 11 mm. The single failure was caused by tag dislocation to the central airway. Successful localization and wedge resection of pseudolesions with appropriate margins were accomplished in an experimental setting. Our RFID marking system has future applications for accurately locating SPLNs in a clinical setting.

Audio User Interface for Visually Impaired Computer Users: in a Two Dimensional Audio Environment

In this paper we discuss a set of guidelines which could be adapted when designing an audio user interface for the visually impaired. It is based on an audio environment that is focused on audio positioning. Unlike current applications which only interpret Graphical User Interface (GUI) for the visually impaired, this particular audio environment bypasses GUI to provide a direct auditory output. It presents the capability of two dimensional (2D) navigation on audio interfaces. This paper highlights the significance of a 2D audio environment with spatial information in the context of the visually impaired. A thorough usability study has been conducted to prove the applicability of proposed design guidelines for these auditory interfaces. While proving these guidelines, previously unearthed design aspects have been revealed in this study. Keywords—Human Computer Interaction, Audio User Interfaces, 2D Audio Environment, Visually Impaired Users

Touch n' Tag: digital annotation of physical objects with voice tagging

PurposeThis article aims to explore the possibilities and use of a mobile technology‐supported audio annotation system that can be used for attaching free‐formatted audio annotations to physical objects. The solution can help visually impaired people to identify objects and associate additional information with these objects.Design/methodology/approachA human‐centred design approach was adopted in the system's development and potential end‐users were involved in the development process. In order to evaluate the emerging use cases, as well as the usefulness and usability of the application, a qualitative field trial was conducted with ten visually impaired or blind users.FindingsThe findings show that visually impaired users learned to use the application easily and found it easy and robust to use. Most users responded positively towards the idea of tagging items with their own voice messages. Some users found the technology very useful and saw many possibilities for using it in the future. The most common targets for tagging were food items; however, some users had difficulties in integrating the solution with their everyday practices.Originality/valueThis paper presents an innovative mobile phone application with a touch and audio user interface. The actual use cases describe the everyday needs of visually impaired people and this information might be valuable to service providers and technology developers. Also, the experiences gained from these trials can be used when developing software for the visually impaired on other platforms.

Method for Reading Sensors and Controlling Actuators Using Audio Interfaces of Mobile Devices

This article presents a novel closed loop control architecture based on audio channels of several types of computing devices, such as mobile phones and tablet computers, but not restricted to them. The communication is based on an audio interface that relies on the exchange of audio tones, allowing sensors to be read and actuators to be controlled. As an application example, the presented technique is used to build a low cost mobile robot, but the system can also be used in a variety of mechatronics applications and sensor networks, where smartphones are the basic building blocks.

Text Reader for Konkani Language

Text to Speech conversion and Speech Synthesis are two major areas in which lot of research work is currently carried out. In these areas the product design are majorly useful to develop Human Computer Interfaces. In the present era of Human Computer Interaction the visually impaired community in India are deprived of technologies that could help them interact with the sighted world, develop their educational infrastructure and consequently join the mainstream life through employment opportunities. This can be achieved by developing an audio interface using a text to speech system as the basic backbone. . Due to this the products are becoming major tool to inculcate the IT learning in rural part of India. We have emphasized on an Indian Language Speech (Konkani) Interface to be used in different applications that would empower the visually impaired to communicate. The main component of a text to speech processing system is speech synthesis. Speech synthesis is a process in which input text is analysed, understood and then rendered into audible speech (Text To Speech (TTS) Conversion). It is the text processing unit that accepts an arbitrary text and produces appropriate sequence of phonemic units. These phonemic units are realized by a speech generation component either by synthesis from parameters or by selection of a unit from a large speech corpus. A number of tools and resources are developed for text to speech processing. A lot of research is carried out for development of TTS systems for Indian Languages. There are a few text to speech system designed for Indian languages like Dhvani, Vani, etc which are capable of generating intelligible speech for many Indian languages (such as hindi, tamil, urdu etc.) excluding Konkani. This paper contains survey of the tool eSpeak, used for its implementation. The paper presented describes the current state of Text to Speech Conversion and Speech Synthesis techniques and issues regarding Konkani language.

Sounds Relaxing—Looks Cool

This paper considers the design of audio and visual user interface elements for pervasive computer systems that aim to support wellness, specifically for promoting calm, relaxation and for the relief of emotional stress. The methodology included conducting a survey of people’s favourite everyday sounds, as well as those they found the most annoying. The authors then took the most popular of these and correlated them with colours that people associated with those sounds. An adapted repertory grid approach was used for this exercise. The results suggest there is potential for a classification of sound and emotion on a shared scale based on the colour spectrum.

Evaluation of arrow release in highly skilled archers using an acoustic measurement system

During aiming, an archer holds the stretched bow while a small piece of metal - the clicker - presses the arrow lateral against the bow. At the end of the aiming phase, the archer pulls the arrow back until the clicker slips over the arrowhead and causes a click, and then the archer shoots. To investigate the precision of the timing of this motor program, an acoustic measurement system has been developed. The system is composed of a modular microphone (AKG C 480 B comb-ULS/61), and an external FireWire Audio Interface (Focusrite Saffire LE 24Bit / 96kHz). From the audio signal, the time of the clicker's fall, the release of the shot and the hit of the target can be detected. Seven elite archers (two males, five female) performed 30 shots (ten ends of three arrows without any time limit) at 18 m indoors. Means and coefficients of variation (CV) were calculated from average speed and clicker time, i.e. the time between the click and the release of the shot. Forward stepwise multiple linear regression analysis was done to determine mean scores. CV of the clicker time accounted for 66% of the variability in scoring. A “leave-one-out” cross validation procedure revealed consistent estimate of the model (all corrected R2 were in a range of 0.66 and 0.75 with p<0.03). It has been shown that a highly precise timing in arrow release in terms of small CVs of clicker time is important for high mean scoring.

Development of Navigation Skills Through Audio Haptic Videogaming in Learners Who are Blind

This study presents the development of a video game with audio and haptic interfaces that allows for the stimulation of orientation and mobility skills in people who are blind through the use of virtual environments. We evaluate the usability and the impact of the use of an audio and haptic-based videogame on the development of orientation and mobility skills in school-age blind learners. The results show that the interfaces used in the videogame are usable and appropriately designed, and that the haptic interface is as effective as the audio interface for orientation and mobility purposes.

Development of General Embedded Intelligent Monitoring System for Tower Crane

Development of General Embedded Intelligent Monitoring System for Tower Crane

Cost-effective solution to synchronised audio-visual data capture using multiple sensors

Applications such as surveillance and human behaviour analysis require high-bandwidth recording from multiple cameras, as well as from other sensors. In turn, sensor fusion has increased the required accuracy of synchronisation between sensors. Using commercial off-the-shelf components may compromise quality and accuracy due to several challenges, such as dealing with the combined data rate from multiple sensors; unknown offset and rate discrepancies between independent hardware clocks; the absence of trigger inputs or -outputs in the hardware; as well as the different methods for time-stamping the recorded data. To achieve accurate synchronisation, we centralise the synchronisation task by recording all trigger- or timestamp signals with a multi-channel audio interface. For sensors that don't have an external trigger signal, we let the computer that captures the sensor data periodically generate timestamp signals from its serial port output. These signals can also be used as a common time base to synchronise multiple asynchronous audio interfaces. Furthermore, we show that a consumer PC can currently capture 8-bit video data with 1024 × 1024 spatial- and 59.1 Hz temporal resolution, from at least 14 cameras, together with 8 channels of 24-bit audio at 96 kHz. We thus improve the quality/cost ratio of multi-sensor systems data capture systems.