Videophone Systems Research Articles

Voice Driven Emotion Recognizer Mobile Phone

This article proposes an application of emotion recognizer system in telecommunications entitled voice driven emotion recognizer mobile phone (VDERM). The design implements a voice-to-image conversion scheme through a voice-to-image converter that extracts emotion features in the voice, recognizes them, and selects the corresponding facial expression images from image bank. Since it only requires audio transmission, it can support video communication at a much lower bit rate than the conventional videophone. The first prototype of VDERM system has been implemented into a personal computer. The coder, voice-to-image converter, image database, and system interface are preinstalled in the personal computer. In this article, we present and discuss some evaluations that have been conducted in supporting this proposed prototype. The results have shown that both voice and image are important for people to correctly recognize emotion in telecommunications and the proposed solution can provide an alternative to videophone systems. The future works list some modifications that can be done to the proposed prototype in order to make it more practical for mobile applications.

Fast Video Segmentation Algorithm With Shadow Cancellation, Global Motion Compensation, and Adaptive Threshold Techniques

Automatic video segmentation plays an important role in real-time MPEG-4 encoding systems. Several video segmentation algorithms have been proposed; however, most of them are not suitable for real-time applications because of high computation load and many parameters needed to be set in advance. This paper presents a fast video segmentation algorithm for MPEG-4 camera systems. With change detection and background registration techniques, this algorithm can give satisfying segmentation results with low computation load. The processing speed of 40 QCIF frames per second can be achieved on a personal computer with an 800 MHz Pentium-III processor. Besides, it has shadow cancellation mode, which can deal with light changing effect and shadow effect. A fast global motion compensation algorithm is also included in this algorithm to make it applicable in slight moving camera situations. Furthermore, the required parameters can be decided automatically, which can enhance the proposed algorithm to have adaptive threshold ability. It can be integrated into MPEG-4 videophone systems and digital cameras.

Processing of audio and visual speech for telecommunication systems

Most verbal communications use cues from both the vi- sual and acoustic modalities to convey messages. During the pro- duction of speech, the visible information provided by the external articulatory organs can influence the understanding of the language, by interpreting the combined information into meaningful linguistic expressions. The task of integrating speech and image data to emu- late the bimodal human interaction system can be depicted by de- veloping automated systems. These systems have a wide range of applications such as videophone systems, where the interdepen- dencies between image and speech signals can be exploited for data compression and in solving the task of lip synchronization which has been a major problem. Therefore the objective of this work is to investigate and quantify this relationship such that the knowledge gained will assist in longer term multimedia and video- phone research. © 1999 SPIE and IS&T. (S1017-9909(99)00703-5)

Tracking of the motion of important facial features in model-based coding

A new method of tracking the position of important facial features for semantic wire-frame-based moving-image coding is presented. Reliable and fast tracking of the facial features in head-and-shoulders scenes is of paramount importance for reconstruction of the speaker's motion in videophone systems if wire-frame-based coding is used. The proposed method is based on eigenvalue decomposition of the sub-images extracted from subsequent frames of the video sequence. Motion of each facial feature (the left eye, the right eye, the nose and the lips) is tracked separately. No restrictions other than the presence of the speaker's face, were imposed on the actual contents of the scene. The algorithm was tested on widely used head-and-shoulders video sequences containing moderate head pan, rotation and zoom with remarkably good results. Tracking was maintained even when the facial features were occluded. The algorithm can also be used in other semantic-based systems.

Reliable tracking of facial features in semantic-based video coding

A new method of tracking the position of important facial features for semantic-based moving image coding is presented. Reliable and fast tracking of the facial features in head-and-shoulders scenes is of paramount importance for reconstruction of the speaker's motion in videophone systems. The proposed method is based on eigenvalue decomposition of the sub-images extracted from subsequent frames of the video sequence. Motion of each facial feature (the left eye, the right eye, the nose and the lips) is tracked separately; this means that the algorithm can be easily adapted for a parallel machine. No restrictions, other than the presence of the speaker's face, were imposed on the actual contents of the scene. The algorithm was tested on numerous widely used head-and-shoulders video sequences containing moderate head pan, rotation and zoom, with remarkably good results. Tracking was maintained even when the facial features were occluded. The algorithm can also be used in other semantic-based systems.

Frame Rate and Viseme Analysis for Multimedia Applications toAssist Speechreading

Current video conference and phone systems do not provide the necessary temporal resolution and motion for speechreading. In this paper the perceptual boundaries which effect speechreading performance are investigated. Analysis of the relationships between viseme groupings, accuracy of viseme recognition and presentation frame rate is presented based on the results of subject testing. Results reveal a minimum frame rate of 10 frames per second (fps) for distinguishing viseme groupings. Confusion analysis results demonstrate the importance of the tongue and teeth oral features for speechreading. These results are critical to the design of speech-assisted video systems to enhance speechreading for individuals with impaired hearing.

Dual-mode vector-quantized low-rate cordless videophone systems for indoors and outdoors applications

Dual-mode reconfigurable wireless videophone transceivers are proposed for noise-, rather than interference-limited indoors and outdoors applications and their video quality, bit rate, robustness, and complexity issues are analyzed. A suite of fixed, but arbitrarily programmable low-rate, perceptually weighted vector quantized (VQ) codecs with and without run-length compression (RLC) are contrived for quarter common intermediate format (QCIF) videophone sequences. The 11.36-kb/s Codec 1 is Bose-Chaudhuri-Hochquenghem (BCH) (127,71,9) coded to a rate of 20.32 kb/s and this arrangement is comparatively studied along with the 8-kb/s Codec 2 and BCH (127,50,13) scheme, which has the same 20.32-kb/s overall rate. The source-sensitivity matched Systems 1-6 characterized in a table were contrived to comparatively study the range of system design options. For example, using Codec 1 in System 1 and coherent pilot symbol assisted 16-level quadrature amplitude modulation (16-PSAQAM), an overall signaling rate of 9 kBd was yielded, if the noise-limited channel had a signal-to-noise ratio (SNR) in excess of about 22 dB in the vicinity of the basestation or in indoors scenarios. In contrast, over lower quality outdoors channels near the fringes of the cell, the more robust 4-QAM mode of operation had to be invoked, which required twice as many time slots to accommodate the resulting 18-kBd stream and hence, reduced the total number of users supported. The robustness of Systems 2-4, and 6 was increased using automatic repeat requests (ARQ), again, inevitably reducing the number of users supported, which was between 6 and 16. In a bandwidth of 200 kHz, similarly to the Pan-European GSM mobile radio system's speech channel, using Systems 1, 3, 4, or 5, for example, 16 and eight videophone users can be supported in the 16- and 4-QAM modes, respectively, while in dual-mode cells the number of users is between eight and 16. The basic system characteristics are highlighted.

Quadtree-based reconfigurable cordless videophone systems

Arbitrarily programmable, but fixed-rate quadtree (QT) decomposed, parametrically enhanced videophone codecs using quarter common intermediate format (QCIF) video sequences are proposed as a direct replacement for mobile radio voice codecs in second generation systems, such as the Pan-European GSM, the American IS-54 and IS-95, as well as the Japanese systems. The corresponding bit rates are 13, 8, 9.6 and 6.7 kb/s, respectively. As an example, the proposed 11.36 kb/s prototype Codec 1 and the 11 kb/s Codec 2 are embedded in the adaptively reconfigurable wireless videophone systems 1-4 featured in a table and their video quality, bit rate, robustness, and complexity issues are investigated. Coherent reconfigurable 16 or four-level pilot symbol assisted quadrature amplitude modulation (PSAQAM) is used and the system's robustness is improved by a combination of diversity and automatic repeat request (ARQ) techniques. When using a bandwidth of 200 kHz, as in the Pan-European GSM mobile radio system, the number of videophone users supported varies between three and 16, while the minimum required channel signal to noise ratio over Gaussian and Rayleigh channels is in excess of 6 and 8 dB, respectively, assuming a noise-limited, rather than interference-limited scenario. The salient system features are also summarized in a table.

New generation of real-time software-based video codec: Popular Video Coder II (PVC-II)

A new generation of real-time software-based video coder, the Popular Video Coder II (PVC-II), is presented.. The transform and the motion estimation parts are removed and the quantizer and the entropy coder are modified in the PVC-II as compared with the traditional video coder. Moreover, the PVC-II improves the coding performance of its previous version, the Popular Video Coder, by introducing several newly developed efficient coding techniques, such as the adaptive quantizer, the adaptive resolution reduction and the fixed-model intraframe DPCM, into the codec. The coding speed, compression ratio and picture quality of the PVC-II are good enough for applying it to various real-time multimedia applications. Since no compression hardware is needed for the PVC-II to encode and decode video data, the cost and complexity of developing multimedia applications, such as video phone and multimedia e-mail systems, can be largely reduced.

A teleconferencing system capable of multiple person eye contact (MPEC) using half mirrors and cameras placed at common points of extended lines of gaze

In this paper a novel method of providing eye contact for multiple participants in a teleconferencing system is presented. This method proposes a simple idea to fulfill the requirement for eye contact, for two or more local and remote participants simultaneously, by using half mirrors and cameras placed the common points of the extended lines of gaze of each participant. In contrast to the commonly available one to one eye contact of video phone systems, which also use half mirrors, the proposed system has the advantage of serving more than one person a given location, hence named multiple person eye contact (MPEC) teleconferencing system. We observed the resultant images by using a preliminary experiment and also by building a prototype system. The results from both of these experiments proved the merits of the proposed idea. One of the main features of this system is that, it gives any participant, a feeling of looked at if any remote participant is actually gazing him. On the other hand if none of the remote participants are looking him, then he gets the feeling of not being looked at which is also extremely important for effective inter personal communication. Furthermore, this system preserves the spatial continuity of neighboring participants, since all the participants a given location are captured by each video camera. >

Real-time software-based video coder for multimedia communication systems

A novel software-based video compression algorithm, the Popular Video Coder (PVC), is presented in this paper, and a video phone system, the Popular Phone, is also implemented based on the PVC. The PVC simplifies the traditional video coder by removing the transform and the motion estimation parts and modifies the quantizer and entropy coder. Two novel coding algorithms, the adaptive quantizer and the modified windowed Huffman-like coder, are used in the PVC to encode the video data with a quality picture at a high compression ratio. The video quality of the proposed coder is as good as that of the MPEG coder when the input is a low-resolution and slow-motion video, and the computational complexity of the PVC is much lower than that of the Motion Picture Expert Group (MPEG). Since no compression hardware is needed for the PVC to encode and decode video data, the cost and complexity of developing multimedia applications, such as video phone and multimedia e-mail systems, can be greatly reduced. Furthermore, some networking issues, such as error control and flow control, are discussed in connection with applying the PVC to implement the Popular Phone.

Fast two-dimensional inverse discrete cosine transform for HDTV or videophone systems

The authors propose a new fast two-dimensional inverse discrete cosine transform (2D-IDCT) algorithm which is developed on the basis of the DCT computational kernel matrices. With the symmetrical properties of kernel matrices, one can greatly reduce the number of multiplications. With carefully grouping the DCT coefficients, the computational complexity can be further reduced, for example, if the 2D-DCT coefficients are coded and transmitted according to the zig-zag scanning. The proposed fast 2D-IDCT algorithm which takes the advantage of zero-valued DCT elements is suitable for high throughput HDTV receiving systems. The DCT is generally recognized as the best way to encode digital picture information, because it results in virtually the same energy compact performance as the Karhunen-Loeve transform (KLT). The DCT has the advantage of lower computational complexity than the KLT.< <ETX xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">&gt;</ETX>

Contribution of display size to speech intelligibility in videophone systems : Ostberg, O., Lindstrom, B., and Renhall, P.O.Int J Human-Computer Interaction, 1989, 1.2, 149–159, 26 refs

Contribution of display size to speech intelligibility in videophone systems : Ostberg, O., Lindstrom, B., and Renhall, P.O.Int J Human-Computer Interaction, 1989, 1.2, 149–159, 26 refs

Contribution of display size to speech intelligibility in videophone systems

Widespread use of videophones has been limited due to cost factors and human factors. If demands for image size and quality were relaxed while preserving the communicational value added by visual data, low end videophones may become feasible. After a review of pertinent research, we describe an experiment indicating modest visual augmentation significantly aids speech intelligibility. The results suggest several directions for further research.

テレビ電話・テレコンファレンス特集 ＩＩ テレビ電話システム １ テレビ電話システムの概要

テレビ電話・テレコンファレンス特集 ＩＩ テレビ電話システム １ テレビ電話システムの概要

テレビ電話・テレコンファレンス特集 ＩＩ テレビ電話システム ３ 静止画テレビ電話装置

テレビ電話・テレコンファレンス特集 ＩＩ テレビ電話システム ３ 静止画テレビ電話装置

テレビ電話・テレコンファレンス特集 ＩＩ テレビ電話システム ２ テレビ電話用符号化装置

テレビ電話・テレコンファレンス特集 ＩＩ テレビ電話システム ２ テレビ電話用符号化装置