Mean Opinion Score Values Research Articles

Pengembangan Aplikasi Text-to-Speech Bahasa Indonesia Menggunakan Metode Finite State Automata Berbasis Android

In this paper, the development of Android-based Indonesian Text-to-Speech (TTS) utilizing API Google for English TTS as a data voice has been proposed. Using this approach, a lightweight and mobile Indonesian TTS application can be realized. The utilization of English data voice is carried out by cutting off (hyphenate) Indonesian words into syllables using Finite State Automata method, then searching for equivalent syllables in English which have similar pronunciation. Using FSA method, this system is able to hyphenate all Indonesian words up to 95.19%. This application has Mean Opinion Score (MOS) value of understanding the words criteria of 3.31 and MOS value of smoothness pronunciation criteria of 3.46 (in a scale of 5).

MCL-3D: A Database for Stereoscopic Image Quality Assessment using 2D-Image-Plus-Depth Source

A new stereoscopic image quality assessment database rendered using the 2D-image- plus-depth source, called MCL-3D, is described and the performance benchmarking of several known 2D and 3D image quality metrics using the MCL-3D database is presented in this work. Nine image-plus-depth sources are first selected, and a depth image- based rendering (DIBR) technique is used to render stereoscopic image pairs. Distortions applied to either the texture image or the depth image before stereoscopic image rendering include: Gaussian blur, additive white noise, down-sampling blur, JPEG and JPEG-2000 (JP2K) compression and transmission error. Furthermore, the distortion caused by imperfect rendering is also examined. The MCL-3D database contains 693 stereoscopic image pairs, where one third of them are of resolution 1024*768 and two thirds are of resolution 1920*1080. The pair-wise comparison was adopted in the subjective test for user friendliness, and the Mean Opinion Score (MOS) were computed accordingly. Finally, we evaluate the performance of several 2D and 3D image quality metrics applied to MCL-3D. All texture images, depth images, rendered image pairs in MCL- 3D and their MOS values obtained in the subjective test are available to the public (http: //mcl.usc.edu/mcl-3d-database/) for future research and development.

CID2013: a database for evaluating no-reference image quality assessment algorithms.

This paper presents a new database, CID2013, to address the issue of using no-reference (NR) image quality assessment algorithms on images with multiple distortions. Current NR algorithms struggle to handle images with many concurrent distortion types, such as real photographic images captured by different digital cameras. The database consists of six image sets; on average, 30 subjects have evaluated 12-14 devices depicting eight different scenes for a total of 79 different cameras, 480 images, and 188 subjects (67% female). The subjective evaluation method was a hybrid absolute category rating-pair comparison developed for the study and presented in this paper. This method utilizes a slideshow of all images within a scene to allow the test images to work as references to each other. In addition to mean opinion score value, the images are also rated using sharpness, graininess, lightness, and color saturation scales. The CID2013 database contains images used in the experiments with the full subjective data plus extensive background information from the subjects. The database is made freely available for the research community.

Speech Enhancement Using Modified Modulation Magnitude Estimation-Based Spectral Subtraction Algorithm

In this paper, noise variance is estimated from the noisy spectrum using the proposed noise estimation algorithm. In addition, the speech signal is enhanced by the modified modulation magnitude estimation (MMME)-based spectral subtraction algorithm. Chi-square distribution-based noise estimation algorithm is proposed in this work. During the speech presence period, the noise variance is estimated by recursively smoothing and averaging the noisy spectrum. The speech signal is then enhanced by estimating the modulation magnitude spectrum, which depends on a posterior SNR instead of a priori SNR. The optimal speech presence probability q m is found as 0.9, and simulations are carried out for this optimal value. Objective and subjective measures for the various noises are made and compared between the existing and proposed algorithms. From the experimental results, it is observed that the proposed noise estimation method comparably reduces mean square error (MSE) and LogErr to the earlier methods under various noise conditions with different input SNR levels. In addition, the proposed spectral subtraction method noticeably increases the dB value of peak signal to noise ratio, segmental SNR and segmental SNR improvement (ΔSNRseg) contrasting various existing spectral subtraction algorithms with the optimal q m value. Also, it improves the mean opinion score value and reduces the log spectral distance and MSE when compared to that of various existing ones. Hence, it reveals that the MMME-based spectral subtraction with proposed noise estimation algorithm reduces the speech distortion and residual noise as compared to existing methods.

Image database TID2013: Peculiarities, results and perspectives

This paper describes a recently created image database, TID2013, intended for evaluation of full-reference visual quality assessment metrics. With respect to TID2008, the new database contains a larger number (3000) of test images obtained from 25 reference images, 24 types of distortions for each reference image, and 5 levels for each type of distortion. Motivations for introducing 7 new types of distortions and one additional level of distortions are given; examples of distorted images are presented. Mean opinion scores (MOS) for the new database have been collected by performing 985 subjective experiments with volunteers (observers) from five countries (Finland, France, Italy, Ukraine, and USA). The availability of MOS allows the use of the designed database as a fundamental tool for assessing the effectiveness of visual quality. Furthermore, existing visual quality metrics have been tested with the proposed database and the collected results have been analyzed using rank order correlation coefficients between MOS and considered metrics. These correlation indices have been obtained both considering the full set of distorted images and specific image subsets, for highlighting advantages and drawbacks of existing, state of the art, quality metrics. Approaches to thorough performance analysis for a given metric are presented to detect practical situations or distortion types for which this metric is not adequate enough to human perception. The created image database and the collected MOS values are freely available for downloading and utilization for scientific purposes.

Towards Layer Adaptation for Audio Transmission

When audio is transmitted over the wireless channel, the quality of the audio depends on the signal-to-noise ratio (SNR). The purpose of this paper is to investigate if rate adaptation can be avoided, and a system can rely instead on the audio encoder and decoder to alleviate the effect of channel errors. To this end, the paper reports on a set of experiments on various combinations of channel conditions, constellation sizes and audio encoding used and on the final audio quality achieved. The Mean Opinion Score (MOS) is used for performance evaluation. The MOS values are generated using the ITU-T P.862 (PESQ) and P.863 (POLQA) algorithms, and also using tests by experts. The results support the common practice of adapting the physical layer parameters to changing channel conditions. However, in some cases, it is possible to maintain a constant rate without impacting significantly the quality of the audio. This means that the complexity associated with physical layer and audio rate adaptation can be avoided leading to simpler and more robust designs.

Coding ECG beats using multiscale compressed sensing based processing

Compressed sensing recovers a sparse signal from a small set of linear, nonadaptive measurements. A sparse signal can be represented by compressed measurements with a reduced number of projections on a set of random vectors. In this paper, a multiscale compressed sensing based processing is investigated for an electrocardiogram signal which yields coded measurements. In case of an electrocardiogram (ECG) signal, the coded measurements are expected to retain the clinical information. To achieve this, compressed sensing based processing is applied at each wavelet scale and measurements are coded using Huffman coder. The measurements at each scale use random sensing matrix with independent identically distributed (i.i.d.) entries formed by sampling a Gaussian distribution. The proposed method is evaluated using pathological ECG signals from the CSE database, synthetic and normal ECGs. It helps preserve the pathological information and clinical components in compressed signal. The compressed signal quality is evaluated using standard distortion measures and mean opinion score (MOS). The MOS values for the signals range from 5% to 8.3% with a wavelet energy based diagnostic distortion (WEDD) value of 9.46% which falls under the excellent category.

QoE model for video delivered over an LTE network using HTTP adaptive streaming

The end user quality of experience (QoE) of content delivered over a radio network is (mainly) influenced by the radio parameters in the radio access network. This paper will present a QoE model for video delivered over a radio network (e.g., Long Term Evolution (LTE)) using HTTP (Hypertext Transfer Protocol) adaptive streaming (HAS). The model is based on experiments performed in the context of the Next Generation Mobile Networks (NGMN) project P-SERQU (Project Service Quality Definition and Measurement). In the first phase, a set of representative HAS profiles were selected based on a lab experiment where scenarios with typical radio impairments (fading, signal-to-interference-plus-noise ratio, round trip time and competing traffic) were investigated in a test network. Based on these HAS profiles, video files were prepared by concatenating chunks of the corresponding video quality. In a second phase, these video files were downloaded, viewed, and rated by a large number of volunteers. Based on these user scores a mean opinion score (MOS) was determined for each of the video files, and hence, the HAS profiles. Several QoE models that predict the MOS from the HAS profile have been analyzed. Using the preferred QoE model, a range of MOS values can be obtained for each set of initial radio impairments. It is argued that a QoE model based on the radio parameters is necessarily less accurate than a QoE model based on HAS profiles and an indication is given of how much the performance of the former is less than the latter.

A Study on Objective Quality Measure for Bandwidth-Extended Speech in Mobile Voice Communications

SUMMARY In this paper, we study an objective quality measure that approximates the subjective mean opinion score (MOS) for bandwidthextended wideband speech with respect to narrowband speech. Bandwidthextended speech should be widely evaluated by a subjective quality assessment such as MOS. However, such subjective quality assessments are expensive and time-consuming. This paper proposes a new objective quality measure that combines the perceptual evaluation of speech quality (PESQ) and spectral-distortion. We evaluated the correlation between our proposed scheme and MOS using AMR and AMR-WB speech codecs. The coefficient of correlation between the proposed scheme and the MOS value was found to be 0.973. We concluded that the proposed scheme is a valid and effective objective quality measure.

Image Retargeting Quality Assessment: A Study of Subjective Scores and Objective Metrics

This paper presents the result of a recent large-scale subjective study of image retargeting quality on a collection of images generated by several representative image retargeting methods. Owning to many approaches to image retargeting that have been developed, there is a need for a diverse independent public database of the retargeted images and the corresponding subjective scores to be freely available. We build an image retargeting quality database, in which 171 retargeted images (obtained from 57 natural source images of different contents) were created by several representative image retargeting methods. And the perceptual quality of each image is subjectively rated by at least 30 viewers, meanwhile the mean opinion scores (MOS) were obtained. It is revealed that the subject viewers have arrived at a reasonable agreement on the perceptual quality of the retargeted image. Therefore, the MOS values obtained can be regarded as the ground truth for evaluating the quality metric performances. The database is made publicly available (Image Retargeting Subjective Database, [Online]. Available: http://ivp.ee.cuhk.edu.hk/projects/demo/retargeting/index.html) to the research community in order to further research on the perceptual quality assessment of the retargeted images. Moreover, the built image retargeting database is analyzed from the perspectives of the retargeting scale, the retargeting method, and the source image content. We discuss how to retarget the images according to the scale requirement and the source image attribute information. Furthermore, several publicly available quality metrics for the retargeted images are evaluated on the built database. How to develop an effective quality metric for retargeted images is discussed through a specifically designed subjective testing process. It is demonstrated that the metric performance can be further improved, by fusing the descriptors of shape distortion and content information loss.

Investigation of Automatic Speech Recognition Performance and Mean Opinion Scores for Different Standard Speech and Audio Codecs

Usage of Automatic Speech Recognition (ASR) systems is increasing day-by-day for voice centric applications in mobile handheld and Voice over Internet Protocol (VoIP) devices. The necessity is also increasing to find out the ASR performance under different network impediments. Among them, speech and audio coding standards is the one, which affects the ASR performance greatly, when, using them with different sampling and bit rates in the practical systems. Another common impediment which influences the ASR accuracy is the bit errors in the wireless networks and packet drop conditions in the VoIP networks. ASR performance with some of the speech coding standards under noise conditions for the wireless networks is reported in the literature. However, each study is reporting the ASR performance for few narrowband codecs with different speech databases and different ASR toolkits like RAPHEL, HTK, SPHINX, etc. In this paper, the analysis on ASR performance while using both narrowband and wideband speech and audio coding standards, which are currently accepted for GSM mobile and VoIP networks, using the common speech database-TIMIT, and using ASR toolkit-SPHINX, is presented. The Mean Opinion Score (MOS), which is the generally accepted speech quality measurement technique, is also analyzed for all the speech and audio coding standards, using the same speech database. The results of the studies carried out for the ASR word accuracies and MOS values for different narrowband and wideband speech and audio codecs under no-loss conditions are presented. Results for different rates of packet drop condition which is the common noise scenario in wired networks such as VoIP (which is also merging with wireless networks) are also presented. The observation is that though some of the codecs are showing poor MOS performance at lower bit rates, the corresponding ASR performance is comparable with other codecs at higher bit rates.

Real-Time VoIP Quality Measurement for Mobile Devices

The quality of voice over Internet protocol (VoIP) is typically measured by mean opinion score (MOS) that significantly varies in wireless environments. Several approaches have been proposed to evaluate the MOS values of voice paths. However, these solutions require extra servers/gateways to conduct heavy computation for MOS measurement of VoIP calls. We propose two MOS measurement procedures for mobile devices. In our approach, a simple program is installed in each of the mobile devices (through an app-store like mechanism). In the perceptual evaluation of speech quality (PESQ) MOS measurement procedure, a lightweight real-time table lookup solution significantly reduces the computation time of PESQ MOS measurement from 315.4 s to 3 s. In the E-model MOS measurement procedure, the E-model MOS value can be accurately computed in 5.35 s.

Objectification of perceptual image quality for mobile video

This paper presents an objective video quality evaluation method for quantifying the subjective quality of digital mobile video. The proposed method aims to objectify the subjective quality by extracting edgeness and blockiness parameters. To evaluate the performance of the proposed algorithms, we carried out subjective video quality tests with the double-stimulus continuous quality scale method and obtained differential mean opinion score values for 120 mobile video clips. We then compared the performance of the proposed methods with that of existing methods in terms of the differential mean opinion score with 120 mobile video clips. Experimental results showed that the proposed methods were approximately 10% better than the edge peak signal-to-noise ratio of the J.247 method in terms of the Pearson correlation.

Two speaker speech separation by LP residual weighting and harmonics enhancement

This paper presents a method for separating speech of individual speakers from the combined speech of two speakers. The main objective of this work is to demonstrate the significance of the combined excitation source based temporal processing and short-time spectrum based spectral processing method for the separation of speech produced by individual speakers. Speech in a two speaker environment is simultaneously collected over two spatially separated microphones. The speech signals are first subjected to excitation source information (linear prediction residual) based temporal processing. In temporal processing, speech of each speaker is enhanced with respect to the other by relatively emphasizing the speech around the instants of significant excitation of desired speaker by deriving speaker-specific weight function. To further improve the separation, the temporally processed speech is subjected to spectral processing. This involves enhancing the regions around the pitch and harmonic peaks of short time spectra computed from the temporally processed speech. To do so the pitch estimate is obtained from the temporally processed speech. The performance of the proposed method is evaluated using (i) objective quality measures: percentage of energy loss, percentage of noise residue, the signal-to-noise ratio (SNR) gain and perceptual evaluation of speech quality (PESQ), and (ii) subjective quality measure: mean opinion score (MOS). Experimental results are reported for both real and synthetic speech mixtures. The SNR gain and MOS values show that the proposed combined temporal and spectral processing method provides an average improvement in the performance of 5.83% and 8.06% respectively, compared to the best performing individual temporal or spectral processing methods.

디지털 영상의 인지적 무참조 화질 평가 방법

In this paper, a no-reference perceptual metric is proposed for image quality assessment. It measures the amount of overall blockiness and blurring of the image and evaluates the amount of ringing, staircase, and mosaic noises around the strong edges. Finally, the individual scores are combined by a fuzzy integral to generate the final quality score of the image. The quality scores obtained by the proposed algorithm show strong relationship with the MOS(Mean Opinion Score) values by experts.

Effect of degradations' distribution in a corpus test on auditory ratings

Speech quality of telecommunications systems is usually evaluated thanks to auditory tests, which must be carried out in accordance with ITU‐T Recommendations. In these tests, subjects are asked to assess the quality of speech sample by giving a score on a five‐level scale. The averaging of subjects' scores yields the Mean Opinion Score (MOS) which represents the speech quality for a given condition. However, MOS values can be strongly influenced by many factors. In this paper, we focus on a specific bias: the distribution of the impairments in the stimuli set. In effect the rating of a narrow‐band stimulus (300‐3400 Hz) may have a lower quality score in a mixed‐band set (mixed of narrow‐band and wideband (50‐7000 Hz) conditions) than in a purely narrow band set. This effect is detailed through results of auditory tests recently carried out. Consequently, the validity of MOS value is theoretically limited within a test which prevents MOS comparison between different tests. Finally, two solutions are proposed to avoid this effect: first, an improvement of auditory tests methodology and then, a new approach to assess speech quality, based on the subject's behaviour.

Image Quality Analysis for Visible Spectral Imaging Systems

An image quality investigation of visible spectral imaging systems was performed. Spectral images were simulated using different combinations of imaging system parameters with different numbers of imaging channels, wavelength steps, and noise levels based on two practical spectral imaging systems. A mean opinion score (MOS) was determined from a subjective visual assessment scale experiment for image quality of spectral images, rendered to a three-channel LCD display. A set of image distortion measures, including color difference for color images, were defined based on image quality concerns. The relationships between the distortion factors and the combinations of parameters in spectral imaging systems are discussed in detail. The MOS values and distortion measures were highly correlated. The results indicate that the image quality of spectral imaging systems was significantly affected by the number of channels used with noise in the image capture stage. The selection of wavelength steps had no significant impact on final image quality, especially when there was no noise involved. The results also showed that the contrast factor indicates a different impact on image quality for human portraits compared to other relatively complex scene images. An empirical metric is proposed to estimate the scaled image quality. The correlation between this metric and the subjective measure, MOS, was 0.97. The results also indicate that two distortion factor eigenvectors were sufficient to represent four distortion factors used in this experiment. This suggests that further research needs to be performed to find more efficient distortion factors.