Mean Opinion Score Research Articles

Millimeter Wave Massive MIMO Heterogeneous Networks Using Fuzzy-Based Deep Convolutional Neural Network (FDCNN)

Enabling high mobility applications in millimeter wave (mmWave) based systems opens up a slew of new possibilities, including vehicle communications in addition to wireless virtual/augmented reality. The narrow beam usage in addition to the millimeter waves sensitivity might block the coverage along with the reliability of the mobile links. In this research work, the improvement in the quality of experience faced by the user for multimedia-related applications over the millimeter-wave band is investigated. The high attenuation loss in high frequencies is compensated with a massive array structure named Multiple Input and Multiple Output (MIMO) which is utilized in a hyperdense environment called heterogeneous networks (HetNet). The optimization problem which arises while maximizing the Mean Opinion Score (MOS) is analyzed along with the QoE(Quality of Experience) metric by considering the Base Station(BS) powers in addition to the needed Quality of Service (QoS). Most of the approaches related to wireless network communication are not suitable for the millimeter-wave band because of its problems due to high complexity and its dynamic nature. Hence a deep reinforcement learning framework is developed for tackling the same optimization problem. In this work, a Fuzzy-based Deep Convolutional Neural Network (FDCNN) is proposed in addition to a Deep Reinforcing Learning Framework (DRLF) for extracting the features of highly correlated data. The investigational results prove that the proposed method yields the highest satisfaction to the user by increasing the number of antennas in addition with the small-scale antennas at the base stations. The proposed work outperforms in terms of MOS with multiple antennas.

Quadratic fitting model in no-reference image quality assessment

The perceptual quality of image is affected by distortions during compression, delivery and storage. Distortions also impact automatic image quality assessment (IQA) that needs to be highly correlated with subjective scores. In the absence of reference, which is a typical scenario in practice, no-reference (NR) metrics are necessary for quality measurements. Recently such methods are proposed, and they employ natural scene statistics (NSS). The experimental analysis performed in this paper takes into consideration two fitting or regression models of several NR-IQA metrics relying on different distortion types. The results show quadratic model as promising for making relations in terms of difference mean opinion score and Shannon entropy.

BL-JUNIPER: A CNN-Assisted Framework for Perceptual Video Coding Leveraging Block-Level JND

Just Noticeable Distortion (JND) finds the minimum distortion level perceivable by humans. This can be a natural solution for setting the compression for each video region in perceptual video coding. However, existing JND-based solutions estimate JND levels for each video frame and ignore the fact that different video regions have different perceptual importance. To address this issue, we propose a Block-Level Just Noticeable Distortion-based Perceptual (BL-JUNIPER) framework for video coding. The proposed four-stage framework combines different perceptual information to further improve the prediction accuracy. The JND mapping in the first stage derives block-level JNDs from frame-level information without the need to collect a new bock-level JND dataset. In the second stage, an efficient CNN-based model is proposed to predict JND levels for each block according to spatial and temporal characteristics. Unlike existing methods, BL-JUNIPER works on raw video frames and avoids re-encoding each frame several times, making it computationally practical. Third, the visual importance of each block is measured using a visual attention model. Finally, a proposed quantization control algorithm uses both JND levels and visual importance to adjust the Quantization Parameter (QP) for each block. The specific algorithm for each stage of the proposed framework can be changed, as long as the input and output formats of each block are followed, without the need to change other stages, based on any current or future methods, providing a flexible and robust solution. Extensive experimental results demonstrate that BL-JUNIPER achieves a mean bitrate reduction of 27.75% with a Delta Mean Opinion Score (DMOS) close to zero and BD-Rate gains of 25.44% based on MOS, compared to the baseline encoding, and also gains a better performance compared to competing methods.

Full-Reference Image Quality Expression via Genetic Programming.

Full-reference image quality measures are a fundamental tool to approximate the human visual system in various applications for digital data management: from retrieval to compression to detection of unauthorized uses. Inspired by both the effectiveness and the simplicity of hand-crafted Structural Similarity Index Measure (SSIM), in this work, we present a framework for the formulation of SSIM-like image quality measures through genetic programming. We explore different terminal sets, defined from the building blocks of structural similarity at different levels of abstraction, and we propose a two-stage genetic optimization that exploits hoist mutation to constrain the complexity of the solutions. Our optimized measures are selected through a cross-dataset validation procedure, which results in superior performance against different versions of structural similarity, measured as correlation with human mean opinion scores. We also demonstrate how, by tuning on specific datasets, it is possible to obtain solutions that are competitive with (or even outperform) more complex image quality measures.

DIA-TTS: Deep-Inherited Attention-Based Text-to-Speech Synthesizer

Text-to-speech (TTS) synthesizers have been widely used as a vital assistive tool in various fields. Traditional sequence-to-sequence (seq2seq) TTS such as Tacotron2 uses a single soft attention mechanism for encoder and decoder alignment tasks, which is the biggest shortcoming that incorrectly or repeatedly generates words when dealing with long sentences. It may also generate sentences with run-on and wrong breaks regardless of punctuation marks, which causes the synthesized waveform to lack emotion and sound unnatural. In this paper, we propose an end-to-end neural generative TTS model that is based on the deep-inherited attention (DIA) mechanism along with an adjustable local-sensitive factor (LSF). The inheritance mechanism allows multiple iterations of the DIA by sharing the same training parameter, which tightens the token-frame correlation, as well as fastens the alignment process. In addition, LSF is adopted to enhance the context connection by expanding the DIA concentration region. In addition, a multi-RNN block is used in the decoder for better acoustic feature extraction and generation. Hidden-state information driven from the multi-RNN layers is utilized for attention alignment. The collaborative work of the DIA and multi-RNN layers contributes to outperformance in the high-quality prediction of the phrase breaks of the synthesized speech. We used WaveGlow as a vocoder for real-time, human-like audio synthesis. Human subjective experiments show that the DIA-TTS achieved a mean opinion score (MOS) of 4.48 in terms of naturalness. Ablation studies further prove the superiority of the DIA mechanism for the enhancement of phrase breaks and attention robustness.

The Tacotron-Based Signal Synthesis Method for Active Sonar

The importance of active sonar is increasing due to the quieting of submarines and the increase in maritime traffic. However, the multipath propagation of sound waves and the low signal-to-noise ratio due to multiple clutter make it difficult to detect, track, and identify underwater targets using active sonar. To solve this problem, machine learning and deep learning techniques that have recently been in the spotlight are being applied, but these techniques require a large amount of data. In order to supplement insufficient active sonar data, methods based on mathematical modeling are primarily utilized. However, mathematical modeling-based methods have limitations in accurately simulating complicated underwater phenomena. Therefore, an artificial intelligence-based sonar signal synthesis technique is proposed in this paper. The proposed method modified the major modules of the Tacotron model, which is widely used in the field of speech synthesis, in order to apply the Tacotron model to the field of sonar signal synthesis. To prove the validity of the proposed method, spectrograms of synthesized sonar signals are analyzed and the mean opinion score was measured. Through the evaluation, we confirmed that the proposed method can synthesize active sonar data similar to the trained one.

A novel spectrum handoff switching decision scheme for improved performance of secondary users in cognitive radio network

SummarySpectrum handoff (SH) in the cognitive radio network (CRN) is considered as a key challenging area to enhance the performance of secondary users (SUs) in CRN. If the primary user is detected, the SU may pause and stay on the same channel or may perform SH to another idle channel. An accurate and precise handoff decision improves the overall throughput and quality of experience of end‐users. In this paper, we introduce a new SH algorithm and continuous short‐sensing strategy to improve the overall throughput of SUs. In addition, we have derived the minimum length of the target channel sequence based on network‐specific parameters like desired call dropping probability. Further, an optimum channel search time is obtained to minimize the handoff delay. The simulation result shows that the proposed scheme improves the overall throughput of CRN, and the mean opinion score of different video applications increases by 10%, 4.6%, and 1% for rapid motion, gentle walk, and slight motion types of video applications. In the case of VoIP applications, the maximum simultaneous call is improved by 2 times in the case of G.711, 1.72 times in the case of G.729, and 1.66 times in the case of iLBC.

Three-Stage Tone Mapping Algorithm

In this paper, a tone mapping algorithm is presented to map real-world luminance into displayed luminance. Our purpose is to reveal the local contrast of real-world scenes on a conventional monitor. Around this point, we propose a three-stage algorithm to visualize high dynamic range images. All pixels of high dynamic range images are classified into three groups. For the first stage, we introduce piecewise linear mapping as the global tone mapping operator to map the luminance of the first group, which provides overall impressions of luminance. For the second stage, the luminance of the second group is determined by the weighted average of its neighborhood pixels, which are derived from the first group’s pixels. For the third stage, the luminance of the third group is determined by the weighted average of its neighborhood pixels, which are derived from the second group’s pixels. Experimental results on several real-world images and the TMQI database show that our algorithm can improve the visibility of real-world scenes with about 12% and 9% higher scores of mean opinion score and tone-mapped image quality index than the closest competitive tone mapping methods. Compared to the existing tone mapping methods, our algorithm produces visually compelling results without halo artifacts and loss of detail.

End-to-End Kurdish Speech Synthesis Based on Transfer Learning

A text-to-speech (TTS) system converts the texts into speech in a specific language. Several TTS systems generate natural-like speech signals in numerous languages, such as English. On the other hand, the Kurdish language has just been examined. Existing preliminary research on Kurdish speech synthesis has utilized old methods and has generated low-quality speech. They also lack important aspects of speech, including intonation, emphasis, and rhythm. Some approaches were presented to address these challenges, including the use of concatenative systems. For example, the unit selection or statistical parametric methods. On the other hand, they need a great deal of time, effort, and domain knowledge. An additional factor for Kurdish speech synthesizers' low performance is the absence of publicly available speech corpora, unlike English, which has many freely-available corpora and audiobooks. The motivation of this paper is to create a Central Kurdish speech corpus and generate a human-like speech from the Kurdish text. This paper explains how to utilize Tacotron 2, an end-to-end neural network architecture and HiFi-GAN vocoder, to produce a high-quality, realistic, and human-like Kurdish voice. This work utilizes "text, audio" pairings, which contain 10 hours of recorded audio samples and texts collected from the Internet and textbooks. It shows how to use English character embedding as the pre-trained knowledge with Kurdish characters as input and how to preprocess these audio examples to get a great outcome. Our evaluations for various types of texts show a mean opinion score of 4.1, comparable with state-of-the-art synthesizers in other languages.

Intelligent Caching for Mobile Video Streaming in Vehicular Networks with Deep Reinforcement Learning

Caching-enabled multi-access edge computing (MEC) has attracted wide attention to support future intelligent vehicular networks, especially for delivering high-definition videos in the internet of vehicles with limited backhaul capacity. However, factors such as the constrained storage capacity of MEC servers and the mobility of vehicles pose challenges to caching reliability, particularly for supporting multiple bitrate video streaming caching while achieving considerable quality of experience (QoE). Motivated by the above challenges, in this paper, we propose an intelligent caching strategy that takes into account vehicle mobility, time-varying content popularity, and backhaul capability to improve the QoE of vehicle users effectively. First, based on the mobile video mean opinion score (MV-MOS), we designed an average download percentage (ADP) weighted QoE evaluation model. Then, the video content caching problem is formulated as a Markov decision process (MDP) to maximize the ADP weighted MV-MOS. Owing to the prior knowledge of video content popularity and channel state information that may not be available at the road side unit in practical scenarios, we propose a deep reinforcement learning (DRL)-based caching strategy to solve the problem while achieving a maximum ADP weighted MV-MOS. To accelerate its convergence speed, we further integrate the prioritized experience replay, dueling, and double deep Q-network technologies, which improve the performance of DRL algorithm. Numerical results demonstrate that the proposed DRL-based caching strategy significantly improves QoE, and achieves better video delivery reliability compared to existing non-learning approaches.

Registration of photoacoustic tomography vascular images: Comparison and analysis of automatic registration approaches

Image registration is crucial in the clinical application of photoacoustic tomography (PAT) for vascular growth monitoring. Aiming to find an optimized registration scheme for PAT vascular images acquired at different times and with varying imaging conditions, we compared and analyzed different commonly used intensity-based and feature-based automatic registration schemes. To further improve the registration performance, we proposed a new scheme that combines phase correlation with these commonly used intensity-based registration methods and compared their performances. The objective evaluation measures: peak signal-to-noise ratio (PSNR), structural similarity index metric (SSIM), root mean square error (RMSE), and quantitative visual perception (jump percentage P), as well as subjective evaluation using mean opinion score (MOS), were combined to evaluate the registration performance. Results show that the feature-based approaches in this study were not suitable for PAT image registration. And by adding phase correlation as rough registration, the overall registration performance was improved significantly. Among these methods, the proposed scheme of phase correlation combined with mean square error (MSE) similarity measure and regular-step-gradient-descent optimizer provides the best visual effect, accuracy, and efficiency in PAT vascular image registration.

Good view frames from ultrasonography (USG) video containing ONS diameter using state-of-the-art deep learning architectures.

This paper presents an automated method for detection of the diagnostically prominent frames containing optic nerve sheath (ONS) from ocular ultrasonography video using deep learning; such frames are referred to as "Good View" frames in this paper. Vivid acquisition and measurement of diagnostic features during ultrasound imaging is a challenging task; it needs a highly skilled and experienced medical expert. Automated detection of the Good View frame and the subsequent automatic measurement of optic nerve sheath diameter (ONSD), predicting elevated intracranial pressure (ICP) status, will eliminate the need for frequent intervention of a medical expert for continuous monitoring and ICP status in traumatic patients. In the presented work, the proposed model automatically detects the appropriate frames containing ONS, from an ultrasound video, by using faster region-based CNN (Faster R-CNN) object detection model. The region proposal detection network finds the ONS by using bounding boxes. In addition, three CNN-based architectures are used for its feature extraction. Finally, SoftMax classifier classifies the ONS containing Good View frame. The Inceptionv2, ResNet50, and ResNet101 architectures are then compared by utilizing the optimized learning rate and epoch parameters for the CNN model so as to provide better detection of the Good View frame. The performance of the developed module has been analyzed by proposing a grading criterion of the Good View frame. Based on the detection score and mean opinion score, an USG frame is considered a Good View for a 95-99% detection score, and this Good View frame is used for measuring the ONSD value. It is found that Faster R-CNN ResNet101 (model 3) is an optimal model in terms of sensitivity and specificity for Good View frame detection at a learning rate of 0.0003. The sensitivity and specificity of this model are obtained as 90.41 and 91.45, respectively. Furthermore, the ONSD value is measured from Good View-detected frames using an automated algorithm involving image processing and computational methods. Considering the Good View frame (detection score 95-99), the algorithm-generated ONSD values are compared with the radiologist's measured value of ONSD to validate the findings; a small percent root mean square difference (PRD) of 0.501 is found between these values, which is strong indicative of the accuracy of algorithm generated ONSD measurement using automatically detected Good View ocular USG frames.

The Combination of R&D and MDLC Models in WEB-Based Interactive Learning Media

The Technopreneur is a core course offered in semester IV (four) in the Information Systems department, Faculty of Information Technology, Unand. Based on observations, the learning process using the lecture method via video conference makes students more inclined to be subordinated. At the same time, the lecturer position tends to appear as a one-man show. The problems make students more-passive and weak in the creative power. To attract students' interest in learning, it is necessary to develop interactive learning media. This research aims to produce a WEB-based interactive learning media with a combination of R&D and MDLC model. Learning media is designed based result course learning process on the previous semester and user needs. Based on the results of system functionality testing, it shows 100% of the features are functioning properly. While testing user satisfaction on multimedia elements from 20 respondents with the Mean Opinion Score (MOS) scheme was obtained at 4.11, which means the learning media is quite good. For interactive elements, it gets a score of 4.31, which means it is included in the category ready to be applied in the Technopreneur class.

Construction of a Video Transmission Scenario in Software-Defined Networks for QoE Estimation

The services supported by data networks have become widespread, so the architectures of the new data networks are service-oriented. They are endowed with intelligence, flexibility, and programmability. The preceding is with the aim of providing acceptability by users. Thus, this paper presents the construction of a video transmission scenario over a software-defined network (SDN, Software-Defined Networking) using free software and modifying its behavior with background traffic, on which the Quality of Experience (QoE) is estimated. Subjective and objective metrics were used for the QoE estimation. For the first one, the Mean Opinion Score (MOS) was used, while the second one was studied with the Full Reference Image Quality Assessment (FR-IQA). Finally, a correlation between the two types of metrics was proposed.

NNCD-IQA: A new neural networks based compressed database for image quality assessment

Objective and subjective quality assessment is still a challenging problem in various image processing tasks. For instance, in the context of image compression, most of the conducted studies have focused on image datasets encoded using standard algorithms such as JPEG and JPEG2000. In this paper, we propose to further investigate the quality assessment issue in the presence of neural networks-based compressed images. More precisely, a new database of compressed images has been firstly built using JPEG2000 standard as well as four recent neural networks based coding schemes. Then, subjective experiments are performed to obtain the mean opinion scores of the generated distorted images. Finally, an extensive evaluation and analysis of objective image quality assessment metrics is achieved. For instance, in addition to conventional and machine learning metrics, we have considered different deep learning based models, which have been trained on our database. The new subjective database with its associated mean opinion scores as well as the learned models are publicly available at https://github.com/zakopz/NNCD-IQA-Database . The obtained results show the interest of deep learning based metrics in the context of neural networks-based compressed images.

BIQ2021: a large-scale blind image quality assessment database

The assessment of the perceptual quality of digital images is becoming increasingly important as a result of the widespread use of digital multimedia devices. Smartphones and high-speed internet are just two examples of technologies that have multiplied the amount of multimedia content available. Thus, obtaining a representative dataset, which is required for objective quality assessment training, is a significant challenge. The Blind Image Quality Assessment Database, BIQ2021, is presented in this article. By selecting images with naturally occurring distortions and reliable labeling, the dataset addresses the challenge of obtaining representative images for no-reference image quality assessment. The dataset consists of three sets of images: those taken without the intention of using them for image quality assessment, those taken with intentionally introduced natural distortions, and those taken from an open-source image-sharing platform. It is attempted to maintain a diverse collection of images from various devices, containing a variety of different types of objects and varying degrees of foreground and background information. To obtain reliable scores, these images are subjectively scored in a laboratory environment using a single stimulus method. The database contains information about subjective scoring, human subject statistics, and the standard deviation of each image. The dataset's Mean Opinion Scores (MOS) make it useful for assessing visual quality. Additionally, the proposed database is used to evaluate existing blind image quality assessment approaches, and the scores are analyzed using Pearson and Spearman's correlation coefficients. The image database and MOS are freely available for use and benchmarking.

Demystifying Content-Blockers: Measuring Their Impact on Performance and Quality of Experience

With the evolution of the online advertisement and tracking ecosystem, content-blockers have become the reference tool for improving the security, privacy and browsing experience when surfing the Internet. It is also commonly believed that using content-blockers to stop unsolicited content decreases the time needed for loading websites. In this work, we perform a large-scale study on the actual improvements of using content-blockers in terms of performance and quality of experience. For measuring it, we analyze the page size and loading times of the 100K most popular websites, as well as the most relevant QoE metrics, such as the Speed Index, Time to Interactive or the Cumulative Layout Shift, for the subset of the top 10K of them. Our experiments show that using content-blockers results in small improvements in terms of performance. However, contrary to popular belief, this has a negligible impact in terms of loading time and quality of experience. Moreover, in the case of small and lightweight websites, the overhead introduced by content-blockers can even result in decreased performance. Finally, we evaluate the improvement in terms of QoE based on the Mean Opinion Score (MOS) and find that two of the three studied content-blockers present an overall decrease between 3% and 5% instead of the expected improvement.

Parallel voice conversion with limited training data using stochastic variational deep kernel learning

There are two types of voice conversion methods: statistical and deep learning-based. Although statistical methods can train with limited data, they face challenges, including spectral oversmoothing and time-domain discontinuity. On the other hand, extensively researched deep learning-based methods rely primarily on massive amounts of data, which limits their practical applicability. Given that voice conversion is an engineering problem with limited training data, it is crucial to develop techniques that can produce satisfactory results in terms of quality and similarity in the absence of a large amount of data.This paper proposes a voice conversion model based on stochastic variational deep kernel learning (SVDKL), which works with limited training data. The model allows the use of both the deep neural network’s expressive capability and the high flexibility of the Gaussian process, which is a Bayesian and non-parametric method. The model utilizes a cascade of a deep neural network and a conventional kernel as the covariance function, which enables it to estimate non-smooth and more complex functions. Furthermore, the model’s sparse variational Gaussian process solves the scalability problem of exact inference and enables the learning of a global mapping function for the entire acoustic space. One of the most important aspects of the proposed scheme is that the model parameters are trained using marginal likelihood optimization, which takes into account both data fitting and model complexity. Considering model complexity reduces the training data by increasing the robustness to overfitting. To evaluate the proposed scheme, we examined the model’s performance with as little as approximately 80 s of training data. The results indicated that our method obtains a higher mean opinion score, smaller spectral distortion, and better preference tests than the state-of-the-art limited data methods.

Medical image fusion quality assessment based on conditional generative adversarial network.

Multimodal medical image fusion (MMIF) has been proven to effectively improve the efficiency of disease diagnosis and treatment. However, few works have explored dedicated evaluation methods for MMIF. This paper proposes a novel quality assessment method for MMIF based on the conditional generative adversarial networks. First, with the mean opinion scores (MOS) as the guiding condition, the feature information of the two source images is extracted separately through the dual channel encoder-decoder. The features of different levels in the encoder-decoder are hierarchically input into the self-attention feature block, which is a fusion strategy for self-identifying favorable features. Then, the discriminator is used to improve the fusion objective of the generator. Finally, we calculate the structural similarity index between the fake image and the true image, and the MOS corresponding to the maximum result will be used as the final assessment result of the fused image quality. Based on the established MMIF database, the proposed method achieves the state-of-the-art performance among the comparison methods, with excellent agreement with subjective evaluations, indicating that the method is effective in the quality assessment of medical fusion images.

CoCoCs: co-optimized compressive imaging driven by high-level vision.

Compressive imaging senses optically encoded high-dimensional scene data with far fewer measurements and then performs reconstruction via appropriate algorithms. In this paper, we present a novel noniterative end-to-end deep learning-based framework for compressive imaging, dubbed CoCoCs. In comparison to existing approaches, we extend the pipeline by co-optimizing the recovery algorithm with optical coding as well as cascaded high-level computer vision tasks to boost the quality of the reconstruction. We demonstrate the proposed framework on two typical compressive imaging systems, i.e., single pixel imaging and snapshot video compressive imaging. Extensive results, including conventional image quality criteria, mean opinion scores, and accuracy in image classification and motion recognition, confirm that CoCoCs can yield realistic images and videos, which are friendly to both human viewing and computer vision. We hope CoCoCs will give impetus to bridge the gap between compressive imagers and computer vision and the perception of human.