Objective Evaluation Results Research Articles

Sequeval: An Offline Evaluation Framework for Sequence-Based Recommender Systems

Recommender systems have gained a lot of popularity due to their large adoption in various industries such as entertainment and tourism. Numerous research efforts have focused on formulating and advancing state-of-the-art of systems that recommend the right set of items to the right person. However, these recommender systems are hard to compare since the published evaluation results are computed on diverse datasets and obtained using different methodologies. In this paper, we researched and prototyped an offline evaluation framework called Sequeval that is designed to evaluate recommender systems capable of suggesting sequences of items. We provide a mathematical definition of such sequence-based recommenders, a methodology for performing their evaluation, and the implementation details of eight metrics. We report the lessons learned using this framework for assessing the performance of four baselines and two recommender systems based on Conditional Random Fields (CRF) and Recurrent Neural Networks (RNN), considering two different datasets. Sequeval is publicly available and it aims to become a focal point for researchers and practitioners when experimenting with sequence-based recommender systems, providing comparable and objective evaluation results.

Statistical Parametric Speech Synthesis Using Deep Gaussian Processes

This paper proposes a framework of speech synthesis based on deep Gaussian processes DGPs, which is a deep architecture model composed of stacked Bayesian kernel regressions. In this method, we train a statistical model of transformation from contextual features to speech parameters in a similar manner to deep neural network DNN-based speech synthesis. To apply DGPs to a statistical parametric speech synthesis framework, our framework uses an approximation method, doubly stochastic variational inference, which is suitable for an arbitrary amount of data. Since the training of DGPs is based on the marginal likelihood that takes into account not only data fitting, but also model complexity, DGPs are less vulnerable to overfitting compared with DNNs. In experimental evaluations, we investigated a performance comparison of the proposed DGP-based framework with a feedforward DNN-based one. Subjective and objective evaluation results showed that our DGP framework yielded a higher mean opinion score and lower acoustic feature distortions than the conventional framework.

Multimodal Sensor Medical Image Fusion Based on Local Difference in Non-Subsampled Domain

Medical imaging sensors, such as positron emission tomography and single-photon emission computed tomography, can provide rich information, but each has its inherent drawbacks. In this scenario, multimodal sensor medical image fusion becomes an effective solution. The chief objective of medical imaging is to extract as much preponderant and complementary information as possible from the source into a single output that can play a critical role in medical diagnosis and clinical operations. In this paper, a novel fusion method is presented for multimodal sensor medical images, based on local difference (LD) in non-subsampled domain. In this method, the source medical images are first decomposed into low-frequency and high-frequency subimages, via non-subsampled schemes. Then, the coefficients of sub-bands are fused by an operator, called LD. The final fused image is reconstructed, via the inverse non-subsampled schemes, with all composite coefficients. The proposed fusion method was applied in several clinical studies, and the results show that it is a much more straightforward and effective method than some of the state-of-the-art methods, in terms of both subjective visual performance and objective evaluation results. Also, the performance of the proposed method was compared with that of two non-subsampled schemes, namely, non-subsampled contourlet transform and non-subsampled shearlet transform.

Hybrid Subjective and Objective Evaluation Method of the Equipment for First Class Distribution Network

This paper presents a hybrid subjective and objective evaluation method of the equipment for first-class distribution network. Firstly, it analysis the difficulties of the equipment selection in the current first-class distribution network. Secondly, it deeply analysis the influencing factors of equipment selection in the first-class distribution network. The comprehensive evaluation indices system of first-class power distribution network equipment is constructed from five aspects including economy, performance, intelligence, coordination, and standardization of equipment, in order to the features of the equipment for power distribution network, such as diversity of equipment functions, diverse evaluation indicators, various types, and varied access methods. The subjective weighted methods are highly subjective and difficult to fully express the effectiveness of weights for evaluating indicators. The objective weighted methods are easy to lose key information due to limited sample of measurement data. A hybrid subjective and objective weighting method is proposed to combined with principal component analysis and analytic hierarchy process, and the relative entropy principle is used to weight distribution of subjective and objective evaluation results. Finally, the feasibility of the proposed method is verified through the first-class distribution network equipment in a province in southern China. This paper provides reference for comprehensive evaluation of first-class distribution network equipment.

Comprehensive evaluation of renewable energy technical plans based on data envelopment analysis

Nowadays different kinds of renewable energy resources have got rapid development. For renewable energy planning, it is meaningful to assess the comprehensive performance of different candidate renewable energy schemes and distinguish the optimal. In view of this, based on data envelopment analysis(DEA), this paper presents a comprehensive evaluation method of renewable energy technical plans. Firstly, multiple indexes from different aspects are taken into consideration to establish the index system. And in the index processing stage, intuitionistic fuzzy number theory is adopted to describe the fuzziness and ambiguity in qualitative index and convert it into quantitative calculation. Then, super-efficiency DEA model is used to determine the relative ranking of different plans based on the concept of relative efficiency. Finally, a real case of renewable energy development in 2020 from a province in China is used to illustrate the feasibility and effectiveness of the proposed method in this paper. Results show that this proposed method can overcome the shortcoming of traditional basic DEA model and get more objective evaluation results and provide beneficial references for renewable energy planning and development.

Grey Clustering Evaluation of Water Resources Carrying Capacity Based on Triangle Whitening Weight Function

Water resources are an important component of the resource environment. The sustainable carrying capacity of water resources is an essential indicator for the comprehensive evaluation of regional sustainable development. This paper will select 17 water resources carrying capacity impact factors to establish a sustainable water resources assessment system. The grey clustering evaluation based on the triangle whitening weight function is used to obtain the objective evaluation results of water resources carrying capacity.

Sound quality evaluation and prediction for the emitted noise of axial piston pumps

Sound quality has been an important attribute closely related to products’ competitiveness. A neural network model is proposed to predict the sound quality of an axial piston pump based on the objective and subjective sound quality evaluations. Five psychoacoustic metrics of the objective evaluation and six adjective pairs of semantic evaluation method are utilized. The influences of the speed and outlet pressure on the objective and subjective evaluation results are compared. The correlations between the objective and subjective evaluation results are analyzed. The results show that the A-weighted sound pressure levels (AWSPLs) and loudness of the sound take on increasing with the increase of the speed and outlet pressure. There are strong correlations between the AWSPL and loudness and the perceptions of “unpleasant-pleasant” and “like-dislike”, while the correlations between the fluctuation strength and the perception of “unpleasant-pleasant” and “like-dislike” are moderate, and the correlations between the roughness and sharpness and the perception of “unpleasant-pleasant” and “like-dislike” are weak. In addition, the results show that the sound quality model can predict the sound quality of axial piston pumps with good performance. This study lays the foundation for sound quality improvement of hydraulic displacement pumps.

Contrast-enhanced fusion of infrared and visible images

The fusion of infrared and visible images may result in low contrast, which is unsuitable for observation by human eyes. Thus, we propose a contrast-enhanced fusion algorithm with nonsubsampled shearlet transform (NSST) frames, in which the NSST is first employed to decompose each of the source images into one low frequency sub-band and a series of high frequency sub-bands. To improve the fusion performance, we designed two measures for fusion of the low frequency and the high frequency: the low frequency is divided into salient and nonsalient regions in accordance with the human visual system to improve the global contrast by targeted fusion and the high frequency requires a local contrast fusion strategy. Finally, the merged sub-bands are constructed according to the selection principles, and the final fused image is produced by applying the inverse NSST on these merged sub-bands. Experimental results demonstrate the effectiveness and superiority of the proposed method over the state-of-the-art fusion methods in terms of both visual effect and objective evaluation results.

PENGHAMPIRAN ”NEO FUZZY NEURON” WACANA DALAM RANCANGAN KONSEPSUAL SISTEM PENILAIAN KARYAWAN

Computation and information technology progress in the last three decades gives real impact for management practice development. Currently some uncertainties in receiving information on management policy response is easier to be predicted. in terms of staff evaluation, inaccuracy of staff performance evaluation due to lack of good instrument caused low guarantee of objective evaluation result and free of un-bias evaluator. Neo Fuzzy Neuron approach is a thinking method which orient to human intellectual concept and to computation and information technology. This offer an alternative to predict staff behaviour evaluation. As a thinker concept, this approach can be ued as a base in developing a decision to support system in evaluating staff performance objectively and accurately; and has quite good operational feasibility.

A systematic approach for public-private partnership projects CSFs evaluation

PurposePublic–private partnership (PPP) projects are increasingly significant in many countries. The purpose of this paper is to assess the impact of critical success factors (CSFs) on PPP projects and comprehensively consider the interrelations and interaction among dimensions and factors to achieve a better understanding of PPP project management.Design/methodology/approachAn evaluation index system for PPP projects such as the presented case study is proposed based on a literature review and a survey. Then, interpretative structural modeling is used to transform the CSFs dimension into a multi-level hierarchical model to reflect the driven-dependency relation of each dimension; the fuzzy analytic network process model optimized by moment estimation theory is used to investigate the impact of CSFs by considering their internal impact.FindingsRegarding the project used as the case study, the driving force and dependence for driving layer and dependent layer are determined. Moreover, in driving layer, efficient and well-structured payment mechanism is the most important CSF if considering the internship and interaction among CSFs, and efficient and well-structured payment mechanism and good governance provide most positive interaction; in dependent layer, population of beneficiaries is the most important CSF if considering the internship and interaction among CSFs, and public client’s satisfaction provides most positive interaction.Originality/valueThis paper developed an evaluation model to explore the interrelationships of dimensions and factors and then determine the impact of CSFs. The model propose in this paper relaxes the independence assumptions of traditional methods and is more in line with reality; besides, weighting method is optimized to obtain more objective and reasonable evaluation results. Through an empirical study, the validity of the model has been verified; therefore, the study can help project stakeholders better understand the CSFs and further improve project performance.

Human vision system-based structural similarity model for evaluating seismic image quality

The quality of a seismic section is generally evaluated based on relevant qualitative and quantitative criteria. We have developed a new quantitative model to evaluate seismic sections, i.e., a full-reference human visual system based on seismic image structural similarity. With this model, we compared and analyzed two seismic sections before and after data processing in terms of energy intensity, contrast, and seismic reflection configuration similarity measures. We evaluated the quality of the seismic section in combination with the three measures, and their changing trends before and after data processing are represented with relevant quantitative indicators. The numerical simulation used algorithms and actual data, and the results indicate that compared with conventional evaluation methods, our method is easier to understand and simpler to calculate. Our method can also highlight the differences between changing seismic sections and improve the consistency of the objective evaluation results of seismic images with the corresponding human subjective perception.

Deep Learning-Based Noise Reduction Approach to Improve Speech Intelligibility for Cochlear Implant Recipients.

We investigate the clinical effectiveness of a novel deep learning-based noise reduction (NR) approach under noisy conditions with challenging noise types at low signal to noise ratio (SNR) levels for Mandarin-speaking cochlear implant (CI) recipients. The deep learning-based NR approach used in this study consists of two modules: noise classifier (NC) and deep denoising autoencoder (DDAE), thus termed (NC + DDAE). In a series of comprehensive experiments, we conduct qualitative and quantitative analyses on the NC module and the overall NC + DDAE approach. Moreover, we evaluate the speech recognition performance of the NC + DDAE NR and classical single-microphone NR approaches for Mandarin-speaking CI recipients under different noisy conditions. The testing set contains Mandarin sentences corrupted by two types of maskers, two-talker babble noise, and a construction jackhammer noise, at 0 and 5 dB SNR levels. Two conventional NR techniques and the proposed deep learning-based approach are used to process the noisy utterances. We qualitatively compare the NR approaches by the amplitude envelope and spectrogram plots of the processed utterances. Quantitative objective measures include (1) normalized covariance measure to test the intelligibility of the utterances processed by each of the NR approaches; and (2) speech recognition tests conducted by nine Mandarin-speaking CI recipients. These nine CI recipients use their own clinical speech processors during testing. The experimental results of objective evaluation and listening test indicate that under challenging listening conditions, the proposed NC + DDAE NR approach yields higher intelligibility scores than the two compared classical NR techniques, under both matched and mismatched training-testing conditions. When compared to the two well-known conventional NR techniques under challenging listening condition, the proposed NC + DDAE NR approach has superior noise suppression capabilities and gives less distortion for the key speech envelope information, thus, improving speech recognition more effectively for Mandarin CI recipients. The results suggest that the proposed deep learning-based NR approach can potentially be integrated into existing CI signal processors to overcome the degradation of speech perception caused by noise.

Sound Field Reproduction of Focused Virtual Sources with Symmetry Sigmoid Regularization Function

Reproduction of focused sound sources is a fundamental problem in acoustic signal processing. Based on the cylindrical harmonic expansion of higher order Ambisonics (HOA), a novel method for focused sound source reproduction is proposed. To restrict the near-field distortion caused by higher order harmonic components of focused sound sources, a Symmetry sigmoid regularization function (SSRF) is designed, which exhibits a more suitable attenuation characteristic. By using SSRF, we first give the generalized cylindrical harmonic expansion of focused sound sources. The concept of continuous loudspeaker is utilized over a circular array to generate the discrete loudspeaker driving function. Based on the wideband error analysis, the match relationship between the SSRF and the variance pair (i.e., the operating frequency and the expansion order) is established. Objective evaluation results show that the near-field distortion is eliminated significantly by the proposed method. The reproduction errors of interior/exterior sound field are evidently lower than the state-of-the-art techniques. The subjective evaluations also confirm that the proposed approach is able to achieve higher reproduction quality compared to the referenced methods.

PLC: A simple and semi-physical topographic correction method for vegetation canopies based on path length correction

Rugged terrain distorts optical remote sensing signals, and land-cover classification and biophysical parameter retrieval over mountainous regions must account for topographic effects. Therefore, topographic correction is a prerequisite for many remote sensing applications. In this study, we proposed a semi-physically based and simple topographic correction method for vegetation canopies based on path length correction (PLC). The PLC method was derived from the solution to the classic radiative transfer equation, and the influence of terrain on the radiative transfer process within the canopy is explicitly considered, making PLC physically sound. The radiative transfer equation was simplified to make PLC mathematically simple. Near-nadir observations derived from a Landsat 8 Operational Land Imager (OLI) image covering a mountainous region and wide field-of-view observations derived from simulation using a canopy reflectance model were combined to test the PLC correction method. Multi-criteria were used to provide objective evaluation results. The performances were compared to that of five other methods: CC, SCS + C, and SE, which are empirical parameter-based methods, and SCS and D-S, which are semi-physical methods without empirical parameter. All the six methods could significantly reduce the topographic effects. However, SCS showed obvious overcorrection for near-nadir observations. The correction results from D-S showed an obvious positive bias. For near-nadir observations, the performance of PLC was comparable to the well-validated parameter-based methods. For wide field-of-view observations, PLC obviously outperformed all other methods. Because of the physical soundness and mathematical simplicity, PLC provides an efficient approach to correct the terrain-induced canopy BRDF distortion and will facilitate the exploitation of multi-angular information for biophysical parameter retrieval over mountainous regions.

Correlation Between Objective Evaluation Result of Nasal Congestion and Life Quality in Patients with Acute Rhinosinusitis.

Rhinosinusitis is an inflammatory process involving the nasal mucosa and paranasal sinuses. Rhinitis and sinusitis generally occur simultaneously and thus the current terminology used is rhinosinusitis. The blind nose can negatively affect the overall quality of life including the physical or emotional condition of the sufferer and disruption to work or school (reduced productivity and difficulty in concentrating). To analyse correlation between objective evaluation results of nasal congestion through NIPF and life quality based on sinonasal outcome Test-20 score in patients with acute rhinosinusitis. The study was conducted at OtolaryngologyOutpatient Clinic at Dr. Soetomo General Hospital Surabaya. The time of study began in July 2011 until the minimum sample size was met. The sample of the study was patients diagnosed with acute rhinosinusitis treated at OtolaryngologyOutpatientClinic at Dr. Soetomo General Hospital Surabaya and meet the research criteria. Based on statistical analysis with Spearman rank correlation test, correlation between NIPF value and SNOT-20 score on 16 samples got correlation coefficient-0.310 and p = 0.243. The scatter diagram showed the NIPF value variables and the SNOT-20 scores being scattered in uneven spots and erratic patterns. This indicated that an objective evaluation of nasal congestion based on NIPF and subjective assessment of quality of life based on SNOT-20 score was not obtained correlation (p > 0.05). The results of the objective nasal obstruction examination based on Nasal Inspiratory Peak Flow have no correlation with subjective subjective assessment of quality of life based on Sino-Nasal Outcome Test-20 score in patients with acute rhinosinusitis.

Evaluation of Farmland Drainage Water Quality by Fuzzy–Gray Combination Method

The complex relationships between indicators and water conditions cause fuzzy and gray uncertainties in evaluation of water quality. Compared to conventional single-factor evaluation methods, the combination evaluation method can consider these two uncertainties to produce more objective and reasonable evaluation results. In this paper, we propose a combination evaluation method with two main parts: (1) the use of fuzzy comprehensive evaluation and gray correlation analysis as submodels with which to consider the fuzzy and gray uncertainties and (2) the establishment of a combination model based on minimum bias squares. In addition, using this method, we evaluate the water quality of a ditch in a typical rice–wheat system of Yixing city in the Taihu Lake Basin during three rainfall events. The results show that the ditch water quality is not good and we found the chemical oxygen demand to be the key indicator that affects water quality most significantly. The proposed combination evaluation method is more accurate and practical than single-factor evaluation methods in that it considers the uncertainties of fuzziness and grayness.

Spiral-Like Pixel Reconstruction Algorithm for Spatiotemporal Video Error Concealment

This paper proposes efficient error concealment (EC) algorithms that use a spiral-like pixel reconstruction (SPR) scheme on the H.264/AVC joint model simulation platform. The algorithms provide low complexity and high accuracy, and their subjective and objective evaluation results are superior to those of extant EC algorithms. First, edge matching is applied to the boundary of lost macroblocks (MBs). Second, the directional edge group with the highest magnitude is selected and symmetric pixel referencing is performed along its orthogonal symmetry axis. Finally, the lost MBs are estimated using the novel SPR scheme with reference mode selection according to the edge ratio. Experimental results revealed that, compared with the extant edge-oriented PR approaches of EC, the proposed total- and partial-spiral ordering modes in pixel referencing can be used to reconstruct lost MBs, with a desirable decoder peak signal-to-noise ratio and high visual quality.

Research on a new teaching quality evaluation method based on improved fuzzy neural network for college English

For the heavy workload and complicated statistics of college English teaching work, the progress and limitations of neural network, and the existing characteristics of fuzzy information, fuzzy logic and RBF neural network are introduced to integrate the advantages of learning, association, identification, adaptation and fuzzy information processing to propose an improved fuzzy RBF neural network model based on back-propagation learning. Then the teaching quality evaluation method of college English based on improved fuzzy neural network is proposed to obtain the more objective and reasonable evaluation result. To test the effectiveness of the teaching quality evaluation method, the college English teaching in Henan University of Chinese Medicine is selected as study case. The results show that the proposed teaching quality evaluation method can effectively overcome the subjectivity and randomness of the traditional teaching quality evaluation methods, and make the evaluation results more in line with the actual situation.

A Spectra-Based Equalization-Generation Combined Framework for Throat Microphone Speech Enhancement

Non-acoustic sensors are widely used in speech signal processing tasks, and their immunity to the background acoustic noise shows great benefits to traditional speech enhancement. To avoid using acoustic speech disturbed by strong noise, spectra mapping from throat microphone (TM) speech to acoustic microphone (AM) speech has been studied. However, there is a distinguished difference between the spectra of the two kinds of speech, and the mapping relationship is different in the low-band and high-band spectra, which limits the performance of the traditional full-band spectra mapping model. In this paper, to improve the perceived quality and intelligibility of TM speech, we investigate the low-band and high-band spectral structure between TM and AM speech, respectively, and propose a spectra-based band-division mapping framework for TM speech enhancement based on the investigation. In the framework, the low-band target spectra of AM speech are mapped based on the equalization method, and the high-band spectra are mapped from the full-band TM speech spectra, which are lack of high-frequency components. The overall framework can be viewed as a combination of spectra equalization in the low band and spectra generation in the high band. Both the objective and subjective evaluation results show clear advantages over the existing TM speech enhancement method based on the full-band spectra mapping.

An Integrated Tone Mapping for High Dynamic Range Image Visualization

There are two type tone mapping operators for high dynamic range (HDR) image visualization. HDR image mapped by perceptual operators have strong sense of reality, but will lose local details. Empirical operators can maximize local detail information of HDR image, but realism is not strong. A common tone mapping operator suitable for all applications is not available. This paper proposes a novel integrated tone mapping framework which can achieve conversion between empirical operators and perceptual operators. In this framework, the empirical operator is rendered based on improved saliency map, which simulates the visual attention mechanism of the human eye to the natural scene. The results of objective evaluation prove the effectiveness of the proposed solution.