Subjective Evaluation Metrics Research Articles

Examining the Impact of Digital Human Gaze Expressions on Engagement Induction.

With advancements in technology, digital humans are becoming increasingly sophisticated, with their application scope widening to include interactions with real people. However, research on expressions that facilitate natural engagement in interactions between real people and digital humans is scarce. With this study, we aimed to examine the differences in user engagement as measured by subjective evaluations, eye tracking, and electroencephalogram (EEG) responses relative to different gaze expressions in various conversational contexts. Conversational situations were categorized as face-to-face, face-to-video, and digital human interactions, with gaze expressions segmented into eye contact and gaze avoidance. Story stimuli incorporating twelve sentences verified to elicit positive and negative emotional responses were employed in the experiments after validation. A total of 45 participants (31 females and 14 males) underwent stimulation through positive and negative stories while exhibiting eye contact or gaze avoidance under each of the three conversational conditions. Engagement was assessed using subjective evaluation metrics in conjunction with measures of the subjects' gaze and brainwave activity. The findings revealed engagement disparities between the face-to-face and digital-human conversation conditions. Notably, only positive stimuli elicited variations in engagement based on gaze expression across different conversation conditions. Gaze analysis corroborated the engagement differences, aligning with prior research on social sensitivity, but only in response to positive stimuli. This research departs from traditional studies of un-natural interactions with digital humans, focusing instead on interactions with digital humans designed to mimic the appearance of real humans. This study demonstrates the potential for gaze expression to induce engagement, regardless of the human or digital nature of the conversational dyads.

Radiographic response to neoadjuvant therapy in pleural mesothelioma should serve as a guide for patient selection for cytoreductive operations.

Malignant pleural mesothelioma (MPM) is associated with poor prognosis despite advances in multimodal therapeutic strategies. While patients with resectable disease may benefit from added survival with oncologic resection, patient selection for mesothelioma operations often relies on both objective and subjective evaluation metrics. We sought to evaluate factors associated with improved overall survival (OS) in patients with mesothelioma who underwent macroscopic complete resection (MCR). Patients with MPM who received neoadjuvant therapy and underwent MCR were identified in a prospectively maintained departmental database. Clinicopathologic, blood-based, and radiographic variables were collected and included in a Cox regression analysis (CRA). Response to neoadjuvant therapy was characterized by a change in tumor thickness from pretherapy to preoperative scans using the modified RECIST criteria. In this study, 99 patients met the inclusion criteria. The median age of the included patients was 64.7 years, who were predominantly men, had smoking and asbestos exposure, and who received neoadjuvant therapy. The median change in tumor thickness following neoadjuvant therapy was -16.5% (interquartile range of -49.7% to +14.2%). CRA demonstrated reduced OS associated with non-epithelioid histology [hazard ratio (HR): 3.06, 95% confidence interval (CI): 1.62-5.78, p < 0.001] and a response to neoadjuvant therapy inferior to the median (HR: 2.70, CI: 1.55-4.72, p < 0.001). Patients who responded poorly (below median) to neoadjuvant therapy had lower median survival (15.8 months compared to 38.2 months, p < 0.001). Poor response to neoadjuvant therapy in patients with MPM is associated with poor outcomes even following maximum surgical cytoreduction and should warrant a patient-centered discussion regarding goals of care and may therefore help guide further therapeutic decisions.

Low light image enhancement algorithm based on improved multi-objective grey wolf optimization with detail feature enhancement

This article aims to enhance low-light images by proposing a novel algorithm called MoGDF. It departs from the traditional approach of using deep learning with low-light and normal images as training data. Instead, exposure images at five different exposure scales are generated using a proposed model called EIPM, which consists of an improved Gamma function. Then it introduces a feature extraction network called FMEM, which leverages an improved PFPN structure to enhance the fusion of multi-scale information and strengthen the connection between high-level and low-level semantic information. Finally, it optimizes the fusion weights using an improved multiple-objective grey wolf optimization algorithm with comprehensive improvements in the convergence factor updating strategy, adaptive inertia weight strategy, and individual position strategy. The experiments show that the MoGDF outperforms the current state-of-the-art algorithms in terms of color restoration and detail preservation, and has significant advantages in objective and subjective evaluation metrics, with improvements of 5.69%, 2.60%, 1.11%, and 6.67% in PSNR, SSIM, NIQE, and LPIPS, respectively.

Knowledge-Enriched Attention Network With Group-Wise Semantic for Visual Storytelling.

As a technically challenging topic, visual storytelling aims at generating an imaginary and coherent story with narrative multi-sentences from a group of relevant images. Existing methods often generate direct and rigid descriptions of apparent image-based contents, because they are not capable of exploring implicit information beyond images. Hence, these schemes could not capture consistent dependencies from holistic representation, impairing the generation of reasonable and fluent stories. To address these problems, a novel knowledge-enriched attention network with group-wise semantic model is proposed. Three main novel components are designed and supported by substantial experiments to reveal practical advantages. First, a knowledge-enriched attention network is designed to extract implicit concepts from external knowledge system, and these concepts are followed by a cascade cross-modal attention mechanism to characterize imaginative and concrete representations. Second, a group-wise semantic module with second-order pooling is developed to explore the globally consistent guidance. Third, a unified one-stage story generation model with encoder-decoder structure is proposed to simultaneously train and infer the knowledge-enriched attention network, group-wise semantic module and multi-modal story generation decoder in an end-to-end fashion. Substantial experiments on the visual storytelling datasets with both objective and subjective evaluation metrics demonstrate the superior performance of the proposed scheme as compared with other state-of-the-art methods. The source code of this work can be found in https://mic.tongji.edu.cn.

Development of acoustic denoising learning network for communication enhancement in construction sites

This paper presents a novel denoising approach based on deep learning and signal processing to improve communication efficiency. Construction activities take place when different trades come to the site for overlapped periods to perform their works, which may easily produce hazardous noise levels. The existence of noise affects workers' health issues, especially hearing and rhythm of the heart, and impacts communication efficiency between workers. The proposed approach employs signal processing technique to transform the noisy audio into image and utilize neural networks to extract noisy features and denoise the image. The denoised image is then converted to obtain the denoised audio. Experiments on reducing the side effect of several common noises in construction sites were conducted, compared with the performance of denoising using conventional wavelet transform. Standard objective measures, such as signal-to-noise ratio (SNR), and subjective measures, such as listening tests are used for evaluations. Our experimental results show that the proposed algorithm achieved significant improvements over the traditional method, as evidenced by the following quantitative results of median value: MSE of 0.002, RMSE of 0.049, SNR of 5.7 dB, PSNR of 25.8 dB, and SSR of 8.Results indicate that the proposed algorithm outperforms conventional denoising methods in terms of both objective and subjective evaluation metrics and have the potential to facilitate communication between site workers when facing different noise sources inevitably.

MetaMGC: a music generation framework for concerts in metaverse

In recent years, there has been a national craze for metaverse concerts. However, existing meta-universe concert efforts often focus on immersive visual experiences and lack consideration of the musical and aural experience. But for concerts, it is the beautiful music and the immersive listening experience that deserve the most attention. Therefore, enhancing intelligent and immersive musical experiences is essential for the further development of the metaverse. With this in mind, we propose a metaverse concert generation framework — from intelligent music generation to stereo conversion and sound field design for virtual concert stages. First, combining the ideas of reinforcement learning and value functions, the Transformer-XL music generation network is improved and used in training all the music in the POP909 dataset. Experiments show that both improved algorithms have advantages over the original method in terms of objective evaluation and subjective evaluation metrics. In addition, this paper validates a neural rendering method that can be used to generate spatial audio based on a binaural-integrated neural network with a fully convolutional technique. And the purely data-driven end-to-end model performs to be more reliable compared with traditional spatial audio generation methods such as HRTF. Finally, we propose a metadata-based audio rendering algorithm to simulate real-world acoustic environments.

Research on Super-Resolution Relationship Extraction and Reconstruction Methods for Images Based on Multimodal Graph Convolutional Networks

This study constructs a multimodal graph convolutional network model, conducts an in-depth study on image super-resolution relationship extraction and reconstruction methods, and constructs a model of image super-resolution relationship extraction and reconstruction methods based on multimodal graph convolutional networks. In this study, we study the domain adaptation model algorithm based on chart convolutional networks, which constructs a global relevance graph based on all samples using pre-extracted features and performs distribution approximation of sample features in two domains using a diagram convolutional neural network with maximum mean difference loss; with this approach, the model effectively preserves the structural information among the samples. In this study, several comparison experiments are designed based on the COCO and VG datasets; the image space information-based and knowledge graph-based target detection and recognition models substantially improve recognition performance over the baseline model. The super-pixel-based target detection and recognition model can also effectively reduce the number of floating-point operations and the complexity of the model. In this study, we propose a multiscale GAN-based image super-resolution reconstruction algorithm. Aiming at the problems of detail loss or blurring in the reconstruction of detail-rich images by SRGAN, it integrates the idea of the Laplace pyramid to complete the task of multiscale reconstruction of images through staged reconstruction. It incorporates the concept of a discriminative network with patch GAN to effectively improve the recovery effect of graph details and improve the reconstruction quality of images. Using Set5, Set14, BSD100, and Urban100 datasets as test sets, experimental analysis is conducted from objective and subjective evaluation metrics to effectively validate the performance of the improved algorithm proposed in this study.

Sar2color: Learning Imaging Characteristics of SAR Images for SAR-to-Optical Transformation

Optical images are rich in spectral information, but difficult to acquire under all-weather conditions, while SAR images can overcome adverse meteorological conditions, but geometric distortion and speckle noise will reduce the quality of SAR images and thus make image interpretation more challenging. Therefore, transforming SAR images to optical images to assist SAR image interpretation will bring opportunities for SAR image application. With the advancement of deep learning technology, the ability of SAR-to-optical transformation has been greatly improved. However, most of the current mainstream transformation methods do not consider the imaging characteristics of SAR images, and there will be failures such as noisy color spots and regional landform deformation in the generated optical images. Moreover, since the SAR image itself does not contain color information, there also exist many color errors in these results. Aiming at the above problems, Sar2color, an end-to-end general SAR-to-optical transformation model, is proposed based on a conditional generative adversarial network (CGAN). The model uses DCT residual block to reduce the effect of coherent speckle noise on the generated optical images, and constructs the Light atrous spatial pyramid pooling (Light-ASPP) module to mitigate the negative effect of geometric distortion on the generation of optical images. These two designs ensure the precision of texture details when the SAR image is transformed into an optical image, and use the correct color memory block (CCMB) to improve the color accuracy of transformation results. Towards the Sar2color model, we have carried out evaluations on the homologous heterogeneous SAR image and optical image pairing dataset SEN1-2. The experimental results show that, compared with other mainstream transformation models, Sar2color achieves the state-of-the-art effect on all three objective and one subjective evaluation metrics. Furthermore, we have carried out various ablation experiments, and the results show the effectiveness of each designed module of Sar2color.

Machine Learning Approaches for Whisper to Normal Speech Conversion

Whispered speech is a mode of speech that differs from normal speech due to the absence of a periodic component, namely the Fundamental Frequency that characterizes the pitch, among other spectral and temporal differences. Much attention has been given in recent years to the application of Machine Learning techniques for voice conversion tasks. The whisper-to-normal speech conversion is particularly challenging, however, especially with respect to the Fundamental Frequency estimation. Based on the most recent literature, this survey assesses the state-of-the-art regarding Machine Learning based whisper-to-normal speech conversion, identifying trends both on modeling and training approaches. The proposed solutions include Generative Adversarial Network based, Autoencoder based and Bidirectional Long Short-Term Memory based frameworks, among other Deep Neural Network based architectures. In addition to Parallel versus Non-Parallel training, time-alignment requirements and strategies, datasets, vocoder usage, as well as both objective and subjective evaluation metrics are also covered by the present survey.

The VoicePrivacy 2020 Challenge: Results and findings

This paper presents the results and analyses stemming from the first VoicePrivacy 2020 Challenge which focuses on developing anonymization solutions for speech technology. We provide a systematic overview of the challenge design with an analysis of submitted systems and evaluation results. In particular, we describe the voice anonymization task and datasets used for system development and evaluation. Also, we present different attack models and the associated objective and subjective evaluation metrics. We introduce two anonymization baselines and provide a summary description of the anonymization systems developed by the challenge participants. We report objective and subjective evaluation results for baseline and submitted systems. In addition, we present experimental results for alternative privacy metrics and attack models developed as a part of the post-evaluation analysis. Finally, we summarize our insights and observations that will influence the design of the next VoicePrivacy challenge edition and some directions for future voice anonymization research.

IDE-Net: Interactive Driving Event and Pattern Extraction From Human Data

Autonomous vehicles (AVs) need to share the road with multiple, heterogeneous road users in a variety of driving scenarios. It is overwhelming and unnecessary to carefully interact with all observed agents, and AVs need to determine whether and when to interact with each surrounding agent. In order to facilitate the design and testing of prediction and planning modules of AVs, in-depth understanding of interactive behavior is expected with proper representation, and events in behavior data need to be extracted and categorized automatically. Answers to what are the essential patterns of interactions are also crucial for these motivations in addition to answering whether and when . Thus, learning to extract interactive driving events and patterns from human data for tackling the whether-when-what tasks is of critical importance for AVs. There is, however, no clear definition and taxonomy of interactive behavior, and most of the existing works are based on either manual labelling or hand-crafted rules and features. In this letter, we propose the I nteractive D riving event and pattern E xtraction Net work (IDE-Net), which is a deep learning framework to automatically extract interaction events and patterns directly from vehicle trajectories. In IDE-Net, we leverage the power of multi-task learning and proposed three auxiliary tasks to assist the pattern extraction in an unsupervised fashion. We also design a unique spatial-temporal block to encode the trajectory data. Experimental results on the INTERACTION dataset verified the effectiveness of such designs in terms of better generalizability and effective pattern extraction. We find three interpretable patterns of interactions, bringing insights for driver behavior representation, modeling and comprehension. Both objective and subjective evaluation metrics are adopted in our analysis of the learned patterns.

Low-Light Image Enhancement With Semi-Decoupled Decomposition

Low-light image enhancement is important for high-quality image display and other visual applications. However, it is a challenging task as the enhancement is expected to improve the visibility of an image while keeping its visual naturalness. Retinex-based methods have well been recognized as a representative technique for this task, but they still have the following limitations. First, due to less-effective image decomposition or strong imaging noise, various artifacts can still be brought into enhanced results. Second, although the priori information can be explored to partially solve the first issue, it requires to carefully model the priori by a regularization term and usually makes the optimization process complicated. In this paper, we address these issues by proposing a novel Retinex-based low-light image enhancement method, in which the Retinex image decomposition is achieved in an efficient semi-decoupled way. Specifically, the illumination layer I is gradually estimated only with the input image S based on the proposed Gaussian Total Variation model, while the reflectance layer R is jointly estimated by S and the intermediate I. In addition, the imaging noise can be simultaneously suppressed during the estimation of R. Experimental results on several public datasets demonstrate that our method produces images with both higher visibility and better visual quality, which outperforms the state-of-the-art low-light enhancement methods in terms of several objective and subjective evaluation metrics.

Nonsubsampled rotated complex wavelet transform (NSRCxWT) for medical image fusion related to clinical aspects in neurocysticercosis

Neurocysticercosis (NCC) is a parasite infection caused by the tapeworm Taenia solium in its larvae stage which affects the central nervous system of the human body (a definite host). It results in the formation of multiple lesions in the brain at different locations during its various stages. During diagnosis of such symptomatic patients, these lesions can be better visualized using a feature based fusion of Computed Tomography (CT) and Magnetic Resonance Imaging (MRI). This paper presents a novel approach to Multimodality Medical Image Fusion (MMIF) used for the analysis of the lesions for the diagnostic purpose and post treatment review of NCC. The MMIF presented here is a technique of combining CT and MRI data of the same patient into a new slice using a Nonsubsampled Rotated Complex Wavelet Transform (NSRCxWT). The forward NSRCxWT is applied on both the source modalities separately to extract the complementary and the edge related features. These features are then combined to form a composite spectral plane using average and maximum value selection fusion rules. The inverse transformation on this composite plane results into a new, visually better, and enriched fused image. The proposed technique is tested on the pilot study data sets of patients infected with NCC. The quality of these fused images is measured using objective and subjective evaluation metrics. Objective evaluation is performed by estimating the fusion parameters like entropy, fusion factor, image quality index, edge quality measure, mean structural similarity index measure, etc. The fused images are also evaluated for their visual quality using subjective analysis with the help of three expert radiologists. The experimental results on 43 image data sets of 17 patients are promising and superior when compared with the state of the art wavelet based fusion algorithms. The proposed algorithm can be a part of computer-aided detection and diagnosis (CADD) system which assists the radiologists in clinical practices.

Research of Kansei Image Based on Product Appearance Form Deconstruction

Based on kansei engineering theory, with mobile phone model as the research object, on the speciation analysis of deconstruction, study of the correlation between form design elements and subjective feeling. To obtain mobile phone simulation modeling samples by the concept of orthogonal test. Through collecting a large number of perceptual image vocabulary, the multiple scale method, class cluster method was applied to typical filtered emotional vocabulary, as the shape of subjective evaluation metrics. Using the mobile phone modeling sample obtained from the screened and representative emotional semantic difference experiment on the emotional reactions of vocabulary, get the corresponding relation between mobile phones form design elements and subjective evaluation value. Quantitative fuzzy uncertainty of emotional problem, and using kansei engineering theory to improve product that cares consumers emotional has become the direction of product design.

Information Seeking Spoken Dialogue Systems— Part II: Multimodal Dialogue

For pt.1see ibid., vol. 9, p. 3 (2007). In this paper, the task and user interface modules of a multimodal dialogue system development platform are presented. The main goal of this work is to provide a simple, application-independent solution to the problem of multimodal dialogue design for information seeking applications. The proposed system architecture clearly separates the task and interface components of the system. A task manager is designed and implemented that consists of two main submodules: the electronic form module that handles the list of attributes that have to be instantiated by the user, and the agenda module that contains the sequence of user and system tasks. Both the electronic forms and the agenda can be dynamically updated by the user. Next a spoken dialogue module is designed that implements the speech interface for the task manager. The dialogue manager can handle complex error correction and clarification user input, building on the semantics and pragmatic modules presented in Part I of this paper. The spoken dialogue system is evaluated for a travel reservation task of the DARPA Communicator research program and shown to yield over 90% task completion and good performance for both objective and subjective evaluation metrics. Finally, a multimodal dialogue system which combines graphical and speech interfaces, is designed, implemented and evaluated. Minor modifications to the unimodal semantic and pragmatic modules were required to build the multimodal system. It is shown that the multimodal system significantly outperforms the unimodal speech-only system both in terms of efficiency (task success and time to completion) and user satisfaction for a travel reservation task

Digital zooming for color filter array-based image sensors

In this paper, zooming methods which operate directly on color filter array (CFA) data are proposed, analyzed, and evaluated. Under the proposed framework enlarged spatial resolution images are generated directly from the CFA-based image sensors. The reduced computational complexity of the proposed schemes makes them ideal for real-time surveillance systems, industrial strength computer vision solutions, and mobile sensor-based visual systems. Simulation studies reported here indicate that the new methods (i) produce excellent results, in terms of both objective and subjective evaluation metrics, and (ii) outperform conventional zooming schemes operating in the RGB domain.