Perceptual Properties Research Articles

Mapping Quantitative Observer Metamerism of Displays.

Observer metamerism (OM) is the name given to the variability between the color matches that individual observers consider accurate. The standard color imaging approach, which uses color-matching functions of a single representative observer, does not accurately represent every individual observer's perceptual properties. This paper investigates OM in color displays and proposes a quantitative assessment of the OM distribution across the chromaticity diagram. An OM metric is calculated from a database of individual LMS cone fundamentals and the spectral power distributions of the display's primaries. Additionally, a visualization method is suggested to map the distribution of OM across the display's color gamut. Through numerical assessment of OM using two distinct publicly available sets of individual observers' functions, the influence of the selected dataset on the intensity and distribution of OM has been underscored. The case study of digital cinema has been investigated, specifically the transition from xenon-arc to laser projectors. The resulting heatmaps represent the "topography" of OM for both types of projectors. The paper also presents color difference values, showing that achromatic highlights could be particularly prone to disagreements between observers in laser-based cinema theaters. Overall, this study provides valuable resources for display manufacturers and researchers, offering insights into observer metamerism and facilitating the development of improved display technologies.

Configuration of the action observation network depends on the goals of the observer

Observing the actions of others engages a core action observation network (AON) that includes the bilateral inferior frontal cortex (IFC), posterior superior temporal sulcus (pSTS) and inferior parietal lobule (IPL) (Caspers et al., 2010). Each region in the AON has functional properties that are heterogeneous and include representing the perceptual properties of action, predicting action outcomes and making inferences as to the goals of the actor. Critically, recent evidence shows that neural representations within the pSTS are sharpened when attending to the kinematics of the actor, such that the top-down guided attention reshapes underlying neural representations. In this study we evaluate how attention alters network connectivity within the AON as a system. Cues directed participant's attention to the goal, kinematics, or identity depicted in short action animations while brain responses were measured by fMRI. We identified those parcels within the AON with functional connectivity modulated by task. Results show that connectivity between the right pSTS and right IFC, and bilateral extended STS (STS+) were modulated during action observation such that connections were strengthened when the participant was attending to the action than goal. This finding is contrasted by the univariate results, which no univariate modulations in these brain regions except for right IFC. Using the functional networks defined by Yeo et al. (2011), we identified the parcels that are modulated by the attention to consist mainly of the fronto-parietal control network and default mode networks. These results are consistent with models of top-down feedback from executive system in the IFC to pSTS and implicates a right lateralized dual pathway model for action observation when focused on whole-body kinematics.

Preference-Based Human-in-the-Loop Optimization for Perceived Realism of Haptic Rendering.

The fidelity of haptic rendering is characterized by the perceived realism capturing the level of similarity to a corresponding tangible object. Perceived realism depends on the musculoskeletal, mental, and perceptual properties of the individuals that manipulate the system. Human-in-the-loop (HiL) studies provide a feasible means for the concurrent optimization of the performance of the overall haptic rendering process, as the physical limitations of the hardware, the factors affecting the fidelity of the rendering algorithm, and the limitations of human action and perception can all be considered simultaneously. In this study, we propose the use of preference-based HiL optimization techniques based on sample-efficient Bayesian optimization algorithms and qualitative pairwise comparisons to maximize the perceived realism of haptic rendering tasks. We present two HiL optimization studies that maximize the perceived realism of spring and friction rendering and validate our results by comparing the HiL-optimized rendering models with expert-tuned nominal models. We show that the system parameters can effectively be optimized within a reasonable amount of time using a preference-based HiL optimization approach. Furthermore, we demonstrate that the approach provides an efficient means of studying the effect of haptic rendering parameters on perceived realism by capturing the interactions among the parameters, even for relatively high dimensional parameter spaces.

Do cultural differences emerge at different levels of representational hierarchy?

In prior research, Eastern and Western culture groups differ in memory specificity for objects. However, these studies used concrete object stimuli, which carry semantic information that may be confounded with culture. Additionally, the perceptual properties of the stimuli were not tightly controlled. Therefore, it cannot be precisely determined whether the observed cross-cultural differences are generalizable across different stimulus types and memory task demands. In prior studies, Americans demonstrated higher memory specificity than East Asians, but this may be due to Americans being more attuned to the low-level features that distinguish studied items from similar lures, rather than general memory differences. To determine whether this pattern of cross-cultural memory differences emerges irrespective of stimulus properties, we tested American and East Asian young adults using a recognition memory task employing abstract stimuli for which attention to conjunctions of features was critical for discrimination. Additionally, in order to more precisely determine the influence of stimulus and task on culture differences, participants also completed a concrete objects memory task identical to the one used in prior research. The results of the abstract objects task mirror the pattern seen in prior studies with concrete objects: Americans showed generally higher levels of recognition memory performance than East Asians for studied abstract items, whether discriminating them from similar or entirely new items. Results from the current concrete object task generally replicated this pattern. This suggests cross-cultural memory differences generalize across stimulus types and task demands, rather than reflecting differential sensitivity to low-level features or higher-level conjunctions.

Perceptual loss guided Generative adversarial network for saliency detection

In this work, we introduce a novel approach for saliency detection through the utilization of a generative adversarial network guided by perceptual loss. Achieving effective saliency detection through deep learning entails intricate challenges influenced by a multitude of factors, with the choice of loss function playing a pivotal role. Previous studies usually formulate loss functions based on pixel-level distances between predicted and ground-truth saliency maps. However, these formulations don’t explicitly exploit the perceptual attributes of objects, such as their shapes and textures, which serve as critical indicators of saliency. To tackle this deficiency, we propose an innovative loss function that capitalizes on perceptual features derived from the saliency map. Our approach has been rigorously evaluated on six benchmark datasets, demonstrating competitive performance when compared against the forefront methods in terms of both Mean Absolute Error (MAE) and F-measure. Remarkably, our experiments reveal consistent outcomes when assessing the perceptual loss using either grayscale saliency maps or saliency-masked colour images. This observation underscores the significance of shape information in shaping the perceptual saliency cues.The code is available at https://github.com/XiaoxuCai/PerGAN.

Cross-cultural comparison of soundscape evaluation between Japanese and Vietnamese using standardized attributes

The Soundscape Attributes Translation Project (SATP) was initiated to translate English standardized perceptual attributes of the ISO/TS 12913-2:2018 into various languages. This paper outlines the translation and utilization of Vietnamese and Japanese attributes in standardized listening experiments and evaluates their effectiveness in assessing specific sound environments in Vietnam and Japan. The study was conducted in two parts: (1) a listening evaluation experiment, in which Vietnamese and Japanese participants listened to 27 binaural recordings provided by the SATP project to examine the differences in subjective evaluations of the same sound environment between Japanese and Vietnamese; and (2) a survey on Vietnamese and Japanese perceptions of sound environments in public spaces using the standardized attributes and scales proposed by the SATP project. For part 2, twelve study sites were selected in Japan and Vietnam based on their distinct landscape and acoustic characteristics. The research methodology involved analyzing physical indices of each soundscape context from the recording data and subjective evaluation experiments. The results revealed cultural differences in soundscape evaluations between Japanese and Vietnamese, which could potentially impact people's perception of soundscapes. These findings provide a basis for proposing soundscape designs that align with the values and perceptions of residents in specific environments.

Spatial audio reproduction over ad hoc loudspeaker array using near-field compensation in spherical harmonics domain

Spatial audio reproduction using the loudspeaker array introduces the curvature effect leading to a distorted listening experience when the listener is in the near field. In the near-field, the loudspeakers are approximated as point sources (spherical wave) and amplify the mode vectors. Further, the problem becomes more challenging for the irregular loudspeaker arrangement, which causes uneven energy distribution in the reproduction region. In this context, a near-field compensation is applied to the encoded ambisonics coefficients. An optimization problem is formulated, such as the loudspeaker gains encoded with spherical harmonics basis coefficients should match the target ambisonics coefficients. Further, the in-phase and quadrature components of the energy localization vector are imposed as the constraints to direct maximum energy in the reproduction region. The solution to the optimization problem is obtained using a derivative-free optimization solver. The performance of the proposed methods is evaluated for ITU-R recommended loudspeaker layouts using the technical and perceptual evaluation attributes.

Research on the Perceived Quality of Virtual Reality Headsets in Human-Computer Interaction.

The progress of commercial VR headsets largely depends on the progress of sensor technology, the iteration of which often means longer research and development cycles, and also higher costs. With the continuous maturity and increasing competition of VR headsets, designers need to create a balance among user needs, technologies, and costs to achieve commercial competition advantages. To make accurate judgments, consumer feedback and opinions are particularly important. Due to the increasing maturity in the technology of commercial VR headsets in recent years, the cost has been continuously decreasing, and potential consumers have gradually increased. With the increase in consumer demand for virtual reality headsets, it is particularly important to establish a perceptual quality evaluation system. The relationship between consumer perception and product quality determined by evaluations of experience is improving. Using the research method implemented in this work, through semi-structured interviews and big data analysis of VR headset consumption, the perceptual quality elements of VR headsets are proposed, and the order of importance of perceptual quality attributes is determined by questionnaire surveys, quantitative analysis, and verification. In this study, the perceptual quality elements, including technical perceptual quality (TPQ) and value perceptual quality (VPQ), of 14 types of VR headsets were obtained, and the importance ranking of the VR headsets' perceptual quality attributes was constructed. In theory, this study enriches the research on VR headsets. In practice, this study provides better guidance and suggestions for designing and producing VR headsets so that producers can better understand which sensor technology has met the needs of consumers, and which sensor technology still has room for improvement.

Sound-symbolic association between speech sound and spatial meaning in relation to the concepts of up/down and above/below

AbstractResearch has shown sound-symbolic associations between speech sounds and conceptual and/or perceptual properties of a referent. This study used the choice response time method to investigate hypothesized associations between a high/low vowel and spatial concepts of up/down and above/below. The participants were presented with a stimulus that moved either upward or downward (Experiments 1 and 2), or that was located above or below the reference stimulus (Experiment 3), and they had to pronounce a vowel ([i] or [æ]) based on the spatial location of the stimulus. The study showed that the high vowel [i] was produced faster in relation to the up-directed and the above-positioned stimulus, while the low vowel [æ] was produced faster in relation to the down-directed and the below-positioned stimulus. In addition, the study replicated the pitch-elevation effect showing a raising of the vocalization pitch when vocalizations were produced to the up-directed stimulus. The article discusses these effects in terms of the involvement of sensorimotor processes in representing spatial concepts.

Understanding the effects of urban perceptions on housing rent using big data and machine learning

ABSTRACT Urban perceptions are related to residents’ subjective well-being, activities, settlement choice, and physical and mental health, which are vital for promoting sustainable community development. Landsenses ecology proposes the concepts of physical and psychological perceptions to measure the interaction between humans and the urban environment. However, such perceptions and their quantitative relationships with housing rental prices (HRPs) are still not clear. In this study, a novel urban-perception-oriented hedonic price model (UP-HPM) is proposed based on the construction of a multisource geographic database of Beijing’s rental market with rich information on people-oriented perception metrics, neighborhood environment characteristics, and HRPs. Furthermore, we use advanced machine learning (ML) pipelines to measure residents’ multidimensional urban perceptions and investigate their impact on block-level HRPs and their spatial disparities. We also quantify and compare the contribution of different physical and psychological perceptual attributes to HRPs through UP-HPM. The results show that neighborhood perceived greenery and the use of fences tend to bring about positive psychological perceptions and further affect HRPs. By comparing different combinations of hedonic and perceptual features, the UP-HPM performs well in terms of fitting capacity, spatial description, and interpretability and can inspire humanistic thinking and landsenses planning guidelines for the regulation of the allocation of public rental housing and improvement in block-level physical settings and infrastructure services, thereby providing insights into how best to promote residents’ quality of life and well-being.

Mixed Evidence for Gestalt Grouping in Deep Neural Networks

Gestalt psychologists have identified a range of conditions in which humans organize elements of a scene into a group or whole, and perceptual grouping principles play an essential role in scene perception and object identification. Recently, Deep Neural Networks (DNNs) trained on natural images (ImageNet) have been proposed as compelling models of human vision based on reports that they perform well on various brain and behavioural benchmarks. Here we test a total of 16 networks covering a variety of architectures and learning paradigms (convolutional, attention-based, supervised and self-supervised, feed-forward and recurrent) on dots (Experiment 1) and more complex shapes (Experiment 2) stimuli that produce strong Gestalts effects in humans. In Experiment 1 we found that convolutional networks were indeed sensitive in a human-like fashion to the principles of proximity, linearity, and orientation, but only at the output layer. In Experiment 2, we found that most networks exhibited Gestalt effects only for a few sets, and again only at the latest stage of processing. Overall, self-supervised and Vision Transformer appeared to perform worse than convolutional networks in terms of human similarity. Remarkably, no model presented a grouping effect at the early or intermediate stages of processing. This is at odds with the widespread assumption that Gestalts occur prior to object recognition, and indeed, serve to organize the visual scene for the sake of object recognition. Our overall conclusion is that, albeit noteworthy that networks trained on simple 2D images support a form of Gestalt grouping for some stimuli at the output layer, this ability does not seem to transfer to more complex features. Additionally, the fact that this grouping only occurs at the last layer suggests that networks learn fundamentally different perceptual properties than humans.

In-the-wild Material Appearance Editing using Perceptual Attributes

Intuitively editing the appearance of materials, just from a single image, is a challenging task given the complexity and ambiguity of the interactions between light and matter. This problem has been traditionally solved by estimating additional factors of the scene like geometry or illumination, thus solving an inverse rendering problem where the interaction of light and matter needs to be modelled. Instead, we present a single-image appearance editing framework that allows to intuitively modify the material appearance of an object by increasing or decreasing high-level perceptual attributes describing appearance (e.g., glossy or metallic). Our framework uses just an in-the-wild image as input, where geometry or illumination are not controlled.
 We rely on generative neural networks and, in particular, on Selective Transfer Generative Adversarial Networks (STGAN) that allow to preserve high-frequency details from the input image in the edited one. To train our framework we combine pairs of synthetic images, rendered with physically-based ray tracing algorithms, and their corresponding ratings of the high-level attributes, given by humans through crowd-sourced user studies. Last, although trained on synthetic images, we demonstrate the applicability of our method on synthetic video sequences; and real-world photographs downloaded from online catalogs and manually taken using our mobile phones.

A European and Brazilian cross-national investigation into the Portuguese translation of soundscape perceptual attributes within the SATP project

This paper presents a cross-national investigation into the soundscape perceptual attributes translation from English into European and Brazilian Portuguese. It is a study within the scope of a larger project – the Soundscape Attributes Translation Project (SATP) – that involves the translation of the same eight Perceived Affective Qualities (PAQs) into different languages through standard procedures. The structure of the paper reflects two separate moments of data collection and analysis. In the first part, the work done outlines the translation process and the resulting Portuguese-language expressions for the eight PAQs. The translation used a mixed-method approach that involved expert panels and online questionnaires. The second part of the paper outlines the impact of cultural and linguistic factors on soundscape perception and evaluation. This assessment analyzed how Portuguese speakers perceive the eight PAQs of the SATP project and compared it with the results of the English speakers. Additionally, investigations occurred on the differences between the perception of European Portuguese and Brazilian Portuguese speakers. The study found differences in how individuals from Portuguese-speaking countries describe sounds compared to the original English circumplex soundscape model. The Vibrant, Monotonous, Annoying, and Chaotic PAQs did not correlate as in the original English model, indicating the need for further investigation. However, the Pleasant and Calm PAQs had no issues in all analyses and shall remain with their initial translations. These findings have important implications for soundscape developments related to standardization tools, research methods, and test design in Portuguese-speaking contexts, as well as contributions to the growing body of knowledge on the role of culture and language in soundscape perception and evaluation.

Predicting textural attributes and frictional characteristics of topical formulations based on non-linear rheology

A number of analytical procedures (Spectrum Descriptive Analysis (SDA), Quantitative Descriptive Analysis (QDA), and Check All That Apply (CATA)) are used for characterizing sensorial attributes of topical formulations. However, these techniques are evaluated by expert panels/consumers, which are subjective, expensive and time consuming. Despite widespread use of these methods, the techniques do not necessarily aid in the development of innovative formulations and understanding of consumer liking. In addition, the hedonic attributes of a product can significantly dominate the sensation. Therefore, a more rapid, quantitative, and objective approach is required for understanding the formulation factors that governs sensory perception at different points of consumer application. Rheological evaluations of topical formulations were carried out under steady and oscillatory (SAOS and LAOS) rheology using a commercial rheometer. Friction measurements were performed using an in-house built tribometer on non-biological skin model to investigate how surface properties are influenced by application of different topical formulations. Further, a broad range of instrumental texture measurements was performed to characterize the formulations. Principal component analysis was used for dimensionality reduction of the instrumental data. Supervised machine learning was performed on the closely related PCA parameters to develop predictive models. We identified physical parameters relevant to different perceptual attributes by comparing a range of commercial topical formulations with various compositions using rheological and tribological methods. Multi-variate machine learning models were developed where several key instrumental textural attributes and skin feel could be predicted from data obtained from linear and non-linear rheological measurements. Our study shows rheological analysis based on a few material parameters can be effectively used for predicting instrumental sensory attributes that is of relevance to consumer care industry. The machine learning models may benefit development of innovative formulations, valorisation of new materials and serve as a platform for mapping commercial formulations for product optimization. The models can likely shorten design cycle time by screening through large volume of formulations and short list only those with highest potential to be evaluated by a very specialized and expensive human test panel.

Revealing spatio-temporal evolution of urban visual environments with street view imagery

The visual landscape plays a pivotal role in urban planning and healthy cities. Recent studies of visual evaluation focus on either objective or subjective approach, while describing the visual character holistically and monitor its evolution remains challenging. This study introduces an embedding-driven clustering approach that integrates both physical and perceptual attributes to infer the spatial structure of the visual environment, and investigates its spatio-temporal evolution. Singapore, a highly urbanised yet green city, is selected as a case study. Firstly, a visual feature matrix is derived from street view imagery (SVI). Then, a graph neural network is constructed based on road connections to encode visual features and spatial dependency leading to a clustering algorithm that is used to discover the underlying characteristics of the visual environment. The implementation characterises streetscapes of the city-state into six types of clusters. Finally, taking advantage of historical SVI, a longitudinal analysis reveals how visual clusters have evolved in the past decade. Among them, one of the clusters represents high-density visual experience, affirming the work as such streetscape dominates the central business district and it is evolving elsewhere, mirroring the expansion of new towns. In turn, another identified cluster, indicating sparse landscapes, decreased, while areas that are considered to be in the most visually pleasant cluster, increased. For the first time, this study demonstrates a novel method to understand the urban visual structure and analyse its spatio-temporal evolution, which could support future planning decision-making and urban landscape betterment.

Change in preference for vegetables and their perceptual attributes as a function of age and pickiness

Preference for vegetables is influenced by various factors, including demographic, psychological, socio-environmental, and genetic factors. This study confirmed that age, pickiness, and perceptual attributes were predictors of preference for vegetables and examined how preference for vegetables and their perceptual attributes varies with age and pickiness. Children (8–14 years, n = 420), youth (15–34 years, n = 569), middle-aged adults (35–64 years, n = 726), and older adults (65–85 years, n = 270) were asked which vegetables they liked (or disliked) and which perceptual attributes of each vegetable they liked (or disliked). On the basis of their responses, an overall preference score and a preference sub-score for each perceptual attribute were calculated. Participants in each age group were classified into four statuses (non-, mild, moderate, and severe) according to their pickiness scores. Multiple regression analysis revealed that age and preference sub-scores for eight perceptual attributes (sweetness, sourness, bitterness, umami, pungency, orthonasal aroma, texture, and appearance) were positive predictors of overall preference score and that pickiness score and four perceptual attributes (saltiness, astringency, retronasal aroma, and aftertaste) were negative predictors. In addition, overall preference score and preference sub-scores for perceptual attributes other than saltiness increased with increasing age group and decreasing picker status; however, preference sub-scores for at least one of the six perceptual attributes (bitterness, astringency, pungency, orthonasal aroma, retronasal aroma, and aftertaste) exhibited negative values in children, youth, and pickers (mild, moderate, and severe). The increase in preference for these perceptual attributes might be an indicator of the adultization of food perception and the expansion of food acceptance.

Learning a crowd-powered perceptual distance metric for facial blendshapes

It is known that purely geometric distance metrics cannot reflect the human perception of facial expressions. A novel perceptually based distance metric designed for 3D facial blendshape models is proposed in this paper. To develop this metric, comparative evaluations of facial expressions were collected from a crowdsourcing experiment. Then, the weights of a distance metric, based on descriptive features of the models, were optimized to match the results with crowdsourced data, through a metric learning process. The method incorporates perceptual properties such as curvature and visual saliency. A formal analysis of the results proves the high correlation between the metric output and human perception. The effectiveness and success of the proposed metric were also compared to other distance alternatives. The proposed metric will enable intelligent processing of 3D facial blendshapes data in several ways. It will be possible to generate perceptually valid clustering and visualization of 3D facial blendshapes. It will help reduce storage and computational requirements by removing redundant expressions that are perceptually identical from the overall dataset. It can also be used to assist novice animators while creating plausible and expressive facial animations.

Translation of soundscape perceptual attributes from English to Turkish

The International Standard Organization (ISO) published the standard series on soundscape for the identification, data collection and data analysis. However, since all these standards are in English language, the reliable standardized usage in other languages and its applicability is questionable. Thus, this two-staged study aims: i) to determine the Turkish equivalences of the 8 soundscape perceptual attributes that are published in ISO/TS 12913–2:2018 and ISO/TS 12913–3:2019, ii) to analyze if the determined Turkish attributes have concept equivalence to the original ones, and iii) to determine if the translated Turkish scale is reproducible. The first stage involved the translation of the attributes by focus group discussions and finalization by Turkish linguistic experts. As a result, the attributes ‘eventful’, ‘vibrant’, ‘pleasant’, ‘calm’, ‘uneventful’, ‘monotonous’, ‘annoying’, and ‘chaotic’ are translated to Turkish as ‘hareketli’, ‘coşkulu’, ‘keyifli’, ‘sakin’, ‘durağan’, ‘tekdüze’, ‘rahatsız edici’, and ‘karmaşık’, respectively. The second stage involves the analysis of reproducibility in terms of inter-rater reliability and conceptual validity. It is found that the Turkish scale is reproducible based on high inter-rater reliability in all attributes. Context validity at a conceptual level is analyzed both in terms of the difference between the average scores given to the English attributes and their corresponding Turkish equivalences and the correlation between the English and Turkish scores given to each attribute. The highest difference between the average scores (around 10 points on a slider scale of 0 to 100) is found to be in the translation of ‘vibrant’ while the lowest correlated one (slightly lower than 0.5) is found in ‘chaotic’ attribute as in line with literature. Despite this result, when the scores are reduced to 2 dimensions as pleasantness and eventfulness, it is seen that there is a high correlation between the English and Turkish scales. It is considered that the results obtained from this research could act as a base in the future for the establishment of Turkish Standards on soundscape and standardization of the translated and validated Turkish soundscape perceptual attributes and the ‘perceived affective quality’ scale defined under ISO/TS 12913–2:2018 in English.

In‐the‐wild Material Appearance Editing using Perceptual Attributes

AbstractIntuitively editing the appearance of materials from a single image is a challenging task given the complexity of the interactions between light and matter, and the ambivalence of human perception. This problem has been traditionally addressed by estimating additional factors of the scene like geometry or illumination, thus solving an inverse rendering problem and subduing the final quality of the results to the quality of these estimations. We present a single‐image appearance editing framework that allows us to intuitively modify the material appearance of an object by increasing or decreasing high‐level perceptual attributes describing such appearance (e.g., glossy or metallic). Our framework takes as input an in‐the‐wild image of a single object, where geometry, material, and illumination are not controlled, and inverse rendering is not required. We rely on generative models and devise a novel architecture with Selective Transfer Unit (STU) cells that allow to preserve the high‐frequency details from the input image in the edited one. To train our framework we leverage a dataset with pairs of synthetic images rendered with physically‐based algorithms, and the corresponding crowd‐sourced ratings of high‐level perceptual attributes. We show that our material editing framework outperforms the state of the art, and showcase its applicability on synthetic images, in‐the‐wild real‐world photographs, and video sequences.

An Advanced Color Model for Evaluating New Display Technologies

AbstractUnderstanding chromatic stimuli's brightness using experienced color range can help scientists and consumers predict the perceptual attributes of electronic displays.