Visual Signals Research Articles

Evolution of rhodopsin in flatfishes (Pleuronectiformes) is associated with depth and migratory behavior.

Visual signals are involved in many fitness-related tasks and are therefore essential for survival in many species. Aquatic organisms are ideal systems to study visual evolution, as the high diversity of spectral properties in aquatic environments generates great potential for adaptation to different light conditions. Flatfishes are an economically important group, with over 800 described species distributed globally, including halibut, flounder, sole, and turbot. The diversity of flatfish species and wide array of environments they occupy provides an excellent opportunity to understand how this variation translates to molecular adaptation of vision genes. Using models of molecular evolution, we investigated how the light environments inhabited by different flatfish lineages have shaped evolution in the rhodopsin gene, which is responsible for mediating dim-light visual transduction. We found strong evidence for positive selection in rhodopsin, and this was correlated with both migratory behavior and several fundamental aspects of habitat, including depth and freshwater/marine evolutionary transitions. We also identified several mutations that likely affect the wavelength of peak absorbance of rhodopsin, and outline how these shifts in absorbance correlate with the response to the light spectrum present in different habitats. This is the first study of rhodopsin evolution in flatfishes that considers their extensive diversity, and our results highlight how ecologically-driven molecular adaptation has occurred across this group in response to transitions to novel light environments.

The contribution of the vascular architecture and cerebrovascular reactivity to the BOLD signal formation across cortical depth.

Assessment of neuronal activity using blood oxygenation level-dependent (BOLD) is confounded by how the cerebrovascular architecture modulates hemodynamic responses. To understand brain function at the laminar level, it is crucial to distinguish neuronal signal contributions from those determined by the cortical vascular organization. Therefore, our aim was to investigate the purely vascular contribution in the BOLD signal by using vasoactive stimuli and compare that with neuronal-induced BOLD responses from a visual task. To do so, we estimated the hemodynamic response function (HRF) across cortical depth following brief visual stimulations under different conditions using ultrahigh-field (7 Tesla) functional (f)MRI. We acquired gradient-echo (GE)-echo-planar-imaging (EPI) BOLD, containing contributions from all vessel sizes, and spin-echo (SE)-EPI BOLD for which signal changes predominately originate from microvessels, to distinguish signal weighting from different vascular compartments. Non-neuronal hemodynamic changes were induced by hypercapnia and hyperoxia to estimate cerebrovascular reactivity and venous cerebral blood volume ( ). Results show that increases in GE HRF amplitude from deeper to superficial layers coincided with increased macrovascular . -normalized GE-HRF amplitudes yielded similar cortical depth profiles as SE, thereby possibly improving specificity to neuronal activation. For GE BOLD, faster onset time and shorter time-to-peak were observed toward the deeper layers. Hypercapnia reduced the amplitude of visual stimulus-induced signal responses as denoted by lower GE-HRF amplitudes and longer time-to-peak. In contrast, the SE-HRF amplitude was unaffected by hypercapnia, suggesting that these responses reflect predominantly neurovascular processes that are less contaminated by macrovascular signal contributions.

An enhanced speech emotion recognition using vision transformer

In human–computer interaction systems, speech emotion recognition (SER) plays a crucial role because it enables computers to understand and react to users’ emotions. In the past, SER has significantly emphasised acoustic properties extracted from speech signals. The use of visual signals for enhancing SER performance, however, has been made possible by recent developments in deep learning and computer vision. This work utilizes a lightweight Vision Transformer (ViT) model to propose a novel method for improving speech emotion recognition. We leverage the ViT model’s capabilities to capture spatial dependencies and high-level features in images which are adequate indicators of emotional states from mel spectrogram input fed into the model. To determine the efficiency of our proposed approach, we conduct a comprehensive experiment on two benchmark speech emotion datasets, the Toronto English Speech Set (TESS) and the Berlin Emotional Database (EMODB). The results of our extensive experiment demonstrate a considerable improvement in speech emotion recognition accuracy attesting to its generalizability as it achieved 98%, 91%, and 93% (TESS-EMODB) accuracy respectively on the datasets. The outcomes of the comparative experiment show that the non-overlapping patch-based feature extraction method substantially improves the discipline of speech emotion recognition. Our research indicates the potential for integrating vision transformer models into SER systems, opening up fresh opportunities for real-world applications requiring accurate emotion recognition from speech compared with other state-of-the-art techniques.

Comparative transcriptomic analysis primarily explores the molecular mechanism of compound eye formation in Neocaridina denticulata sinensis

Compound eyes formation in decapod crustaceans occurs after the nauplius stage. However, the key genes and regulatory mechanisms of compound eye development during crustacean embryonic development have not yet been clarified. In this study, RNA-seq was used to investigate the gene expression profiles of Neocaridina denticulata sinensis from nauplius to zoea stage. Based on RNA-seq data analysis, the phototransduction and insect hormone biosynthesis pathways were enriched, and molting-related neuropeptides were highly expressed. There was strong cell proliferation in the embryo prior to compound eye development. The formation of the visual system and the hormonal regulation of hatching were the dominant biological events during compound eye development. The functional analysis of DEGs across all four developmental stages showed that cuticle formation, muscle growth and the establishment of immune system occurred from nauplius to zoea stage. Key genes related to eye development were discovered, including those involved in the determination and differentiation of the eye field, eye-color formation, and visual signal transduction. In conclusion, the results increase the understanding of the molecular mechanism of eye formation in crustacean embryonic stage.

TRPV4 affects visual signals in photoreceptors and rod bipolar cells.

Mechanical sensitive channels expressed in mammalian retinas are effectors of elevated pressure stresses, but it is unclear how their activation affects visual function in pressure-related retinal disorders. This study investigated the role of the transient potential channel vanilloid TRPV4 in photoreceptors and rod bipolar cells (RBCs) with immunohistochemistry, confocal microscopy, electroretinography (ERG), and patch-clamp techniques. TRPV4 immunoreactivity (IR) was found in the outer segments of photoreceptors, dendrites and somas of PKCα-positive RBCs and other BCs, plexiform layers, and retinal ganglion cells (RGCs) in wild-type mice. TRPV4-IR was largely diminished in the retinas of homozygous TRPV4 transgenic mice. Genetically suppressing TRPV4 expression moderately but significantly enhanced the amplitude of ERG a- and b-waves evoked by scotopic and mesopic lights (0.55 to 200 Rh*rod-1 s-1) and photopic lights (105-106 Rh*rod-1 s-1) compared to wild-type mice in fully dark-adapted conditions. The implicit time evoked by dim lights (0.55 to 200 Rh*rod-1 s-1) was significantly decreased for b-waves and elongated for a-waves in the transgenic mice. ERG b-wave evoked by dim lights is primarily mediated by RBCs, and under voltage-clamp conditions, the latency of the light-evoked cation current in RBCs of the transgenic mice was significantly shorter compared to wild-type mice. About 10% of the transgenic mice had one eye undeveloped, and the percentage was significantly higher than in wild-type mice. The data indicates that TRPV4 involves ocular development and is expressed and active in outer retinal neurons, and interventions of TRPV4 can variably affect visual signals in rods, cones, RBCs, and cone ON BCs.

74‐1: Display Reflectance Measurements Finally Made Simple, Comprehensive and Affordable

Analysis of the point‐spread function (PSF) yields the directional variations of scattered light with high resolution in the vicinity of the specular direction and now, finally, at reduced but sufficient resolution, also at off‐specular directions, thus providing a basis for the determination of a set of complete in‐plane BRD/BTD functions. This approach is applied to both identification and separation of mirror, haze and Lambertian reflection components which then can be classified into such components that are forwarding visual information to the observer (i. e. visual signal) and disturbing reflections (i.e. visual noise).

Stopping Speed in Response to Auditory and Visual Stop Signals Depends on Go Signal Modality.

Past research has found that the speed of the action cancellation process is influenced by the sensory modality of the environmental change that triggers it. However, the effect on selective stopping processes (where participants must cancel only one component of a multicomponent movement) remains unknown, despite these complex movements often being required as we navigate our busy modern world. Thirty healthy adults (mean age = 31.1 years, SD = 10.5) completed five response-selective stop signal tasks featuring different combinations of "go signal" modality (the environmental change baring an imperative to initiate movement; auditory or visual) and "stop signal" modality (the environmental change indicating that action cancellation is required: auditory, visual, or audiovisual). EMG recordings of effector muscles allowed detailed comparison of the characteristics of voluntary action and cancellation between tasks. Behavioral and physiological measures of stopping speed demonstrated that the modality of the go signal influenced how quickly participants cancelled movement in response to the stop signal: Stopping was faster in two cross-modal experimental conditions (auditory go - visual stop; visual go - auditory stop), than in two conditions using the same modality for both signals. A separate condition testing for multisensory facilitation revealed that stopping was fastest when the stop signal consisted of a combined audiovisual stimulus, compared with all other go-stop stimulus combinations. These findings provide novel evidence regarding the role of attentional networks in action cancellation and suggest modality-specific cognitive resources influence the latency of the stopping process.

Query-Oriented Micro-Video Summarization.

Query-oriented micro-video summarization task aims to generate a concise sentence with two properties: (a) summarizing the main semantic of the micro-video and (b) being expressed in the form of search queries to facilitate retrieval. Despite its enormous application value in the retrieval area, this direction has barely been explored. Previous studies of summarization mostly focus on the content summarization for traditional long videos. Directly applying these studies is prone to gain unsatisfactory results because of the unique features of micro-videos and queries: diverse entities and complex scenes within a short time, semantic gaps between modalities, and various queries in distinct expressions. To specifically adapt to these characteristics, we propose a query-oriented micro-video summarization model, dubbed QMS. It employs an encoder-decoder-based transformer architecture as the skeleton. The multi-modal (visual and textual) signals are passed through two modal-specific encoders to obtain their representations, followed by an entity-aware representation learning module to identify and highlight critical entity information. As to the optimization, regarding the large semantic gaps between modalities, we assign different confidence scores according to their semantic relevance in the optimization process. Additionally, we develop a novel strategy to sample the effective target query among the diverse query set with various expressions. Extensive experiments demonstrate the superiority of the QMS scheme, on both the summarization and retrieval tasks, over several state-of-the-art methods.

Survey on Visual Signal Coding and Processing With Generative Models: Technologies, Standards, and Optimization

Survey on Visual Signal Coding and Processing With Generative Models: Technologies, Standards, and Optimization

Guest Editorial Advances in Generative Visual Signal Coding and Processing

Guest Editorial Advances in Generative Visual Signal Coding and Processing

A potential-controlled electrochromic visual biosensor based on distance readout for zearalenone detection

In this work, a potential-controlled electrochromic visual biosensor was developed for detecting zearalenone (ZEN) using a distance readout strategy. The sensor chip includes a square detection area and a folded signal output area created with laser etching technology. The detection area is modified with graphene oxide and ZEN aptamer, while Prussian blue (PB) is electrodeposited onto the signal output channel. When an appropriate voltage is applied, PB in the signal output area is reduced to colorless Prussian white (PW). The target ZEN molecules have the capability to release aptamers from graphene oxide (GO) surface in the detection area, resulting in a subsequent change in the potential of the visual signal output channel. This change determines the length of the channel that changes from blue to colorless, with the color change distance being proportional to the ZEN concentration. Using this distance readout strategy, ZEN detection within the range of 1 ng/mL to 300 ng/mL was achieved, with a detection limit of 0.29 ng/mL.

Shining a light on UV-fluorescent floral nectar after 50 years

Nature is aglow with numerous captivating examples of UV-fluorescence in the animal kingdom. Despite a putative role as a visual signal, exploration of UV-fluorescence in plants and its role in plant-animal interactions is lagging in comparison. Almost 50 years ago, UV-fluorescence of floral nectar, a crucial reward for pollinators, was reported for 23 flowering plant species. Since this intriguing discovery, UV-fluorescent nectar has only seldom been addressed in the scientific literature and has not been scrutinized in a phylogenetic or ecological context. Here, we report the prevalence of vibrant UV-fluorescent floral nectar across the family Cleomaceae, including the first photographic documentation in vivo colour for flowering plants. Though Cleomaceae flowers are morphologically diverse varying in colour, nectary prominence, and nectar volume, UV-fluorescent floral nectar may be a ubiquitous characteristic of the family. Fluorescence spectra show that the identity and number of fluorescent compounds in floral nectar may differ among Cleomaceae species. As Cleomaceae pollinators range from insects to bats and birds, we suggest that the UV-fluorescent floral nectar not only functions as a visual cue for the diurnal pollinators but also for the nocturnal/crepuscular pollinators in low light settings.

Male proboscis monkey cranionasal size and shape is associated with visual and acoustic signalling

The large nose adorned by adult male proboscis monkeys is hypothesised to serve as an audiovisual signal of sexual selection. It serves as a visual signal of male quality and social status, and as an acoustic signal, through the expression of loud, low-formant nasalised calls in dense rainforests, where visibility is poor. However, it is unclear how the male proboscis monkey nasal complex, including the internal structure of the nose, plays a role in visual or acoustic signalling. Here, we use cranionasal data to assess whether large noses found in male proboscis monkeys serve visual and/or acoustic signalling functions. Our findings support a visual signalling function for male nasal enlargement through a relatively high degree of nasal aperture sexual size dimorphism, the craniofacial region to which nasal soft tissue attaches. We additionally find nasal aperture size increases beyond dental maturity among male proboscis monkeys, consistent with the visual signalling hypothesis. We show that the cranionasal region has an acoustic signalling role through pronounced nasal cavity sexual shape dimorphism, wherein male nasal cavity shape allows the expression of loud, low-formant nasalised calls. Our findings provide robust support for the male proboscis monkey nasal complex serving both visual and acoustic functions.

Color-advertising strategies of invasive plants through the bee eye.

Invasive plants represent a significant global challenge as they compete with native plants for limited resources such as space, nutrients and pollinators. Here, we focused on four invasive species that are widely spread in the French Pyrenees, Buddleja davidii, Reynoutria japonica, Spiraea japonica and Impatiens glandulifera, and analyzed their visual advertisement signals with respect to those displayed by their surrounding native species using a perceptual approach based on the neural mechanisms of bee vision given that bees are regular pollinators of these plants. We collected 543 spectral reflections from the 4 invasive species, and 66 native species and estimated achromatic and chromatic similarities to the bee eye. R. japonica, S. japonica and B. davidii were inconspicuous against the foliage background and could be hardly discriminated in terms of color from their surrounding native plants. These characteristics promote generalization, potentially attracting pollinators foraging on similar native species. Two morphs of I. glandulifera were both highly salient in chromatic and achromatic terms and different from their surrounding native species. This distinctive identity facilitates detection and learning in association with rich nectar. While visual signals are not the only sensory cue accounting for invasive-plant success, our study reveals new elements for understanding biological invasion processes from the perspective of pollinator perceptual processes.

Asymmetric distribution of color-opponent response types across mouse visual cortex supports superior color vision in the sky.

Color is an important visual feature that informs behavior, and the retinal basis for color vision has been studied across various vertebrate species. While many studies have investigated how color information is processed in visual brain areas of primate species, we have limited understanding of how it is organized beyond the retina in other species, including most dichromatic mammals. In this study, we systematically characterized how color is represented in the primary visual cortex (V1) of mice. Using large-scale neuronal recordings and a luminance and color noise stimulus, we found that more than a third of neurons in mouse V1 are color-opponent in their receptive field center, while the receptive field surround predominantly captures luminance contrast. Furthermore, we found that color-opponency is especially pronounced in posterior V1 that encodes the sky, matching the statistics of natural scenes experienced by mice. Using unsupervised clustering, we demonstrate that the asymmetry in color representations across cortex can be explained by an uneven distribution of green-On/UV-Off color-opponent response types that are represented in the upper visual field. Finally, a simple model with natural scene-inspired parametric stimuli shows that green-On/UV-Off color-opponent response types may enhance the detection of "predatory"-like dark UV-objects in noisy daylight scenes. The results from this study highlight the relevance of color processing in the mouse visual system and contribute to our understanding of how color information is organized in the visual hierarchy across species.

Differences in small-breed dogs' body language and vocalizations in a negative context

A dog's emotional state is communicated primarily through body language, readily observable by humans and other animals. Additionally, dogs express visual and auditory signals differently according to the situation they are experiencing. However, there is a need for more research on smaller breeds since they tend to display more behavioral issues than larger breeds. The objective of the present study was to identify the behavioral characteristics of 25 dogs of three different small breeds (Chihuahua, Toy Poodle, and Miniature Schnauzer) in a negative context. We used a web-based tool called the Dog Actions Video Annotation Interface (DAVAI) to evaluate the movements of the tail, legs, snout, and vocalizations when the dogs were exposed to unfamiliar people. Our results showed no significant differences in dog behavior associated with breed. However, we identified variations in dog vocalizations explained by the sex, with male dogs whining more frequently than females. We highlight the importance of understanding dog body language and vocalizations, regardless of their breed. Our study is particularly relevant in negative contexts, where dogs may exhibit distress signals that require prompt intervention to alleviate their suffering.

Oculomotor behavior can be adjusted on the basis of artificial feedback signals indicating externally caused errors.

Whether a saccade is accurate and has reached the target cannot be evaluated during its execution, but relies on post-saccadic feedback. If the eye has missed the target object, a secondary corrective saccade has to be made to align the fovea with the target. If a systematic post-saccadic error occurs, adaptive changes to the oculomotor behavior are made, such as shortening or lengthening the saccade amplitude. Systematic post-saccadic errors are typically attributed internally to erroneous motor commands. The corresponding adaptive changes to the motor command reduce the error and the need for secondary corrective saccades, and, in doing so, restore accuracy and efficiency. However, adaptive changes to the oculomotor behavior also occur if a change in saccade amplitude is beneficial for task performance, or if it is rewarded. Oculomotor learning thus is more complex than reducing a post-saccadic position error. In the current study, we used a novel oculomotor learning paradigm and investigated whether human participants are able to adapt their oculomotor behavior to improve task performance even when they attribute the error externally. The task was to indicate the intended target object among several objects to a simulated human-machine interface by making eye movements. The participants were informed that the system itself could make errors. The decoding process depended on a distorted landing point of the saccade, resulting in decoding errors. Two different types of visual feedback were added to the post-saccadic scene and we compared how participants used the different feedback types to adjust their oculomotor behavior to avoid errors. We found that task performance improved over time, regardless of the type of feedback. Thus, error feedback from the simulated human-machine interface was used for post-saccadic error evaluation. This indicates that 1) artificial visual feedback signals and 2) externally caused errors might drive adaptive changes to oculomotor behavior.

Coded aperture compressive temporal imaging via unsupervised lightweight local-global networks with geometric characteristics

Coded aperture compressive temporal imaging (CACTI) utilizes compressive sensing (CS) theory to compress three dimensional (3D) signals into 2D measurements for sampling in a single snapshot measurement, which in turn acquires high-dimensional (HD) visual signals. To solve the problems of low quality and slow runtime often encountered in reconstruction, deep learning has become the mainstream for signal reconstruction and has shown superior performance. Currently, however, impressive networks are typically supervised networks with large-sized models and require vast training sets that can be difficult to obtain or expensive. This limits their application in real optical imaging systems. In this paper, we propose a lightweight reconstruction network that recovers HD signals only from compressed measurements with noise and design a block consisting of convolution to extract and fuse local and global features, stacking multiple features to form a lightweight architecture. In addition, we also obtain unsupervised loss functions based on the geometric characteristics of the signal to guarantee the powerful generalization capability of the network in order to approximate the reconstruction process of real optical systems. Experimental results show that our proposed network significantly reduces the model size and not only has high performance in recovering dynamic scenes, but the unsupervised video reconstruction network can approximate its supervised version in terms of reconstruction performance.

FedCMD: A Federated Cross-modal Knowledge Distillation for Drivers’ Emotion Recognition

Emotion recognition has attracted a lot of interest in recent years in various application areas such as healthcare and autonomous driving. Existing approaches to emotion recognition are based on visual, speech, or psychophysiological signals. However, recent studies are looking at multimodal techniques that combine different modalities for emotion recognition. In this work, we address the problem of recognizing the user’s emotion as a driver from unlabeled videos using multimodal techniques. We propose a collaborative training method based on cross-modal distillation, i.e., “FedCMD” (Federated Cross-Modal Distillation). Federated Learning (FL) is an emerging collaborative decentralized learning technique that allows each participant to train their model locally to build a better generalized global model without sharing their data. The main advantage of FL is that only local data is used for training, thus maintaining privacy and providing a secure and efficient emotion recognition system. The local model in FL is trained for each vehicle device with unlabeled video data by using sensor data as a proxy. Specifically, for each local model, we show how driver emotional annotations can be transferred from the sensor domain to the visual domain by using cross-modal distillation. The key idea is based on the observation that a driver’s emotional state indicated by a sensor correlates with facial expressions shown in videos. The proposed “FedCMD” approach is tested on the multimodal dataset “BioVid Emo DB” and achieves state-of-the-art performance. Experimental results show that our approach is robust to non-identically distributed data, achieving 96.67% and 90.83% accuracy in classifying five different emotions with IID (independently and identically distributed) and non-IID data, respectively. Moreover, our model is much more robust to overfitting, resulting in better generalization than the other existing methods.

Transcriptome and proteome revealed the differences in 3 colors of earlobe in Jiangshan Black-bone chicken

The earlobe is a featherless, exposed thickening located beneath the ear canal of chickens, which plays a visual signaling role in age, performance, mental vitality, reproduction, and other aspects. However, despite its importance, there have been few studies on the color differences and formation mechanisms of chicken earlobes, particularly the structurally blue earlobes characteristic of the Jiangshan black-bone chicken. In this study, we explored the physiological mechanisms that may influence the formation of differently colored earlobes using 3 types of earlobes from Jiangshan black-bone chickens: light peacock green (Green group), dark peacock green (Blue group), and dark reddish purple (Black group). All 3 earlobe colors exhibited positive melanin Masson-Fontana staining, and the thickness of collagen fibers in the dermis decreased in the order of Green, Blue, and Black groups. A total of 1,953 differentially expressed genes (DEGs) were detected in the 3 earlobes through mRNA sequencing, among which the GO term "collagen trimer" was significantly enriched in DEGs between groups. Additionally, 716 differentially expressed proteins (DEPs) were identified in the 3 earlobes using 4D-DIA proteomics, with the term "collagen fibril organization" being significantly enriched in DEPs between the Green and Black groups. Integrated analysis of transcriptome and proteome data revealed that 12 DEGs and DEPs were commonly differentially expressed between the Green and Black groups, including the gene LUM (corneal keratan sulfate proteoglycan), which was significantly enriched in the "collagen fibril organization" GO term. In conclusion, our study suggests that LUM plays a crucial role in the formation of peacock green earlobes in Jiangshan black-bone chickens. The high level of LUM in peacock green (Green and Blue groups) may affect collagen nanostructures, leading to a stronger effect of melanin-supported dermal collagen on the production of non-iridescent structural colors through coherent scattering, resulting in a bright structural blue color in Jiangshan black-bone chickens. In contrast, the low expression of LUM in dark reddish purple (Black group) reduces the reflection of non-iridescent structural colors, making the earlobe color appear almost black, similar to melanin.