Visual Signals Research Articles

Survey on Visual Signal Coding and Processing With Generative Models: Technologies, Standards, and Optimization

Survey on Visual Signal Coding and Processing With Generative Models: Technologies, Standards, and Optimization

Guest Editorial Advances in Generative Visual Signal Coding and Processing

Guest Editorial Advances in Generative Visual Signal Coding and Processing

A potential-controlled electrochromic visual biosensor based on distance readout for zearalenone detection

In this work, a potential-controlled electrochromic visual biosensor was developed for detecting zearalenone (ZEN) using a distance readout strategy. The sensor chip includes a square detection area and a folded signal output area created with laser etching technology. The detection area is modified with graphene oxide and ZEN aptamer, while Prussian blue (PB) is electrodeposited onto the signal output channel. When an appropriate voltage is applied, PB in the signal output area is reduced to colorless Prussian white (PW). The target ZEN molecules have the capability to release aptamers from graphene oxide (GO) surface in the detection area, resulting in a subsequent change in the potential of the visual signal output channel. This change determines the length of the channel that changes from blue to colorless, with the color change distance being proportional to the ZEN concentration. Using this distance readout strategy, ZEN detection within the range of 1 ng/mL to 300 ng/mL was achieved, with a detection limit of 0.29 ng/mL.

Shining a light on UV-fluorescent floral nectar after 50 years

Nature is aglow with numerous captivating examples of UV-fluorescence in the animal kingdom. Despite a putative role as a visual signal, exploration of UV-fluorescence in plants and its role in plant-animal interactions is lagging in comparison. Almost 50 years ago, UV-fluorescence of floral nectar, a crucial reward for pollinators, was reported for 23 flowering plant species. Since this intriguing discovery, UV-fluorescent nectar has only seldom been addressed in the scientific literature and has not been scrutinized in a phylogenetic or ecological context. Here, we report the prevalence of vibrant UV-fluorescent floral nectar across the family Cleomaceae, including the first photographic documentation in vivo colour for flowering plants. Though Cleomaceae flowers are morphologically diverse varying in colour, nectary prominence, and nectar volume, UV-fluorescent floral nectar may be a ubiquitous characteristic of the family. Fluorescence spectra show that the identity and number of fluorescent compounds in floral nectar may differ among Cleomaceae species. As Cleomaceae pollinators range from insects to bats and birds, we suggest that the UV-fluorescent floral nectar not only functions as a visual cue for the diurnal pollinators but also for the nocturnal/crepuscular pollinators in low light settings.

Male proboscis monkey cranionasal size and shape is associated with visual and acoustic signalling

The large nose adorned by adult male proboscis monkeys is hypothesised to serve as an audiovisual signal of sexual selection. It serves as a visual signal of male quality and social status, and as an acoustic signal, through the expression of loud, low-formant nasalised calls in dense rainforests, where visibility is poor. However, it is unclear how the male proboscis monkey nasal complex, including the internal structure of the nose, plays a role in visual or acoustic signalling. Here, we use cranionasal data to assess whether large noses found in male proboscis monkeys serve visual and/or acoustic signalling functions. Our findings support a visual signalling function for male nasal enlargement through a relatively high degree of nasal aperture sexual size dimorphism, the craniofacial region to which nasal soft tissue attaches. We additionally find nasal aperture size increases beyond dental maturity among male proboscis monkeys, consistent with the visual signalling hypothesis. We show that the cranionasal region has an acoustic signalling role through pronounced nasal cavity sexual shape dimorphism, wherein male nasal cavity shape allows the expression of loud, low-formant nasalised calls. Our findings provide robust support for the male proboscis monkey nasal complex serving both visual and acoustic functions.

Color-advertising strategies of invasive plants through the bee eye.

Invasive plants represent a significant global challenge as they compete with native plants for limited resources such as space, nutrients and pollinators. Here, we focused on four invasive species that are widely spread in the French Pyrenees, Buddleja davidii, Reynoutria japonica, Spiraea japonica and Impatiens glandulifera, and analyzed their visual advertisement signals with respect to those displayed by their surrounding native species using a perceptual approach based on the neural mechanisms of bee vision given that bees are regular pollinators of these plants. We collected 543 spectral reflections from the 4 invasive species, and 66 native species and estimated achromatic and chromatic similarities to the bee eye. R. japonica, S. japonica and B. davidii were inconspicuous against the foliage background and could be hardly discriminated in terms of color from their surrounding native plants. These characteristics promote generalization, potentially attracting pollinators foraging on similar native species. Two morphs of I. glandulifera were both highly salient in chromatic and achromatic terms and different from their surrounding native species. This distinctive identity facilitates detection and learning in association with rich nectar. While visual signals are not the only sensory cue accounting for invasive-plant success, our study reveals new elements for understanding biological invasion processes from the perspective of pollinator perceptual processes.

Asymmetric distribution of color-opponent response types across mouse visual cortex supports superior color vision in the sky.

Color is an important visual feature that informs behavior, and the retinal basis for color vision has been studied across various vertebrate species. While many studies have investigated how color information is processed in visual brain areas of primate species, we have limited understanding of how it is organized beyond the retina in other species, including most dichromatic mammals. In this study, we systematically characterized how color is represented in the primary visual cortex (V1) of mice. Using large-scale neuronal recordings and a luminance and color noise stimulus, we found that more than a third of neurons in mouse V1 are color-opponent in their receptive field center, while the receptive field surround predominantly captures luminance contrast. Furthermore, we found that color-opponency is especially pronounced in posterior V1 that encodes the sky, matching the statistics of natural scenes experienced by mice. Using unsupervised clustering, we demonstrate that the asymmetry in color representations across cortex can be explained by an uneven distribution of green-On/UV-Off color-opponent response types that are represented in the upper visual field. Finally, a simple model with natural scene-inspired parametric stimuli shows that green-On/UV-Off color-opponent response types may enhance the detection of "predatory"-like dark UV-objects in noisy daylight scenes. The results from this study highlight the relevance of color processing in the mouse visual system and contribute to our understanding of how color information is organized in the visual hierarchy across species.

Differences in small-breed dogs' body language and vocalizations in a negative context

A dog's emotional state is communicated primarily through body language, readily observable by humans and other animals. Additionally, dogs express visual and auditory signals differently according to the situation they are experiencing. However, there is a need for more research on smaller breeds since they tend to display more behavioral issues than larger breeds. The objective of the present study was to identify the behavioral characteristics of 25 dogs of three different small breeds (Chihuahua, Toy Poodle, and Miniature Schnauzer) in a negative context. We used a web-based tool called the Dog Actions Video Annotation Interface (DAVAI) to evaluate the movements of the tail, legs, snout, and vocalizations when the dogs were exposed to unfamiliar people. Our results showed no significant differences in dog behavior associated with breed. However, we identified variations in dog vocalizations explained by the sex, with male dogs whining more frequently than females. We highlight the importance of understanding dog body language and vocalizations, regardless of their breed. Our study is particularly relevant in negative contexts, where dogs may exhibit distress signals that require prompt intervention to alleviate their suffering.

Oculomotor behavior can be adjusted on the basis of artificial feedback signals indicating externally caused errors.

Whether a saccade is accurate and has reached the target cannot be evaluated during its execution, but relies on post-saccadic feedback. If the eye has missed the target object, a secondary corrective saccade has to be made to align the fovea with the target. If a systematic post-saccadic error occurs, adaptive changes to the oculomotor behavior are made, such as shortening or lengthening the saccade amplitude. Systematic post-saccadic errors are typically attributed internally to erroneous motor commands. The corresponding adaptive changes to the motor command reduce the error and the need for secondary corrective saccades, and, in doing so, restore accuracy and efficiency. However, adaptive changes to the oculomotor behavior also occur if a change in saccade amplitude is beneficial for task performance, or if it is rewarded. Oculomotor learning thus is more complex than reducing a post-saccadic position error. In the current study, we used a novel oculomotor learning paradigm and investigated whether human participants are able to adapt their oculomotor behavior to improve task performance even when they attribute the error externally. The task was to indicate the intended target object among several objects to a simulated human-machine interface by making eye movements. The participants were informed that the system itself could make errors. The decoding process depended on a distorted landing point of the saccade, resulting in decoding errors. Two different types of visual feedback were added to the post-saccadic scene and we compared how participants used the different feedback types to adjust their oculomotor behavior to avoid errors. We found that task performance improved over time, regardless of the type of feedback. Thus, error feedback from the simulated human-machine interface was used for post-saccadic error evaluation. This indicates that 1) artificial visual feedback signals and 2) externally caused errors might drive adaptive changes to oculomotor behavior.

Coded aperture compressive temporal imaging via unsupervised lightweight local-global networks with geometric characteristics

Coded aperture compressive temporal imaging (CACTI) utilizes compressive sensing (CS) theory to compress three dimensional (3D) signals into 2D measurements for sampling in a single snapshot measurement, which in turn acquires high-dimensional (HD) visual signals. To solve the problems of low quality and slow runtime often encountered in reconstruction, deep learning has become the mainstream for signal reconstruction and has shown superior performance. Currently, however, impressive networks are typically supervised networks with large-sized models and require vast training sets that can be difficult to obtain or expensive. This limits their application in real optical imaging systems. In this paper, we propose a lightweight reconstruction network that recovers HD signals only from compressed measurements with noise and design a block consisting of convolution to extract and fuse local and global features, stacking multiple features to form a lightweight architecture. In addition, we also obtain unsupervised loss functions based on the geometric characteristics of the signal to guarantee the powerful generalization capability of the network in order to approximate the reconstruction process of real optical systems. Experimental results show that our proposed network significantly reduces the model size and not only has high performance in recovering dynamic scenes, but the unsupervised video reconstruction network can approximate its supervised version in terms of reconstruction performance.

FedCMD: A Federated Cross-modal Knowledge Distillation for Drivers’ Emotion Recognition

Emotion recognition has attracted a lot of interest in recent years in various application areas such as healthcare and autonomous driving. Existing approaches to emotion recognition are based on visual, speech, or psychophysiological signals. However, recent studies are looking at multimodal techniques that combine different modalities for emotion recognition. In this work, we address the problem of recognizing the user’s emotion as a driver from unlabeled videos using multimodal techniques. We propose a collaborative training method based on cross-modal distillation, i.e., “FedCMD” (Federated Cross-Modal Distillation). Federated Learning (FL) is an emerging collaborative decentralized learning technique that allows each participant to train their model locally to build a better generalized global model without sharing their data. The main advantage of FL is that only local data is used for training, thus maintaining privacy and providing a secure and efficient emotion recognition system. The local model in FL is trained for each vehicle device with unlabeled video data by using sensor data as a proxy. Specifically, for each local model, we show how driver emotional annotations can be transferred from the sensor domain to the visual domain by using cross-modal distillation. The key idea is based on the observation that a driver’s emotional state indicated by a sensor correlates with facial expressions shown in videos. The proposed “FedCMD” approach is tested on the multimodal dataset “BioVid Emo DB” and achieves state-of-the-art performance. Experimental results show that our approach is robust to non-identically distributed data, achieving 96.67% and 90.83% accuracy in classifying five different emotions with IID (independently and identically distributed) and non-IID data, respectively. Moreover, our model is much more robust to overfitting, resulting in better generalization than the other existing methods.

Transcriptome and proteome revealed the differences in 3 colors of earlobe in Jiangshan Black-bone chicken

The earlobe is a featherless, exposed thickening located beneath the ear canal of chickens, which plays a visual signaling role in age, performance, mental vitality, reproduction, and other aspects. However, despite its importance, there have been few studies on the color differences and formation mechanisms of chicken earlobes, particularly the structurally blue earlobes characteristic of the Jiangshan black-bone chicken. In this study, we explored the physiological mechanisms that may influence the formation of differently colored earlobes using 3 types of earlobes from Jiangshan black-bone chickens: light peacock green (Green group), dark peacock green (Blue group), and dark reddish purple (Black group). All 3 earlobe colors exhibited positive melanin Masson-Fontana staining, and the thickness of collagen fibers in the dermis decreased in the order of Green, Blue, and Black groups. A total of 1,953 differentially expressed genes (DEGs) were detected in the 3 earlobes through mRNA sequencing, among which the GO term "collagen trimer" was significantly enriched in DEGs between groups. Additionally, 716 differentially expressed proteins (DEPs) were identified in the 3 earlobes using 4D-DIA proteomics, with the term "collagen fibril organization" being significantly enriched in DEPs between the Green and Black groups. Integrated analysis of transcriptome and proteome data revealed that 12 DEGs and DEPs were commonly differentially expressed between the Green and Black groups, including the gene LUM (corneal keratan sulfate proteoglycan), which was significantly enriched in the "collagen fibril organization" GO term. In conclusion, our study suggests that LUM plays a crucial role in the formation of peacock green earlobes in Jiangshan black-bone chickens. The high level of LUM in peacock green (Green and Blue groups) may affect collagen nanostructures, leading to a stronger effect of melanin-supported dermal collagen on the production of non-iridescent structural colors through coherent scattering, resulting in a bright structural blue color in Jiangshan black-bone chickens. In contrast, the low expression of LUM in dark reddish purple (Black group) reduces the reflection of non-iridescent structural colors, making the earlobe color appear almost black, similar to melanin.

Exploring AKAPs in visual signaling.

The complex nature of the retina demands well-organized signaling to uphold signal accuracy and avoid interference, a critical aspect in handling a variety of visual stimuli. A-kinase anchoring proteins (AKAPs), known for binding protein kinase A (PKA), contribute to the specificity and efficiency of retinal signaling. They play multifaceted roles in various retinal cell types, influencing photoreceptor sensitivity, neurotransmitter release in bipolar cells, and the integration of visual information in ganglion cells. AKAPs like AKAP79/150 and AKAP95 exhibit distinct subcellular localizations, impacting synaptic transmission and receptor sensitivity in photoreceptors and bipolar cells. Furthermore, AKAPs are involved in neuroprotective mechanisms and axonal degeneration, particularly in retinal ganglion cells. In particular, AKAP6 coordinates stress-specific signaling and promotes neuroprotection following optic nerve injury. As our review underscores the therapeutic potential of targeting AKAP signaling complexes for retinal neuroprotection and enhancement, it acknowledges challenges in developing selective drugs that target complex protein-protein interactions. Overall, this exploration of AKAPs provides valuable insights into the intricacies of retinal signaling, offering a foundation for understanding and potentially addressing retinal disorders.

Laccase is a multitasking protein for synthetic gene circuits in the yeast Saccharomycescerevisiae

Laccase is a multicopper oxidase enzyme that oxidizes a variety of substrates, including polyphenols and polycyclic aromatic hydrocarbons (PAHs). It catalyzes the four-electron reduction of molecular oxygen that results in the production of water as a by-product. Thus, laccase can play an important role in environmental care. Previously, we have successfully expressed Trametes trogii laccase (TtLcc1) in the yeast Saccharomyces cerevisiae. In this work, we have expressed in yeast another laccase, LacA from Trametes sp. AH28-2, and tested its function on PAHs. Yeast cells engineered to produce the two laccases performed efficient PAH degradation. Both TtLcc1 and LacA led to the construction of spatiotemporal fluorescence-pulse generators when combined with a benzoate/salicylate yeast biosensor in a two-population system. Moreover, laccases returned a visual output signal in yeast synthetic circuits—upon reacting with ABTS (2,2′-azino-bis (3-ethylbenzthiazoline-6-sulfonic acid)). Thus, in S. cerevisiae, laccases are a powerful alternative to fluorescent reporter proteins.

Can Large Language Models Be Good Companions?

Developing chatbots as personal companions has long been a goal of artificial intelligence researchers. Recent advances in Large Language Models (LLMs) have delivered a practical solution for endowing chatbots with anthropomorphic language capabilities. However, it takes more than LLMs to enable chatbots that can act as companions. Humans use their understanding of individual personalities to drive conversations. Chatbots also require this capability to enable human-like companionship. They should act based on personalized, real-time, and time-evolving knowledge of their users. We define such essential knowledge as the common ground between chatbots and their users, and we propose to build a common-ground-aware dialogue system from an LLM-based module, named OS-1, to enable chatbot companionship. Hosted by eyewear, OS-1 can sense the visual and audio signals the user receives and extract real-time contextual semantics. Those semantics are categorized and recorded to formulate historical contexts from which the user's profile is distilled and evolves over time, i.e., OS-1 gradually learns about its user. OS-1 combines knowledge from real-time semantics, historical contexts, and user-specific profiles to produce a common-ground-aware prompt input into the LLM module. The LLM's output is converted to audio, spoken to the wearer when appropriate. We conduct laboratory and in-field studies to assess OS-1's ability to build common ground between the chatbot and its user. The technical feasibility and capabilities of the system are also evaluated. Our results show that by utilizing personal context, OS-1 progressively develops a better understanding of its users. This enhances user satisfaction and potentially leads to various personal service scenarios, such as emotional support and assistance.

Seeing without a Scene: Neurological Observations on the Origin and Function of the Dorsal Visual Stream.

In all vertebrates, visual signals from each visual field project to the opposite midbrain tectum (called the superior colliculus in mammals). The tectum/colliculus computes visual salience to select targets for context-contingent visually guided behavior: a frog will orient toward a small, moving stimulus (insect prey) but away from a large, looming stimulus (a predator). In mammals, visual signals competing for behavioral salience are also transmitted to the visual cortex, where they are integrated with collicular signals and then projected via the dorsal visual stream to the parietal and frontal cortices. To control visually guided behavior, visual signals must be encoded in body-centered (egocentric) coordinates, and so visual signals must be integrated with information encoding eye position in the orbit-where the individual is looking. Eye position information is derived from copies of eye movement signals transmitted from the colliculus to the frontal and parietal cortices. In the intraparietal cortex of the dorsal stream, eye movement signals from the colliculus are used to predict the sensory consequences of action. These eye position signals are integrated with retinotopic visual signals to generate scaffolding for a visual scene that contains goal-relevant objects that are seen to have spatial relationships with each other and with the observer. Patients with degeneration of the superior colliculus, although they can see, behave as though they are blind. Bilateral damage to the intraparietal cortex of the dorsal stream causes the visual scene to disappear, leaving awareness of only one object that is lost in space. This tutorial considers what we have learned from patients with damage to the colliculus, or to the intraparietal cortex, about how the phylogenetically older midbrain and the newer mammalian dorsal cortical visual stream jointly coordinate the experience of a spatially and temporally coherent visual scene.

Tunable Negative and Positive Photoconductance in Van Der Waals Heterostructure for Image Preprocessing.

The processing of visual information occurs mainly in the retina, and the retinal preprocessing function greatly improves the transmission quality and efficiency of visual information. The artificial retina system provides a promising path to efficient image processing. Here, graphene/InSe/h-BN heterogeneous structure is proposed, which exhibits negative and positive photoconductance (NPC and PPC) effects by altering the strength of a single wavelength laser. Moreover, a modified theoretical model is presented based on the power-dependent photoconductivity effect of laser: , which can reveal the internal physical mechanism of negative/positive photoconductance effects. The present 2D structure design allows the field effect transistor (FET) to exhibit excellent photoelectric performance (RNPC=1.1× 104AW-1, RPPC=13AW-1) and performance stability. Especially, the retinal pretreatment process is successfully simulated based on the negative and positive photoconductive effects. Moreover, the pulse signal input improves the device responsivity by 167%, and the transmission quality and efficiency of the visual signal can also be enhanced. This work provides a new design idea and direction for the construction of artificial vision, and lay a foundation for the integration of the next generation of optoelectronicdevices.

DESIGNING CLOTHING INSPIRED BY AVERSION THERAPY TO CREATE AWARENESS

To assist people in kicking cancer-causing habits, Fashion Fights Cancer presents a cutting-edge fashion collection that is modelled by aversion treatment methods. By combining psychology and fashion, this innovative method offers a special tool to encourage healthy lives and reduce the incidence of cancer. This collection offers an innovative and captivating approach to assist people in breaking bad habits by implementing the ideas of aversion therapy, which involves linking unpleasant stimuli with negative behaviours to prevent their recurrence. Every article of clothing in this collection is designed with attention to arouse feelings or visual signals that set up subconscious unpleasant reactions associated with particular actions that cause cancer. With thoughtfully chosen textures, colours, patterns, and design aspects, these pieces serve as concrete deterrents and reminders, encouraging wearers to reevaluate their decisions and make good transitions toward healthier ones. Fashion Fights Cancer allows people to address their habits in a novel and effective way by incorporating psychological concepts into fashion design. This compilation increases awareness of the link between lifestyle decisions and cancer risk while also promoting personal development. Fashion Fights Cancer promotes proactive health behaviours and emphasizes prevention in the fight against cancer by starting conversations and questioning social conventions through fashion. People are inspired to take control of their well-being and make wise decisions that prioritize long-term health and vitality thanks to this creative marriage of fashion and psychology. In this study, the clothes in the collection are designed to make people feel uncomfortable about habits that can cause cancer. They use colours, textures, and designs to remind people of these habits, encouraging them to think about making healthier choices instead. The goal is to use creativity to help people change their habits and improve their health. Combining fashion with psychology wants to help people understand how their lifestyle choices can affect their chances of getting cancer. They hope to inspire people to take better care of themselves and prevent cancer by starting conversations and encouraging healthy behaviours. This method also shows how using creative activities, like making clothes, can support people who are dealing with cancer.

From torches to trees: Political party logo changes and voter perception

Logos are a prominent visual signal used by virtually every European political party. Most of these parties have also changed their logo at some point in their history. Despite the ubiquitousness of both logos and logo changes, little is known about voter attitudes towards this element of the party brand. To examine these questions, we first provide a theoretical framework that can be used to understand party logos and logo changes. We then implement a survey experiment in the United Kingdom to determine if voters notice these logo changes, how these may affect the issues that voters associate with the party, and how competent they perceive the party to be on these issues. Our findings provide suggestive evidence that voters notice and consider changes to the party logo, and that these changes can alter the issues that voters associate with the party. These findings provide the groundwork for future studies on party logos.

How Biofeedback With Surface EMG Can Contribute to the Diagnosis and Treatment of AMI in the Knee

Background: Arthrogenic muscle inhibition (AMI) is a complex neurological phenomenon that is actually very common. AMI can appear following a knee injury or knee surgery. If it is not addressed, the patient's function can be negatively affected. There is currently no specific tool for the diagnosis and treatment of AMI. Indications: Following surgery or injury to the knee, AMI causes defective activation of the quadriceps, sometimes combined with a knee extension deficit. A clinical classification of AMI has been proposed recently to help with making the diagnosis and to guide treatment. In this article, we describe how biofeedback and surface electromyography (EMG) can be used in the early diagnosis and treatment of knee AMI. Technique Description: Biofeedback is based on transformation of physiological variables into a visual or auditory signal. This allows the patient to learn how to control physical and bodily functions, which were previously considered as involuntary processes. This technique requires the use of a specific device that transforms the measured physiological signals—neuromuscular activation in this instance—into visual signals. This tool incorporates surface EMG, with electrodes connected to a measuring unit. This unit is connected to a screen that shows the neuromuscular activation in real time. Thus, EMG biofeedback can be used to help clinicians make a diagnosis and to specifically treat the AMI. Results: We have used biofeedback in patients who have a knee injury or who have undergone knee surgery. Both the therapist and patient were able to objectively see the presence of AMI using the biofeedback device. For treatment, the therapist works with a patient by providing clear instructions and increasingly more challenging goals. We show examples of how patients were able to eliminate their AMI by using biofeedback. Discussion/Conclusion: EMG biofeedback is a promising tool for the diagnosis and treatment of AMI after an anterior cruciate ligament (ACL) tear and ACL reconstruction surgery. The next steps are to define thresholds and a rehabilitation protocol that will be used in a clinical trial. Patient Consent Disclosure Statement: The author(s) attests that consent has been obtained from any patient(s) appearing in this publication. If the individual may be identifiable, the author(s) has included a statement of release or other written form of approval from the patient(s) with this submission for publication.