Visual Information Research Articles

Tomato pose estimation using the association of tomato body and sepal

In facility horticulture smart farms, harvesting robotic systems to automate harvesting tasks are challenging due to the complex environment, irregular growth and fruits pose. To harvest a fruits, the vision information required by a harvesting robot is accurate position and pose. Especially, fruit pose information is essential for planning paths to avoid damaging stems, leaves, branches, and obstacles, preventing damage to the fruit and harvesting robots, and planning efficient harvest sequences. This paper presents a method that uses information from tomato sepals to estimate the orientation of tomatoes, which are a commonly grown crop in horticulture. First, we train a YOLOv8 model to detect and segment bodies and sepals of tomatoes. For robust training of the model, the training data is constructed using effective data augmentation, which synthesizes segmented foreground objects and inserting them into the background. Then, for finding association between the body and sepal of a tomato, we apply IoU score based matching and the Hungarian algorithm. Consequently, we obtain point clouds for both parts using RGB-D data. Finally, we compute the center point of each object by using spherical fitting and the statistics of the point cloud, respectively. Then, we estimate the pose of the tomato as a vector between two center points of the body and sepal. To accurately and practically evaluate the proposed method, we generated ground truth data using calibration patterns in tomato greenhouse. As the experimental results show, The segmentation results show that AP50,sepal is 94.7, AP50,tomato is 96.3, and mAP is 61.5. The result of the pose estimation for the pose validation dataset is a mean error angle of 6.79 ± 3.18 and angle errors of less than 10 degrees account for 87.2%. The total algorithm proposed in this paper requires about 0.038s for each tomato greenhouse image. As a result, our proposed pose estimation can be practically utilized in robot systems for tomato harvesting.

Solution-Processed Ultralow Voltage Organic Transistors with Sharp Switching for Adaptive Visual Perception.

Neuromorphic visual systems can emulate biological retinal systems to perceive visual information under different levels of illumination, making them considerable potential for future intelligent vehicles and vision automation. However, the complex circuits and high operating voltages of conventional artificial vision systems present great challenges for device integration and power consumption. Here, we report bioinspired synaptic transistors based on organic single crystal phototransistors, which exhibit excitation and inhibition synaptic plasticity with time-varying. By manipulating the charge dynamics of the trapping centers of organic crystal-electret vertical stacks, organic transistors can operate below 1V with record high on/off ratios close to 108 and sharp switching with a subthreshold swing of 59.8mV dec-1. Moreover, the approach offers visual adaptation with highly localized modulation and over 98.2% recognition accuracyunder different illumination levels. These bioinspired visual adaptation transistors offer great potential for simplifying the circuitry of artificial vision systems and will contribute to the development of machine vision applications. This article is protected by copyright. All rights reserved.

Contrast enhancement algorithm for infrared images based on atmospheric scattering model

With the rapid development of infrared technology, infrared cameras are widely used in the military, medical, and industrial fields. However, the thermal radiation information received by infrared cameras often undergoes degradation due to random factors such as scattering or reflection during transmission. Consequently, the final infrared image suffers from poor contrast and blurred vision. To address these problems, this paper proposes an infrared image contrast enhancement algorithm based on the atmospheric scattering model. Firstly, we modify the hypothesis of the dark channel prior theory, and the pseudo dark channel prior theory is deduced to meet the processing requirements of single-channel images. Secondly, we propose anti-target interference and filtering threshold method to accurately estimate the atmospheric illumination value and transmittance of the atmospheric scattering model. These methods effectively mitigate the influence of infrared targets on the atmospheric illumination value and eliminate the blocking artifact effect caused by pixel value jumps in the transmittance images. Furthermore, we employ adaptive histogram equalization with contrast limitation, highlighting the global contrast. This paper conducts experiments on the M3FD dataset and employs objective evaluations using Visible Edge (E), Figure Definition (FD), Peak Signal-to-Noise Ratio (PSNR), Information Entropy (IE), Structure Similarity Index Measure (SSIM), and Visual Information Fidelity (VIF). The results indicate that the proposed algorithm enhances the overall contrast of images while highlighting fine texture details, exhibiting good visual effects. Both subjective and objective evaluations outperform seven other contrast algorithms, demonstrating the effectiveness of the proposed method.

Adversarial Item Promotion on Visually-Aware Recommender Systems by Guided Diffusion

Visually-aware recommender systems have found widespread applications in domains where visual elements significantly contribute to the inference of users’ potential preferences. While the incorporation of visual information holds the promise of enhancing recommendation accuracy and alleviating the cold-start problem, it is essential to point out that the inclusion of item images may introduce substantial security challenges. Some existing works have shown that the item provider can manipulate item exposure rates to its advantage by constructing adversarial images. However, these works cannot reveal the real vulnerability of visually-aware recommender systems because (1) the generated adversarial images are markedly distorted, rendering them easily detected by human observers; (2) the effectiveness of these attacks is inconsistent and even ineffective in some scenarios or datasets. To shed light on the real vulnerabilities of visually-aware recommender systems when confronted with adversarial images, this paper introduces a novel attack method, IPDGI (Item Promotion by Diffusion Generated Image). Specifically, IPDGI employs a guided diffusion model to generate adversarial samples designed to promote the exposure rates of target items (e.g., long-tail items). Taking advantage of accurately modeling benign images’ distribution by diffusion models, the generated adversarial images have high fidelity with original images, ensuring the stealth of our IPDGI. To demonstrate the effectiveness of our proposed methods, we conduct extensive experiments on two commonly used e-commerce recommendation datasets (Amazon Beauty and Amazon Baby) with several typical visually-aware recommender systems. The experimental results show that our attack method significantly improves both the performance of promoting the long-tailed (i.e., unpopular) items and the quality of generated adversarial images.

Room corners and how they influence the memory of visual information arranged on walls

In formal and informal learning settings corresponding information units are often arranged in the space around the viewer. For example, teachers pin task relevant information on classroom walls, or museum curators arrange exhibits in museum halls. Often learners and visitors are expected to see meaningful relationships between these information units. Theoretically, Gestalt Psychology has been examining the effects of connecting and separating elements in visual information displays, leading to the question of whether these findings also hold in three-dimensional environments. Does the mostly rectangular form of our rooms also either highlight or downplay relations between information dispersed across a room? Three experiments using virtual rooms showed that the matching pairs of pictures were memorized better if both pictures were arranged on the same wall instead of across two adjacent walls: that is, the presence of a room corner between matching pairs decreased memory (Experiments 1–3). Additionally, the findings showed that the participants’ orientation of their central field of view during learning influenced the effect of corners on memory. When initially looking around in rooms, participants most often oriented the center of their field of view toward the middle of a wall (Experiments 1 and 2); however, if they were restricted to orienting their field of view toward corners, the corner effect on memory vanished (Experiment 3). These findings suggest that room characteristics influence the exploratory behavior of viewers, thereby also affecting their memories of the presented information.

Smart Assistive System for Visually Impaired using PI

Visually impaired individuals face significant challenges when navigating and engaging with their surroundings independently. Our solution, "Smart Assistive System for visually Impaired using pi" employs a Raspberry Pi and camera for real-time image capture, precise object classification (with over 90% accuracy), and auditory feedback. The project addresses a pressing need for greater inclusion and accessibility for the visually impaired, offering a cost-effective and innovative solution that converts visual information into non-visual cues. The "Caption-Speak" system holds the potential to significantly enhance the independence, mobility, and overall quality of life for visually impaired individuals

Discriminating orientation information with phase consistency in alpha and low-gamma frequency bands: an EEG study

Recent studies suggest that noninvasive imaging methods (EEG, MEG) in the human brain scalp can decode the content of visual features information (orientation, color, motion, etc.) in Visual-Working Memory (VWM). Previous work demonstrated that with the sustained low-frequency Event-Related Potential (ERP under 6 Hz) of scalp EEG distributions, it is possible to accurately decode the content of orientation information in VWM during the delay interval. In addition, previous studies showed that the raw data captured by a combination of the occi-parietal electrodes could be used to decode the orientation. However, it is unclear whether the orientation information is available in other frequency bands (higher than 6 Hz) or whether this information is feasible with fewer electrodes. Furthermore, the exploration of orientation information in the phase values of the signal has not been well-addressed. In this study, we propose that orientation information is also accessible through the phase consistency of the occipital region in the alpha band frequency. Our results reveal a significant difference between orientations within 200 ms after stimulus offset in early visual sensory processing, with no apparent effect in power and Event-Related Oscillation (ERO) during this period. Additionally, in later periods (420–500 ms after stimulus offset), a noticeable difference is observed in the phase consistency of low gamma-band activity in the occipital area. Importantly, our findings suggest that phase consistency between trials of the orientation feature in the occipital alpha and low gamma-band can serve as a measure to obtain orientation information in VWM. Furthermore, the study demonstrates that phase consistency in the alpha and low gamma band can reflect the distribution of orientation-selective neuron numbers in the four main orientations in the occipital area.

Special Issue on Methods in Visual Politics and Protest

This special issue forms the second part of a double issue on methods in visual politics and protest. It draws together five articles that provide new pathways for deconstructing visual political narratives and offers reflexive and nuanced accounts for researching visual data and information shared on social media platforms (here: TikTok, Instagram, Twitter/X, Facebook). They do so through the application of feminist mixed methods (femmix), cross-platform analysis, and context-aware, comparative, and triangulated approaches. Taken together, the double issue offers a substantive compendium of articles exploring the latest methodological developments in visual politics and protest.

Visual perception of group fatness as a precursor to social identity threat and safety

ABSTRACT Across two studies (N = 190), we evaluate how fat people perceive groups of bodies and how this visual processing predicts social identity threat and safety. We examined how fat people ensemble code – rapidly and simultaneously extract visual information from a group – body size from groups of bodies. Stimuli included sets of 3-d modeled bodies varying in body mass index (BMI; Study 1) and real photographs of groups of people with known BMI (Study 2). As hypothesized, we found that fat people were generally accurate at ensemble coding the average body size of arrays after 500 ms exposure, and that, particularly among fat people with medium and high stigma consciousness, arrays of increasing average BMI were associated with heightened feelings of identity safety. Our work suggests that the representation of fatness in one’s environment may be an important precursor to social identity threat and safety processes.

Envisioning Sight Beyond Sight: Revolutionizing Visionary Assistive Smart Glasses for Visually Challenged

This paper presents a groundbreaking development in assistive innovation through the improvement of visionary smart glasses custom-made for people with visual impedances. These smart glasses are equipped with a Raspberry Pi Camera module designed to capture images of printed materials such as daily papers, books, and other objects of interest. The captured visual information is then handled through advanced optical character recognition (OCR) technology, changing over it into text. Hence, the text information is consistently transformed into audible content using text-to-speech (TTS), which the user can tune in to through an associated headset.The main objective of these visionary smart glasses is to enable the visually challenged by giving real-time access to printed materials and improving their ability to independently engage with composed data. Furthermore, these smart glasses incorporate a second module outlined for cash reading, permitting users to precisely distinguish and manage distinctive categories of currency. This dual-functionality aims to extend availability and independence, subsequently cultivating more prominent inclusivity and quality of life for individuals with visual impairments.The advancement of these visionary smart glasses represents a critical step towards bridging the data gap that frequently hinders the visually challenged community. By consistently converting printed materials into audio and empowering quick and reliable cash identification, these inventive smart glasses are poised to revolutionise the way people with visual disabilities interact with their environment, eventually enhancing their freedom and overall well-being. Keywords— Envisioning, Revolutionizing, Raspberry Pi, Camera Module, OCR, TTS, Currency.

Empathy in schizophrenia: neural alterations during emotion recognition and affective sharing

IntroductionDeficits in emotion recognition and processing are characteristic for patients with schizophrenia [SCZ].MethodsWe targeted both emotion recognition and affective sharing, one in static and one in dynamic facial stimuli, during functional magnetic resonance imaging [fMRI] in 22 SCZ patients and 22 matched healthy controls [HC]. Current symptomatology and cognitive deficits were assessed as potential influencing factors.ResultsBehaviorally, patients only showed a prolonged response time in age-discrimination trials. For emotion-processing trials, patients showed a difference in neural response, without an observable behavioral correlate. During emotion and age recognition in static stimuli, a reduced activation of the bilateral anterior cingulate cortex [ACC] and the right anterior insula [AI] emerged. In the affective sharing task, patients showed a reduced activation in the left and right caudate nucleus, right AI and inferior frontal gyrus [IFG], right cerebellum, and left thalamus, key areas of empathy.DiscussionWe conclude that patients have deficits in complex visual information processing regardless of emotional content on a behavioral level and that these deficits coincide with aberrant neural activation patterns in emotion processing networks. The right AI as an integrator of these networks plays a key role in these aberrant neural activation patterns and, thus, is a promising candidate area for neurofeedback approaches.

“Love looks not with the eyes”: supranormal processing of emotional speech in individuals with late-blindness versus preserved processing in individuals with congenital-blindness

ABSTRACT Processing of emotional speech in the absence of visual information relies on two auditory channels: semantics and prosody. No study to date has investigated how blindness impacts this process. Two theories, Perceptual Deficit, and Sensory Compensation, yiled different expectations about the role of visual experience (or its lack thereof) in processing emotional speech. To test the effect of vision and early visual experience on processing of emotional speech, we compared individuals with congenital blindness (CB, n = 17), individuals with late blindness (LB, n = 15), and sighted controls (SC, n = 21) on identification and selective-attention of semantic and prosodic spoken-emotions. Results showed that individuals with blindness performed at least as well as SC, supporting Sensory Compensation and the role of cortical reorganisation. Individuals with LB outperformed individuals with CB, in accordance with Perceptual Deficit, supporting the role of early visual experience. The LB advantage was moderated by executive functions (working-memory). Namely, the advantage was erased for individuals with CB who showed higher levels of executive functions. Results suggest that vision is not necessary for processing of emotional speech, but early visual experience could improve it. The findings support a combination of the two aforementioned theories and reject a dichotomous view of deficiencies/enhancements of blindness.

An IoT-Based Automatic Vehicle Accident Detection and Visual Situation Reporting System

Road accidents are a major cause of injuries and deaths worldwide. Many accident victims lose their lives because of the late arrival of the emergency response team (ERT) at the accident site. Moreover, the ERT often lacks crucial visual information about the victims and the condition of the vehicles involved in the accident, leading to a less effective rescue operation. To address these challenges, a new Internet of Things (IoT)-based system is proposed that uses on-vehicle sensors to detect and report the accident to rescue operator without any human involvement. The sensor data are automatically transmitted to a remote server to create a visual representation of the accident vehicles (which existing systems lack), facilitating the situation-based rescue operation. The system tackles any false reporting issue and also sends alerts to the victim’s family. A mobile application has also been developed for eyewitnesses to manually report the accident. The proposed system is evaluated in a simulated environment using a remote-controlled car. The results show that the system is robust and effective, automatically generating visuals of accident vehicles to facilitate informed rescue operation. The system has the potential to aid the ERT in providing timely first aid and, thus, saving human lives.

Complexity matters: Normalization to prototypical viewpoint induces memory distortion along the vertical axis of scenes.

Scene memory is prone to systematic distortions potentially arising from experience with the external world. Boundary transformation, a well-known memory distortion effect along the near-far axis of the three-dimensional space, represents the observer's erroneous recall of scenes' viewing distance. Researchers argued that normalization to the prototypical viewpoint with the high-probability viewing distance influenced this phenomenon. Herein, we hypothesized that the prototypical viewpoint also exists in the vertical Angle of View (AOV) dimension and could cause memory distortion along scenes' vertical axis. Human subjects of both sexes were recruited to test this hypothesis and two behavioral experiments were conducted, revealing a systematic memory distortion in the vertical AOV in both the forced choice (n = 79) and free adjustment (n = 30) tasks. Furthermore, the regression analysis implied that the complexity information asymmetry in scenes' vertical axis and the independent subjective AOV ratings from a large set of online participants (n = 1208) could jointly predict AOV biases. Furthermore, in a functional Magnetic Resonance Imaging (fMRI) experiment (n = 24), we demonstrated the involvement of areas in the ventral visual pathway (V3/V4, PPA, and OPA) in AOV bias judgment. Additionally, in a Magnetoencephalography (MEG) experiment (n = 20), we could significantly decode the subjects' AOV bias judgments ∼140 ms after scene onset and the low-level visual complexity information around the similar temporal interval. These findings suggest that AOV bias is driven by the normalization process and associated with the neural activities in the early stage of scene processing.Significance Statement Perceiving a scene with high precision is critical for our navigation and interaction with the surrounding environment. However, systematic memory distortion is quite common. Herein, we discovered that scene memory could be distorted to the upper or lower visual field. According to the behavioral results, multiple measures of scenes' complexity information were critically involved in the formation of this memory distortion. Furthermore, the results support the normalization theory regarding the existence of a high-probability prototypical viewpoint in scene processing. Our findings also suggested that the normalization process-induced scene memory distortion could be beneficial for the observer's future action selection. The identified complexity measures could be used in designing Artificial Intelligence (AI) systems with navigational functions.

Crowdsourcing the Impact of Illustration in Cleft Surgery Education.

This study aimed to (1) assess layperson preferences for how surgical information is presented; (2) evaluate how the format of visual information relates to layperson comfort with undergoing surgery, perceptions of surgeon character traits, and beliefs about artistic skill impacting plastic surgery practice; and (3) identify sociodemographic characteristics associated with these outcomes. A survey was developed in which one of five standardized sets of information depicting a unilateral cleft lip repair was presented as (1) text alone, (2) quick sketches, (3) simple drawings, (4) detailed illustrations, or (5) photographs. Online crowdsourcing platform. Raters aged 18 years and older from the United States. None. After viewing the surgical information, participants answered three sets of Likert scale questions. Ratings were averaged to produce three composite scores assessing (1) comfort with undergoing surgery (2) perceptions of surgeon character traits, and (3) beliefs about plastic surgery and artistry. Four hundred seventy-nine participants were included. Surgeon character traits score was highest among participants who viewed detailed illustrations at 4.46 ± 0.59, followed by photographs at 4.43 ± 0.54, text alone at 4.28 ± 0.59, simple drawings at 4.17 ± 0.67, and quick sketches at 4.17 ± 0.71 (p = 0.0014). Participants who viewed detailed illustrations rated surgical comfort score and plastic surgery and artistry score highest, although differences did not achieve statistical significance. Viewing detailed cleft lip repair illustrations was significantly associated with positive perceptions of surgeon character traits. Our data help to contextualize methods of communication and education valued by the public when seeking cleft care.

WuMKG: a Chinese painting and calligraphy multimodal knowledge graph

Chinese Painting and Calligraphy (ChP&C) holds significant cultural value, representing integral aspects of both Chinese culture and global art. A considerable amount of ChP&C works are dispersed worldwide. With the emergence of digital humanities, a vast collection of cultural artifacts data is now available online. However, the online databases of these artifacts remain decentralized and diverse, posing significant challenges to their effective organization and utilization. Addressing this, our paper focuses on the Wu Men School of Painting and proposes a framework for constructing a multimodal knowledge graph for the ChP&C domain. We construct the domain ontology by analyzing the ChP&C knowledge schema. Then, we acquire knowledge from diverse data sources, including textual and visual information. To enhance data collection around collecting historical context and subject matter, we propose methods for seal extraction and subject extraction specific to ChP&C. We validate the effectiveness of these methods on the constructed dataset. Finally, we construct the Wu Men Multimodal Knowledge Graph (WuMKG) and implement applications such as cross-modal retrieval, knowledge-based question-answering and visualization.

Трансляция эмоций в аудиодескрипции к немецкоязычным детским анимационным фильмам

The aim of the study is to identify linguistic means and methods for representing emotions in specialized sound descriptions and commentaries (audio descriptions) intended for blind and visually impaired recipients and accompanying feature films. The article examines the features of a children’s ani-mated fairy tale film as a cinematographic genre. Considerable attention is paid to the characteristics of the film text as a semiotic system in which emotional meanings are formed at the verbal and non-verbal level. The scientific novelty of the study is due to the systematization of linguistic means that manifest basic emotions in the German-language audio description to the children’s animated film adaptation. As a result of the study, it was found that the transformation of visual emotional information into verbal information in the audio description text is based on the use of simple syntactic constructions, the models “verb + adjective”, “verb + adverb of manner”, “verb + participle”, as well as the use of phraseological units, colloquial expressions, similes and epithets, which is due to the age and sensory characteristics of young recipients with visual impairment, and serves to facilitate the perception of the transmitted information.

Text and image generation from intracranial electroencephalography using an embedding space for text and images

Objective. Invasive brain–computer interfaces (BCIs) are promising communication devices for severely paralyzed patients. Recent advances in intracranial electroencephalography (iEEG) coupled with natural language processing have enhanced communication speed and accuracy. It should be noted that such a speech BCI uses signals from the motor cortex. However, BCIs based on motor cortical activities may experience signal deterioration in users with motor cortical degenerative diseases such as amyotrophic lateral sclerosis. An alternative approach to using iEEG of the motor cortex is necessary to support patients with such conditions. Approach. In this study, a multimodal embedding of text and images was used to decode visual semantic information from iEEG signals of the visual cortex to generate text and images. We used contrastive language-image pretraining (CLIP) embedding to represent images presented to 17 patients implanted with electrodes in the occipital and temporal cortices. A CLIP image vector was inferred from the high-γ power of the iEEG signals recorded while viewing the images. Main results. Text was generated by CLIPCAP from the inferred CLIP vector with better-than-chance accuracy. Then, an image was created from the generated text using StableDiffusion with significant accuracy. Significance. The text and images generated from iEEG through the CLIP embedding vector can be used for improved communication.

The Evolution of Simplifying Heuristics in Visual Cognition: Categorization, Specialization, and Visual Illusions.

Animals live in visually complex environments. As a result, visual systems have evolved mechanisms that simplify visual processing and allow animals to focus on the information that is most relevant to adaptive decision making. This review explores two key mechanisms that animals use to efficiently process visual information: categorization and specialization. Categorization occurs when an animal's perceptual system sorts continuously varying stimuli into a set of discrete categories. Specialization occurs when particular classes of stimuli are processed using distinct cognitive operations that are not used for other classes of stimuli. We also describe a nonadaptive consequence of simplifying heuristics: visual illusions, where visual perception consistently misleads the viewer about the state of the external world or objects within it. We take an explicitly comparative approach by exploring similarities and differences in visual cognition across human and nonhuman taxa. Considering areas of convergence and divergence across taxa provides insight into the evolution and function of visual systems and associated perceptual strategies.

Visuotactile integration in individuals with fibromyalgia

Our brain constantly integrates afferent information, such as visual and tactile information, to perceive the world around us. According to the maximum-likelihood estimation (MLE) model, imprecise information will be weighted less than precise, making the multisensory percept as precise as possible. Individuals with fibromyalgia (FM), a chronic pain syndrome, show alterations in the integration of tactile information. This could lead to a decrease in their weight in a multisensory percept or a general disruption of multisensory integration, making it less beneficial. To assess multisensory integration, 15 participants with FM and 18 pain-free controls performed a temporal-order judgment task in which they received pairs of sequential visual, tactile (unisensory conditions), or visuotactile (multisensory condition) stimulations on the index and the thumb of the non-dominant hand and had to determine which finger was stimulated first. The task enabled us to measure the precision and accuracy of the percept in each condition. Results indicate an increase in precision in the visuotactile condition compared to the unimodal conditions in controls only, although we found no intergroup differences. The observed visuotactile precision was correlated to the precision predicted by the MLE model in both groups, suggesting an optimal integration. Finally, the weights of the sensory information were not different between the groups; however, in the group with FM, higher pain intensity was associated with smaller tactile weight. This study shows no alterations of the visuotactile integration in individuals with FM, though pain may influence tactile weight in these participants.