Acquisition Of Visual Information Research Articles

Using eye tracking to assess learning of a multifunction prosthetic hand: an exploratory study from a rehabilitation perspective

BackgroundEye tracking technology not only reveals the acquisition of visual information at fixation but also has the potential to unveil underlying cognitive processes involved in learning to use a multifunction prosthetic hand. It also reveals gaze behaviours observed during standardized tasks and self-chosen tasks. The aim of the study was to explore the use of eye tracking to track learning progress of multifunction hands at two different time points in prosthetic rehabilitation.MethodsThree amputees received control training of a multifunction hand with new control strategy. Detailed description of control training was collected first. They wore Tobii Pro2 eye-tracking glasses and performed a set of standardized tasks (required to switch to different grips for each task) after one day of training and at one-year-follow-up (missing data for Subject 3 at the follow up due to socket problem). They also performed a self-chosen task (free to use any grip for any object) and were instructed to perform the task in a way how they would normally do at home. The gaze-overlaid videos were analysed using the Tobii Pro Lab and the following metrics were extracted: fixation duration, saccade amplitude, eye-hand latency, fixation count and time to first fixation.ResultsDuring control training, the subjects learned 3 to 4 grips. Some grips were easier, and others were more difficult because they forgot or were confused with the switching strategies. At the one-year-follow-up, a decrease in performance time, fixation duration, eye-hand latency, and fixation count was observed in Subject 1 and 2, indicating an improvement in the ability to control the multifunction hand and a reduction of cognitive load. An increase in saccade amplitude was observed in both subjects, suggesting a decrease in difficulty to control the prosthetic hand. During the standardized tasks, the first fixation of all three subjects were on the multifunction hand in all objects. During the self-chosen tasks, the first fixations were mostly on the objects first.ConclusionThe qualitative data from control training and the quantitative eye tracking data from clinical standardized tasks provided a rich exploration of cognitive processing in learning to control a multifunction hand. Many prosthesis users prefer multifunction hands and with this study we have demonstrated that a targeted prosthetic training protocol with reliable assessment methods will help to lay the foundation for measuring functional benefits of multifunction hands.

Visual sequence encoding is modulated by music schematic structure and familiarity.

Music is omnipresent in daily life and may interact with critical cognitive processes including memory. Despite music's presence during diverse daily activities including studying, commuting, or working, existing literature has yielded mixed results as to whether music improves or impairs memory for information experienced in parallel. To elucidate how music memory and its predictive structure modulate the encoding of novel information, we developed a cross-modal sequence learning task during which participants acquired sequences of abstract shapes accompanied with paired music. Our goal was to investigate whether familiar and structurally regular music could provide a "temporal schema" (rooted in the organized and hierarchical structure of music) to enhance the acquisition of parallel temporally-ordered visual information. Results revealed a complex interplay between music familiarity and music structural regularity in learning paired visual sequences. Notably, compared to a control condition, listening to well-learned, regularly-structured music (music with high predictability) significantly facilitated visual sequence encoding, yielding quicker learning and retrieval speed. Conversely, learned but irregular music (where music memory violated musical syntax) significantly impaired sequence encoding. While those findings supported our mechanistic framework, intriguingly, unlearned irregular music-characterized by the lowest predictability-also demonstrated memory enhancement. In conclusion, this study demonstrates that concurrent music can modulate visual sequence learning, and the effect varies depending on the interaction between both music familiarity and regularity, offering insights into potential applications for enhancing human memory.

A Study on the Willingness of “Generation Z” Consumers to Use Online Virtual Try-On Shopping Services Based on the S-O-R Framework

Virtual try-on services, which significantly reduce return rates and enhance user shopping experiences, pose a crucial question: how can user willingness to use these services be increased? Additionally, “Generation Z” consumers, known for their unique traits and significant consumption potential, have been relatively understudied in this context. To address this theoretical gap, this study employs the Stimulus–Organism–Response (S-O-R) theory as its research framework, using online virtual shoe try-on services as a case study. Focusing on “Generation Z” consumers, this study utilizes literature review, user research, factor analysis, and linear regression to establish a user experience evaluation scale and behavior model. The user experience evaluation scale identifies positive elements such as convenience, price value, visual information acquisition, emotional value, and social interaction. The negative elements include technical limitations, personalized service deficiencies, and uncertainty. In the behavior model, all elements influence user attitudes. Notably, only the negative elements directly affect users’ willingness to use the service, except social interaction among the positive elements, which positively impacts usage intention. This study also reveals that “Generation Z” consumers have novel requirements for social interaction. This research effectively addresses existing theoretical gaps and provides a foundational theory for the development of related services and technologies.

WatchCap: Improving Scanning Efficiency in People with Low Vision through Compensatory Head Movement Stimulation

Individuals with low vision (LV) frequently face challenges in scanning performance, which in turn complicates daily activities requiring visual recognition. Although those with PVL can theoretically compensate for these scanning deficiencies through the use of active head movements, few practical applications have sought to capitalize on this potential, especially during visual recognition tasks. In this paper, we present WatchCap, a novel device that leverages the hanger reflex phenomenon to naturally elicit head movements through stimulation feedback. Our user studies, conducted with both sighted individuals in a simulated environment and people with glaucoma-related PVL, demonstrated that WatchCap's scanning-contingent stimulation enhances visual exploration. This improvement is evidenced by the fixation and saccade-related features and positive feedback from participants, which did not cause discomfort to the users. This study highlights the promise of facilitating head movements to aid those with LVs in visual recognition tasks. Critically, since WatchCap functions independently of predefined or task-specific cues, it has a wide scope of applicability, even in ambient task situations. This independence positions WatchCap to complement existing tools aimed at detailed visual information acquisition, allowing integration with existing tools and facilitating a comprehensive approach to assisting individuals with LV.

Enhancing Computational Efficiency in Event-Based Optical Camera Communication Using N-Pulse Modulation

Event cameras are bio-inspired devices that have revolutionized the acquisition of visual information by mimicking the neural architecture of the eye. These cameras respond asynchronously to changes in scene illumination at the pixel level, providing high-precision time information with low latency, typically in the order of microseconds. In this work, we experimentally evaluate an optical camera communication (OCC) link using an LED-based transmitter and an event camera as the receiver. We propose n-pulse modulation to encode data, adapting the system to the specific characteristics and operational principles of event cameras. The proposed scheme significantly reduces the demodulation complexity compared to other alternatives found in the literature. Furthermore, a set of experiments considering different camera bias sensitivities, encoding duty cycles, and LED radiant fluxes were carried out. The results showed that the BER performance was strongly dependent on the received optical power and the bias sensitivity. In addition, duty cycles between 0.3 and 0.7 at a 200 Hz transmission frequency presented the best performance, with a BER below 1.25×10−4, which is under the forward error correction (FEC) limit. This work showcases the cutting-edge capabilities of event-camera-based OCC technology and contributes to the ongoing revolution in optical wireless communication (OWC).

Image-free single-pixel keypoint detection for privacy preserving human pose estimation.

Computer vision technology has been applied in various fields such as identification, surveillance, and robot vision. However, computer vision algorithms used for human-related tasks operate on human images, which raises data security and privacy concerns. In this Letter, we propose an image-free human keypoint detection technique using a few coded illuminations and a single-pixel detector. Our proposed method can complete the keypoint detection task at an ultralow sampling rate on a measured one-dimensional sequence without image reconstruction, thus protecting privacy from the data collection stage and preventing the acquisition of detailed visual information from the source. The network is designed to optimize both the illumination patterns and the human keypoint predictor with an encoder-decoder framework. For model training and validation, we used 2000 images from Leeds Sport Dataset and COCO Dataset. By incorporating EfficientNet backbone, the inference time is reduced from 4 s to 0.10 s. In the simulation, the proposed network achieves 91.7% average precision. Our experimental results show an average precision of 88.4% at a remarkably low sampling rate of 0.015. In summary, our proposed method has the advantages of privacy protection and resource efficiency, which can be applied to many monitoring and healthcare tasks, such as clinical monitoring, construction site monitoring, and home service robots.

Comprehensive visual information acquisition for tomato picking robot based on multitask convolutional neural network

The tomato picking robot's vision system faces two difficult tasks: precise tomato pose acquisition and stem location. Tomato pose and stem location can help determine the end effector pose and achieve collision-free picking. To realise efficient crop picking, the tasks of target location, pose detection, and obstacle semantic segmentation should be completed in one model to obtain comprehensive visual information. Therefore, the multitask convolutional neural network YOLO-MCNN is proposed, a new method to complete the above tasks in one model. By fusing multi-scale features and determining the optimal locations for the semantic segmentation branch, four strategies are proposed for enhancing the segmentation ability. The experiment results show that fusing the semantic segmentation branch with the second layer of shallow feature maps and placing the branch after the 17th layer can result in the best segmentation performance. Fusing shallow feature maps improves small target detection while merging multi-scale feature maps enhances semantic segmentation performance. Moreover, ablation experiments are conducted to understand the influence between multitask convolutional and single task networks. It proves that running multiple tasks on the same backbone network does not affect their performance. The YOLO-MCNN's target detection performance F1 is 87.8%, the semantic segmentation performance mIoU is 74.8%, the keypoint detection performance dlmk is 6.95 pixels, the network size is 15.2 MB, and the inference speed is 19.9ms. Compared with other target detection and semantic segmentation networks, it shows that the comprehensive performance of the YOLO-MCNN is the best. The method provides theoretical foundation for constructing multitask convolutional neural networks.

Information Difference of Transfer Entropies between Head Motion and Eye Movement Indicates a Proxy of Driving.

Visual scanning is achieved via head motion and gaze movement for visual information acquisition and cognitive processing, which plays a critical role in undertaking common sensorimotor tasks such as driving. The coordination of the head and eyes is an important human behavior to make a key contribution to goal-directed visual scanning and sensorimotor driving. In this paper, we basically investigate the two most common patterns in eye-head coordination: "head motion earlier than eye movement" and "eye movement earlier than head motion". We utilize bidirectional transfer entropies between head motion and eye movements to determine the existence of these two eye-head coordination patterns. Furthermore, we propose a unidirectional information difference to assess which pattern predominates in head-eye coordination. Additionally, we have discovered a significant correlation between the normalized unidirectional information difference and driving performance. This result not only indicates the influence of eye-head coordination on driving behavior from a computational perspective but also validates the practical significance of our approach utilizing transfer entropy for quantifying eye-head coordination.

Online defect detection method of optical cable pitch based on machine vision technology and deep learning algorithms

In the field of optical cable manufacturing, the pitch error of finished optical cable exceeding 2 % is considered a defect. Currently, there is no method for online detection of pitch defect in optical cable. Therefore, this paper proposes an online intelligent detection method for cable pitch defects. This method utilizes a multi-sensor fusion image acquisition platform consisting of a Doppler laser velocimeter and an industrial line scan CCD to achieve visual information acquisition of ultra long measured objects. A mathematical model that integrates cascaded dual deep convolutional networks and iterative approximation algorithms based on deep learning technology is proposed for defect detection of optical cable pitch. In order to verify the accuracy and stability of the on-line detection method, 6 types of optical cables were prepared for experimental testing. The experimental results show that the method is robust in the field of engineering detection, and the detection results meet the accuracy requirements.

Eye and head movements while encoding and recognizing panoramic scenes in virtual reality.

One approach to studying the recognition of scenes and objects relies on the comparison of eye movement patterns during encoding and recognition. Past studies typically analyzed the perception of flat stimuli of limited extent presented on a computer monitor that did not require head movements. In contrast, participants in the present study saw omnidirectional panoramic scenes through an immersive 3D virtual reality viewer, and they could move their head freely to inspect different parts of the visual scenes. This allowed us to examine how unconstrained observers use their head and eyes to encode and recognize visual scenes. By studying head and eye movement within a fully immersive environment, and applying cross-recurrence analysis, we found that eye movements are strongly influenced by the content of the visual environment, as are head movements-though to a much lesser degree. Moreover, we found that the head and eyes are linked, with the head supporting, and by and large mirroring the movements of the eyes, consistent with the notion that the head operates to support the acquisition of visual information by the eyes.

Eye drift during fixation predicts visual acuity

Visual acuity is commonly assumed to be determined by the eye optics and spatial sampling in the retina. Unlike a camera, however, the eyes are never stationary during the acquisition of visual information; a jittery motion known as ocular drift incessantly displaces stimuli over many photoreceptors. Previous studies have shown that acuity is impaired in the absence of retinal image motion caused by eye drift. However, the relation between individual drift characteristics and acuity remains unknown. Here, we show that a) healthy emmetropes exhibit a large variability in their amount of drift and that b) these differences profoundly affect the structure of spatiotemporal signals to the retina. We further show that c) the spectral distribution of the resulting luminance modulations strongly correlates with individual visual acuity and that d) natural intertrial fluctuations in the amount of drift modulate acuity. As a consequence, in healthy emmetropes, acuity can be predicted from the motor behavior elicited by a simple fixation task, without directly measuring it. These results shed new light on how oculomotor behavior contributes to fine spatial vision.

Impact of dual tasking on gaze behaviour and locomotor strategies adopted while circumventing virtual pedestrians during a collision avoidance task.

We investigated gaze behaviour and collision avoidance strategies in 16 healthy young individuals walking towards a goal while exposed to virtual pedestrians (VRPs) approaching from different directions (left, middle, right). This locomotor task and an auditory-based cognitive task were performed under single and dual-task conditions. Longer gaze fixation durations were observed on the approaching vs. other VRPs, with longer fixations devoted to the upper trunk and head compared to other body segments. Compared to other pedestrian approaches, the middle pedestrian received longer fixations and elicited faster walking speeds, larger onset distances of trajectory devitation and smaller obstacle clearances. Gaze and locomotor behaviours were similar between single and dual-task conditions but dual-task costs were observed for the cognitive task. The longer gaze fixations on approaching vs. other pedestrians suggest that enhanced visual attention is devoted to pedestrians posing a greater risk of collision. Likewise, longer gaze fixations for the middle pedestrians may be due to the greater collision risk entailed by this condition, and/or to the fact that this pedestrian was positioned in front of the end goal. Longer fixations on approaching VRPs' trunk and head may serve the purpose of anticipating their walking trajectory. Finally, the dual-task effects that were limited to the cognitive task suggest that healthy young adults prioritize the locomotor task and associated acquisition of visual information. The healthy patterns of visuomotor behaviour characterized in this study will serve as a basis for comparison to further understand defective collision avoidance strategies in patient populations.

이미지 기반 소셜 네트워크 서비스(SNS)의 디자인 활용에 관한 연구

Today, various communication activities represented by communication and exchange between people are being used as an important method for creative problem-solving in design. A social network service (SNS) called 'Pinterest' has the advantage of active communication and acquisition of visual information in the design field based on images, and is increasingly emerging. The purpose of this study is to examine the use cases and behaviors in the design field, where creative work is prioritized, in that design is the transmission and exchange of image information, and to explore the influence and direction it will have on basic design education. Accordingly, a questionnaire survey was conducted on 261 college students majoring in design, and meaningful data on the recent use of Pinterest was obtained. As a result of the survey, it was found that Pinterest, as a vast image-based SNS, has the ease of use to obtain design ideas suitable for one's major field, and actively participates in its use by logging in at least once a day. In addition, it was confirmed that most of the subjects recognized that Pinterest was more helpful in improving problem-solving ability and creativity than when not using Pinterest. It can be said that the significance of this study is that it has deduced a certain research result of acquiring basic data that can contemplate new possibilities of basic design education process and subject design in universities through this study.

Advance of Target Visual Information Acquisition Technology for Fresh Fruit Robotic Harvesting: A Review

In view of the continuous increase in labor costs for complex picking tasks, there is an urgent demand for intelligent harvesting robots in the global fresh fruit cultivation industry. Fruit visual information is essential to guide robotic harvesting. However, obtaining accurate visual information about the target is critical in complex agricultural environments. The main challenges include the image color distortion under changeable natural light, occlusions from the interlaced plant organs (stems, leaves, and fruits), and the picking point location on fruits with variable shapes and poses. On top of summarizing the current status of typical fresh fruit harvesting robots, this paper outlined the state-of-the-art advance of visual information acquisition technology, including image acquisition in the natural environment, fruit recognition from the complex backgrounds, target stereo locating and measurement, and fruit search among the plants. It then analyzed existing problems and raised future potential research trends from two aspects, multiple images fusion and self-improving algorithm model.

Abnormal Prefrontal Functional Connectivity Is Associated with Inflexible Information Processing in Patients with Autism Spectrum Disorder (ASD): An fNIRS Study.

Individuals with autism spectrum disorder (ASD) are characterized by impairments in flexibly acquiring and maintaining new information, as well as in applying learned information for problem solving. However, the neural mechanism underpinning such impairments remains unclear. This study investigated the flexibility in the acquisition and application of visual information in ASD (aged 14–21) when they performed the Wisconsin Card Sorting Test (WCST). Behavioral data including response accuracy and latency, and prefrontal hemodynamic data measured by functional near-infrared spectroscopy (fNIRS), were collected when individuals performed WCST. Canonical general linear model and functional connectivity analyses were performed to examine the prefrontal activation and synchronization patterns, respectively. Results showed that although ASD individuals (n = 29) achieved comparable accuracy rates when compared with age- and intelligence quotient (IQ)-matched typically developing (TD; n = 26) individuals (F1,53 = 3.15, p = 0.082), ASD individuals needed significantly more time to acquire and apply WCST card sorting rules (F1,53 = 17.92, p < 0.001). Moreover, ASD individuals showed significantly lower prefrontal functional connectivity than TD individuals during WCST (F1,42 = 9.99, p = 0.003). The hypoconnectivity in ASD individuals was highly significant in the right lateral PFC in the acquisition condition (p = 0.005) and in the bilateral lateral PFC in the application condition (ps = 0.006). Furthermore, slower WCST reaction time was correlated with lower bilateral lateral PFC functional connectivity only in the application condition (ps = 0.003) but not the acquisition condition. Impairment in information acquisition and application is evident in ASD individuals and is mediated by processing speed, which is associated with lower functional connectivity in the bilateral lateral PFC when these individuals apply learned rules to solve novel problems.

A Study on the Application of Computer-Aided Dual-Coding Theory in English Vocabulary Teaching

Learning English needs a huge amount of vocabulary recalls as well as multiple grammatical structures and practices. With the development and improvement of the humanization and quality of English teaching, the multidimensional teaching of English has formed a powerful learning model and teaching method. The teaching of dual-coding theory assisted by computer teaching software mainly focuses on the dual activation representation of language combined with nonverbal graphic recognition to complete the recognition of information. According to the dual-coding hypothesis, in Gilford’s intelligence theory, the capacity of spatial and information recognition and reading is highly significant. With the large amount of visual recognition and information acquisition, images involve spatial domain information reading, which is a learning method based on the principle of information feature recognition. Based on the principle of dual-coding theory, it is a very effective way to present information and recall specific information in the form of vision and language. In this study, the application research for the purpose of English vocabulary teaching is adopted that has strong innovative and practical characteristics. According to the steps and contents of English vocabulary learning, the study is carried out step by step from the context in detail, so it continuously improves the learning quality and efficiency of English vocabulary teaching. Vocabulary teaching can help the students better improve their cognitive level of vocabulary and meet the vocabulary test requirements of the new institution entrance examination. Further, how to improve the effect of senior high school English vocabulary teaching is a major topic in the field of senior high school English teaching. In order to sort out this issue, I proposed a computer-aided dual-coding theory to help quickly understand vocabulary usage in order to increase learning efficiency. The Paivio–Desrochers model is used in this study. In addition, this study also attempts to explore the application of dual-coding theory in senior high school English vocabulary teaching.

What These Findings Tell Us. Reply to Kelly et al. What Do These Findings Tell Us? Comment on "Tinella et al. Cognitive Efficiency and Fitness-to-Drive along the Lifespan: The Mediation Effect of Visuospatial Transformations. Brain Sci. 2021, 11, 1028".

The study of the contribution of spatial transformation skills to driving behavior is a research topic substantiated by scarce evidence. In previous studies, we found that mental rotation and perspective-taking skills have an influence on performance in driving tasks by conveying the distal effects of the general cognitive efficiency on the execution of driving maneuvers. Studies have provided evidence on the relevance of the cognitive processes of encoding, imagined rotation, and spatial orientation in the accuracy of both the vehicle management during stressful driving situations and the acquisition of visual information on the traffic scenario. Results can find applications in both the training and the assessment of fitness to drive, as well as in the study of interaction between the drivers and in-vehicle devices. The lack of cross-validations in path analysis models cannot be assumed, a priori, to be capitalizing on chance and as an example of bad science. The non-replicability of a study should be demonstrated before it is proclaimed. The purpose of this reply was to address the questions raised by Kelly et al. (2022)—that is, “Do these results seem replicable?” and “How do these results advance our understanding of brain function and/or human behavior?”—by providing additional information on the study in question.

Trasmigrazioni: cultura materiale e sviluppo dell'identità. Alcune riflessioni sulla produzione di San Vincenzo al Volturno

The text analyzes the ornamental culture of the monastery of San Vincenzo al Volturno, through the comparison between different classes of materials. Through the analogies between the ornamental motifs, we tried to reconstruct their creative process, on a perceptual basis. The study, in fact, focuses on the relationship between the individual and the environment, exploiting the theories of Gibson, applying them to a particular context such as monasteries. The goal is to demonstrate how the continuous use of space determines the acquisition of visual information, which, stored in memory, is re-proposed during the creative process. Especially, it is the daily journey of the rooms that guarantees the memorization of the signs and, at the same time, their updating, through the change of perspective that occurs with movement. In the last part, we discuss the perceptive effects of the aniconic paintings in the crypt of Joshua.

Approach Direction Prior to Landing Explains Patterns of Colour Learning in Bees.

Gaze direction is closely coupled with body movement in insects and other animals. If movement patterns interfere with the acquisition of visual information, insects can actively adjust them to seek relevant cues. Alternatively, where multiple visual cues are available, an insect’s movements may influence how it perceives a scene. We show that the way a foraging bumblebee approaches a floral pattern could determine what it learns about the pattern. When trained to vertical bicoloured patterns, bumblebees consistently approached from below centre in order to land in the centre of the target where the reward was located. In subsequent tests, the bees preferred the colour of the lower half of the pattern that they predominantly faced during the approach and landing sequence. A predicted change of learning outcomes occurred when the contrast line was moved up or down off-centre: learned preferences again reflected relative frontal exposure to each colour during the approach, independent of the overall ratio of colours. This mechanism may underpin learning strategies in both simple and complex visual discriminations, highlighting that morphology and action patterns determines how animals solve sensory learning tasks. The deterministic effect of movement on visual learning may have substantially influenced the evolution of floral signals, particularly where plants depend on fine-scaled movements of pollinators on flowers.

When cats need to see to step accurately?

Locomotion on complex terrains often requires vision. However, how vision serves locomotion is not well understood. Here, we asked when visual information necessary for accurate stepping is collected and how its acquisition relates to the step cycle. In cats of both sexes, we showed that a brief (200-400 ms) interruption of visual input can rapidly influence cat's walking along a horizontal ladder. Depending on the phase within the step cycle, a 200 ms period of darkness could be tolerated fully without any changes to the strides or could lead to minor increases of stride duration. The effects of 300-400 ms of visual input denial, which typically prolonged stances and/or swings, also depended on the phase of the darkness onset. The increase of the duration of strides was always shorter than the duration of darkness. We conclude that visual information for planning a swing is collected starting from the middle of the preceding stance until the beginning of the current swing. For a stance (and/or a swing of the other paw), visual information is collected starting from the end of the previous stance and until the middle of the current stance. Acquisition of visual information during these windows is not uniform but depends on the phase of the step cycle. Notably, both the extension of these windows and their non-homogeneity are closely related to the pattern of gaze behaviour in cats, described previously. This new knowledge will help to guide research and understanding of neuronal mechanisms of visuomotor integration and modulation of visual function by strides during locomotion. KEY POINTS: Cats, like humans, rely on vision to navigate in complex environments. In cats walking along a horizontally placed ladder, we show that visual information required for accurate stepping is collected in a non-uniform manner throughout the stride cycle. Brief denial of visual input during a swing prolongs the next stance of that forelimb. Denial of visual input during a stance prolongs this stance, as well as the next swing and stance. Denial during the first half of a stance has a greater effect than during the second half. The phase dependence of the use of vision for accurate stepping and the pattern of affected swings and stances are closely related to the previously described pattern of gaze behaviour in cats. This new knowledge opens new perspectives for research into neuronal mechanisms of visuomotor coordination and visual function during walking and for understanding related disorders.