Visual Research Articles

Automated Identification of the Retinogeniculate Visual Pathway Using a High-Dimensional Tractography Atlas

Automated Identification of the Retinogeniculate Visual Pathway Using a High-Dimensional Tractography Atlas

Individual thalamic inhibitory interneurons are functionally specialized toward distinct visual features

Inhibitory interneurons in the dorsolateral geniculate nucleus (dLGN) are situated at the first central synapse of the image-forming visual pathway, but little is known about their function. Given their anatomy, they are expected to be multiplexors, integrating many different retinal channels along their dendrites. Here, using targeted single-cell-initiated rabies tracing, we found that mouse dLGN interneurons exhibit a degree of retinal input specialization similar to thalamocortical neurons. Some are anatomically highly specialized, for example, toward motion-selective information. Two-photon calcium imaging performed invivo revealed that interneurons are also functionally specialized. In mice lacking retinal horizontal direction selectivity, horizontal direction selectivity is reduced in interneurons, suggesting a causal link between input and functional specialization. Functional specialization is not only present at interneuron somata but also extends into their dendrites. Altogether, inhibitory interneurons globally display distinct visual features which reflect their retinal input specialization and are ideally suited to perform feature-selective inhibition.

JsonCurer: Data Quality Management for JSON Based on an Aggregated Schema.

High-quality data is critical to deriving useful and reliable information. However, real-world data often contains quality issues undermining the value of the derived information. Most existing research on data quality management focuses on tabular data, leaving semi-structured data under-exploited. Due to the schema-less and hierarchical features of semi-structured data, discovering and fixing quality issues is challenging and time-consuming. To address the challenge, this paper presents JsonCurer, an interactive visualization system to assist with data quality management in the context of JSON data. To have an overview of quality issues, we first construct a taxonomy based on interviews with data practitioners and a review of 119 real-world JSON files. Then we highlight a schema visualization that presents structural information, statistical features, and quality issues of JSON data. Based on a similarity-based aggregation technique, the visualization depicts the entire JSON data with a concise tree, where summary visualizations are given above each node, and quality issues are illustrated using Bubble Sets across nodes. We evaluate the effectiveness and usability of JsonCurer with two case studies. One is in the domain of data analysis while the other concerns quality assurance in MongoDB documents.

Type II RAF inhibitor tovorafenib in relapsed/refractory pediatric low-grade glioma (pLGG): Reversible decreases in growth velocity in the phase 2 FIREFLY-1 trial.

10036 Background: Tovorafenib is an investigational, selective, CNS-penetrant, type II RAF inhibitor. Results from the ongoing FIREFLY-1 (NCT04775485) phase 2 study (1) in BRAF-altered pLGG showed clinically meaningful tumor responses and a manageable safety profile with tovorafenib monotherapy. Decreased growth velocity (GV) was observed; this is an update on changes in GV in skeletally immature children receiving tovorafenib. Methods: A planned safety analysis was completed on August 8, 2023 and included 137 patients (Arm 1: 77 & Arm 2: 60) treated with ≥1 dose of tovorafenib in FIREFLY-1. This report provides additional follow-up on all cases of decreased growth velocity (GV), an adverse event of special interest (AESI), reported to the Sponsor’s global safety database (GSDB) as of January 19, 2024. Results: Decreased GV was reported in 40 (29.2%) of 137 patients; 26 (19%) had GV reduction ≥50% from baseline (BL). Of those with decreased GV, 30 (75.0%) had pre-existing neuromuscular or endocrine comorbidities that may affect normal growth, including 6 (15%) with precocious puberty being treated with a gonadotropin-releasing hormone analogue, and 9 (22.5%) with BL heights 2 standard deviations above or below average for age and sex. Nineteen of the 40 patients with this AESI had on-treatment bone age assessments; none showed advancement of bone age from BL or premature closure of growth plates. No osteopenia or abnormal fractures were reported. Of the 35 patients with growth hormone (GH) testing, only 2 had signs of GH deficiency (both had optic pathway tumors; 1 had GH deficiency at BL, the other, a new tumor-associated GH deficiency). Post-treatment heights ≥3 months off treatment were reported for 10 of 40 patients who had interrupted or discontinued treatment for any reason (mean follow up: 5.8 months), including 2 (1.5%) permanent discontinuations and 3 (2.2%) interruptions due to decreased GV. All 10 showed post-treatment recovery in annualized GV (AGV) (average on-treatment AGV 1.1 cm/y vs. average post-treatment AGV 8 cm/y), in some cases exceeding expected average AGV for age. One additional 4-year-old boy with on-treatment AGV of 1.2 cm/y had an off-treatment AGV of 12.3 cm/y after 2 months of off-treatment follow-up. Decreased GV was reported to the GSDB for 5 (11.4%) of 44 patients who received above the maximum dose of tovorafenib used in FIREFLY-1 in an ongoing investigator-initiated study. Four of the 5 patients from this study had ≥3 months of off-treatment follow-up reported; all 4 showed evidence of GV recovery. Conclusions: Decreased GV has been observed in patients treated with tovorafenib. Early follow-up data from patients whose treatment was interrupted show consistent evidence for GV recovery and preservation of growth potential on bone age studies. 1. Kilburn LK, et al. Nat Med.2023. Clinical trial information: NCT04775485 .

Music Beats - A Music Recommendation System using Facial Expressions

Abstract: Interest in the nexus between music recommendation systems and affective computing has grown recently. This article examines the state of the art in music recommendation systems that use facial expression analysis to improve customer satisfaction. Facial expression detection has been incorporated into computer vision and machine learning systems, opening up new possibilities for personalized music curating. The basic theories and techniques supporting facial expression analysis in the context of emotional computing are first examined in this study. It explores the developments in emotion identification technology, emphasizing important datasets, algorithms, and their advantages and disadvantages. Additionally, this research assesses how well these systems capture user preferences while resolving issues with scalability, accuracy, and privacy. It talks about the effects on user satisfaction and engagement of using facial expressions as a channel for feedback in recommendation systems. In order to improve and develop facial expression-based music recommendation systems, this paper concludes by outlining future directions and possible research routes and highlighting the necessity of multidisciplinary cooperation between computer science, psychology, and musicology. To sum up, this paper offers a thorough analysis of the current state-of-the-art in facial expression-driven music recommendation systems, outlining their innovations, difficulties, and potential directions for growth of multidisciplinary cooperation between computer science, psychology, and musicology

Child Missing with Surveillance Analysis with Multi Phase Extraction Face Monitoring

Abstract: Visual Surveillance has been placed in everywhere for any incidents identification. With the wide developments missing child identification with the computer vision system is the main focus. The implementation carried out with face based detection with rapid advance digital connectivity. Face recognition plays as a major developmental process where the concern person can be identified with face based identification system. Machine learning with artificial intelligence plays a vital role in facial detection and recognition. The paper proposes Haar Cascade on feature extraction from facial points and Hybrid Recurrent Neural System(HRNN) with Long Short Term Memory(LSTM) algorithm, on multi face recognition over public crowded area. Missing child images will be pre- trained in a database where the face matching templates will be provided. On matching the database labeled points with the feature point an E-mail or SMS based alert system can be generated on notification to the authority with cam points. Thus the development carried will give best accuracy solution on finding out from multiple face regions. Mainly identification of the missing child over the road side areas will be done with computer vision frame targeting system.

Fine-Grained Multi-class Road Segmentation using MultiScale Probability Learning

Abstract: Driven road segmentation, a crucial part of advanced driver assistance systems, looks at the surroundings to keep vehicles within safe driving limits (ADASs). It begins by outlining the automatic lane detection shortfalls of conventional computer vision systems, pointing out problems such subpar segmentation, insufficient mask edge contours, sluggish processing, and restricted flexibility in intricate urban environments. Next, a multi-step procedure using deep learning networks is offered as a solution. This involves extracting vector skeletons, computing neighbouring pixels, assigning proportional weights depending on endpoints, and getting binary prediction masks. The conversation also touches on how self-driving technology will affect society, emphasizing how it could provide safe and intelligent transportation choices in the face of an increase in traffic accidents caused by careless drivers. Self-driving cars could be the first practical example of socially conscious robots interacting with humans, the story implies, even if it acknowledges possible public opposition. Furthermore, the story highlights the latest developments in autonomous driving technology, highlighting the vital requirement for strong sensing, perception, and cognitive technologies to enable completely autonomous vehicles that can adjust to changing road conditions.

A YOLO-based deep learning model for Real-Time face mask detection via drone surveillance in public spaces

Automating face mask detection in public areas is paramount to maintaining public health, especially in the context of the COVID-19 pandemic. Utilization of technologies such as deep learning and computer vision systems enables effective monitoring of mask compliance, thereby minimizing the risk of virus spread. Real-time detection helps in prompt intervention for and enforcement of the use of masks, thereby preventing potential outbreaks and ensuring compliance with public health guidelines. This method helps save human resources and makes the reinforcement of wearing masks in public areas consistent and objective. Automatic detection of face masks serves as a key tool for preventing the spread of contagious diseases, protecting public health, and creating a safer environment for every person. This study addresses the challenges of real-time face mask detection via drone surveillance in public spaces, with reference to three categories: wearing of mask, incorrect wearing of mask, and no mask. Addressing these challenges entails an efficient and robust object detection and recognition algorithm. This algorithm can deal with a crowd of multiple faces via a mobile camera carried by a mini drone, and performs real-time video processing. Accordingly, this study proposes a You Only Look Once (YOLO) based deep learning C-Mask model for real-time face mask detection and recognition via drone surveillance in public spaces. The C-Mask model aims to operate within a mini drone surveillance system and provide efficient and robust face mask detection. The C-Mask model performs preprocessing, feature extraction, feature generation, feature enhancement, feature selection, and multivariate classification tasks for each face mask detection cycle. The preprocessing task prepares the training and testing data in the form of images for further processing. The feature extraction task is performed using a Convolutional Neural Network (CNN). Moreover, Cross-Stage Partial (CSP) DarkNet53 is used to improve the feature extraction and to facilitate the model’s object detection ability. A data augmentation algorithm is used for feature generation to enhance the model’s training robustness. The feature enhancement task is performed by applying the Path Aggregation Network (PANet) and Spatial Pyramid Pooling Network (SPPNet) algorithms, which are deployed to enhance the extracted and generated features. The classification task is performed through multi-label classification, wherein each object in an image can belong to multiple classes simultaneously, and the network generates a grid of bounding boxes and corresponding confidence scores for each class. The YOLO-based C-Mask model testing is performed by experimenting with various face mask detection scenarios and with varying mask colors and types, to ensure the efficiency and robustness of the proposed model. The C-Mask model test results show that this model can correctly and effectively detect face masks in real-time video streams under various conditions with an overall accuracy of 92.20%, precision of 92.04, recall of 90.83%, and F1-score of 89.95%, for all the three classes. These high scores have been obtained despite mini drone mobility and camera orientation adjustment substantially affecting face mask detection performance.

Advancements in Visual Prosthetics: The Visionary Bionic Model

Abstract: The Visionary Bionic Model represents a paradigm shift in the field of visual prosthetics, offering new hope to individuals afflicted by retinal degenerative diseases. This journal paper provides a comprehensive overview of the foundational framework, operational mechanics, and advancements associated with the Visionary Bionic Model. Beginning with a historical context, the paper traces the evolution of bionic eyes from early conceptualizations to contemporary innovations. It explores the technological underpinnings of the Visionary Bionic Model, including microelectrode arrays, image acquisition systems, and signal processing algorithms. Clinical trials and case studies are examined to evaluate the safety and efficacy of the Visionary Bionic Model in restoring sight to individuals with visual impairments. The paper also discusses challenges and limitations inherent in current bionic vision systems, such ` resolution and compatibility issues, and proposes avenues for future research and development. Ethical considerations surrounding accessibility, affordability, and informed consent are addressed, emphasizing the importance of responsible innovation. Ultimately, the Visionary Bionic Model holds promise as a transformative solution for individuals living with retinal degenerative diseases, offering renewed possibilities for enhanced vision and quality of life.

An ethologically relevant paradigm to assess defensive response to looming visual contrast stimuli

In the animal kingdom, threat information is perceived mainly through vision. The subcortical visual pathway plays a critical role in the rapid processing of visual information-induced fear, and triggers a response. Looming-evoked behavior in rodents, mimicking response to aerial predators, allowed identify the neural circuitry underlying instinctive defensive behaviors; however, the influence of disk/background contrast on the looming-induced behavioral response has not been examined, either in rats or mice. We studied the influence of the dark disk/gray background contrast in the type of rat and mouse defensive behavior in the looming arena, and we showed that rat and mouse response as a function of disk/background contrast adjusted to a sigmoid-like relationship. Both sex and age biased the contrast-dependent response, which was dampened in rats submitted to retinal unilateral or bilateral ischemia. Moreover, using genetically manipulated mice, we showed that the three type of photoresponsive retinal cells (i.e., cones, rods, and intrinsically photoresponsive retinal ganglion cells (ipRGCs)), participate in the contrast-dependent response, following this hierarchy: cones > > rods > > > ipRGCs. The cone and rod involvement was confirmed using a mouse model of unilateral non-exudative age-related macular degeneration, which only damages canonical photoreceptors and significantly decreased the contrast sensitivity in the looming arena.

What Is a Visual Stream?

The dual stream model of the human and non-human primate visual systems remains Leslie Ungerleider's (1946-2020) most indelible contribution to visual neuroscience. In this model, a dorsal "where" stream specialized for visuospatial representation extends through occipitoparietal cortex, whereas a ventral "what" stream specialized for representing object qualities extends through occipito-temporal cortex. Over time, this model underwent a number of revisions and expansions. In one of her last scientific contributions, Leslie proposed a third visual stream specialized for representing dynamic signals related to social perception. This alteration invites the question: What is a visual stream, and how are different visual streams individuated? In this article, we first consider and reject a simple answer to this question based on a common idealizing visualization of the model, which conflicts with the complexities of the visual system that the model was intended to capture. Next, we propose a taxonomic answer that takes inspiration from the philosophy of science and Leslie's body of work, which distinguishes between neural mechanisms, pathways, and streams. In this taxonomy, visual streams are superordinate to pathways and mechanisms and provide individuation conditions for determining whether collections of cortical connections delineate different visual streams. Given this characterization, we suggest that the proposed third visual stream does not yet meet these conditions, although the tripartite model still suggests important revisions to how we think about the organization of the human and non-human primate visual systems.

The evolution of AI: What does the future hold in the next two years

Artificial intelligence (AI) has evolved rapidly in recent years, transforming several industries and reshaping cultural norms. This article examines the current state of AI technology, focusing on significant trends and developments that have accelerated its growth. This paper sheds light on the evolving environment of AI innovation by examining current breakthroughs in machine learning, deep learning, and ethical concerns surrounding AI deployment. The following two years should see substantial advances in natural language processing, computer vision, and autonomous systems. Despite the excitement, difficulties such as ethical considerations, regulatory frameworks, and societal repercussions loom large, needing careful study and decisive action. This study aims to inform stakeholders about the transformative potential of AI and the methods required to properly navigate its growing landscape by conducting a balanced examination of opportunities and challenges.

A power-aware vision-based virtual sensor for real-time edge computing

Graphics processing units and tensor processing units coupled with tiny machine learning models deployed on edge devices are revolutionizing computer vision and real-time tracking systems. However, edge devices pose tight resource and power constraints. This paper proposes a real-time vision-based virtual sensors paradigm to provide power-aware multi-object tracking at the edge while preserving tracking accuracy and enhancing privacy. We thoroughly describe our proposed system architecture, focusing on the Dynamic Inference Power Manager (DIPM). Our proposed DIPM is based on an adaptive frame rate to provide energy savings. We implement and deploy the virtual sensor and the DIPM on the NVIDIA Jetson Nano edge platform to prove the effectiveness and efficiency of the proposed solution. The results of extensive experiments demonstrate that the proposed virtual sensor can achieve a reduction in energy consumption of about 36% in videos with relatively low dynamicity and about 21% in more dynamic video content while simultaneously maintaining tracking accuracy within a range of less than 1.2%.

An Evaluation of Various Deep Convolutional Networks for the Development of a Vision System for the Classification of Domestic Solid Street Waste

An Evaluation of Various Deep Convolutional Networks for the Development of a Vision System for the Classification of Domestic Solid Street Waste

Insights from the Evolving Model of Two Cortical Visual Pathways.

The two cortical visual pathways framework has had a profound influence on theories and empirical studies of the visual system for over 40 years. By grounding physiological responses and behavior in neuroanatomy, the framework provided a critical guide for understanding vision. Although the framework has evolved over time, as our understanding of the physiology and neuroanatomy expanded, cortical visual processing is still often conceptualized as two separate pathways emerging from the primary visual cortex that support distinct behaviors ("what" vs. "where/how"). Here, we take a historical perspective and review the continuing evolution of the framework, discussing key and often overlooked insights. Rather than a functional and neuroanatomical bifurcation into two independent serial, hierarchical pathways, the current evidence points to two highly recurrent heterarchies with heterogeneous connections to cortical regions and subcortical structures that flexibly support a wide variety of behaviors. Although many of the simplifying assumptions of the framework are belied by the evidence gathered since its initial proposal, the core insight of grounding function and behavior in neuroanatomy remains fundamental. Given this perspective, we highlight critical open questions and the need for a better understanding of neuroanatomy, particularly in the human.

A vision chip with complementary pathways for open-world sensing.

Image sensors face substantial challenges when dealing with dynamic, diverse and unpredictable scenes in open-world applications. However, the development of image sensors towards high speed, high resolution, large dynamic range and high precision is limited by power and bandwidth. Here we present a complementary sensing paradigm inspired by the human visual system that involves parsing visual information into primitive-based representations and assembling these primitives to form two complementary vision pathways: a cognition-oriented pathway for accurate cognition and an action-oriented pathway for rapid response. To realize this paradigm, a vision chip called Tianmouc is developed, incorporating a hybrid pixel array and a parallel-and-heterogeneous readout architecture. Leveraging the characteristics of the complementary vision pathway, Tianmouc achieves high-speed sensing of up to 10,000 fps, a dynamic range of 130 dB and an advanced figure of merit in terms of spatial resolution, speed and dynamic range. Furthermore, it adaptively reduces bandwidth by 90%. We demonstrate the integration of a Tianmouc chip into an autonomous driving system, showcasing its abilities to enable accurate, fast and robust perception, even in challenging corner cases on open roads. The primitive-based complementary sensing paradigm helps in overcoming fundamental limitations in developing vision systems for diverse open-world applications.

Differential cortical and subcortical visual processing with eyes shut.

Closing our eyes largely shuts down our ability to see. That said, our eyelids still pass some light, allowing our visual system to coarsely process information about visual scenes, such as changes in luminance. However, the specific impact of eye closure on processing within the early visual system remains largely unknown. To understand how visual processing is modulated when eyes are shut, we used functional magnetic resonance imaging (fMRI) to measure responses to a flickering visual stimulus at high (100%) and low (10%) temporal contrasts, while participants viewed the stimuli with their eyes open or closed. Interestingly, we discovered that eye closure produced a qualitatively distinct pattern of effects across the visual thalamus and visual cortex. We found that with eyes open, low temporal contrast stimuli produced smaller responses across the lateral geniculate nucleus (LGN), primary (V1) and extrastriate visual cortex (V2). However, with eyes closed, we discovered that the LGN and V1 maintained similar blood oxygenation level-dependent (BOLD) responses as the eyes open condition, despite the suppressed visual input through the eyelid. In contrast, V2 and V3 had strongly attenuated BOLD response when eyes were closed, regardless of temporal contrast. Our findings reveal a qualitatively distinct pattern of visual processing when the eyes are closed-one that is not simply an overall attenuation but rather reflects distinct responses across visual thalamocortical networks, wherein the earliest stages of processing preserve information about stimuli but are then gated off downstream in visual cortex.NEW & NOTEWORTHY When we close our eyes coarse luminance information is still accessible by the visual system. Using functional magnetic resonance imaging, we examined whether eyelid closure plays a unique role in visual processing. We discovered that while the LGN and V1 show equivalent responses when the eyes are open or closed, extrastriate cortex exhibited attenuated responses with eye closure. This suggests that when the eyes are closed, downstream visual processing is blind to this information.

A Dynamic Visual SLAM System Incorporating Object Tracking for UAVs

The capability of unmanned aerial vehicles (UAVs) to capture and utilize dynamic object information assumes critical significance for decision making and scene understanding. This paper presents a method for UAV relative positioning and target tracking based on a visual simultaneousocalization and mapping (SLAM) framework. By integrating an object detection neural network into the SLAM framework, this method can detect moving objects and effectively reconstruct the 3D map of the environment from image sequences. For multiple object tracking tasks, we combine the region matching of semantic detection boxes and the point matching of the optical flow method to perform dynamic object association. This joint association strategy can prevent trackingoss due to the small proportion of the object in the whole image sequence. To address the problem ofacking scale information in the visual SLAM system, we recover the altitude data based on a RANSAC-based plane estimation approach. The proposed method is tested on both the self-created UAV dataset and the KITTI dataset to evaluate its performance. The results demonstrate the robustness and effectiveness of the solution in facilitating UAV flights.

A visual inspection and diagnosis system for bridge rivets based on a convolutional neural network

A visual inspection and diagnosis system for bridge rivets based on a convolutional neural network

Avian eye-inspired perovskite artificial vision system for foveated and multispectral imaging.

Avian eyes have deep central foveae as a result of extensive evolution. Deep foveae efficiently refract incident light, creating a magnified image of the target object and making it easier to track object motion. These features are essential for detecting and tracking remote objects in dynamic environments. Furthermore, avian eyes respond to a wide spectrum of light, including visible and ultraviolet light, allowing them to efficiently distinguish the target object from complex backgrounds. Despite notable advances in artificial vision systems that mimic animal vision, the exceptional object detection and targeting capabilities of avian eyes via foveated and multispectral imaging remain underexplored. Here, we present an artificial vision system that capitalizes on these aspects of avian vision. We introduce an artificial fovea and vertically stacked perovskite photodetector arrays whose designs were optimized by theoretical simulations for the demonstration of foveated and multispectral imaging. The artificial vision system successfully identifies colored and mixed-color objects and detects remote objects through foveated imaging. The potential for use in uncrewed aerial vehicles that need to detect, track, and recognize distant targets in dynamic environments is also discussed. Our avian eye-inspired perovskite artificial vision system marks a notable advance in bioinspired artificial visions.