Retinal Prosthesis Research Articles

Research on key technology of cooled infrared bionic compound eye camera based on small lens array

Traditional 2D imaging technologies are limited by the need for a large field of view and their sensitivity to small target motion. Inspired by the characteristics of insect compound eye structure, we propose an infrared bionic compound eye camera based on a small lens array. The camera is composed of 61 small lens arrays mounted on a curved spherical shell and a relay optical system. The imaging device is a high-performance cooled mid-wave infrared detector. This is an innovative design for an infrared biomimetic compound eye camera system that provides a wide field of view and all-day detection capability. Aimed to meet the specified requirements. The optical system achieves a 100% cold-membrane match between the infrared optical system and the cooled detector, and the relay optical system optimizes the large-field aberration by introducing a higher-order aspheric surface and modifying the geometric surface of the lenses. Our entire system enables an observation field angle of 108∘×108∘\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$108^\\circ \ imes 108^\\circ$$\\end{document}. The experiments showed that the image quality of the system is high, each ommatidium was effective within the imaging range of the compound eye camera, resulting in an improved signal-to-noise ratio in various scenes.

Bio-inspired foveal super-resolution method for multi-focal-length images based on local gradient constraints

Most existing super-resolution (SR) imaging systems, inspired by the bionic compound eye, utilize image registration and reconstruction algorithms to overcome the angular resolution limitations of individual imaging systems. This article introduces a multi-aperture multi-focal-length imaging system and a multi-focal-length image super-resolution algorithm, mimicking the foveal imaging of the human eye. Experimental results demonstrate that with the proposed imaging system and an SR imaging algorithm inspired by the human visual system, the proposed method can enhance the spatial resolution of the foveal region by up to 4 × compared to the original acquired image. These findings validate the effectiveness of the proposed imaging system and computational imaging algorithm in enhancing image texture and spatial resolution.

Tunable Negative and Positive Photoconductance in Van Der Waals Heterostructure for Image Preprocessing.

The processing of visual information occurs mainly in the retina, and the retinal preprocessing function greatly improves the transmission quality and efficiency of visual information. The artificial retina system provides a promising path to efficient image processing. Here, graphene/InSe/h-BN heterogeneous structure is proposed, which exhibits negative and positive photoconductance (NPC and PPC) effects by altering the strength of a single wavelength laser. Moreover, a modified theoretical model is presented based on the power-dependent photoconductivity effect of laser: , which can reveal the internal physical mechanism of negative/positive photoconductance effects. The present 2D structure design allows the field effect transistor (FET) to exhibit excellent photoelectric performance (RNPC=1.1× 104AW-1, RPPC=13AW-1) and performance stability. Especially, the retinal pretreatment process is successfully simulated based on the negative and positive photoconductive effects. Moreover, the pulse signal input improves the device responsivity by 167%, and the transmission quality and efficiency of the visual signal can also be enhanced. This work provides a new design idea and direction for the construction of artificial vision, and lay a foundation for the integration of the next generation of optoelectronicdevices.

Numerical Design of a Micro-LED Based Optogenetic Stimulator for Visual Cortical Prosthesis

Numerical Design of a Micro-LED Based Optogenetic Stimulator for Visual Cortical Prosthesis

Learning Scene Representations for Human-assistive Displays Using Self-attention Networks

Video-see-through (VST) augmented reality (AR) is widely used to present novel augmentative visual experiences by processing video frames for viewers. Among VST AR systems, assistive vision displays aim to compensate for low vision or blindness, presenting enhanced visual information to support activities of daily living for the vision impaired/deprived. Despite progress, current assistive displays suffer from a visual information bottleneck, limiting their functional outcomes compared to healthy vision. This motivates the exploration of methods to selectively enhance and augment salient visual information. Traditionally, vision processing pipelines for assistive displays rely on hand-crafted, single-modality filters, lacking adaptability to time-varying and environment-dependent needs. This article proposes the use of Deep Reinforcement Learning (DRL) and Self-attention (SA) networks as a means to learn vision processing pipelines for assistive displays. SA networks selectively attend to task-relevant features, offering a more parameter—and compute-efficient approach to RL-based task learning. We assess the feasibility of using SA networks in a simulation-trained model to generate relevant representations of real-world states for navigation with prosthetic vision displays. We explore two prosthetic vision applications, vision-to-auditory encoding, and retinal prostheses, using simulated phosphene visualisations. This article introduces SA-px, a general-purpose vision processing pipeline using self-attention networks, and SA-phos, a display-specific formulation targeting low-resolution assistive displays. We present novel scene visualisations derived from SA image patches importance rankings to support mobility with prosthetic vision devices. To the best of our knowledge, this is the first application of self-attention networks to the task of learning vision processing pipelines for prosthetic vision or assistive displays.

Optimizing Image Enhancement: Feature Engineering for Improved Classification in AI-Assisted Artificial Retinas.

Artificial retinas have revolutionized the lives of many blind people by enabling their ability to perceive vision via an implanted chip. Despite significant advancements, there are some limitations that cannot be ignored. Presenting all objects captured in a scene makes their identification difficult. Addressing this limitation is necessary because the artificial retina can utilize a very limited number of pixels to represent vision information. This problem in a multi-object scenario can be mitigated by enhancing images such that only the major objects are considered to be shown in vision. Although simple techniques like edge detection are used, they fall short in representing identifiable objects in complex scenarios, suggesting the idea of integrating primary object edges. To support this idea, the proposed classification model aims at identifying the primary objects based on a suggested set of selective features. The proposed classification model can then be equipped into the artificial retina system for filtering multiple primary objects to enhance vision. The suitability of handling multi-objects enables the system to cope with real-world complex scenarios. The proposed classification model is based on a multi-label deep neural network, specifically designed to leverage from the selective feature set. Initially, the enhanced images proposed in this research are compared with the ones that utilize an edge detection technique for single, dual, and multi-object images. These enhancements are also verified through an intensity profile analysis. Subsequently, the proposed classification model's performance is evaluated to show the significance of utilizing the suggested features. This includes evaluating the model's ability to correctly classify the top five, four, three, two, and one object(s), with respective accuracies of up to 84.8%, 85.2%, 86.8%, 91.8%, and 96.4%. Several comparisons such as training/validation loss and accuracies, precision, recall, specificity, and area under a curve indicate reliable results. Based on the overall evaluation of this study, it is concluded that using the suggested set of selective features not only improves the classification model's performance, but aligns with the specific problem to address the challenge of correctly identifying objects in multi-object scenarios. Therefore, the proposed classification model designed on the basis of selective features is considered to be a very useful tool in supporting the idea of optimizing image enhancement.

A novel GCaMP6f-RCS rat model for studying electrical stimulation in the degenerated retina.

Background: Retinal prostheses aim to restore vision by electrically stimulating the remaining viable retinal cells in Retinal Degeneration (RD) cases. Research in this field necessitates a comprehensive analysis of retinal ganglion cells' (RGCs) responses to assess the obtained visual acuity and quality. Here we present a novel animal model which facilitates the optical recording of RGCs activity in an RD rat. This model can significantly enhance the functional evaluation of vision restoration treatments. Methods: The development of the novel rat model is based on crossbreeding a retinal degenerated Royal College of Surgeons (RCS) rat with a transgenic line expressing the genetic calcium indicator GCaMP6f in the RGCs. Characterization of the model was achieved using Optical Coherence Tomography (OCT) imaging, histology, and electroretinography (ERG) at the ages of 4, 8, and 12weeks. Additionally, optical recordings of RGCs function in response to ex-vivo subretinal electrical stimulations were performed. Results: Histological investigations confirmed the high expression of GCaMP6f in the RGCs and minimal expression in the inner nuclear layer (INL). OCT imaging and histological studies revealed the expected gradual retinal degeneration, as evident by the decrease in retinal thickness with age and the formation of subretinal debris. This degeneration was further confirmed by ERG recordings, which demonstrated a significant decrease in the b-wave amplitude throughout the degeneration process, culminating in its absence at 12weeks in the GCaMP6f-RCS rat. Importantly, the feasibility of investigating subretinal stimulation was demonstrated, revealing a consistent increase in activation threshold throughout degeneration. Furthermore, an increase in the diameter of the activated area with increasing currents was observed. The spatial spread of the activation area in the GCaMP6f-RCS rat was found to be smaller and exhibited faster activation dynamics compared with the GCaMP6f-LE strain. Conclusion: This novel animal model offers an opportunity to deepen our understanding of prosthetically induced retinal responses, potentially leading to significant advancements in prosthetic interventions in visual impairments.

Perceptography unveils the causal contribution of inferior temporal cortex to visual perception

Neurons in the inferotemporal (IT) cortex respond selectively to complex visual features, implying their role in object perception. However, perception is subjective and cannot be read out from neural responses; thus, bridging the causal gap between neural activity and perception demands independent characterization of perception. Historically, though, the complexity of the perceptual alterations induced by artificial stimulation of IT cortex has rendered them impossible to quantify. To address this old problem, we tasked male macaque monkeys to detect and report optical impulses delivered to their IT cortex. Combining machine learning with high-throughput behavioral optogenetics, we generated complex and highly specific images that were hard for the animal to distinguish from the state of being cortically stimulated. These images, named “perceptograms” for the first time, reveal and depict the contents of the complex hallucinatory percepts induced by local neural perturbation in IT cortex. Furthermore, we found that the nature and magnitude of these hallucinations highly depend on concurrent visual input, stimulation location, and intensity. Objective characterization of stimulation-induced perceptual events opens the door to developing a mechanistic theory of visual perception. Further, it enables us to make better visual prosthetic devices and gain a greater understanding of visual hallucinations in mental disorders.

A bionic self-driven retinomorphic eye with ionogel photosynaptic retina.

Bioinspired bionic eyes should be self-driving, repairable and conformal to arbitrary geometries. Such eye would enable wide-field detection and efficient visual signal processing without requiring external energy, along with retinal transplantation by replacing dysfunctional photoreceptors with healthy ones for vision restoration. A variety of artificial eyes have been constructed with hemispherical silicon, perovskite and heterostructure photoreceptors, but creating zero-powered retinomorphic system with transplantable conformal features remains elusive. By combining neuromorphic principle with retinal and ionoelastomer engineering, we demonstrate a self-driven hemispherical retinomorphic eye with elastomeric retina made of ionogel heterojunction as photoreceptors. The receptor driven by photothermoelectric effect shows photoperception with broadband light detection (365 to 970 nm), wide field-of-view (180°) and photosynaptic (paired-pulse facilitation index, 153%) behaviors for biosimilar visual learning. The retinal photoreceptors are transplantable and conformal to any complex surface, enabling visual restoration for dynamic optical imaging and motion tracking.

Self-powered photoelectric sensors based on hydrogel diodes doped with photoacid

Artificial retina requires photo-sensory units that convert light into electric signals without relying on any external power supply. Recent development on hydrogel diodes paves a new avenue for creating photo-electric signals that can be picked up by human optic nerves. Here, we report a self-powered photoelectric sensor based on hydrogel diodes doped with photoacid, which produces electric signal relying on no external power supply. The photoelectric sensor is self-powered due to the fact that the photoacid-released protons are mobilized under the influence of the built-in electric field in the hydrogel PN junction. The photo-induced open-circuit voltage reaches to 1.6 mV under 50 mW cm−2. We found that the highest electric response occurred when photoacid was presented at the cationic side of the hydrogel PN junction. A prototype consisting 9-element sensor array is prepared, and we have demonstrated its potential for spatially resolved visualization upon light irradiation.

Optoelectronic dual-synapse based on wafer-level GaN-on-Si device incorporating embedded SiO2 barrier layers

Optoelectronic synapses possess the potential to seamlessly integrate perception, storage, and neuromorphic computation, thereby offering advantages in constructing artificial vision systems. Gallium nitride (GaN) materials, renowned for their exceptional optoelectronic characteristics and the compatibility of Silicon-based GaN (GaN-on-Si) with CMOS technology, contribute significantly to realizing large-scale optoelectronic neuromorphic circuits. In this study, we present an optoelectronic dual-synapse based on a structurally-improved AlGaN/GaN MIS-HEMT with ring-shaped SiO2 barriers embedded surrounding the source and drain regions. This device facilitates simultaneous signal transmission to the source and drain through current-limiting electrical breakdown beneath the gate. This integration of two discrete optoelectronic synapses within a single device enhances neural signal transmission nodes and branching connectivity, facilitating the construction of large-scale neuromorphic circuits. Under the co-influence of electrical and optical pulses, the device not only demonstrates excellent optoelectronic synaptic characteristics but also exhibits the ability to modulate the critical wavelength of synaptic responses by varying the source/drain voltage, effectively simulating the light signal recognition observed in biological retinas. The synergistic operation of the two sub-synapses enables a successful transition from short-term potentiation (STP) to long-term potentiation (LTP). Furthermore, leveraging the device's STP, we emulate the function of artificial retina through a GaN optoelectronic neuromorphic array, successfully achieving image recognition in conjunction with artificial neural networks (ANNs). The devices developed in this study, based on GaN-on-Si allow wafer-level manufacturing, provding a promising avenue for the large-scale production of optoelectronic synapse devices and the realization of artificial vision systems.

Axonal stimulation affects the linear summation of single-point perception in three Argus II users.

Objective.Retinal implants use electrical stimulation to elicit perceived flashes of light ('phosphenes'). Single-electrode phosphene shape has been shown to vary systematically with stimulus parameters and the retinal location of the stimulating electrode, due to incidental activation of passing nerve fiber bundles. However, this knowledge has yet to be extended to paired-electrode stimulation.Approach.We retrospectively analyzed 3548 phosphene drawings made by three blind participants implanted with an Argus II Retinal Prosthesis. Phosphene shape (characterized by area, perimeter, major and minor axis length) and number of perceived phosphenes were averaged across trials and correlated with the corresponding single-electrode parameters. In addition, the number of phosphenes was correlated with stimulus amplitude and neuroanatomical parameters: electrode-retina and electrode-fovea distance as well as the electrode-electrode distance to ('between-axon') and along axon bundles ('along-axon'). Statistical analyses were conducted using linear regression and partial correlation analysis.Main results.Simple regression revealed that each paired-electrode shape descriptor could be predicted by the sum of the two corresponding single-electrode shape descriptors (p < .001). Multiple regression revealed that paired-electrode phosphene shape was primarily predicted by stimulus amplitude and electrode-fovea distance (p < .05). Interestingly, the number of elicited phosphenes tended to increase with between-axon distance (p < .05), but not with along-axon distance, in two out of three participants.Significance.The shape of phosphenes elicited by paired-electrode stimulation was well predicted by the shape of their corresponding single-electrode phosphenes, suggesting that two-point perception can be expressed as the linear summation of single-point perception. The impact of the between-axon distance on the perceived number of phosphenes provides further evidence in support of the axon map model for epiretinal stimulation. These findings contribute to the growing literature on phosphene perception and have important implications for the design of future retinal prostheses.

Gaze-contingent processing improves mobility, scene recognition and visual search in simulated head-steered prosthetic vision.

Objective.The enabling technology of visual prosthetics for the blind is making rapid progress. However, there are still uncertainties regarding the functional outcomes, which can depend on many design choices in the development. In visual prostheses with a head-mounted camera, a particularly challenging question is how to deal with the gaze-locked visual percept associated with spatial updating conflicts in the brain. The current study investigates a recently proposed compensation strategy based on gaze-contingent image processing with eye-tracking. Gaze-contingent processing is expected to reinforce natural-like visual scanning and reestablished spatial updating based on eye movements. The beneficial effects remain to be investigated for daily life activities in complex visual environments.Approach.The current study evaluates the benefits of gaze-contingent processing versus gaze-locked and gaze-ignored simulations in the context of mobility, scene recognition and visual search, using a virtual reality simulated prosthetic vision paradigm with sighted subjects.Main results.Compared to gaze-locked vision, gaze-contingent processing was consistently found to improve the speed in all experimental tasks, as well as the subjective quality of vision. Similar or further improvements were found in a control condition that ignores gaze-dependent effects, a simulation that is unattainable in the clinical reality.Significance.Our results suggest that gaze-locked vision and spatial updating conflicts can be debilitating for complex visually-guided activities of daily living such as mobility and orientation. Therefore, for prospective users of head-steered prostheses with an unimpaired oculomotor system, the inclusion of a compensatory eye-tracking system is strongly endorsed.

Tau-Cell-Based Analog Silicon Retina With Spatio- Temporal Filtering and Contrast Gain Control.

Developing precise artificial retinas is crucial because they hold the potential to restore vision, improve visual prosthetics, and enhance computer vision systems. Emulating the luminance and contrast adaption features of the retina is essential to improve visual perception and efficiency to provide an environment realistic representation to the user. In this article, we introduce an artificial retina model that leverages its potent adaptation to luminance and contrast to enhance vision sensing and information processing. The model has the ability to achieve the realization of both tonic and phasic cells in the simplest manner. We have implemented the retina model using 0.18 μm process technology and validated the accuracy of the hardware implementation through circuit simulation that closely matches the software retina model. Additionally, we have characterized a single pixel fabricated using the same 0.18 μm process. This pixel demonstrates an 87.7-% ratio of variance with the temporal software model and operates with a power consumption of 369 nW.

Robot-assisted implantation of a microelectrode array in the occipital lobe as a visual prosthesis: technical note.

The prospect of direct interaction between the brain and computers has been investigated in recent decades, revealing several potential applications. One of these is sight restoration in profoundly blind people, which is based on the ability to elicit visual perceptions while directly stimulating the occipital cortex. Technological innovation has led to the development of microelectrodes implantable on the brain surface. The feasibility of implanting a microelectrode on the visual cortex has already been shown in animals, with promising results. Current research has focused on the implantation of microelectrodes into the occipital brain of blind volunteers. The technique raises several technical challenges. In this technical note, the authors suggest a safe and effective approach for robot-assisted implantation of microelectrodes in the occipital lobe for sight restoration.

Honeycomb-Patterned Graphene Microelectrodes: A Promising Approach for Safe and Effective Retinal Stimulation Based on Electro-Thermo-Mechanical Modeling and Simulation.

The main objective of the present study is to use graphene as electrode neural interface material to design novel microelectrodes topology for retinal prosthesis and investigate device operation safety based on the computational framework. The study's first part establishes the electrode material selection based on electrochemical impedance and the equivalent circuit model. The second part of the study is modeling at the microelectrode-tissue level to investigate the potential distribution, generated resistive heat dissipation, and thermally induced stress in the tissue due to electrical stimulation. The formulation of Joule heating and thermal expansion between microelectrode-tissue-interface employing finite element method modeling is based on the three coupled equations, specifically Ohm's law, Navier's equation, and Fourier equation. Electrochemical simulation results of electrode material reveal that single-layer and few-layer graphene-based microelectrode has a specific impedance in the range of 0.02- [Formula: see text], comparable to platinum counterparts. The microelectrode of [Formula: see text] size can stimulate retinal tissue with a threshold current in the range of 8.7- [Formula: see text]. Such stimulation with the observed microelectrode size indicates that both microelectrodes and retinal tissue stay structurally intact, and the device is thermally and mechanically stable, functioning within the safety limit. The results reveal the viability of high-density graphene-based microelectrodes for improved interface as stimulating electrodes to acquire higher visual acuity. Furthermore, the novel microelectrodes design configuration in the honeycomb pattern gives the retinal tissue non-invasive heating and minimal stress upon electrical stimulation. Thus, it paves the path to designing a graphene-based microelectrode array for retinal prosthesis for further in vitro or in vivo studies.

Double‐Sided, Thin‐Film Microelectrode Array with Hemispheric Electrodes for Subretinal Stimulation

AbstractComponents in neural implants, such as the electrode array and stimulator circuit, are often fabricated discretely. This modular fabrication scheme offers flexibility during development but poses difficulties during assembly, as components must be compactly integrated for implantation. It is particularly difficult in cases where the electrode array is required to have a high number of channels, such as in retinal prostheses. This paper presents the development of a parylene C‐based, double‐sided microelectrode array with 294 hemispheric electrodes for subretinal stimulation. The bonding pads on the bottom side of the double‐sided array are connected with electrodes through vias, eliminating the interconnection lines. The array can be integrated with a stimulator circuit through pad‐to‐pad bonding, resulting in a compact implant. The hemispheric electrodes are fabricated using thermally reflowed photoresist infillings, through which the height and width of the hemispheres can be easily controlled. The long‐term stability and biocompatibility of the materials and methods used to fabricate and package the electrodes are demonstrated in in vitro and in vivo environments over months. Finally, subretinal stimulation by the developed electrodes is successfully demonstrated using in vitro retinal patches from mice and monkeys.

Nanoparticle-based optical interfaces for retinal neuromodulation: a review.

Degeneration of photoreceptors in the retina is a leading cause of blindness, but commonly leaves the retinal ganglion cells (RGCs) and/or bipolar cells extant. Consequently, these cells are an attractive target for the invasive electrical implants colloquially known as "bionic eyes." However, after more than two decades of concerted effort, interfaces based on conventional electrical stimulation approaches have delivered limited efficacy, primarily due to the current spread in retinal tissue, which precludes high-acuity vision. The ideal prosthetic solution would be less invasive, provide single-cell resolution and an ability to differentiate between different cell types. Nanoparticle-mediated approaches can address some of these requirements, with particular attention being directed at light-sensitive nanoparticles that can be accessed via the intrinsic optics of the eye. Here we survey the available known nanoparticle-based optical transduction mechanisms that can be exploited for neuromodulation. We review the rapid progress in the field, together with outstanding challenges that must be addressed to translate these techniques to clinical practice. In particular, successful translation will likely require efficient delivery of nanoparticles to stable and precisely defined locations in the retinal tissues. Therefore, we also emphasize the current literature relating to the pharmacokinetics of nanoparticles in the eye. While considerable challenges remain to be overcome, progress to date shows great potential for nanoparticle-based interfaces to revolutionize the field of visual prostheses.

Explainable machine learning predictions of perceptual sensitivity for retinal prostheses

Objective. Retinal prostheses evoke visual precepts by electrically stimulating functioning cells in the retina. Despite high variance in perceptual thresholds across subjects, among electrodes within a subject, and over time, retinal prosthesis users must undergo ‘system fitting’, a process performed to calibrate stimulation parameters according to the subject’s perceptual thresholds. Although previous work has identified electrode-retina distance and impedance as key factors affecting thresholds, an accurate predictive model is still lacking. Approach. To address these challenges, we (1) fitted machine learning models to a large longitudinal dataset with the goal of predicting individual electrode thresholds and deactivation as a function of stimulus, electrode, and clinical parameters (‘predictors’) and (2) leveraged explainable artificial intelligence (XAI) to reveal which of these predictors were most important. Main results. Our models accounted for up to 76% of the perceptual threshold response variance and enabled predictions of whether an electrode was deactivated in a given trial with F1 and area under the ROC curve scores of up to 0.732 and 0.911, respectively. Our models identified novel predictors of perceptual sensitivity, including subject age, time since blindness onset, and electrode-fovea distance. Significance. Our results demonstrate that routinely collected clinical measures and a single session of system fitting might be sufficient to inform an XAI-based threshold prediction strategy, which has the potential to transform clinical practice in predicting visual outcomes.

Femtosecond Laser Microfabrication of Artificial Compound Eyes

Over millions of years of evolution, arthropods have intricately developed and fine-tuned their highly sophisticated compound eye visual systems, serving as a valuable source of inspiration for human emulation and tracking. Femtosecond laser processing technology has attracted attention for its excellent precision, programmable design capabilities, and advanced three-dimensional processing characteristics, especially in the production of artificial bionic compound eye structures, showing unparalleled advantages. This comprehensive review initiates with a succinct introduction to the operational principles of biological compound eyes, providing essential context for the design of biomimetic counterparts. It subsequently offers a concise overview of crucial manufacturing methods for biomimetic compound eye structures. In addition, the application of femtosecond laser technology in the production of biomimetic compound eyes is also briefly introduced. The review concludes by highlighting the current challenges and presenting a forward-looking perspective on the future of this evolving field.