Cortical Visual Prosthesis Research Articles

A Wearable Real-Time System for Simultaneous Wireless Power and Data Transmission to Cortical Visual Prosthesis.

Wireless, miniaturised and distributed neural interfaces are emerging neurotechnologies. Although extensive research efforts contribute to their technological advancement, the need for real-time systems enabling simultaneous wireless information and power transfer toward distributed neural implants remains crucial. Here we present a complete wearable system including a software for real-time image capturing, processing and digital data transfer; an hardware for high radiofrequency generation and modulation via amplitude shift keying; and a 3-coil inductive link adapt to operate with multiple miniaturised receivers. The system operates in real-time with a maximum frame rate of 20 Hz, reconstructing each frame with a matrix of 32 × 32 pixels. The device generates a carrier frequency of 433.92 MHz. It transmits the highest power of 32 dBm with a data rate of 6 Mbps and a variable modulation index as low as 8 %, thus potentially enabling wireless communication with 1024 miniaturised and distributed intracortical microstimulators. The system is primarily conceived as an external wearable device for distributed cortical visual prosthesis covering a visual field of 20 °. At the same time, it is modular and versatile, being suitable for multiple applications requiring simultaneous wireless information and power transfer to large-scale neural interfaces.

Comparison of electrical microstimulation artifact removal methods for high-channel-count prostheses

BackgroundNeuroprostheses are used to electrically stimulate the brain, modulate neural activity and restore sensory and motor function following injury or disease, such as blindness, paralysis, and other movement and psychiatric disorders. Recordings are often made simultaneously with stimulation, allowing the monitoring of neural signals and closed-loop control of devices. However, stimulation-evoked artifacts may obscure neural activity, particularly when stimulation and recording sites are nearby. Several methods have been developed to remove stimulation artifacts, but it remains challenging to validate and compare these methods because the ‘ground-truth’ of the neuronal signals may be contaminated by artifacts. New methodHere, we delivered stimulation to the visual cortex via a high-channel-count prosthesis while recording neuronal activity and stimulation artifacts. We quantified the waveforms and temporal properties of stimulation artifacts from the cortical visual prosthesis (CVP) and used them to build a dataset, in which we simulated the neuronal activity and the stimulation artifacts. We illustrate how to use the simulated data to evaluate the performance of six software-based artifact removal methods (Template subtraction, Linear interpolation, Polynomial fitting, Exponential fitting, SALPA and ERAASR) in a CVP application scenario. ResultsWe here focused on stimulation artifacts caused by electrical stimulation through a high-channel-count cortical prosthesis device. We find that the Polynomial fitting and Exponential fitting methods outperform the other methods in recovering spikes and multi-unit activity. Linear interpolation and Template subtraction recovered the local-field potentials. ConclusionPolynomial fitting and Exponential fitting provided a good trade-off between the quality of the recovery of spikes and multi-unit activity (MUA) and the computational complexity for a cortical prosthesis.

Numerical Design of a Micro-LED Based Optogenetic Stimulator for Visual Cortical Prosthesis

Numerical Design of a Micro-LED Based Optogenetic Stimulator for Visual Cortical Prosthesis

Towards biologically plausible phosphene simulation for the differentiable optimization of visual cortical prostheses.

Blindness affects millions of people around the world. A promising solution to restoring a form of vision for some individuals are cortical visual prostheses, which bypass part of the impaired visual pathway by converting camera input to electrical stimulation of the visual system. The artificially induced visual percept (a pattern of localized light flashes, or 'phosphenes') has limited resolution, and a great portion of the field's research is devoted to optimizing the efficacy, efficiency, and practical usefulness of the encoding of visual information. A commonly exploited method is non-invasive functional evaluation in sighted subjects or with computational models by using simulated prosthetic vision (SPV) pipelines. An important challenge in this approach is to balance enhanced perceptual realism, biologically plausibility, and real-time performance in the simulation of cortical prosthetic vision. We present a biologically plausible, PyTorch-based phosphene simulator that can run in real-time and uses differentiable operations to allow for gradient-based computational optimization of phosphene encoding models. The simulator integrates a wide range of clinical results with neurophysiological evidence in humans and non-human primates. The pipeline includes a model of the retinotopic organization and cortical magnification of the visual cortex. Moreover, the quantitative effects of stimulation parameters and temporal dynamics on phosphene characteristics are incorporated. Our results demonstrate the simulator's suitability for both computational applications such as end-to-end deep learning-based prosthetic vision optimization as well as behavioral experiments. The modular and open-source software provides a flexible simulation framework for computational, clinical, and behavioral neuroscientists working on visual neuroprosthetics.

Invited Session I: Vision restoration: The New Generation of Visual Cortical Prosthetics.

Since the 1960s, the prospect of developing a visual cortical prosthesis for restoration of useful vision to the blind has tantalized scientists, engineers, and clinicians. After two decades of relative inactivity, interest in this approach has been renewed. There are now multiple active efforts to develop next generation visual prostheses and to bring these devices to clinical implementation. We will review the current state of the field, with a particular emphasis on our own work, and will highlight advances in the field as well challenges that must be overcome to produce a clinically useful prosthetic device.

A novel simulation paradigm utilising MRI-derived phosphene maps for cortical prosthetic vision

Objective. We developed a realistic simulation paradigm for cortical prosthetic vision and investigated whether we can improve visual performance using a novel clustering algorithm. Approach. Cortical visual prostheses have been developed to restore sight by stimulating the visual cortex. To investigate the visual experience, previous studies have used uniform phosphene maps, which may not accurately capture generated phosphene map distributions of implant recipients. The current simulation paradigm was based on the Human Connectome Project retinotopy dataset and the placement of implants on the cortices from magnetic resonance imaging scans. Five unique retinotopic maps were derived using this method. To improve performance on these retinotopic maps, we enabled head scanning and a density-based clustering algorithm was then used to relocate centroids of visual stimuli. The impact of these improvements on visual detection performance was tested. Using spatially evenly distributed maps as a control, we recruited ten subjects and evaluated their performance across five sessions on the Berkeley Rudimentary Visual Acuity test and the object recognition task. Main results. Performance on control maps is significantly better than on retinotopic maps in both tasks. Both head scanning and the clustering algorithm showed the potential of improving visual ability across multiple sessions in the object recognition task. Significance. The current paradigm is the first that simulates the experience of cortical prosthetic vision based on brain scans and implant placement, which captures the spatial distribution of phosphenes more realistically. Utilisation of evenly distributed maps may overestimate the performance that visual prosthetics can restore. This simulation paradigm could be used in clinical practice when making plans for where best to implant cortical visual prostheses.

Utilization of brain scans to create realistic phosphene maps for cortical visual prosthesis simulation studies.

It has been shown that we can restore sensations of light by stimulating the visual cortex. Cortical prosthetic vision consists of light perception in the visual field named phosphenes. Phosphenes are like pixels on a monitor which we can control to form the desired perception. However, the locations of phosphenes evoked vary between individuals. One of the biggest challenges is how to utilize phosphenes to present recognizable patterns that represent real-world scenes. Because of the difficulties of recruiting participants, and the risks of neurosurgery, researchers have used computer simulations to investigate the outcome of cortical visual prostheses. Previous simulations used regular phosphene maps, which may overestimate the visual ability cortical visual prosthesis can provide. This study aims to develop a more realistic simulation for cortical visual prostheses. We derived realistic phosphene maps using an existing cortical retinotopy dataset and decided implant placement by considering neurosurgery restrictions. We rendered some visual stimuli to evaluate the usability of those phosphene maps. The results indicate that presenting information on phosphenes maps may be more challenging than previously estimated.

Cortical layering disrupts multi-electrode current steering

Objective. Blindness affects approximately 40 million people worldwide and has inspired the development of cortical visual prostheses for restoring sight. Cortical visual prostheses electrically stimulate neurons of the visual cortex to artificially evoke visual percepts. Of the 6 layers of the visual cortex, layer 4 contains neurons that are likely to evoke a visual percept. Intracortical prostheses therefore aim to target layer 4; however, this can be difficult due to cortical curvature, inter-subject cortical variability, blindness-induced anatomical changes in cortex, and electrode placement variations. We investigated the feasibility of using current steering to stimulate specific cortical layers between electrodes in the laminar column. Approach. We explored whether the multiunit neural activity peak can be manipulated between two simultaneously stimulating electrodes in different layers of the cortical column. A 64-channel, 4-shank electrode array was implanted into the visual cortex of Sprague–Dawley rats (n = 7) orthogonal to the cortical surface. A remote return electrode was positioned over the frontal cortex in the same hemisphere. Charge was supplied to two stimulating electrodes along a single shank. Differing ratios of charge (100:0, 75:25, 50:50) and separation distances (300–500 μm) were tested. Results. Current steering across the cortical layers did not result in a consistent shift of the neural activity peak. Both single-electrode and dual-electrode stimulation induced activity throughout the cortical column. This contrasts observations that current steering evoked a controllable peak of neural activity between electrodes implanted at similar cortical depths. However, dual-electrode stimulation across the layers did reduce the stimulation threshold at each site compared to single-electrode stimulation. Significance. Multi-electrode stimulation is not suitable for targeted activation of layers using current steering. However, it can be used to reduce activation thresholds at adjacent electrodes within a given cortical layer. This may be applied to reduce the stimulation side effects of neural prostheses, such as seizures.

Trends and Challenges of BCI-based Visual Prostheses

The development of visual prostheses is a popular field of research in recent years. Several approaches had been suggested to create artificial vision, with varying progress to success. This work attempts to briefly review the current state of research and its implication on patients. We mainly focus on the novel retinal and cortical visual prostheses, and investigate effect of various stimulation regions. Eventually, we also presented our concerns and challenges about this field.

FROM CAMERA AS PART OF THE BODY, TO CAMERA – THE BODY PART

The digital world is where 3.4 billion people communicate via digital images through social media. On average, each one of us spends 6 hours each day in front of a screen. The digital images rapidly appearing and disappearing under our scrolling fingers shape our perception, time and body movements, brain structure, and responses in a complex neurological way. The camera enables the formation of our identity, social development, aesthetic value, and cultural participation. Without this camera image, one is isolated, unable to speak. Moreover, without storage where we save images, we are exempt from personal history, narratives, and memory. Once we integrated a camera into the smartphone, we produced an extra organ. With the Orion visual cortical prosthesis medical study that implants a camera into the eyes to restore vision, face emotional surveillance project VibeCheck, and BeAnotherLab software, swooping through virtual reality the bodies, this article follows the socio-philosophical and technological shifts through the history of the camera and photography in object production, body perception, their definition, interaction, and final merge between body and camera object. Through the reference of the theoretical works of Roland Barthes, Maurice Merleau-Ponty, Karen Barad and Tim Ingold, the article follows how we have produced camera images through time and how camera images have been producing us. The camera transforms from a separate room or device to a prosthetic/inherent part of the body, without which the body is not fully functional. Today, a person without a phone camera is marginalized and handicapped. The deficit of not having an image-generating tool turns out to be a sensual deprivation, social limitation and discrepancy in identity.

Toward a personalized closed-loop stimulation of the visual cortex: Advances and challenges.

Current cortical visual prosthesis approaches are primarily unidirectional and do not consider the feed-back circuits that exist in just about every part of the nervous system. Herein, we provide a brief overview of some recent developments for better controlling brain stimulation and present preliminary human data indicating that closed-loop strategies could considerably enhance the effectiveness, safety, and long-term stability of visual cortex stimulation. We propose that the development of improved closed-loop strategies may help to enhance our capacity to communicate with the brain.

Percepts evoked by multi-electrode stimulation of human visual cortex

BackgroundDirect electrical stimulation of early visual cortex evokes the perception of small spots of light known as phosphenes. Previous studies have examined the location, size, and brightness of phosphenes evoked by stimulation of single electrodes. While it has been envisioned that concurrent stimulation of many electrodes could be used as the basis for a visual cortical prosthesis, the percepts resulting from multi-electrode stimulation have not been fully characterized. ObjectiveTo understand the rules governing perception of phosphenes evoked by multi-electrode stimulation of visual cortex. MethodsMulti-electrode stimulation was conducted in human epilepsy patients. We examined the number and spatial arrangement of phosphenes evoked by stimulation of individual multi-electrode groups (n = 8), and the ability of subjects to discriminate between the pattern of phosphenes generated by stimulation of different multi-electrode groups (n = 7). ResultsSimultaneous stimulation of pairs of electrodes separated by greater than 4 mm tended to produce perception of two distinct phosphenes. Simultaneous stimulation of three electrodes gave rise to a consistent spatial pattern of phosphenes, but with significant variation in the absolute location, size, and orientation of that pattern perceived on each trial. Although multi-electrode stimulation did not produce perception of recognizable forms, subjects could use the pattern of phosphenes evoked by stimulation to perform simple discriminations. ConclusionsThe number of phosphenes produced by multi-electrode stimulation can be predicted using a model for spread of activity in early visual cortex, but there are additional subtle effects that must be accounted for.

Add-on optical mask to declutter visual information based on depth for visual prostheses.

One of the challenges in retinal and cortical visual prostheses is decluttering the visual information of the scene. Current device use brightness information, therefore, bright faraway targets can dominate the perception and nearby obstacles may be missed. Therefore, the aim is to ensure that visual information of interest is delivered to the implant. To achieve this, it is essential to obtain information regarding the distance of each object in a scene. Owing to limited power requirements, it is problematic to use an active depth sensor such as Lidar, which requires strong illumination. specifically, in outdoor environments. Depth sensors based on stereo imaging require heavy computational resources, which is also problematic in wearable devices. There is a need for simple passive and robust depth sensor. Herein, we demonstrate the usability of an optical mask that can be placed in front of any RGB camera to create a depth sensor. The mask introduces chromatic aberrations and artificially separates the colors by creating pre-defined phase errors that shift the focal plane as a function of the wavelength. We designed and fabricated a mask that, in combination with the lens of the camera, allows each color to be in focus at a different distance. The red channel produces a focused image for objects that are in the near range, the green channel focuses objects at the mid-range, and the blue channel focuses distant objects. Because realistic targets contain radiation at all wavelengths, they appear in focus at the color channel that matches their distance. Finally, a standard deviation image filter is used to extract only the information that is in focus and to convert the color information to depth information. Clinical Relevance-The proposed method can enhance the usability and functionality of current and future visual prostheses, specifically in outdoor environments, by providing depth information using a passive miniature optical device.

Micro-magnetic stimulation of primary visual cortex induces focal and sustained activation of secondary visual cortex.

Cortical visual prostheses that aim to restore sight to the blind require the ability to create neural activity in the visual cortex. Electric stimulation delivered via microelectrodes implanted in the primary visual cortex (V1) has been the most common approach, although conventional electrodes may not effectively confine activation to focal regions and thus the acuity they create may be limited. Magnetic stimulation from microcoils confines activation to single cortical columns of V1 and thus may prove to be more effective than conventional microelectrodes, but the ability of microcoils to drive synaptic connections has not been explored. Here, we show that magnetic stimulation of V1 using microcoils induces spatially confined activation in the secondary visual cortex (V2) in mouse brain slices. Single-loop microcoils were fabricated using platinum-iridium flat microwires, and their effectiveness was evaluated using calcium imaging and compared with that of monopolar and bipolar electrodes. Our results show that compared to the electrodes, the microcoils better confined activation to a small region in V1. In addition, they produced more precise and sustained activation in V2. The finding that microcoil-based stimulation propagates to higher visual centres raises the possibility that complex visual perception, e.g. that requiring sustained synaptic inputs, may be achievable. This article is part of the theme issue 'Advanced neurotechnologies: translating innovation for health and well-being'.

A narrative review of cortical visual prosthesis systems: the latest progress and significance of nanotechnology for the future.

Background and ObjectiveWe sought to review the latest developments in cortical visual prosthesis (CVP) systems and the significance of nanotechnology for the future. Over the past century, CVP systems have been researched and developed, resulting in various unique surgical and mechanical techniques. Research findings indicate that partial vision recovery is possible, with improvements in coarse target functions and performance in routine activities.MethodsThis review discusses the architecture and physiology of the visual cortex, the neuroplasticity of the blind brain, and the history of CVP development, and also provides an update on the CVP systems currently being examined in research and clinical trials. Due to advances in nanotechnology, it is possible to make CVPs that are smaller, more efficient, and more biocompatible than ever before.Key Content and FindingsCurrently, 3 CVPs have entered clinical trials, and several additional systems are undergoing preclinical reviews to determine the safety of the devices for chronic implantation. This development provides the first indication that the area of cortical vision restoration medication may be able to meaningfully benefit blind people. However, several significant technical and biological challenges need to be solved before the gap between artificial and natural eyesight can be reconciled. Rapid breakthroughs in nanotechnology have considerably increased its use in biological domains.ConclusionsThis paper summarizes the recent progress of CVP in recent years and its future development direction. It is forecasted that nanotechnology can provide better technical support for the development of CVP.

Modern concepts of bionic vision

Loss of vision is a pressing medical and social problem leading to profound disability, loss of ability to work, serious alterations in the psycho-emotional state, and a decline of the quality of life. When conservative or surgical treatment can not help restore vision, the use of visual prosthesis - bionic eye - can be an effective solution. This review covers the main modern approaches to the development of visual prosthetic systems. Analysis of publications revealed that there are several main approaches to visual prosthesis differing primarily by the anatomical structure targeted for stimulation in order to activate visual sensations. The most significant among them are retinal prostheses, optic nerve stimulation, and cortical visual prostheses. Currently, retinal prostheses such as ARGUS II demonstrate the most successful results, since the stimulation of the surviving neural structures of the retina is a relatively easy task, but their field of application is limited to diseases associated with pathological changes in photoreceptors. The development of cortical visual prostheses is more difficult, but in the future they may allow using more stimulation channels to obtain a more detailed visual perception. In addition, cortical visual prostheses are universal, as they do not require preservation of any structures of the visual organ, only the primary visual cortex.

Non-invasive ultrasound stimulation on the retina and visual cortex for visual restoration

Nearly 30% of the U.S. population aged over 75 have age-related macular degeneration and retinitis pigmentosa. 10% of them may become legally blind. Currently, blindness cannot be cured and patients’ living quality can be compromised severely. Electrical visual prostheses are used as the best near-term strategy. However, they have challenges including invasive devices, complex implantation surgeries, and limited resolution. Non-invasive ultrasonic neuromodulation can be a promising technology for non-invasive visual prostheses. Our work showed that direct ultrasound stimulation on the retina can evoke neuron activities in either normal-sighted or retinal degenerated blind rats in vivo, indicating a promising future of ultrasound stimulation as a novel and non-invasive visual prosthesis for translational applications in blind patients. For patients with damaged optic nerves, retinal prostheses are not effective. Thus, we also investigated the feasibility of transcranial focused ultrasound for non-invasive stimulation of the visual cortex to develop a non-invasive cortical visual prosthesis. Besides stimulations, ultrasound has been used to non-invasively monitor brain activity by imaging cortical blood flow. Our study shows that ultrasound-induced visual responses can be monitored by ultrasound flow imaging of the brain, constructing an ultrasound stimulation-recording closed-loop system.

Eye movements and the perceived location of phosphenes generated by intracranial primary visual cortex stimulation in the blind

BackgroundRestoring sight for the blind using electrical stimulation of the visual pathways is feasible but demands an understanding of the spatial mapping of the visual world at the site of targeted stimulation, whether in the retina, thalamus, or cortex. While a visual cortex stimulator can bypass the eye and create visual percepts, there is an inherent dissociation between this stimulation and eye movements. It is unknown whether and how robustly the brain maintains the oculomotor circuitry in patients with bare- or no-light perception. ObjectiveTo critically and quantitatively evaluate the effect of eye movements have on phosphene locations elicited by cortical stimulation that bypasses the eyes in order to restore sight in blind subjects. MethodsThe NeuroPace Responsive Neurostimulator (RNS) and the Orion visual cortical prosthesis devices were used to electrically stimulate the visual cortex of blind subjects with bare or no light perception. Eye positions were recorded synchronized with stimulation and the location of the percepts were measured using a handheld marker. ResultsThe locations of cortical stimulation-evoked percepts are shifted based on the eye position at the time of stimulation. Measured responses can be remapped based on measured eye positions to determine the retinotopic locations associated with the implanted electrodes, with remapped responses having variance limited by pointing error. ConclusionsEye movements dominate the perceived location of cortical stimulation-evoked phosphenes, even after years of blindness. By accounting for eye positions, we can mimic retinal mapping as in natural sight.

Postmortem investigation of a human cortical visual prosthesis that was implanted for 36 years

ObjectivePostmortem analysis of the brain from a blind human subject who had a cortical visual prosthesis implanted for 36 years (Dobelle 2000 Asaio J. 46 3–9)ApproachThis provided insight into the design requirements for a successful human cortical visual prosthesis by revealing, (a) unexpected rotation of the electrode array 25 to 40 degrees away from the midsagittal plane, thought to be due to the torque of the connecting cable, (b) degradation of the platinum electrodes, and (c) only partial coverage of the primary visual cortex by the rectangular array. The electrode array only overlapped with the anterior 45% of primary visual cortex (identified by the line of Gennari), largely missing the posterior foveal representation of visual cortex.Main resultsA significantly greater proportions of electrodes outside of V1 elicited phosphenes than did electrodes within of V1. Histology did not reveal appreciable loss of neurons in cortex that surrounded the migrated array, perhaps due to the very slow rotation of this implant.SignificanceThis pioneering effort to develop a cortical visual prosthesis suggests that to maximize efficacy, the long-term effects of implanted alien materials on nervous tissue, and vice versa, need to be considered in detail, and that electrode array design considerations need to optimally match the electrodes to the patient’s cortical anatomy. Modern pre-implant imaging can help optimize future implants by identifying the location and extent of bridging veins with MRI and even map the location of the V1/V2 border in vivo with PET.

Local field potential phase modulates neural responses to intracortical electrical stimulation.

Cortical visual prostheses could one day help restore sight to the blind by targeting the visual cortex with electrical stimulation. However, power consumption and limited spatial resolution impose limits on performance, while large amounts of electrical charge sometimes necessary to evoke phosphenes can cause seizures. Here, we propose the use of the local field potential as a control signal for the timing of stimulation to reduce charge requirements. In Sprague-Dawley rats, visual cortex was electrically stimulated at random times, and neural responses recorded. Electrical stimulation at specific phases of the local field potential required smaller amounts of charge to elicit spikes than naïve stimulation. Incorporating this into prosthesis design could improve their safety and efficacy.