Cortical Prosthesis Research Articles

A virtual patient simulation modeling the neural and perceptual effects of human visual cortical stimulation, from pulse trains to percepts

The field of cortical sight restoration prostheses is making rapid progress with three clinical trials of visual cortical prostheses underway. However, as yet, we have only limited insight into the perceptual experiences produced by these implants. Here we describe a computational model or ‘virtual patient’, based on the neurophysiological architecture of V1, which successfully predicts the perceptual experience of participants across a wide range of previously published human cortical stimulation studies describing the location, size, brightness and spatiotemporal shape of electrically induced percepts in humans. Our simulations suggest that, in the foreseeable future the perceptual quality of cortical prosthetic devices is likely to be limited by the neurophysiological organization of visual cortex, rather than engineering constraints.

Comparison of electrical microstimulation artifact removal methods for high-channel-count prostheses

BackgroundNeuroprostheses are used to electrically stimulate the brain, modulate neural activity and restore sensory and motor function following injury or disease, such as blindness, paralysis, and other movement and psychiatric disorders. Recordings are often made simultaneously with stimulation, allowing the monitoring of neural signals and closed-loop control of devices. However, stimulation-evoked artifacts may obscure neural activity, particularly when stimulation and recording sites are nearby. Several methods have been developed to remove stimulation artifacts, but it remains challenging to validate and compare these methods because the ‘ground-truth’ of the neuronal signals may be contaminated by artifacts. New methodHere, we delivered stimulation to the visual cortex via a high-channel-count prosthesis while recording neuronal activity and stimulation artifacts. We quantified the waveforms and temporal properties of stimulation artifacts from the cortical visual prosthesis (CVP) and used them to build a dataset, in which we simulated the neuronal activity and the stimulation artifacts. We illustrate how to use the simulated data to evaluate the performance of six software-based artifact removal methods (Template subtraction, Linear interpolation, Polynomial fitting, Exponential fitting, SALPA and ERAASR) in a CVP application scenario. ResultsWe here focused on stimulation artifacts caused by electrical stimulation through a high-channel-count cortical prosthesis device. We find that the Polynomial fitting and Exponential fitting methods outperform the other methods in recovering spikes and multi-unit activity. Linear interpolation and Template subtraction recovered the local-field potentials. ConclusionPolynomial fitting and Exponential fitting provided a good trade-off between the quality of the recovery of spikes and multi-unit activity (MUA) and the computational complexity for a cortical prosthesis.

Numerical Design of a Micro-LED Based Optogenetic Stimulator for Visual Cortical Prosthesis

Numerical Design of a Micro-LED Based Optogenetic Stimulator for Visual Cortical Prosthesis

Towards biologically plausible phosphene simulation for the differentiable optimization of visual cortical prostheses.

Blindness affects millions of people around the world. A promising solution to restoring a form of vision for some individuals are cortical visual prostheses, which bypass part of the impaired visual pathway by converting camera input to electrical stimulation of the visual system. The artificially induced visual percept (a pattern of localized light flashes, or 'phosphenes') has limited resolution, and a great portion of the field's research is devoted to optimizing the efficacy, efficiency, and practical usefulness of the encoding of visual information. A commonly exploited method is non-invasive functional evaluation in sighted subjects or with computational models by using simulated prosthetic vision (SPV) pipelines. An important challenge in this approach is to balance enhanced perceptual realism, biologically plausibility, and real-time performance in the simulation of cortical prosthetic vision. We present a biologically plausible, PyTorch-based phosphene simulator that can run in real-time and uses differentiable operations to allow for gradient-based computational optimization of phosphene encoding models. The simulator integrates a wide range of clinical results with neurophysiological evidence in humans and non-human primates. The pipeline includes a model of the retinotopic organization and cortical magnification of the visual cortex. Moreover, the quantitative effects of stimulation parameters and temporal dynamics on phosphene characteristics are incorporated. Our results demonstrate the simulator's suitability for both computational applications such as end-to-end deep learning-based prosthetic vision optimization as well as behavioral experiments. The modular and open-source software provides a flexible simulation framework for computational, clinical, and behavioral neuroscientists working on visual neuroprosthetics.

Invited Session I: Vision restoration: The New Generation of Visual Cortical Prosthetics.

Since the 1960s, the prospect of developing a visual cortical prosthesis for restoration of useful vision to the blind has tantalized scientists, engineers, and clinicians. After two decades of relative inactivity, interest in this approach has been renewed. There are now multiple active efforts to develop next generation visual prostheses and to bring these devices to clinical implementation. We will review the current state of the field, with a particular emphasis on our own work, and will highlight advances in the field as well challenges that must be overcome to produce a clinically useful prosthetic device.

FROM CAMERA AS PART OF THE BODY, TO CAMERA – THE BODY PART

The digital world is where 3.4 billion people communicate via digital images through social media. On average, each one of us spends 6 hours each day in front of a screen. The digital images rapidly appearing and disappearing under our scrolling fingers shape our perception, time and body movements, brain structure, and responses in a complex neurological way. The camera enables the formation of our identity, social development, aesthetic value, and cultural participation. Without this camera image, one is isolated, unable to speak. Moreover, without storage where we save images, we are exempt from personal history, narratives, and memory. Once we integrated a camera into the smartphone, we produced an extra organ. With the Orion visual cortical prosthesis medical study that implants a camera into the eyes to restore vision, face emotional surveillance project VibeCheck, and BeAnotherLab software, swooping through virtual reality the bodies, this article follows the socio-philosophical and technological shifts through the history of the camera and photography in object production, body perception, their definition, interaction, and final merge between body and camera object. Through the reference of the theoretical works of Roland Barthes, Maurice Merleau-Ponty, Karen Barad and Tim Ingold, the article follows how we have produced camera images through time and how camera images have been producing us. The camera transforms from a separate room or device to a prosthetic/inherent part of the body, without which the body is not fully functional. Today, a person without a phone camera is marginalized and handicapped. The deficit of not having an image-generating tool turns out to be a sensual deprivation, social limitation and discrepancy in identity.

Restoring vision in blind patients: From optogenetic to sonogenetic therapies

AbstractVisual restoration is certainly the greatest challenge for brain‐machine interfaces with the high pixel number and high refreshing rate. In the recent year, we brought retinal prostheses and optogenetic therapy up to successful clinical trials. Optogenetics relies on the expression of a microbial opsin in residual neurons by gene therapy. These strategies can provide some vision in blind patients with diseases of photoreceptors such as retinal dystrophies or age‐related macular degeneration. They can reintroduce some information in at the retinal level by activating residual neurons after the loss of photoreceptors. For patients with optic nerve atrophy as in glaucoma and diabetic retinopathy, visual restoration requires the direct activation of the visual cortex. However, cortical prostheses have shown efficacy for limited periods of time and limited pixel numbers.In the present project, we are investigating a novel approach to restore vision at the cortical level with sonogenetic therapy. This approach relies on expressing an ultrasound‐sensitive ionic channel in cortical neurons to imprint images at this level by ultrasound stimulations.The Mscl ionic channel fused to the tdTomato reporter gene was expressed in the visual cortex of rats following a cortical AAV injection. The channel expression was indicated by the red fluorescence of tdTomato. Recording with penetrating electrodes indicated responses to cortical ultrasound stimulation with a high temporal resolution but a shorter latency than natural visual responses. In the absence of AAV injection, the visual cortex was not responsive to ultrasounds. Moving the ultrasound stimulator moved the activated area as indicated by Ecog electrode array recording. Finally, mice having learned to associate a flash of light to a water delivery responded similarly to an ultrasound cortical stimulation if the MscL protein was expressed in their cortex.These results indicate that the MscL channel induces an ultrasound sensitivity of cortical neurons generating a cortical activations with a spatiotemporal resolution suitable for vision restoration and perceived as phosphenes. Further studies will have to demonstrate true vision of forms by this sonogenetic approach prior to a clinical translation.

Multichannel stimulation module as a tool for animal studies on cortical neural prostheses.

Intracortical microstimulation to the visual cortex is thought to be a feasible technique for inducing localized phosphenes in patients with acquired blindness, and thereby for visual prosthesis. In order to design effective stimuli for the prosthesis, it is important to elucidate relationships between the spatio-temporal patterns of stimuli and the resulting neural responses and phosphenes through pre-clinical animal studies. However, the physiological basis of effective spatial patterns of the stimuli for the prosthesis has been little investigated in the literature, at least partly because that the previously developed multi-channel stimulation systems were designed specifically for the clinical use. In the present, a 64-channel stimulation module was developed as a scalable tool for animal experiments. The operations of the module were verified by not only dry-bench tests but also physiological animal experiments in vivo. The results demonstrated its usefulness for examining the stimulus-response relationships in a quantitative manner, and for inducing the multi-site neural excitations with a multi-electrode array. In addition, this stimulation module could be used to generate spatially patterned stimuli with up to 4,096 channels in a dynamic way, in which the stimulus patterns can be updated at a certain frame rate in accordance with the incoming visual scene. The present study demonstrated that our stimulation module is applicable to the physiological and other future studies in animals on the cortical prostheses.

Percepts evoked by multi-electrode stimulation of human visual cortex

BackgroundDirect electrical stimulation of early visual cortex evokes the perception of small spots of light known as phosphenes. Previous studies have examined the location, size, and brightness of phosphenes evoked by stimulation of single electrodes. While it has been envisioned that concurrent stimulation of many electrodes could be used as the basis for a visual cortical prosthesis, the percepts resulting from multi-electrode stimulation have not been fully characterized. ObjectiveTo understand the rules governing perception of phosphenes evoked by multi-electrode stimulation of visual cortex. MethodsMulti-electrode stimulation was conducted in human epilepsy patients. We examined the number and spatial arrangement of phosphenes evoked by stimulation of individual multi-electrode groups (n = 8), and the ability of subjects to discriminate between the pattern of phosphenes generated by stimulation of different multi-electrode groups (n = 7). ResultsSimultaneous stimulation of pairs of electrodes separated by greater than 4 mm tended to produce perception of two distinct phosphenes. Simultaneous stimulation of three electrodes gave rise to a consistent spatial pattern of phosphenes, but with significant variation in the absolute location, size, and orientation of that pattern perceived on each trial. Although multi-electrode stimulation did not produce perception of recognizable forms, subjects could use the pattern of phosphenes evoked by stimulation to perform simple discriminations. ConclusionsThe number of phosphenes produced by multi-electrode stimulation can be predicted using a model for spread of activity in early visual cortex, but there are additional subtle effects that must be accounted for.

Stereoscopic Distance Filtering Plus Thermal Imaging Glasses Design.

The authors present the development of eyewear that incorporates stereoscopic and thermal imaging cameras for the purpose of highlighting objects/views of interest. Image processing algorithms that simplify complex elements in a scene have the ability to improve the utility of blind and low vision aids. Thermal imaging can be used to highlight important objects such as people or animals, while stereoscopic imaging can be used to filter background imagery beyond a certain distance. The methods used have been successful in providing utility to retinal prosthesis users. The stereoscopic camera systems involved strict requirements on the relative orientation of the cameras for calibrated distance filtering. A mechanical design is presented that fixes the relative camera locations on a 3D printed titanium structure that can float in the frame to maintain orientations even when the eyewear is flexed during wearing. Clinical Relevance - The design presented has utility in improving perceived spatial resolution in implantable retinal prostheses, implantable visual cortical prostheses, direct-vision prostheses, and wearable low vision aids.

Sequence of visual cortex stimulation affects phosphene brightness in blind subjects

BackgroundVisual cortical prostheses (VCP) could potentially benefit a majority of the blind population. Feasibility testing of these VCP opens new avenues to characterize stimulation of visual cortex in blind subjects. Objective/hypothesisTo determine if sequential stimulation of visual cortex produces a perception bias in phosphene brightness. MethodsWe stimulated three blind subjects implanted with the Orion array with sequences of two and three electrodes and asked them to determine the brighter phosphene, using interval forced-choice paradigms. We selected a set of reference electrodes as the constant stimuli across sequences and compared across three different amplitude levels keeping all other stimulation parameters fixed across electrodes. ResultsFor two subjects, we measured a significant increase in the probability of perceiving a lower-level amplitude just as bright or brighter than a higher-level amplitude when stimulated later in the sequence (p < 0.001, Wilcoxon rank sum test). The probability of reference electrodes selected as brighter was also higher during the second phase, across most amplitude comparisons. For the third subject, there were measurable but not significant changes, where the first stimuli were perceived as brighter. The effects were consistent within subjects in the three-electrode sequences, where the probability of the reference electrode selected as brighter was correlated to when it was presented in the sequence. ConclusionsWe showed evidence of temporal interactions in non-overlapping sequences of electrodes, where the direction of the effect was subject specific but consistent across a variety of electrode locations and current amplitude levels.

Multi-electrode stimulation evokes consistent spatial patterns of phosphenes and improves phosphene mapping in blind subjects

BackgroundVisual cortical prostheses (VCPs) have the potential to restore visual function to patients with acquired blindness. Successful implementation of VCPs requires the ability to reliably map the location of the phosphene produced by stimulation of each implanted electrode. ObjectiveTo evaluate the efficacy of different approaches to phosphene mapping and propose simple improvements to mapping strategy. MethodsWe stimulated electrodes implanted in the visual cortex of five blind and fifteen sighted patients. We tested two fixation strategies, unimanual fixation, where subjects placed a single index finger on a tactile fixation point and bimanual fixation, where subjects overlaid their right index finger over their left on the tactile point. In addition, we compared absolute mapping in which a single electrode was stimulated on each trial, and relative mapping with sequences containing stimulation of three to five phosphenes on each trial. Trial-to-trial variability present in relative mapping sequences was quantified. ResultsPhosphene mapping was less precise in blind subjects than in sighted subjects (2DRMS, 16 ± 2.9° vs. 1.9 ± 0.93°; t (18) = 18, p = <0.001). Within blind subjects, bimanual fixation resulted in more consistent phosphene localization than unimanual fixation (BS1: 4.0 ± 2.6° vs. 19 ± 4.7°, t (79) = 24, p < 0.001; BS2 4.1 ± 2.0° vs. 12 ± 2.7°, t (65) = 19, p < 0.001). Multi-point relative mapping had similar baseline precision to absolute mapping (BS1: 4.7 ± 2.6° vs. 3.9 ± 2.0°; BS2: 4.1 ± 2.0° vs. 3.2 ± 1.1°) but improved significantly when trial-to-trial translational variability was removed. Although multi-point mapping methods did reveal more of the functional organization expected in early visual cortex, subjects tended to artificially regularize the spacing between phosphenes. We attempt to address this issue by fitting a standard logarithmic map to relative multi-point sequences. ConclusionsRelative mapping methods, combined with bimanual fixation, resulted in the most precise estimates of phosphene organization. These techniques, combined with use of a standard logarithmic model of visual cortex, may provide a practical way to improve the implementation of a VCP.

Eye movements and the perceived location of phosphenes generated by intracranial primary visual cortex stimulation in the blind

BackgroundRestoring sight for the blind using electrical stimulation of the visual pathways is feasible but demands an understanding of the spatial mapping of the visual world at the site of targeted stimulation, whether in the retina, thalamus, or cortex. While a visual cortex stimulator can bypass the eye and create visual percepts, there is an inherent dissociation between this stimulation and eye movements. It is unknown whether and how robustly the brain maintains the oculomotor circuitry in patients with bare- or no-light perception. ObjectiveTo critically and quantitatively evaluate the effect of eye movements have on phosphene locations elicited by cortical stimulation that bypasses the eyes in order to restore sight in blind subjects. MethodsThe NeuroPace Responsive Neurostimulator (RNS) and the Orion visual cortical prosthesis devices were used to electrically stimulate the visual cortex of blind subjects with bare or no light perception. Eye positions were recorded synchronized with stimulation and the location of the percepts were measured using a handheld marker. ResultsThe locations of cortical stimulation-evoked percepts are shifted based on the eye position at the time of stimulation. Measured responses can be remapped based on measured eye positions to determine the retinotopic locations associated with the implanted electrodes, with remapped responses having variance limited by pointing error. ConclusionsEye movements dominate the perceived location of cortical stimulation-evoked phosphenes, even after years of blindness. By accounting for eye positions, we can mimic retinal mapping as in natural sight.

Vision restoration: science fiction or reality?

Visual prostheses aim at restoring useful vision to patients who have become blind. This useful vision should enable them to regain autonomy in society for navigation, face recognition or reading. Two retinal prostheses have already obtained market authorization for patients affected by retinal dystrophies while a new device is in clinical trials for patients affected by age-related macular degeneration. Various prostheses, in particular cortical prostheses, are currently in clinical trials for optic neuropathies (glaucoma). Optogenetic therapy, an alternative strategy, has now reached the stage of clinical trials at the retinal level while moving forward at the cortical level. Other innovating strategies have obtained proofs of concepts in rodents but require a further validation in large animals prior to their evaluation on patients. Restoring vision should therefore become a reality for many patients even if this vision will not be as extensive and perfect as natural vision.

Spatially confined responses of mouse visual cortex to intracortical magnetic stimulation from micro-coils

Objective. Electrical stimulation via microelectrodes implanted in cortex has been suggested as a potential treatment for a wide range of neurological disorders. Despite some success however, the effectiveness of conventional electrodes remains limited, in part due to an inability to create specific patterns of neural activity around each electrode and in part due to challenges with maintaining a stable interface. The use of implantable micro-coils to magnetically stimulate the cortex has the potential to overcome these limitations because the asymmetric fields from coils can be harnessed to selectively activate some neurons, e.g. vertically-oriented pyramidal neurons while avoiding others, e.g. horizontally-oriented passing axons. In vitro experiments have shown that activation is indeed confined with micro-coils but their effectiveness in the intact brain of living animals has not been evaluated. Approach. To assess the efficacy of stimulation, a 128-channel custom recording microelectrode array was positioned on the surface of the visual cortex (ECoG) in anesthetized mice and responses to magnetic and electric stimulation were compared. Stimulation was delivered from electrodes or micro-coils implanted through a hole in the center of the recording array at a rate of 200 pulses per second for 100 ms. Main results. Both electric and magnetic stimulation reliably elicited cortical responses, although activation from electric stimulation was spatially expansive, often extending more than 1 mm from the stimulation site, while activation from magnetic stimulation was typically confined to a ∼300 µm diameter region around the stimulation site. Results were consistent for stimulation of both cortical layer 2/3 and layer 5 as well as across a range of stimulus strengths. Significance. The improved focality with magnetic stimulation suggests that the effectiveness of cortical stimulation can be improved. Improved focality may be particularly attractive for cortical prostheses that require high spatial resolution, e.g. devices that target sensory cortex, as it may lead to improved acuity.

The Monash Vision Cortical Prosthesis Quality Assurance

Monash software was developed for the people with no eye so they came up with idea of bionic eye. For every tile have 43 dynamic terminals on its base, and a remotely controlled electronic framework to translate control signals and drive the cathodes with biphasic heartbeats. Refined picture handling, depicted in a buddy paper, guarantees that the client encounters most extreme profit by the modest number of cathodes.. And quality will be assured after the results to make it more feasible and easy to use for the laymen.

Dynamic Stimulation of Visual Cortex Produces Form Vision in Sighted and Blind Humans

A visual cortical prosthesis (VCP) has long been proposed as a strategy for restoring useful vision to the blind, under the assumption that visual percepts of small spots of light produced with electrical stimulation of visual cortex (phosphenes) will combine into coherent percepts of visual forms, like pixels on a video screen. We tested an alternative strategy in which shapes were traced on the surface of visual cortex by stimulating electrodes in dynamic sequence. In both sighted and blind participants, dynamic stimulation enabled accurate recognition of letter shapes predicted by the brain's spatial map of the visual world. Forms were presented and recognized rapidly by blind participants, up to 86 forms per minute. These findings demonstrate that a brain prosthetic can produce coherent percepts of visual forms.

A preliminary implementation of an active intraocular prosthesis as a new image acquisition device for a cortical visual prosthesis.

An active intraocular prosthesis is herein proposed as a new image acquisition device for a cortical visual prosthesis. A conventional intraocular prosthesis is a passive device that helps blind patients underwent eye enucleation to maintain the shape of an eyeball. In contrast, an active intraocular prosthesis, which works as an implantable wireless camera, can capture real-time images and transmit them to a cortical visual prosthesis to restore partial vision of the patients. This active device has distinct advantages in that it can garner a variety of image information while focusing on objects in accordance with natural eye movements, compared with a glasses-mounted camera and implanted micro-photodiodes in typical artificial vision systems. Coated with an epoxy and sealed by an elastomer for biocompatibility as well as durability, the active intraocular prosthesis was fabricated in a spherical form miniaturized enough to be inserted into the eye. Its operation was evaluated by wireless image acquisition displaying a processed gray-scale image. Furthermore, signal-to-noise ratio measurements were conducted to find a reliable communication range of the fabricated prosthesis, while it was covered by an 8-mm-thick biological medium that mimicked in vivo environments. In conclusion, the feasibility of the active intraocular prosthesis to cooperate with a cortical visual prosthesis is discussed.

The Monash Vision Cortical Prosthesis Quality Assurance

The Monash Vision Cortical Prosthesis Quality Assurance

Early Feasibility Study of a Neurostimulator to Create Artificial Vision

Abstract INTRODUCTION Stimulation of human visual cortex is known to elicit visual perceptions that could potentially be used for creating artificial vision. The Orion Visual Cortical Prosthesis is a new device that is intended to restore some functional vision to blind patients. METHODS The device comprises an implant (consisting of an electronics package, receiving antenna, and an electrode array with 60 electrodes); glasses with a video camera; and a video processing unit (VPU). The camera collects real-time visual information, which is then processed by the VPU and converted to stimulation patterns sent to electrode array. The electronics are skull mounted with the subdural array implanted through a posterior interhemispheric approach. This is a 5-yr study of subjects who are bilaterally blind with bare light or no light perception due to non-cortical etiology. RESULTS A total of 6 subjects have been implanted in 2 centers. As of May 1, 2019, average implant duration was 11.1 mo (range 3.4-15.0 mo). Average age at time of implant was 50.3 yr. Cause of blindness included trauma (2), pediatric glaucoma (2), optic neuropathy (1), and endophthalmitis (1). One serious adverse device event (seizure) has been reported. Average thresholds ranged from 1.6 to 3.7 mA across the 6 subjects. At 6 mo postimplant, 3 of 5 subjects performed significantly better with the system on than off on a light localization task; 2 subjects performed better on a direction of motion task, and no subjects had measurable visual acuity. All 5 subjects were rated as receiving “positive” or “mild positive” benefit on a functional vision assessment. One-year adverse event and visual performance data for the first 5 subjects will be presented and compared to results from a commercially-available retinal prosthesis. CONCLUSION Safety and performance results of the first 5 subjects as of 6 mo postimplantation appear encouraging.