Deep Spike Research Articles

Deep Spike Learning With Local Classifiers.

Backpropagation has been successfully generalized to optimize deep spiking neural networks (SNNs), where, nevertheless, gradients need to be propagated back through all layers, resulting in a massive consumption of computing resources and an obstacle to the parallelization of training. A biologically motivated scheme of local learning provides an alternative to efficiently train deep networks but often suffers a low performance of accuracy on practical tasks. Thus, how to train deep SNNs with the local learning scheme to achieve both efficient and accurate performance still remains an important challenge. In this study, we focus on a supervised local learning scheme where each layer is independently optimized with an auxiliary classifier. Accordingly, we first propose a spike-based efficient local learning rule by only considering the direct dependencies in the current time. We then propose two variants that additionally incorporate temporal dependencies through a backward and forward process, respectively. The effectiveness and performance of our proposed methods are extensively evaluated with six mainstream datasets. Experimental results show that our methods can successfully scale up to large networks and substantially outperform the spike-based local learning baselines on all studied benchmarks. Our results also reveal that gradients with temporal dependencies are essential for high performance on temporal tasks, while they have negligible effects on rate-based tasks. Our work is significant as it brings the performance of spike-based local learning to a new level with the computational benefits being retained.

A Deep Neural Network-Based Spike Sorting With Improved Channel Selection and Artefact Removal

In order to implement highly efficient brain-machine interface (BMI) systems, high-channel count sensing is often used to record extracellular action potentials. However, the extracellular recordings are typically severely contaminated by artefacts and various noise sources, rendering the separation of multi-unit neural recordings an immensely challenging task. Removing artefact and noise from neural events can improve the spike sorting performance and classification accuracy. This paper presents a deep learning technique called deep spike detection (DSD) with a strong learning ability of high-dimensional vectors for neural channel selection and artefacts removal from the selected neural channel. The proposed method significantly improves spike detection compared to the conventional methods by sequentially diminishing the noise level and discarding the active artefacts in the recording channels. The simulated and experimental results show that there is considerably better performance when the extracellular raw recordings are cleaned prior to assigning individual spikes to the neurons that generated them. The DSD achieves an overall classification accuracy of 91.53% and outperformes Wave_clus by 3.38% on the simulated dataset with various noise levels and artefacts.

Contractive Slab and Spike Convolutional Deep Belief Network

Convolutional Deep Belief Network (CDBN) is typically classified into deep generative model. Although CDBN has demonstrated the powerful capacity of feature extraction in unsupervised learning, there still remain diverse challenges in the robust and high-quality feature extraction. This paper designs an advanced hierarchical generative model in order to tackle with these troubles. First, we modify conventional Convolutional Restricted Boltzmann Machine (CRBM) through inducing Gaussian hidden units subsequently following point-wise multiplication with the original binary spike hidden units for high-order feature extraction of the local patch. We theoretically derive entire inferences of this novel model. Second, we attempt to learn more robust features by minimizing L2 norm of the jacobian of the extracted features producing from the modified model as novel regularization trick. This can introduce a localized space contraction benefit for robust feature extraction in turn. Finally, this paper construct a novel deep generative model, Contractive Slab and Spike Convolutional Deep Belief Network (CssCDBN), based on the modified CRBM, in order to learn deeper and more abstract features. The performances on diverse visual tasks indicate that CssCDBN is a more powerful model achieving impressive results over many currently excellent models.

Comparison of dense array EEG with simultaneous intracranial EEG for Interictal spike detection and localization

To evaluate the clinical utility of dense array electroencephalography (dEEG) for the detection yield and localization of interictal spikes in mesial temporal lobe epilepsy. We simultaneously recorded 256-channel dEEG and intracranial electroencephalography (icEEG) implanted over the lateral and mesial temporal lobe in patients with intractable epilepsy. We calculated the dEEG spike detection rate for mesial temporal spikes which were confirmed by icEEG and applied source estimation to dEEG to compare noninvasive localization to the invasive recordings. 339 of 760 interictal spikes (45%) were detected by dEEG examining the 256-channel head surface array. The average icEEG amplitude of dEEG detectable spikes was 1083 μV, and that of dEEG undetectable spikes was 780 μV (P<0.05). All spikes detected in dEEG were localized to the temporal lobe. 295 of 339 spikes (87%) were well localized in mesial temporal lobe, close to the position confirmed by subdural electrodes. 256-channel dEEG may provide more precise information for the localization of interictal epileptiform discharges than conventional EEG or MEG in patients with deep spike foci. 256-channel dEEG may be clinically useful in the presurgical work-up for epilepsy, providing accurate noninvasive guidance for the placement of intracranial electrodes.

Membrane properties and synaptic connectivity of fast-spiking interneurons in rat ventral striatum

In vitro patch-clamp recordings were made to study the membrane properties and synaptic connectivity of fast-spiking interneurons in rat ventral striatum. Using a whole-cell configuration in acutely prepared slices , fast-spiking interneurons were recognized based on their firing properties and their morphological phenotype was confirmed by immunocytochemistry. Membrane properties of fast-spiking interneurons were distinguished from those of medium-sized spiny neurons by their more depolarized resting membrane potential, lower action potential amplitude and shorter half-width, short spike repolarization time and deep spike afterhyperpolarization. Firing patterns of interneurons could be subdivided in a bursting and non-bursting mode. Simultaneous dual whole-cell recordings revealed a high degree of connectivity of fast-spiking interneurons to medium-sized spiny neurons via unidirectional synapses. Burst firing in fast-spiking interneurons that were presynaptic to medium-sized spiny neurons resulted in barrages of postsynaptic potentials showing an initial amplitude increment, rapidly followed by a decrement. In conclusion, ventral striatal fast-spiking interneurons can be clearly distinguished from medium-sized spiny neurons by their membrane properties and their firing patterns can be subdivided in bursting and non-bursting modes. Their synaptic connectivity to medium-sized spiny neurons is unidirectional and characterized by frequency-dependent, dynamic changes in postsynaptic amplitude.

Correlation Between the Microstructures and the Electrical Properties of Ni/Au/Te/Au Contacts on n-GaAs

ABSTRACTThe structural evolution of Ni/Au/Te/Au contacts on n-GaAs (001) was examined, in correlation with their electrical properties as a function of rapid thermal annealing in the temperature range 350 - 600°C. It was found that heating at temperatures ≥ 550°C results in the formation of ohmic contacts, while contacts annealed at lower temperatures remain nonohmic. Transmission electron microscopy revealed that heating ≥ 450°C leads to extensive reactions between Ni/Au/Te/Au and GaAs and deep spike formation into the GaAs. The major reaction products were identified as NiAs and β-AuGa. Ga2Te3 grains, growing epitaxially on GaAs, were detected only in 550°C annealed samples. Heating to 600°C caused considerable Ga2Te3 loss. Implications of these results concerning the ohmic contact formation mechanism are discussed.

Lateral growth of titanium silicide over a silicon dioxide layer

If a silicon dioxide step on a single crystal of silicon covered with titanium is annealed, then, following vertical growth on the silicon part, lateral growth of titanium silicide takes place over the oxide layer. In the temperature range 750–950 °C this lateral growth of Ti–silicide was found to be a linear function of the square root of the annealing time. The activation energy of the lateral growth was found to be 1.89 eV. During lateral growth the Ti–silicide forms a deep spike in the silicon crystal at the SiO2 step.

Surface and deep EEG correlates of surgical outcome in temporal lobe epilepsy.

Interictal and ictal EEG characteristics derived from limited surface montages and medial temporal lobe sites were compared with long-term seizure relief following anterior temporal lobectomy in 52 epileptics. Patients were classified into one of four surgical outcome groups, ranging from seizure free to no clinical improvement. For each patient, interictal records were analyzed according to deep and surface spike characteristics and background activity. Ictal records were analyzed according to the proportion of episodes initiated in a unilateral or bilaterally synchronous fashion, the proportion of surface or deep onsets, the variability of onset location, and the morphology of seizures onsets. Interictal EEG variables that correlates with surgical outcome included: (a) various types of bilaterally synchronous surface/deep spikes; (b) diffuse background slowing; (c) sharp waves; and (d) the presence of multiple independent deep spike patterns in the lobe chosen for resection. Relevant ictal EEG variables included: (a) episodes initiated in a bilaterally synchronous fashion; (b) variability in seizure onset location; (c) the proportion of precisely focal onsets from deep sites; (d) the proportion of surface onsets; and (e) the proportion of onsets from the side chosen for resection. Multivariate analysis of these data with linear, stepwise, discriminate analysis and adaptive, nonlinear, distribution-free pattern recognition demonstrated that: (a) both interictal and ictal EEG characteristics can independently predict surgical outcome at levels significantly better than chance; (b) ictal and interictal EEG data contain nonredundant information for making such predictions; and (c) nonlinear pattern recognition techniques are capable of deriving the most accurate rules for predicting the effects of surgery.

MULTICHANNEL Z‐TRANSFORMS AND MINIMUM‐DELAY

In the standard deterministic model of water reverberation generation, the reverberation pulse‐train resulting from a deep reflection is minimum‐delay. Even in the more complex physical situations encountered in the field, there is evidence that in many cases the reverberation pulse‐train waveforms are minimum‐delay, or at least approximately so. The reason for this minimum‐delay property is that a pulse‐train waveform results from multiple reflections and transmissions within the layered earth; because reflection coefficients are less than unity in magnitude, the concentration of energy in a pulse‐train must appear at its beginning rather than its end; this early concentration of energy is the condition that pulse‐train waveform be minimum‐delay. Each deep reflection horizon contributes a minimum‐delay reverberation pulse‐train waveform to a seismic trace. If we let a spike series represent the deep horizons in the sense that the timing of a spike represents the direct arrival time of a reflection and the amplitude of the spike represents the strength of the reflection, then the seismic trace may be considered as the convolution of the spike series with the reverberation pulse‐train waveform. Because the reverberation pulse‐train waveform is minimum‐delay, and because at least approximately the deep horizon spike series represents a statistically uncorrelated series, the two conditions required for the application of the method of predictive deconvolution (Robinson, 1954, 1957) are met, and hence this method can be used as a practical digital data processing method to eliminate water reverberations on field seismic traces. The concept of minimum‐delay therefore is an important link in chaining together the deterministic approach and the statistical approach to seismic record analysis in the single‐channel case. The concept of minimum‐delay can be extended to the multichannel case. The theory of multichannel digital filters can be regarded as the matrix‐valued counterpart of single‐channel digital filter theory. A reflection seismogram consists of many traces; these traces are interrelated. A multichannel filter operates simultaneously on all these traces, and thus it can take advantage of the seismogram structure between traces as well as along a single trace. An important objective of seismogram analysis is to increase the resolution of overlapping waveforms by deconvolution. This goal can be accomplished through the use of inverse multichannel digital filters. Without proper design, an inverse multichannel filter can have the undesirable property that its impulse response function is unstable. In the case when there is the same number of input channels as output channels, then each of the coefficients of a multichannel digital filter may be regarded as a square matrix, and the z‐transform of the filter coefficients is a matrix‐valued polynomial in z. The determinant of its matrix‐valued z‐transform plays a central role in the classification of the delay properties of such a multichannel filter. This determinant is a scalar‐valued polynomial in z. If the coefficients of this polynomial represents a single‐channel minimum‐delay filter, then the original multichannel filter is also minimum‐delay; if they represent a single‐channel maximum‐delay filter, then the multichannel filter is also maximum‐delay; if they represent a single‐channel mixed‐delay filter, then the multichannel filter is also mixed‐delay. A minimum‐delay multichannel digital filter has an inverse which is a stable memory function. On the other hand, a maximum‐delay multichannel digital filter has an inverse that consists of a stable anticipation function. A mixed‐delay multichannel digital filter has a stable inverse, this inverse being made up of a memory component and an anticipation component.

OLFACTORY BULB RESPONSES IN THE TURTLE, WITH SPECIAL REFERENCE TO THE DEEP NEGATIVE SPIKE

The orthodromic cortical response induced by the olfactory nerve stimulation was analysed by the depth recording and also the commissural and the antidromic stimulations. The following results were obtained;1. The orthodromic cortical response (OCR) consists of two components: the initial potential and the slow potential, negative in sign, at the cortical surface. When recording in the external plexiform layer, the negative deep spike of 3-5 msec duration occurred, following by the slow potential. The latency is about 15 msec from the onset of the initial potential and remains constant throughout the external plexiform layer. The recovery process of deep spike is extremely slow, requiring 2 sec or more for full recovery. In the deeper layer, unit discharge appeared.2. In the internal granular layer the commissural cortical response (CCR) consists of positive-negative slow potential having a long latency of 60 msec. The deep spike does not interact with the preceding CCR.3. The antidromic stimulation induces the grouped spike discharges and slow potential in the depth recording. The both negative slow potentials of OCR and antidromic cortical response (ACR) are depressed by the commissural stimulation, but the deep spike of OCR and the grouped spikes of ACR do not interact with the commissural stimulation.From the above findings, it seems likely that the deep spike is generated in the radially oriented secondary element, namely the apical dendrites of the mitral cell.

Olfactory bulb response to antidromic olfactory tract stimulation in the rabbit.

Antidromic olfactory tract stimulation in the rabbit evoked a response which was composed of an initial positive-negative spike, a second negative component and a slow negative wave in shallow layers. In deeper layers the two early components were followed by an initial negative spike and a slow positive wave, while the slow negative wave increased in size.The two early components had almost the same refractoriness; absolute refractory period was 4-6msec. and relative refractory period was in the range of 30-60msec. The slow positive wave had about 50msec. refractoriness, while the slow negative wave had no definite refractoriness, but it was of a summating nature.The deep initial spike was an antidromic volley in mitral cell axons, and it became through the mitral cell bodies the shallower initial positive-negative spike, which was propagated at the conduction rate of about 0.5m/sec. towards the apical dendrites.From the effects of asphyxia, topical curarization, and the behavior to repetitive stimulation of the anterior commissure and of the olfactory tract itself, the second negative component was revealed as activities of abundant basal dendrites activated synaptically through the recurrent axon collaterals of the mitral cells. During observation on unit discharges, double discharges were sometimes found. The second spike of the double discharge behaved like the second negative component in some respects and appeared to be elicited synaptically through the recurrent axon collateral of a mitral cell. This finding gives a favorable support for the above interpretation of the second negative component. It was found, moreover, that the deep slow positive wave was the condensed ‘source’ of the second negative component of the whole bulb cortex.Finally, the slow negative wave is considered as the activity of a large number of internal granular cells because of its similarity to the anterior commissure response. In addition, the depth relationship and interaction among the bulb surface response, the olfactory tract response and the anterior commissure response were examined.