Neural Computation Research Articles

Estimating crash risk and injury severity considering multiple traffic conflict and crash types: A bivariate extreme value approach

Traffic conflicts are generally considered independent events in existing extreme value theory models to estimate the risk of total or single types of crashes. However, traffic events at a road entity are not necessarily independent interactions and can lead to multiple traffic conflicts with shared common unobserved factors. A comprehensive estimation of crash risks in a road entity needs to consider the correlation of potential traffic conflicts to avoid possible bias in prediction performance and the problem of undetected deficiencies. This study proposes a Bayesian non-stationary bivariate generalised extreme value modelling framework to estimate the severe and non-severe crash risks accounting for the correlation between right-turn and rear-end conflicts at signalised intersections. A deep neural network-based computer vision technique was applied to extract the traffic conflicts from 77 h of video recordings over two right-turn approaches at two signalised intersections in Cairns, Australia. Post encroachment time and modified time to collision were used to characterise right-turn and rear-end conflicts, respectively, while an expected post-collision velocity difference was combined with post encroachment time and modified time to collision for crash risk estimation by injury severity levels. Several covariates were used to address the time-varying heterogeneity of traffic conflict extremes and to estimate the differential crash risks at signal cycles. Results showed a significant correlation between right-turn and rear-end crashes at signal cycle levels, indicating the importance of accounting for the dependency among traffic conflict types. Overall, the bivariate models considering the correlation among traffic conflict types were found to understandably perform better than their univariate counterparts. This study provides a demonstration of a correlated crash risk modelling framework that addresses issues related to the suitable traffic conflict measures, time varying risks (non-stationarity), heterogeneity, and injury severity levels of different crash types.

A Brain-Inspired Way of Reducing the Network Complexity via Concept-Regularized Coding for Emotion Recognition

The human brain can effortlessly and reliably perceive emotions, whereas existing facial emotion recognition (FER) methods suffer from drawbacks such as complex model structures, high storage requirements, and poor interpretability. Inspired by the role of emotion concepts in visual perception coding within the human brain, we propose a dual-pathway framework emulating the neural computation of emotion recognition. Specifically, these two pathways are designed to model the representation of emotion concepts in the brain and the visual perception process, respectively. For the former, we adopt a disentangled approach to extract emotion concepts from complex facial geometric attributes; for the latter, we employ an emotional confidence evaluation strategy to determine which concept is optimal for regularizing the perceptual coding. The proposed concept-regularized coding strategy endows the framework with flexibility and interpretability as well as good performances on several benchmarking FER datasets.

Convolutional Neural Network-Based Deep Learning Engine for Mastoidectomy Instrument Recognition and Movement Tracking.

To develop a convolutional neural network-based computer vision model to recognize and track 2 mastoidectomy surgical instruments-the drill and the suction-irrigator-from intraoperative video recordings of mastoidectomies. Technological development and model validation. Academic center. Ten 1-minute videos of mastoidectomies done for cochlear implantation by varying levels of resident surgeons were collected. For each video, containing 900 frames, an open-access computer vision annotation tool was used to annotate the drill and suction-irrigator class images with bounding boxes. A mastoidectomy instrument tracking module, which extracts the center coordinates of bounding boxes, was developed using a feature pyramid network and layered with DETECTRON, an open-access faster-region-based convolutional neural network. Eight videos were used to train the model, and 2 videos were used for testing. Outcome measures included Intersection over Union (IoU) ratio, accuracy, and average precision. For an IoU of 0.5, the mean average precision for the drill was 99% and 86% for the suction-irrigator. The model proved capable of generating maps of drill and suction-irrigator stroke direction and distance for the entirety of each video. This computer vision model can identify and track the drill and suction-irrigator from videos of intraoperative mastoidectomies performed by residents with excellent precision. It can now be employed to retrospectively study objective mastoidectomy measures of expert and resident surgeons, such as drill and suction-irrigator stroke concentration, economy of motion, speed, and coordination, setting the stage for characterization of objective expectations for safe and efficient mastoidectomies.

Combining graph neural networks and computer vision methods for cell nuclei classification in lung tissue

The detection of tumoural cells from whole slide images is an essential task in medical diagnosis and research. In this article, we propose and analyse a novel approach that combines computer vision-based models with graph neural networks to improve the accuracy of automated tumoural cell detection in lung tissue. Our proposal leverages the inherent structure and relationships between cells in the tissue. Experimental results on our own curated dataset show that modelling the problem with graphs gives the model a clear advantage over just working at pixel level. This change in perspective provides extra information that makes it possible to improve the performance. The reduction of dimensionality that comes from working with the graph also allows us to increase the field of view with low computational requirements. Code is available at https://github.com/Jerry-Master/lung-tumour-study, models are uploaded to https://huggingface.co/Jerry-Master/Hovernet-plus-Graphs, and the dataset is published on Zenodo https://zenodo.org/doi/10.5281/zenodo.8368122.

Identification of Early Hippocampal Dynamics during Recognition Memory with Independent Component Analysis.

The hippocampus is generally considered to have relatively late involvement in recognition memory, its main electrophysiological signature being between 400 and 800 ms after stimulus onset. However, most electrophysiological studies have analyzed the hippocampus as a single responsive area, selecting only a single-site signal exhibiting the strongest effect in terms of amplitude. These classical approaches may not capture all the dynamics of this structure, hindering the contribution of other hippocampal sources that are not located in the vicinity of the selected site. We combined intracerebral electroencephalogram recordings from epileptic patients with independent component analysis during a recognition memory task involving the recognition of old and new images. We identified two sources with different responses emerging from the hippocampus: a fast one (maximal amplitude at ∼250 ms) that could not be directly identified from raw recordings and a latter one, peaking at ∼400 ms. The former component presented different amplitudes between old and new items in 6 out of 10 patients. The latter component had different delays for each condition, with a faster activation (∼290 ms after stimulus onset) for recognized items. We hypothesize that both sources represent two steps of hippocampal recognition memory, the faster reflecting the input from other structures and the latter the hippocampal internal processing. Recognized images evoking early activations would facilitate neural computation in the hippocampus, accelerating memory retrieval of complementary information. Overall, our results suggest that the hippocampal activity is composed of several sources with an early activation related to recognition memory.

Natural language instructions induce compositional generalization in networks of neurons

A fundamental human cognitive feat is to interpret linguistic instructions in order to perform novel tasks without explicit task experience. Yet, the neural computations that might be used to accomplish this remain poorly understood. We use advances in natural language processing to create a neural model of generalization based on linguistic instructions. Models are trained on a set of common psychophysical tasks, and receive instructions embedded by a pretrained language model. Our best models can perform a previously unseen task with an average performance of 83% correct based solely on linguistic instructions (that is, zero-shot learning). We found that language scaffolds sensorimotor representations such that activity for interrelated tasks shares a common geometry with the semantic representations of instructions, allowing language to cue the proper composition of practiced skills in unseen settings. We show how this model generates a linguistic description of a novel task it has identified using only motor feedback, which can subsequently guide a partner model to perform the task. Our models offer several experimentally testable predictions outlining how linguistic information must be represented to facilitate flexible and general cognition in the human brain.

Simultaneous, cortex-wide dynamics of up to 1 million neurons reveal unbounded scaling of dimensionality with neuron number

The brain's remarkable properties arise from the collective activity of millions of neurons. Widespread application of dimensionality reduction to multi-neuron recordings implies that neural dynamics can be approximated by low-dimensional "latent" signals reflecting neural computations. However, can such low-dimensional representations truly explain the vast range of brain activity, and if not, what is the appropriate resolution and scale of recording to capture them? Imaging neural activity at cellular resolution and near-simultaneously across the mouse cortex, we demonstrate an unbounded scaling of dimensionality with neuron number in populations up to 1 million neurons. Although half of the neural variance is contained within sixteen dimensions correlated with behavior, our discovered scaling of dimensionality corresponds to an ever-increasing number of neuronal ensembles without immediate behavioral or sensory correlates. The activity patterns underlying these higher dimensions are fine grained and cortex wide, highlighting that large-scale, cellular-resolution recording is required to uncover the full substrates of neuronal computations.

Research and Application of Visual Object Recognition System Based on Deep Learning and Neural Morphological Computation

The development of advanced optoelectronic vision sensors for high-level image recognition and data preprocessing is poised to accelerate the progress of machine vision and mobile electronic technology. Compared to traditional sensory computing methods, such as analog-to-digital signal conversion and digital logic computation tasks (i.e., Von Neumann computing), neural morphological vision computing can significantly improve energy efficiency and data processing speed by minimizing unnecessary raw data transmission between front-end photosensitive sensors and back-end processors. Neural morphological vision sensors are typically designed for tasks such as denoising, edge enhancement, spectral filtering, and visual information recognition. These methods can be categorized into approaches using near-sensor and sensor-internal computing processors based on whether preprocessing can be performed in situ. In near-sensor computing approaches, the image sensor for capturing visual information and the memory computing processor for preprocessing captured images are separate. A memory computing processor can simultaneously perform memory and computing tasks based on analog memory functions. Neural morphological vision sensors for in-sensor computing can be constructed using single-element image sensors, enabling both the reception of visual information and the execution of memory computing processes to be achieved in the same device. This represents an ideal scenario for future artificial intelligence machines and mobile electronic devices in visual computing systems.

Design of a Convolutional Neural Network Accelerator Based on On-Chip Data Reordering

Convolutional neural networks have been widely applied in the field of computer vision. In convolutional neural networks, convolution operations account for more than 90% of the total computational workload. The current mainstream approach to achieving high energy-efficient convolution operations is through dedicated hardware accelerators. Convolution operations involve a significant amount of weights and input feature data. Due to limited on-chip cache space in accelerators, there is a significant amount of off-chip DRAM memory access involved in the computation process. The latency of DRAM access is 20 times higher than that of SRAM, and the energy consumption of DRAM access is 100 times higher than that of multiply–accumulate (MAC) units. It is evident that the “memory wall” and “power wall” issues in neural network computation remain challenging. This paper presents the design of a hardware accelerator for convolutional neural networks. It employs a dataflow optimization strategy based on on-chip data reordering. This strategy improves on-chip data utilization and reduces the frequency of data exchanges between on-chip cache and off-chip DRAM. The experimental results indicate that compared to the accelerator without this strategy, it can reduce data exchange frequency by up to 82.9%.

Hybrid Stochastic Number and Its Neural Network Computation

Hybrid Stochastic Number and Its Neural Network Computation

Advanced Techniques for Geospatial Referencing in Online Media Repositories

In the digital transformation era, video media libraries’ untapped potential is immense, restricted primarily by their non-machine-readable nature and basic search functionalities limited to standard metadata. This study presents a novel multimodal methodology that utilizes advances in artificial intelligence, including neural networks, computer vision, and natural language processing, to extract and geocode geospatial references from videos. Leveraging the geospatial information from videos enables semantic searches, enhances search relevance, and allows for targeted advertising, particularly on mobile platforms. The methodology involves a comprehensive process, including data acquisition from ARD Mediathek, image and text analysis using advanced machine learning models, and audio and subtitle processing with state-of-the-art linguistic models. Despite challenges like model interpretability and the complexity of geospatial data extraction, this study’s findings indicate significant potential for advancing the precision of spatial data analysis within video content, promising to enrich media libraries with more navigable, contextually rich content. This advancement has implications for user engagement, targeted services, and broader urban planning and cultural heritage applications.

Decoding face recognition abilities in the human brain.

Why are some individuals better at recognizing faces? Uncovering the neural mechanisms supporting face recognition ability has proven elusive. To tackle this challenge, we used a multimodal data-driven approach combining neuroimaging, computational modeling, and behavioral tests. We recorded the high-density electroencephalographic brain activity of individuals with extraordinary face recognition abilities-super-recognizers-and typical recognizers in response to diverse visual stimuli. Using multivariate pattern analyses, we decoded face recognition abilities from 1 s of brain activity with up to 80% accuracy. To better understand the mechanisms subtending this decoding, we compared representations in the brains of our participants with those in artificial neural network models of vision and semantics, as well as with those involved in human judgments of shape and meaning similarity. Compared to typical recognizers, we found stronger associations between early brain representations of super-recognizers and midlevel representations of vision models as well as shape similarity judgments. Moreover, we found stronger associations between late brain representations of super-recognizers and representations of the artificial semantic model as well as meaning similarity judgments. Overall, these results indicate that important individual variations in brain processing, including neural computations extending beyond purely visual processes, support differences in face recognition abilities. They provide the first empirical evidence for an association between semantic computations and face recognition abilities. We believe that such multimodal data-driven approaches will likely play a critical role in further revealing the complex nature of idiosyncratic face recognition in the human brain.

Influence of reaction cell electrodes on organic electrochemical transistors

Organic electrochemical transistors (OECTs) hold great potential in various applications, including biosensing and neural network computation. Traditional “all-in-one” OECT device architecture faces the problems of unclear amplification mechanisms and complex side reactions, to name a few. The reaction cell OECT (RC-OECT) device architecture, proposed by Ting et al. [Adv. Funct. Mater. 31(19), 2010868 (2021)], effectively resolves these problems. Furthermore, in many applications of OECTs, such as high throughput (bio)sensing, an OECT device array instead of a single OECT is needed. Therefore, the size (area) of the OECT device, which represents the amount of occupied chip real estate and the integration of the device, matters. In this paper, we developed a hydrogen peroxide sensor based on the RC-OECT. We utilized an RC cathode modified by the poly(3,4-ethylenedioxythiophene) polystyrene sulfonate film and a Ag/AgCl OECT gate electrode for facilitating device integration, varied RC anode area, and achieved optimization of at least one of the two parameters, sensitivity and lower limit of detection (LLoD), in conjunction with the level of integration of the device. Multiple quantitative sensitivity metrics have been adopted in this work. We also evaluated the correlation between RC anode area and LLoD. In addition, a mechanistic analysis of the RC-OECT device structure, in terms of faradic and non-faradaic currents, was carried out to illustrate the interplay between sensing performance and the electrode area of the reaction cell. This mechanistic analysis provides insights for miniaturizing OECT devices with the RC-OECT architecture.

Limited restoration of contrast sensitivity with training after V1 damage in humans.

Stroke damage to the primary visual cortex (V1) causes severe visual deficits, which benefit from perceptual retraining. However, whereas training with high-contrast stimuli can locally restore orientation and motion direction discrimination abilities at trained locations, it only partially restores luminance contrast sensitivity (CS). Recent work revealed that high-contrast discrimination abilities may be preserved in the blind field of some patients early after stroke. Here, we asked if CS for orientation and direction discrimination is similarly preserved inside the blind field, to what extent, and whether it could benefit from a visual training intervention. Thirteen subacute patients (<3 months post-V1-stroke) and 12 chronic patients (>6 months post-V1-stroke) were pre-tested, then trained to discriminate either orientation or motion direction of Gabor patches of progressively lower contrasts as their performance improved. At baseline, more subacute than chronic participants could correctly discriminate the orientation of high-contrast Gabors in their blind field, but all failed to perform this task at lower contrasts, even when 10Hz flicker or motion direction were added. Training improved CS in a greater portion of subacute than chronic participants, but no-one attained normal CS, even when stimuli contained flicker or motion. We conclude that, unlike the near-complete training-induced restoration of high-contrast orientation and motion direction discrimination abilities, V1 damage in adulthood may severely limit the residual visual system's ability to regain normal CS. Our results support the notion that CS involves different neural substrates and computations than those required for orientation and direction discrimination in V1-damaged visual systems.Significance statement Stroke-induced V1 damage in adult humans induces a rapid and severe impairment of contrast sensitivity for orientation and motion direction discrimination in the affected hemifield, although discrimination of high-contrast stimuli can persist for several months. Adaptive training with Gabor patches of progressively lower contrasts improves contrast sensitivity for both orientation and motion discriminations in the blind-field of subacute (<3 months post-stroke) and chronic (>6 months post-stroke) participants; however, it fails to restore normal contrast sensitivity. Nonetheless, more subacute than chronic stroke participants benefit from such training, particularly when discriminating the orientation of static, non-flickering targets. Thus, contrast sensitivity appears critically dependent on processing within V1, with perceptual training displaying limited potential to fully restore it after V1 damage.

A Bio-Inspired Probabilistic Neural Network Model for Noise-Resistant Collision Perception.

Bio-inspired models based on the lobula giant movement detector (LGMD) in the locust's visual brain have received extensive attention and application for collision perception in various scenarios. These models offer advantages such as low power consumption and high computational efficiency in visual processing. However, current LGMD-based computational models, typically organized as four-layered neural networks, often encounter challenges related to noisy signals, particularly in complex dynamic environments. Biological studies have unveiled the intrinsic stochastic nature of synaptic transmission, which can aid neural computation in mitigating noise. In alignment with these biological findings, this paper introduces a probabilistic LGMD (Prob-LGMD) model that incorporates a probability into the synaptic connections between multiple layers, thereby capturing the uncertainty in signal transmission, interaction, and integration among neurons. Comparative testing of the proposed Prob-LGMD model and two conventional LGMD models was conducted using a range of visual stimuli, including indoor structured scenes and complex outdoor scenes, all subject to artificial noise. Additionally, the model's performance was compared to standard engineering noise-filtering methods. The results clearly demonstrate that the proposed model outperforms all comparative methods, exhibiting a significant improvement in noise tolerance. This study showcases a straightforward yet effective approach to enhance collision perception in noisy environments.

Quasi-optimal domain decomposition method for neural network-based computation of the time-dependent Schrödinger equation

In this paper, we derive and analyze the performance of optimal/quasi-optimal Schwarz Waveform Relaxation (SWR) domain decomposition methods (DDM) for the time-dependent Schrödinger equation when implement with neural network-based Partial Differential Equations (PDE) solvers. Optimal SWR methods, which are based on Dirichlet-to-Neumann operators, are known to have a higher convergence rate than classical or optimized SWR methods. However, they are usually considered prohibitive due to their computational costs with standard PDE solvers. Thanks to Physics Informed Neural Network acceleration within the Schwarz waveform relaxation process and an efficient computation of Dirichlet-to-Neumann transmission operators, we demonstrate that optimal and quasi-optimal SWR methods can be performed almost as efficiently as classical or optimized SWR methods while maintaining a faster convergence rate. We present a few numerical examples to illustrate the performance and convergence of the proposed method.

Errata to "A Tutorial on the Spectral Theory of Markov Chains" by Eddie Seabrook and Laurenz Wiskott (Neural Computation, November 2023, Vol. 35, No. 11, pp. 1713-1796, https://doi.org/10.1162/neco_a_01611).

Errata to "A Tutorial on the Spectral Theory of Markov Chains" by Eddie Seabrook and Laurenz Wiskott (Neural Computation, November 2023, Vol. 35, No. 11, pp. 1713-1796, https://doi.org/10.1162/neco_a_01611).

A unifying theory explains seemingly contradictory biases in perceptual estimation.

Perceptual biases are widely regarded as offering a window into the neural computations underlying perception. To understand these biases, previous work has proposed a number of conceptually different, and even seemingly contradictory, explanations, including attraction to a Bayesian prior, repulsion from the prior due to efficient coding and central tendency effects on a bounded range. We present a unifying Bayesian theory of biases in perceptual estimation derived from first principles. We demonstrate theoretically an additive decomposition of perceptual biases into attraction to a prior, repulsion away from regions with high encoding precision and regression away from the boundary. The results reveal a simple and universal rule for predicting the direction of perceptual biases. Our theory accounts for, and yields, new insights regarding biases in the perception of a variety of stimulus attributes, including orientation, color and magnitude. These results provide important constraints on the neural implementations of Bayesian computations.

High-Reliability and Self-Rectifying Alkali Ion Memristor through Bottom Electrode Design and Dopant Incorporation.

Ionic memristor devices are crucial for efficient artificial neural network computations in neuromorphic hardware. They excel in multi-bit implementation but face challenges like device reliability and sneak currents in crossbar array architecture (CAA). Interface-type ionic memristors offer low variation, self-rectification, and no forming process, making them suitable for CAA. However, they suffer from slow weight updates and poor retention and endurance. To address these issues, the study demonstrated an alkali ion self-rectifying memristor with an alkali metal reservoir formed by a bottom electrode design. By adopting Li metal as the adhesion layer of the bottom electrode, an alkali ion reservoir was formed at the bottom of the memristor layer by diffusion occurring during the atomic layer deposition process for the Na:TiO2 memristor layer. In addition, Al dopant was used to improve the retention characteristics by suppressing the diffusion of alkali cations. In the memristor device with optimized Al doping, retention characteristics of more than 20 h at 125 °C, endurance characteristics of more than 5.5 × 105, and high linearity/symmetry of weight update characteristics were achieved. In reliability tests on 100 randomly selected devices from a 32 × 32 CAA device, device-to-device and cycle-to-cycle variations showed low variation values within 81% and 8%, respectively.

Dynamic electrical synapses rewire brain networks for persistent oscillations and epileptogenesis

One of the very fundamental attributes for telencephalic neural computation in mammals involves network activities oscillating beyond the initial trigger. The continuing and automated processing of transient inputs shall constitute the basis of cognition and intelligence but may lead to neuropsychiatric disorders such as epileptic seizures if carried so far as to engross part of or the whole telencephalic system. From a conventional view of the basic design of the telencephalic local circuitry, the GABAergic interneurons (INs) and glutamatergic pyramidal neurons (PNs) make negative feedback loops which would regulate the neural activities back to the original state. The drive for the most intriguing self-perpetuating telencephalic activities, then, has not been posed and characterized. We found activity-dependent deployment and delineated functional consequences of the electrical synapses directly linking INs and PNs in the amygdala, a prototypical telencephalic circuitry. These electrical synapses endow INs dual (a faster excitatory and a slower inhibitory) actions on PNs, providing a network-intrinsic excitatory drive that fuels the IN-PN interconnected circuitries and enables persistent oscillations with preservation of GABAergic negative feedback. Moreover, the entities of electrical synapses between INs and PNs are engaged in and disengaged from functioning in a highly dynamic way according to neural activities, which then determine the spatiotemporal scale of recruited oscillating networks. This study uncovers a special wide-range and context-dependent plasticity for wiring/rewiring of brain networks. Epileptogenesis or a wide spectrum of clinical disorders may ensue, however, from different scales of pathological extension of this unique form of telencephalic plasticity.