Distribution Of Synaptic Weights Research Articles

The spinal cord facilitates cerebellar upper limb motor learning and control; inputs from neuromusculoskeletal simulation.

Complex interactions between brain regions and the spinal cord (SC) govern body motion, which is ultimately driven by muscle activation. Motor planning or learning are mainly conducted at higher brain regions, whilst the SC acts as a brain-muscle gateway and as a motor control centre providing fast reflexes and muscle activity regulation. Thus, higher brain areas need to cope with the SC as an inherent and evolutionary older part of the body dynamics. Here, we address the question of how SC dynamics affects motor learning within the cerebellum; in particular, does the SC facilitate cerebellar motor learning or constitute a biological constraint? We provide an exploratory framework by integrating biologically plausible cerebellar and SC computational models in a musculoskeletal upper limb control loop. The cerebellar model, equipped with the main form of cerebellar plasticity, provides motor adaptation; whilst the SC model implements stretch reflex and reciprocal inhibition between antagonist muscles. The resulting spino-cerebellar model is tested performing a set of upper limb motor tasks, including external perturbation studies. A cerebellar model, lacking the implemented SC model and directly controlling the simulated muscles, was also tested in the same. The performances of the spino-cerebellar and cerebellar models were then compared, thus allowing directly addressing the SC influence on cerebellar motor adaptation and learning, and on handling external motor perturbations. Performance was assessed in both joint and muscle space, and compared with kinematic and EMG recordings from healthy participants. The differences in cerebellar synaptic adaptation between both models were also studied. We conclude that the SC facilitates cerebellar motor learning; when the SC circuits are in the loop, faster convergence in motor learning is achieved with simpler cerebellar synaptic weight distributions. The SC is also found to improve robustness against external perturbations, by better reproducing and modulating muscle cocontraction patterns.

Self-Lateral Propagation Elevates Synaptic Modifications in Spiking Neural Networks for the Efficient Spatial and Temporal Classification.

The brain's mystery for efficient and intelligent computation hides in the neuronal encoding, functional circuits, and plasticity principles in natural neural networks. However, many plasticity principles have not been fully incorporated into artificial or spiking neural networks (SNNs). Here, we report that incorporating a novel feature of synaptic plasticity found in natural networks, whereby synaptic modifications self-propagate to nearby synapses, named self-lateral propagation (SLP), could further improve the accuracy of SNNs in three benchmark spatial and temporal classification tasks. The SLP contains lateral pre ( SLPpre ) and lateral post ( SLPpost ) synaptic propagation, describing the spread of synaptic modifications among output synapses made by axon collaterals or among converging synapses on the postsynaptic neuron, respectively. The SLP is biologically plausible and can lead to a coordinated synaptic modification within layers that endow higher efficiency without losing much accuracy. Furthermore, the experimental results showed the impressive role of SLP in sharpening the normal distribution of synaptic weights and broadening the more uniform distribution of misclassified samples, which are both considered essential for understanding the learning convergence and network generalization of neural networks.

Formation of Cluster-Structured Metallic Filaments in Organic Memristors for Wearable Neuromorphic Systems with Bio-Mimetic Synaptic Weight Distributions.

With increasing demand for wearable electronics capable of computing huge data, flexible neuromorphic systems mimicking brain functions have been receiving much attention. Despite considerable efforts in developing practical neural networks utilizing several types of flexible artificial synapses, it is still challenging to develop wearable systems for complex computations due to the difficulties in emulating continuous memory states in a synaptic component. In this study, polymer conductivity is analyzed as a crucial factor in determining the growth dynamics of metallic filaments in organic memristors. Moreover, flexible memristors with bio-mimetic synaptic functions such as linearly tunable weights are demonstrated by engineering the polymer conductivity. In the organic memristor, the cluster-structured filaments are grown within the polymer medium in response to electric stimuli, resulting in gradual resistive switching and stable synaptic plasticity. Additionally, the device exhibits the continuous and numerous non-volatile memory states due to its low leakage current. Furthermore, complex hardware neural networks including ternary logic operators and a noisy image recognitions system are successfully implemented utilizing the developed memristor arrays. This promising concept of creating flexible neural networks with bio-mimetic weight distributions will contribute to the development of a new computing architecture for energy-efficient wearable smart electronics.

Microcircuit Synchronization and Heavy-Tailed Synaptic Weight Distribution Augment preBötzinger Complex Bursting Dynamics.

The preBötzinger Complex (preBötC) encodes inspiratory time as rhythmic bursts of activity underlying each breath. Spike synchronization throughout a sparsely connected preBötC microcircuit initiates bursts that ultimately drive the inspiratory motor patterns. Using minimal microcircuit models to explore burst initiation dynamics, we examined the variability in probability and latency to burst following exogenous stimulation of a small subset of neurons, mimicking experiments. Among various physiologically plausible graphs of 1000 excitatory neurons constructed using experimentally determined synaptic and connectivity parameters, directed Erdős-Rényi graphs with a broad (lognormal) distribution of synaptic weights best captured the experimentally observed dynamics. preBötC synchronization leading to bursts was regulated by the efferent connectivity of spiking neurons that are optimally tuned to amplify modest preinspiratory activity through input convergence. Using graph-theoretic and machine learning-based analyses, we found that input convergence of efferent connectivity at the next-nearest neighbor order was a strong predictor of incipient synchronization. Our analyses revealed a crucial role of synaptic heterogeneity in imparting exceptionally robust yet flexible preBötC attractor dynamics. Given the pervasiveness of lognormally distributed synaptic strengths throughout the nervous system, we postulate that these mechanisms represent a ubiquitous template for temporal processing and decision-making computational motifs.SIGNIFICANCE STATEMENT Mammalian breathing is robust, virtually continuous throughout life, yet is inherently labile: to adapt to rapid metabolic shifts (e.g., fleeing a predator or chasing prey); for airway reflexes; and to enable nonventilatory behaviors (e.g., vocalization, breathholding, laughing). Canonical theoretical frameworks-based on pacemakers and intrinsic bursting-cannot account for the observed robustness and flexibility of the preBötzinger Complex rhythm. Experiments reveal that network synchronization is the key to initiate inspiratory bursts in each breathing cycle. We investigated preBötC synchronization dynamics using network models constructed with experimentally determined neuronal and synaptic parameters. We discovered that a fat-tailed (non-Gaussian) synaptic weight distribution-a manifestation of synaptic heterogeneity-augments neuronal synchronization and attractor dynamics in this vital rhythmogenic network, contributing to its extraordinary reliability and responsiveness.

Local accumulation times in a diffusion-trapping model of receptor dynamics at proximal axodendritic synapses.

The lateral diffusion and trapping of neurotransmitter receptors within the postsynaptic membrane of a neuron play a key role in determining synaptic strength and plasticity. Trapping is mediated by the reversible binding of receptors to scaffolding proteins (slots) within a synapse. In this paper we introduce a method for analyzing the transient dynamics of proximal axodendritic synapses in a diffusion-trapping model of receptor trafficking. Given a population of spatially distributed synapses, each of which has a fixed number of slots, we calculate the rate of relaxation to the steady-state distribution of bound slots (synaptic weights) in terms of a set of local accumulation times. Assuming that the rates of exocytosis and endocytosis are sufficiently slow, we show that the steady-state synaptic weights are independent of each other (purely local). On the other hand, the local accumulation time of a given synapse depends on the number of slots and the spatial location of all the synapses, indicating a form of transient heterosynaptic plasticity. This suggests that local accumulation time measurements could provide useful information regarding the distribution of synaptic weights within a dendrite.

Heterogeneous Responses to Changes in Inhibitory Synaptic Strength in Networks of Spiking Neurons

How does the dynamics of neurons in a network respond to changes in synaptic weights? Answer to this question would be important for a full understanding of synaptic plasticity. In this article, we report our numerical study of the effects of changes in inhibitory synaptic weights on the spontaneous activity of networks of spiking neurons with conductance-based synapses. Networks with biologically realistic features, which were reconstructed from multi-electrode array recordings taken in a cortical neuronal culture, and their modifications were used in the simulations. The magnitudes of the synaptic weights of all the inhibitory connections are decreased by a uniform amount subjecting to the condition that inhibitory connections would not be turned into excitatory ones. Our simulation results reveal that the responses of the neurons are heterogeneous: while the firing rate of some neurons increases as expected, the firing rate of other neurons decreases or remains unchanged. The same results show that heterogeneous responses also occur for an enhancement of inhibition. This heterogeneity in the responses of neurons to changes in inhibitory synaptic strength suggests that activity-induced modification of synaptic strength does not necessarily generate a positive feedback loop on the dynamics of neurons connected in a network. Our results could be used to understand the effects of bicuculline on spiking and bursting activities of neuronal cultures. Using reconstructed networks with biologically realistic features enables us to identify a long-tailed distribution of average synaptic weights for outgoing links as a crucial feature in giving rise to bursting in neuronal networks and in determining the overall response of the whole network to changes in synaptic strength. For networks whose average synaptic weights for outgoing links have a long-tailed distribution, bursting is observed and the average firing rate of the whole network increases upon inhibition suppression or decreases upon inhibition enhancement. For networks whose average synaptic weights for outgoing links are approximately normally distributed, bursting is not found and the average firing rate of the whole network remains approximately constant upon changes in inhibitory synaptic strength.

Effect of steady-state response versus excitatory/inhibitory balance on spiking synchronization in neural networks with log-normal synaptic weight distribution

Synchronization of neural activity, especially at the gamma band, contributes to perceptual functions. In several psychiatric disorders, deficits of perceptual functions are reflected in synchronization abnormalities. Plausible cause of this impairment is an alteration in the balance between excitation and inhibition (E/I balance); a disruption in the E/I balance leads to abnormal neural interactions reminiscent of pathological states. Moreover, the local lateral excitatory-excitatory synaptic connections in the cortex exhibit excitatory postsynaptic potentials (EPSPs) that follow a log-normal amplitude distribution. This long-tailed distribution is considered an important factor for the emergence of spatiotemporal neural activity. In this context, we hypothesized that manipulating the EPSP distribution under abnormal E/I balance conditions would provide insights into psychiatric disorders characterized by deficits in perceptual functions, potentially revealing the mechanisms underlying pathological neural behaviors. In this study, we evaluated the synchronization of neural activity with external periodic stimuli in spiking neural networks in cases of both E/I balance and imbalance with or without a long-tailed EPSP amplitude distribution. The results showed that external stimuli of a high frequency lead to a decrease in the degree of synchronization with an increasing ratio of excitatory to inhibitory neurons in the presence, but not in the absence, of high-amplitude EPSPs. This monotonic reduction can be interpreted as an autonomous, strong-EPSP-dependent spiking activity selectively interfering with the responses to external stimuli. This observation is consistent with pathological findings. Thus, our modeling approach has potential to improve the understanding of the steady-state response in both healthy and pathological states.

Distinct invivo dynamics of excitatory synapses onto cortical pyramidal neurons and parvalbumin-positive interneurons.

SUMMARYCortical function relies on the balanced activation of excitatory and inhibitory neurons. However, little is known about the organization and dynamics of shaft excitatory synapses onto cortical inhibitory interneurons. Here, we use the excitatory postsynaptic marker PSD-95, fluorescently labeled at endogenous levels, as a proxy for excitatory synapses onto layer 2/3 pyramidal neurons and parvalbumin-positive (PV+) interneurons in the barrel cortex of adult mice. Longitudinal in vivo imaging under baseline conditions reveals that, although synaptic weights in both neuronal types are log-normally distributed, synapses onto PV+ neurons are less heterogeneous and more stable. Markov model analyses suggest that the synaptic weight distribution is set intrinsically by ongoing cell-type-specific dynamics, and substantial changes are due to accumulated gradual changes. Synaptic weight dynamics are multiplicative, i.e., changes scale with weights, although PV+ synapses also exhibit an additive component. These results reveal that cell-type-specific processes govern cortical synaptic strengths and dynamics.

Astrocyte GluN2C NMDA receptors control basal synaptic strengths of hippocampal CA1 pyramidal neurons in the stratum radiatum.

Experience-dependent plasticity is a key feature of brain synapses for which neuronal N-Methyl-D-Aspartate receptors (NMDARs) play a major role, from developmental circuit refinement to learning and memory. Astrocytes also express NMDARs, although their exact function has remained controversial. Here, we identify in mouse hippocampus, a circuit function for GluN2C NMDAR, a subtype highly expressed in astrocytes, in layer-specific tuning of synaptic strengths in CA1 pyramidal neurons. Interfering with astrocyte NMDAR or GluN2C NMDAR activity reduces the range of presynaptic strength distribution specifically in the stratum radiatum inputs without an appreciable change in the mean presynaptic strength. Mathematical modeling shows that narrowing of the width of presynaptic release probability distribution compromises the expression of long-term synaptic plasticity. Our findings suggest a novel feedback signaling system that uses astrocyte GluN2C NMDARs to adjust basal synaptic weight distribution of Schaffer collateral inputs, which in turn impacts computations performed by the CA1 pyramidal neuron.

Forward and backward locomotion patterns in C. elegans generated by a connectome-based model simulation

Caenorhabditis elegans (C. elegans) can produce various motion patterns despite having only 69 motor neurons and 95 muscle cells. Previous studies successfully elucidate the connectome and role of the respective motor neuron classes related to movement. However, these models have not analyzed the distribution of the synaptic and gap connection weights. In this study, we examined whether a motor neuron and muscle network can generate oscillations for both forward and backward movement and analyzed the distribution of the trained synaptic and gap connection weights through a machine learning approach. This paper presents a connectome-based neural network model consisting of motor neurons of classes A, B, D, AS, and muscle, considering both synaptic and gap connections. A supervised learning method called backpropagation through time was adapted to train the connection parameters by feeding teacher data composed of the command neuron input and muscle cell activation. Simulation results confirmed that the motor neuron circuit could generate oscillations with different phase patterns corresponding to forward and backward movement, and could be switched at arbitrary times according to the binary inputs simulating the output of command neurons. Subsequently, we confirmed that the trained synaptic and gap connection weights followed a Boltzmann-type distribution. It should be noted that the proposed model can be trained to reproduce the activity patterns measured for an animal (HRB4 strain). Therefore, the supervised learning approach adopted in this study may allow further analysis of complex activity patterns associated with movements.

A Non-spiking Neuron Model With Dynamic Leak to Avoid Instability in Recurrent Networks.

Recurrent circuitry components are distributed widely within the brain, including both excitatory and inhibitory synaptic connections. Recurrent neuronal networks have potential stability problems, perhaps a predisposition to epilepsy. More generally, instability risks making internal representations of information unreliable. To assess the inherent stability properties of such recurrent networks, we tested a linear summation, non-spiking neuron model with and without a “dynamic leak”, corresponding to the low-pass filtering of synaptic input current by the RC circuit of the biological membrane. We first show that the output of this neuron model, in either of its two forms, follows its input at a higher fidelity than a wide range of spiking neuron models across a range of input frequencies. Then we constructed fully connected recurrent networks with equal numbers of excitatory and inhibitory neurons and randomly distributed weights across all synapses. When the networks were driven by pseudorandom sensory inputs with varying frequency, the recurrent network activity tended to induce high frequency self-amplifying components, sometimes evident as distinct transients, which were not present in the input data. The addition of a dynamic leak based on known membrane properties consistently removed such spurious high frequency noise across all networks. Furthermore, we found that the neuron model with dynamic leak imparts a network stability that seamlessly scales with the size of the network, conduction delays, the input density of the sensory signal and a wide range of synaptic weight distributions. Our findings suggest that neuronal dynamic leak serves the beneficial function of protecting recurrent neuronal circuitry from the self-induction of spurious high frequency signals, thereby permitting the brain to utilize this architectural circuitry component regardless of network size or recurrency.

Tuning the analog synaptic properties of forming free SiO2 memristors by material engineering

The development of alternative brain-inspired neuromorphic computing architectures is anticipated to play a key role in addressing the strict requirements of the artificial intelligence era. In order to obtain a high degree of learning accuracy within an artificial neural network (ANN) that operates with the backpropagation algorithm, a highly symmetric synaptic weight distribution is desired. Along these lines, we present here a detailed device engineering approach that enables analog synaptic properties in completely forming free SiO2-conductive bridge memories. This is achieved by either incorporating a dense layer of Pt nanoparticles as a bottom electrode or fabricating bilayer structures using a second switching layer of VOx. Interestingly, compared with the reference sample that manifests both threshold and bipolar switching modes, the Pt NC sample exhibits only the threshold switching pattern, whereas the bilayer configuration operates only under the bipolar switching mode, as illustrated by direct current measurements. These characteristics have a direct, while different impact, on the conductance modulation pattern and determine the analog nature of the synaptic weight distribution. Valuable insights regarding the origin of these effects and, in particular, of the symmetric and linear conductance modulation processes are gained through the implementation of a self-consistent numerical model that takes into account both the impact of the electrodes' thermal conductivity on the switching pattern and the different diffusion barriers for silver ion migration. Our approach provides useful guidelines toward the realization of high yield ANNs with biological-like dynamic behavior by controlling the conducting filament growth mechanism.

Comparison of Pattern Discrimination Mechanisms of Hebbian and Spatiotemporal Learning Rules in Self-Organization.

The spatiotemporal learning rule (STLR) proposed based on hippocampal neurophysiological experiments is essentially different from the Hebbian learning rule (HEBLR) in terms of the self-organization mechanism. The difference is the self-organization of information from the external world by firing (HEBLR) or not firing (STLR) output neurons. Here, we describe the differences of the self-organization mechanism between the two learning rules by simulating neural network models trained on relatively similar spatiotemporal context information. Comparing the weight distributions after training, the HEBLR shows a unimodal distribution near the training vector, whereas the STLR shows a multimodal distribution. We analyzed the shape of the weight distribution in response to temporal changes in contextual information and found that the HEBLR does not change the shape of the weight distribution for time-varying spatiotemporal contextual information, whereas the STLR is sensitive to slight differences in spatiotemporal contexts and produces a multimodal distribution. These results suggest a critical difference in the dynamic change of synaptic weight distributions between the HEBLR and STLR in contextual learning. They also capture the characteristics of the pattern completion in the HEBLR and the pattern discrimination in the STLR, which adequately explain the self-organization mechanism of contextual information learning.

Distinct <i>in vivo</i> Dynamics of Excitatory Synapses Onto Cortical Pyramidal Neurons and Inhibitory Interneurons

Cortical function relies on the balanced activation of excitatory and inhibitory neurons. However, little is known about the organization and dynamics of shaft excitatory synapses onto cortical inhibitory interneurons, which cannot be easily identified morphologically. Here, we fluorescently visualize the excitatory postsynaptic marker PSD-95 at endogenous levels as a proxy for excitatory synapses onto layer 2/3 pyramidal neurons and parvalbumin-positive (PV+) inhibitory interneurons in the mouse barrel cortex. Longitudinal in vivo imaging reveals that, while synaptic weights in both neuronal types are log-normally distributed, synapses onto PV+ neurons are less heterogeneous and more stable. Markov-model analyses suggest that the synaptic weight distribution is set intrinsically by ongoing cell type-specific dynamics, and substantial changes are due to accumulated gradual changes. Synaptic weight dynamics are multiplicative, i.e., changes scale with weights, though PV+ synapses also exhibit an additive component. These results reveal that cell type-specific processes govern cortical synaptic strengths and dynamics.

Variational mean-field theory for training restricted Boltzmann machines with binary synapses.

Unsupervised learning requiring only raw data is not only a fundamental function of the cerebral cortex, but also a foundation for a next generation of artificial neural networks. However, a unified theoretical framework to treat sensory inputs, synapses, and neural activity together is still lacking. The computational obstacle originates from the discrete nature of synapses, and complex interactions among these three essential elements of learning. Here, we propose a variational mean-field theory in which the distribution of synaptic weights is considered. The unsupervised learning can then be decomposed into two intertwined steps: A maximization step is carried out as a gradient ascent of the lower bound on the data log-likelihood, in which the synaptic weight distribution is determined by updating variational parameters, and an expectation step is carried out as a message passing procedure on an equivalent or dual neural network whose parameter is specified by the variational parameters of the weight distribution. Therefore, our framework provides insights on how data (or sensory inputs), synapses, and neural activities interact with each other to achieve the goal of extracting statistical regularities in sensory inputs. This variational framework is verified in restricted Boltzmann machines with planted synaptic weights and handwritten-digits learning.

Edge of Chaos and Avalanches in Neural Networks with Heavy-Tailed Synaptic Weight Distribution.

We propose an analytically tractable neural connectivity model with power-law distributed synaptic strengths. When threshold neurons with biologically plausible number of incoming connections are considered, our model features a continuous transition to chaos and can reproduce biologically relevant low activity levels and scale-free avalanches, i.e., bursts of activity with power-law distributions of sizes and lifetimes. In contrast, the Gaussian counterpart exhibits a discontinuous transition to chaos and thus cannot be poised near the edge of chaos. We validate our predictions in simulations of networks of binary as well as leaky integrate-and-fire neurons. Our results suggest that heavy-tailed synaptic distribution may form a weakly informative sparse-connectivity prior that can be useful in biological and artificial adaptive systems.

Deterministic characteristics of spontaneous activity detected by multi-fractal analysis in a spiking neural network with long-tailed distributions of synaptic weights.

Cortical neural networks maintain autonomous electrical activity called spontaneous activity that represents the brain's dynamic internal state even in the absence of sensory stimuli. The spatio-temporal complexity of spontaneous activity is strongly related to perceptual, learning, and cognitive brain functions; multi-fractal analysis can be utilized to evaluate the complexity of spontaneous activity. Recent studies have shown that the deterministic dynamic behavior of spontaneous activity especially reflects the topological neural network characteristics and changes of neural network structures. However, it remains unclear whether multi-fractal analysis, recently widely utilized for neural activity, is effective for detecting the complexity of the deterministic dynamic process. To verify this point, we focused on the log-normal distribution of excitatory postsynaptic potentials (EPSPs) to evaluate the multi-fractality of spontaneous activity in a spiking neural network with a log-normal distribution of EPSPs. We found that the spiking activities exhibited multi-fractal characteristics. Moreover, to investigate the presence of a deterministic process in the spiking activity, we conducted a surrogate data analysis against the time-series of spiking activity. The results showed that the spontaneous spiking activity included the deterministic dynamic behavior. Overall, the combination of multi-fractal analysis and surrogate data analysis can detect deterministic complex neural activity. The multi-fractal analysis of neural activity used in this study could be widely utilized for brain modeling and evaluation methods for signals obtained by neuroimaging modalities.

Heterogeneous Synaptic Weighting Improves Neural Coding in the Presence of Common Noise.

Simultaneous recordings from the cortex have revealed that neural activity is highly variable and that some variability is shared across neurons in a population. Further experimental work has demonstrated that the shared component of a neuronal population's variability is typically comparable to or larger than its private component. Meanwhile, an abundance of theoretical work has assessed the impact that shared variability has on a population code. For example, shared input noise is understood to have a detrimental impact on a neural population's coding fidelity. However, other contributions to variability, such as common noise, can also play a role in shaping correlated variability. We present a network of linear-nonlinear neurons in which we introduce a common noise input to model-for instance, variability resulting from upstream action potentials that are irrelevant to the task at hand. We show that by applying a heterogeneous set of synaptic weights to the neural inputs carrying the common noise, the network can improve its coding ability as measured by both Fisher information and Shannon mutual information, even in cases where this results in amplification of the common noise. With a broad and heterogeneous distribution of synaptic weights, a population of neurons can remove the harmful effects imposed by afferents that are uninformative about a stimulus. We demonstrate that some nonlinear networks benefit from weight diversification up to a certain population size, above which the drawbacks from amplified noise dominate over the benefits of diversification. We further characterize these benefits in terms of the relative strength of shared and private variability sources. Finally, we studied the asymptotic behavior of the mutual information and Fisher information analytically in our various networks as a function of population size. We find some surprising qualitative changes in the asymptotic behavior as we make seemingly minor changes in the synaptic weight distributions.

Initial synaptic weight distribution for fast learning speed and high recognition rate in STDP-based spiking neural network

We analyze that the initial synaptic weight distribution affects the performance, such as the learning speed, recognition rate and the power consumption in the spiking neural networks (SNNs) based on spike-timing-dependent plasticity (STDP) learning rule. A thin-film transistor (TFT)-type NOR flash memory is used as a synaptic device. In this fully connected two-layer neuromorphic system using the proposed pulse scheme, the results with and without the homeostasis functionality were analyzed separately. In addition, power consumption of the network in various initial synaptic weight distributions, and recognition rate that varies with the number of output neurons are also investigated. In pattern recognition for 28 × 28 MNIST handwritten patterns, higher performance is achieved in various aspects when the initial synaptic weights are distributed near the maximum value.

Temporal-specific complexity of spiking patterns in spontaneous activity induced by a dual complex network structure

Temporal fluctuation of neural activity in the brain has an important function in optimal information processing. Spontaneous activity is a source of such fluctuation. The distribution of excitatory postsynaptic potentials (EPSPs) between cortical pyramidal neurons can follow a log-normal distribution. Recent studies have shown that networks connected by weak synapses exhibit characteristics of a random network, whereas networks connected by strong synapses have small-world characteristics of small path lengths and large cluster coefficients. To investigate the relationship between temporal complexity spontaneous activity and structural network duality in synaptic connections, we executed a simulation study using the leaky integrate-and-fire spiking neural network with log-normal synaptic weight distribution for the EPSPs and duality of synaptic connectivity, depending on synaptic weight. We conducted multiscale entropy analysis of the temporal spiking activity. Our simulation demonstrated that, when strong synaptic connections approach a small-world network, specific spiking patterns arise during irregular spatio-temporal spiking activity, and the complexity at the large temporal scale (i.e., slow frequency) is enhanced. Moreover, we confirmed through a surrogate data analysis that slow temporal dynamics reflect a deterministic process in the spiking neural networks. This modelling approach may improve the understanding of the spatio-temporal complex neural activity in the brain.