Threshold-linear Units Research Articles

An Internet of Medical Things Based Liver Tumor Detection System using Semantic Segmentation

<p>Internet of Medical Things (IOMT) based systems provide a framework for remote health monitoring. Liver tumors tend to have parallel intensities with neighboring lesions and may have an abnormal apparent form that directly depends on the stage, state, type, and luminosity setup. In this research, a segmentation model based on improved UNet has been deployed to segment the tumors by incorporating the side-by-side convolution layers based on Filter Response normalization layers (FRN) along with Threshold Linear Units (TLU). This combination of FRN along with TLU has a very strong impact on the performance of the model as the FRN layer operates on each batch sample and each response filter during training, and thus it eliminates the problem of batch dependence. Furthermore, we have also switched from the traditional up-sampling layers to fractionally strided convolutions in UNet which performs up-sampling of the required image with proper learning. Moreover, the tumors are directly segmented by the proposed framework from the given CT scan without any extraction of ROIs. To evaluate the performance of our proposed method, we use a publicly available 3DIRCADb dataset. The proposed technique has shown excellent results with 93.0% accuracy and 71.2% Jaccard score.</p> <p> </p>

Efficiency of Local Learning Rules in Threshold-Linear Associative Networks.

We derive the Gardner storage capacity for associative networks of threshold linear units, and show that with Hebbian learning they can operate closer to such Gardner bound than binary networks, and even surpass it. This is largely achieved through a sparsification of the retrieved patterns, which we analyze for theoretical and empirical distributions of activity. As reaching the optimal capacity via nonlocal learning rules like back propagation requires slow and neurally implausible training procedures, our results indicate that one-shot self-organized Hebbian learning can be just as efficient.

Synthesis of parameterized families of correctly functioning sigma-pi neurons

The ΣΠ-neuron is a biologically inspired formal model for logical information processing. The ΣΠ-neuron model adequately reflects information processing processes in the cerebral cortex and in the dendritic trees of neurons. The advantage of the ΣΠ-neuron model is the ability to accurately represent any Boolean function and the possibility of constructive learning (direct construction) in a single pass of the training sample. Another possibility is the direct construction of an ensemble of ΣΠ-neurons that function correctly on the training sample. This article discusses a new algorithm for constructing such ensemble of ΣΠ-neurons in parameterized form. This form can also be easily represented as a single ΣΠ-neuron with a hidden layer of linear and threshold linear units. In some cases, this makes it easier to retrain on new inputs by setting the appropriate control parameter values.

Autoassociative memory retrieval and spontaneous activity bumps in small-world networks of integrate-and-fire neurons

The metric structure of synaptic connections is obviously an important factor in shaping the properties of neural networks, in particular the capacity to retrieve memories, with which are endowed autoassociative nets operating via attractor dynamics. Qualitatively, some real networks in the brain could be characterized as ‘small worlds’, in the sense that the structure of their connections is intermediate between the extremes of an orderly geometric arrangement and of a geometry-independent random mesh. Small worlds can be defined more precisely in terms of their mean path length and clustering coefficient; but is such a precise description useful for a better understanding of how the type of connectivity affects memory retrieval? We have simulated an autoassociative memory network of integrate-and-fire units, positioned on a ring, with the network connectivity varied parametrically between ordered and random. We find that the network retrieves previously stored memory patterns when the connectivity is close to random, and displays the characteristic behavior of ordered nets (localized ‘bumps’ of activity) when the connectivity is close to ordered. Recent analytical work shows that these two behaviors can coexist in a network of simple threshold-linear units, leading to localized retrieval states. We find that they tend to be mutually exclusive behaviors, however, with our integrate-and-fire units. Moreover, the transition between the two occurs for values of the connectivity parameter which are not simply related to the notion of small worlds.

An associative network with spatially organized connectivity

We investigate the properties of an autoassociative network of threshold-linear units whosesynaptic connectivity is spatially structured and asymmetric. Since the methods ofequilibrium statistical mechanics cannot be applied to such a network due to the lack of aHamiltonian, we approach the problem through a signal-to-noise analysis, that we adapt tospatially organized networks. The conditions are analysed for the appearance of stable,spatially non-uniform profiles of activity with large overlaps with one of the storedpatterns. It is also shown, with simulations and analytic results, that the storagecapacity does not decrease much when the connectivity of the network becomesshort range. In addition, the method used here enables us to calculate exactly thestorage capacity of a randomly connected network with arbitrary degree of dilution.

Disappearance of spurious states in analog associative memories.

We show that symmetric n-mixture states, when they exist, are almost never stable in autoassociative networks with threshold-linear units. Only with a binary coding scheme, we could find a limited region of the parameter space in which either 2-mixture or 3-mixture states are stable attractors of the dynamics.

Graded-response neurons and information encodings in autoassociative memories.

A general mean-field theory is presented for an attractor neural network in which each elementary unit is described by one input and one output real variable, and whose synaptic strengths are determined by a covariance imprinting rule. In the case of threshold-linear units, a single equation is shown to yield the storage capacity for the retrieval of random activity patterns drawn from any given probability distribution. If this distribution produces binary patterns, the storage capacity is essentially the same as for networks of binary units. To explore the effects of storing more structured patterns, the case of a ternary distribution is studied. It is shown that the number of patterns that can be stored can be much higher than in the binary case, whereas the total amount of retrievable information does not exceed the limit obtained with binary patterns.

Threshold-linear formal neurons in auto-associative nets

Most analytical results concerning the long-time behaviour of associative memory networks have been obtained by using binary elementary units. The use of alternative types of neuron-like processing elements is considered as a way of testing the generality of those results and of approaching biological realism. In particular, threshold-linear units are proposed as appropriate in models designed to reproduce low firing rates, in which long-time stability does not rely on single unit saturation. Such units are simple enough to allow detailed analytical understanding of the properties of the network. This is demonstrated by analysing the attractor states of a network operating at low rates. It is shown that while the interesting retrieval behaviour persists, the roles of the different parameters as well as the nature of the stable states change completely with respect to the binary implementation.