Nonlinear Separation Functions Research Articles

Solving classification tasks by a receptron based on nonlinear optical speckle fields

Among several approaches to tackle the problem of energy consumption in modern computing systems, two solutions are currently investigated: one consists of artificial neural networks (ANNs) based on photonic technologies, the other is a different paradigm compared to ANNs and it is based on random networks of non-linear nanoscale junctions resulting from the assembling of nanoparticles or nanowires as substrates for neuromorphic computing. These networks show the presence of emergent complexity and collective phenomena in analogy with biological neural networks characterized by self-organization, redundancy, and non-linearity. Starting from this background, we propose and formalize a generalization of the perceptron model to describe a classification device based on a network of interacting units where the input weights are non-linearly dependent. We show that this model, called “receptron”, provides substantial advantages compared to the perceptron as, for example, the solution of non-linearly separable Boolean functions with a single device. The receptron model is used as a starting point for the implementation of an all-optical device that exploits the non-linearity of optical speckle fields produced by a solid scatterer. By encoding these speckle fields we generated a large variety of target Boolean functions. We demonstrate that by properly setting the model parameters, different classes of functions with different multiplicity can be solved efficiently. The optical implementation of the receptron scheme opens the way for the fabrication of a completely new class of optical devices for neuromorphic data processing based on a very simple hardware.

Design for an Individual: Connectionist Approaches to the Evolutionary Transitions in Individuality

The truly surprising thing about evolution is not how it makes individuals better adapted to their environment, but how it makes individuals. All individuals are made of parts that used to be individuals themselves, e.g., multicellular organisms from unicellular organisms. In such evolutionary transitions in individuality, the organised structure of relationships between component parts causes them to work together, creating a new organismic entity and a new evolutionary unit on which selection can act. However, the principles of these transitions remain poorly understood. In particular, the process of transition must be explained by “bottom-up” selection, i.e., on the existing lower-level evolutionary units, without presupposing the higher-level evolutionary unit we are trying to explain. In this hypothesis and theory manuscript we address the conditions for evolutionary transitions in individuality by exploiting adaptive principles already known in learning systems.Connectionistlearning models, well-studied in neural networks, demonstrate how networks of organised functional relationships between components, sufficient to exhibit information integration and collective action, can be produced via fully-distributed and unsupervised learning principles, i.e., without centralised control or an external teacher. Evolutionary connectionism translates these distributed learning principles into the domain of natural selection, and suggests how relationships among evolutionary units could become adaptively organised by selection from below without presupposing genetic relatedness or selection on collectives. In this manuscript, we address how connectionist models with a particular interaction structure might explain transitions in individuality. We explore the relationship between the interaction structures necessary for (a) evolutionary individuality (where the evolution of the whole is a non-decomposable function of the evolution of the parts), (b) organismic individuality (where the development and behaviour of the whole is a non-decomposable function of the behaviour of component parts) and (c) non-linearly separable functions, familiar in connectionist models (where the output of the network is a non-decomposable function of the inputs). Specifically, we hypothesise that the conditions necessary to evolve a new level of individuality are described by the conditions necessary to learn non-decomposable functions of this type (or deep model induction) familiar in connectionist models of cognition and learning.

A binary classifier based on a reconfigurable dense network of metallic nanojunctions

Major efforts to reproduce the brain performances in terms of classification and pattern recognition have been focussed on the development of artificial neuromorphic systems based on top-down lithographic technologies typical of highly integrated components of digital computers. Unconventional computing has been proposed as an alternative exploiting the complexity and collective phenomena originating from various classes of physical substrates. Materials composed of a large number of non-linear nanoscale junctions are of particular interest: these systems, obtained by the self-assembling of nano-objects like nanoparticles and nanowires, results in non-linear conduction properties characterized by spatiotemporal correlation in their electrical activity. This appears particularly useful for classification of complex features: nonlinear projection into a high-dimensional space can make data linearly separable, providing classification solutions that are computationally very expensive with digital computers. Recently we reported that nanostructured Au films fabricated from the assembling of gold clusters by supersonic cluster beam deposition show a complex resistive switching behaviour. Their non-linear electric behaviour is remarkably stable and reproducible allowing the facile training of the devices on precise resistive states. Here we report about the fabrication and characterization of a device that allows the binary classification of Boolean functions by exploiting the properties of cluster-assembled Au films interconnecting a generic pattern of electrodes. This device, that constitutes a generalization of the perceptron, can receive inputs from different electrode configurations and generate a complete set of Boolean functions of n variables for classification tasks. We also show that the non-linear and non-local electrical conduction of cluster-assembled gold films, working at room temperature, allows the classification of non-linearly separable functions without previous training of the device.

Topological Equivalence Classification of Balanced Linearly Separable Boolean Functions on n-Dimensional Hypercube

The concept of conformal transformation is proposed through the study of the spatial structure of [Formula: see text]-dimensional hypercubes. Based on conformal transformation, a novel algorithm, called topological equivalence classification algorithm, is proposed for classifying balanced linearly separable Boolean functions. By the proposed algorithm, the topological equivalence classes of all balanced linearly separable Boolean functions and the number of Boolean functions in each of the topological equivalence classes are obtained. In addition, the properties of conformal transformation also show an application prospect for decomposing nonlinearly separable Boolean functions.

Optimality conditions for efficient solutions of nonconvex constrained multiobjective optimization problems via image space analysis

This paper mainly focuses on optimality conditions for efficient solutions of a nonconvex constrained multiobjective optimization problem via image space analysis. By virtue of the Gerstewitz function and the sum of the components of the objective function, a nonlinear regular weak separation function for efficient solutions, is constructed. In order to illustrate the importance of the sum of the components of the objective function in the nonlinear regular weak separation function for efficient solutions, a nonlinear regular weak separation function for weak efficient solutions, is also given. Moreover, a nonlinear strong separation function for efficient and weak efficient solutions, is introduced. Then, some theorems of the weak and strong alternative and a global necessary and sufficient optimality condition for efficient solutions are derived by means of the nonlinear regular weak and strong separation functions for efficient solutions. A saddle point sufficient optimality condition for efficient solutions in terms of a generalized Lagrangian function associated with the nonlinear regular weak separation function, is established. Finally, under some suitable assumptions, a saddle point necessary optimality condition is obtained, which further yields a Karush/Kuhn–Tucker necessary optimality condition for efficient solutions by the Clarke subdifferential.

Characterizations of robust optimality conditions via image space analysis

ABSTRACT In this paper, we consider general scalar robust optimization problems and study the characterizations for optimality conditions in the general vector spaces where we do not require any topology on the considered space. By using the image space analysis and nonlinear separation function, we derive some necessary and sufficient optimality conditions, especially saddle point sufficient optimality conditions for scalar robust optimization problems. Moreover, we discuss the validity and effectiveness of our results for the shortest path problem.

Nonlinear separation methods and applications for vector equilibrium problems using improvement sets

ABSTRACT In this paper, the image space analysis is applied to investigate a vector equilibrium problem using improvement sets and with matrix inequality constraints. First, a nonlinear scalar regular weak separation function is constructed by using the oriented distance function and the norm function. Then, a global saddle-point condition for a generalized Lagrange function is investigated. It is shown that the existence of a saddle point is equivalent to a nonlinear separation of two suitable subsets in the image space. Furthermore, a gap function and an error bound are obtained in terms of the nonlinear scalar regular weak separation function under suitable assumptions. As applications, an optimality condition, a gap function and an error bound for a strategic game with vector payoffs are also given.

Scalarizations and Optimality of Constrained Set-Valued Optimization Using Improvement Sets and Image Space Analysis

In this paper, we aim at applying improvement sets and image space analysis to investigate scalarizations and optimality conditions of the constrained set-valued optimization problem. Firstly, we use the improvement set to introduce a new class of generalized convex set-valued maps. Secondly, under suitable assumptions, some scalarization results of the constrained set-valued optimization problem are obtained in the sense of (weak) optimal solution characterized by the improvement set. Finally, by considering two classes of nonlinear separation functions, we present the separation between two suitable sets in image space and derive some optimality conditions for the constrained set-valued optimization problem. It shows that the existence of a nonlinear separation is equivalent to a saddle point condition of the generalized Lagrangian set-valued functions.

Characterizations of multiobjective robustness on vectorization counterparts

ABSTRACTIn this paper, we characterize robust solutions for uncertain multiobjective optimization problems on the basis of vectorization models by virtue of image space analysis. By introducing corrected image of original uncertain problem or the selected and corrected images of its robust counterpart, an equivalent relation between multiobjective robustness and the separation of two sets in the image space is well established. Moreover, by means of linear (vector/scalar) separation functions, some Lagrangian-type sufficient robust optimality conditions are presented. Especially, under suitable restriction assumptions, we derive Lagrangian-type necessary robust optimality conditions in terms of nonlinear separation functions. These results obtained in this paper extend and improve some existing ones recently. Finally, several examples are given to show the effectiveness of the conclusions.

Domain Wall Motion-Based Dual-Threshold Activation Unit for Low-Power Classification of Non-Linearly Separable Functions.

Recently, a great deal of scientific endeavour has been devoted to developing spin-based neuromorphic platforms owing to the ultra-low-power benefits offered by spin devices and the inherent correspondence between spintronic phenomena and the desired neuronal, synaptic behavior. While domain wall motion-based threshold activation unit has previously been demonstrated for neuromorphic circuits, it remains well known that neurons with threshold activation cannot completely learn nonlinearly separable functions. This paper addresses this fundamental limitation by proposing a novel domain wall motion-based dual-threshold activation unit with additional nonlinearity in its function. Furthermore, a new learning algorithm is formulated for a neuron with this activation function. We perform 100 trials of tenfold training and testing of our neural networks on real-world datasets taken from the UCI machine learning repository. On an average, the proposed algorithm achieves [Formula: see text] lower misclassification rate (MCR) than the traditional perceptron learning algorithm. In a circuit-level simulation, the neural networks with the proposed activation unit are observed to outperform the perceptron networks by as much as [Formula: see text] MCR. The energy consumption of a neuron having the proposed domain wall motion-based activation unit averages to [Formula: see text] approximately.

A fast training method for memristor crossbar based multi-layer neural networks

Memristor crossbar arrays carry out multiply–add operations in parallel in the analog domain which is the dominant operation in a neural network application. On-chip training of memristor neural network systems have the significant advantage of being able to get around device variability and faults. This paper presents a novel technique for on-chip training of multi-layer neural networks implemented using a single crossbar per layer and two memristors per synapse. Using two memristors per synapse provides double the synaptic weight precision when compared to a design that uses only one memristor per synapse. Proposed system utilizes a novel variant of the back-propagation (BP) algorithm to reduce both circuit area and training time. During training, all the memristors in a crossbar are updated in four steps in parallel. We evaluated the training of the proposed system with some nonlinearly separable datasets through detailed SPICE simulations which take crossbar wire resistance and sneak-paths into consideration. The proposed training algorithm trained the nonlinearly separable functions with a slight loss in accuracy compared to training with the traditional BP algorithm.

Nonlinear separation concerning E-optimal solution of constrained multi-objective optimization problems

This paper aims at investigating optimality conditions in terms of E-optimal solution for constrained multi-objective optimization problems in a general scheme, where E is an improvement set with respect to a nontrivial closed convex point cone with apex at the origin. In the case where E is not convex, nonlinear vector regular weak separation functions and scalar weak separation functions are introduced respectively to realize the separation between the two sets in the image space, and Lagrangian-type optimality conditions are established. These results extend and improve the convex ones in the literature.

Separation Functions and Optimality Conditions in Vector Optimization

In this paper, we propose weak separation functions in the image space for general constrained vector optimization problems on strong and weak vector minimum points. Gerstewitz function is applied to construct a special class of nonlinear separation functions as well as the corresponding generalized Lagrangian functions. By virtue of such nonlinear separation functions, we derive Lagrangian-type sufficient optimality conditions in a general context. Especially for nonconvex problems, we establish Lagrangian-type necessary optimality conditions under suitable restriction conditions, and we further deduce Karush–Kuhn–Tucker necessary conditions in terms of Clarke subdifferentials.

Nonlinear separation approach to inverse variational inequalities

In this paper, we employ the image space analysis to investigate an inverse variational inequality (for short, IVI) with a cone constraint. By virtue of the nonlinear scalarization function commonly known as the Gerstewitz function, three nonlinear weak separation functions, two nonlinear regular weak separation functions and a nonlinear strong separation function are first introduced. Then, by these nonlinear separation functions, theorems of the weak and strong alternative and some optimality conditions for IVI with a cone constraint are derived without any convexity. In particular, a global saddle-point condition for a nonlinear function is investigated. It is shown that the existence of a saddle point is equivalent to a nonlinear separation of two suitable subsets of the image space. Finally, two gap functions and an error bound for IVI with a cone constraint are obtained.

Ultrahigh Density Memristor Neural Crossbar for On-Chip Supervised Learning

Although there are many candidates for future computing systems, memristor-based neural crossbar (NC) is considered especially promising, thanks to their low power consumption, high density, and fault tolerance. However, their implementation is still hindered by the limitations of CMOS neuron and learning cells. In this paper, we present a memristor-based NC that implements on-chip supervised learning. Instead of using a standard CMOS neuron, a simple CMOS inverter realizes the activation function. More importantly, we propose a compact learning cell that consists of a crossbar latch of two antiparallel oriented binary memristors. This design allows for higher density integration and could be naturally extended to a multilayer neural network. Using the CMOS 40-nm design kit and a physics-based compact model of high-performance ferroelectric tunnel memristor, we performed transient simulations to validate the function of the proposed neural crossbar. Then, we construct a multilayer NC by cascading monolayer networks; thereby, enabling the network to learn nonlinearly separable functions (e.g., XOR function). Finally, the fault tolerance is evaluated with Monte Carlo simulation. Analysis of simulation results demonstrates promising applications of our proposed neural crossbar for on-chip supervised learning.

A new approach to constrained optimization via image space analysis

In this article, by introducing a class of nonlinear separation functions, the image space analysis is employed to investigate a class of constrained optimization problems. Furthermore, the equivalence between the existence of nonlinear separation function and a saddle point condition for a generalized Lagrangian function associated with the given problem is proved.

Nonlinear separation functions, optimality conditions and error bounds for Ky Fan quasi-inequalities

In this paper, we employ the image space analysis to investigate a Ky Fan quasi-inequality with cone constraints. By means of the oriented distance function, a new nonlinear weak (regular) separation function is introduced. Some necessary and sufficient optimality conditions, especially, a saddle-point sufficient optimality condition for the Ky Fan quasi-inequality with cone constraints, are obtained. By virtue of the nonlinear regular weak separation function, a gap function for the Ky Fan quasi-inequality with cone constraints is obtained. Moreover, we get an error bound for the solution set of the Ky Fan quasi-inequality with respect to the gap function under strongly monotone assumptions.

Memristor‐based neuron circuit and method for applying learning algorithm in SPICE

The learning of nonlinearly separable functions in cascaded memristor crossbar circuits is described and the feasibility of using them to develop low-power neuromorphic processors is demonstrated. This is the first study evaluating the training of memristor crossbars through SPICE simulations. It is important to capture the alternate current paths and wire resistance inherent in these circuits. The simulations show that neural network learning algorithms are able to train in the presence of alternate current paths and wire resistances. The fact that the approach reduces the area by three times and power by two orders of magnitude compared with the existing approaches that use virtual ground opamps to eliminate alternate current paths is demonstrated.

Memristor-Based Neural Logic Blocks for Nonlinearly Separable Functions

Neural logic blocks (NLBs) enable the realization of biologically inspired reconfigurable hardware. Networks of NLBs can be trained to perform complex computations such as multilevel Boolean logic and optical character recognition (OCR) in an area- and energy-efficient manner. Recently, several groups have proposed perceptron-based NLB designs with thin-film memristor synapses. These designs are implemented using a static threshold activation function, limiting the set of learnable functions to be linearly separable. In this work, we propose two NLB designs-robust adaptive NLB (RANLB) and multithreshold NLB (MTNLB)-which overcome this limitation by allowing the effective activation function to be adapted during the training process. Consequently, both designs enable any logic function to be implemented in a single-layer NLB network. The proposed NLBs are designed, simulated, and trained to implement ISCAS-85 benchmark circuits, as well as OCR. The MTNLB achieves 90 percent improvement in the energy delay product (EDP) over lookup table (LUT)-based implementations of the ISCAS-85 benchmarks and up to a 99 percent improvement over a previous NLB implementation. As a compromise, the RANLB provides a smaller EDP improvement, but has an average training time of only ≈ 4 cycles for 4-input logic functions, compared to the MTNLBs ≈ 8-cycle average training time.

Nonlinear separation functions and constrained extremum problems

In this paper, the image space analysis is employed to investigate constrained extremum problems. A new nonlinear separation function for the constrained extremum problems is given. Some optimality conditions and a strong duality theorem for the constrained extremum problem are obtained. These results extend and improve the corresponding ones in the literature.