Learning Scheme For Neural Networks Research Articles

Data-driven discoveries of Bäcklund transformations and soliton evolution equations via deep neural network learning schemes

We introduce a deep neural network learning scheme to discover the Bäcklund transforms (BTs) of soliton evolution equations and an enhanced deep learning scheme for data-driven soliton equation discovery based on the known BTs, respectively. The first deep learning scheme takes advantage of some solution (or soliton equation) informations to train the data-driven BT discovery, and is valid in the study of the BT of the sine-Gordon equation, and complex and real Miura transforms between the defocusing (focusing) mKdV equation and KdV equation, as well as the data-driven mKdV equation discovery via the Miura transforms. The second deep learning scheme uses the higher-order solitons generated by the explicit/implicit BTs to study the data-driven discoveries of mKdV and sine-Gordon equations, in which the high-order soliton informations are more powerful for the enhanced leaning soliton equations with higher accuracies.

Accurate 3D Shape Reconstruction from Single Structured-Light Image via Fringe-to-Fringe Network

Accurate three-dimensional (3D) shape reconstruction of objects from a single image is a challenging task, yet it is highly demanded by numerous applications. This paper presents a novel 3D shape reconstruction technique integrating a high-accuracy structured-light method with a deep neural network learning scheme. The proposed approach employs a convolutional neural network (CNN) to transform a color structured-light fringe image into multiple triple-frequency phase-shifted grayscale fringe images, from which the 3D shape can be accurately reconstructed. The robustness of the proposed technique is verified, and it can be a promising 3D imaging tool in future scientific and industrial applications.

Joint Interpolation and Representation Learning for Irregularly Sampled Satellite-Derived Geophysical Fields

Earth observation satellite missions provide invaluable global observations of geophysical processes in play in the atmosphere and the oceans. Due to sensor technologies (e.g., infrared satellite sensors), atmospheric conditions (e.g., clouds and heavy rains), and satellite orbits (e.g., polar-orbiting satellites), satellite-derived observations often involve irregular space–time sampling patterns and large missing data rates. Given the current development of learning-based schemes for earth observation, the question naturally arises whether one might learn some representation of the underlying processes as well as solve interpolation issues directly from these observation datasets. In this article, we address these issues and introduce an end-to-end neural network learning scheme, which relies on an energy-based formulation of the interpolation problem. This scheme investigates different learning-based priors for the underlying geophysical field of interest. The end-to-end learning procedure jointly solves the reconstruction of gap-free fields and the training of the considered priors. Through different case studies, including observing system simulation experiments for sea surface geophysical fields, we demonstrate the relevance of the proposed framework compared with optimal interpolation and other state-of-the-art data-driven schemes. These experiments also support the relevance of energy-based representations learned to characterize the underlying processes.

Distributed Min-Max Learning Scheme for Neural Networks With Applications to High-Dimensional Classification.

In this article, a novel learning methodology is introduced for the problem of classification in the context of high-dimensional data. In particular, the challenges introduced by high-dimensional data sets are addressed by formulating a L1 regularized zero-sum game where optimal sparsity is estimated through a two-player game between the penalty coefficients/sparsity parameters and the deep neural network weights. In order to solve this game, a distributed learning methodology is proposed where additional variables are utilized to derive layerwise cost functions. Finally, an alternating minimization approach developed to solve the problem where the Nash solution provides optimal sparsity and compensation through the classifier. The proposed learning approach is implemented in a parallel and distributed environment through a novel computational algorithm. The efficiency of the approach is demonstrated both theoretically and empirically with nine data sets.

HJB-Equation-Based Optimal Learning Scheme for Neural Networks With Applications in Brain–Computer Interface

This paper proposes a novel method for training neural networks (NNs). It uses an approach from optimal control theory, namely, Hamilton–Jacobi–Bellman equation, which optimizes system performance along the trajectory. This formulation leads to a closed-form solution for an optimal weight update rule, which has been combined with per-parameter adaptive scheme AdaGrad to further enhance its performance. To evaluate the proposed method, the NNs are trained and tested on two problems related to EEG classification, namely, mental imagery classification (multiclass) and eye state recognition (binary class). In addition, a novel dataset with the name EEG eye state, for benchmarking learning methods, is presented. The convergence proof for the proposed approach is also included, and performance is validated on many small to large scale, synthetic datasets (UCI, LIBSVM datasets). The performance of NNs trained with the proposed scheme is compared with other state-of-the-art approaches. Evaluation results substantiate the improvements brought about by the proposed scheme regarding faster convergence and better accuracy.

실내 환경에서의 측위를 위한 전파 지문 기반의 병렬 인공신경망 학습 기법

실내 측위 분야에서 가장 보편적으로 사용되는 전파 지문 기반의 측위 기법은 데이터 비교 알고리즘이 성능에 가장 큰 영향을 미친다. 본 논문에서는 인공신경망 학습을 통해 전파 지문 측위를 수행하며, 제안한 학습 방법은 기존의 유클리드 거리 비교 기반의 전파 지문 측위 알고리즘에 비해 높은 성능을 보인다. 본 논문에서는 실내 측위에 적합한 학습 구조뿐만 아니라, 데이터 확장 기법도 제안하며, 제안한 데이터 확장 기법은 다양한 측위 기술에 부분적으로 적용이 가능하다. 실험을 통해 제안한 기술이 기존의 유클리드 거리 기반의 측위 기법보다 높은 성능을 보여주는 것을 확인할 수 있다.

Unsupervised learning in general connectionist systems.

There is a common framework in which different connectionist systems may be treated in a unified way. The general system in which they may all be mapped is a network which, in addition to the connection strengths, has an adaptive node parameter controlling the output intensity. In this paper we generalize two neural network learning schemes to networks with node parameters. In generalized Hebbian learning we find improvements to the convergence rate for small eigenvalues in principal component analysis. For competitive learning the use of node parameters also seems useful in that, by emphasizing or de-emphasizing the dominance of winning neurons, either improved robustness or discrimination is obtained.

A neural networks software for the control engineers

When using the popular back-propagation neural networks for control applications, the training part was found to be most prohibitive misfortune of the first time control engineers. This paper attempts to contribute towards the description of techniques on how to handle critical situations during training to achieve a faster and smooth learning curve. The feed forward neural network learning scheme and recurrent neural models were extensively studied and programmed in the LEarning COntrol SOftware (LECOSO) software. In this learning control software the art of training was reviewed and carefully implemented with critical user interactive facilities during learning. This software is very useful especially for control engineers who wish to spend little effort in the time consuming process of learning but rather concentrate on the stability and plant feedback compensation problems.

Switched-capacitor realisations of artificial neural network learning schemes

Switched-capacitor realisations of learning schemes commonly used to train artificial neural networks are presented. These switched-capacitor realisations are modular in the sense that the same switched-capacitor modules can be used for the sum of products operation and the weight updating. The modularity of the realisations makes them attractive for VLSI implementations. Computer simulation results confirm all expectations.

Daugman's gabor transform as a simple generative back propagation network

Much work has been performed on learning mechanisms for neural networks. A particular area of interest has been the use of neural networks for image processing problems. Two important pieces of work in this area are unified. An architecture and learning scheme for neural networks called generative back propagation has been previously developed and a system for image compression and filtering based on 2-D Gabor transformations which used a neural network type architecture described. Daugman's procedure is exactly replicated, a procedure which used a four layer neural network as a two-layer generative back propagation network with half of the units. The GBP update rule is shown to perform the same change as Daugman's rule, but more efficiently.

Distributed Parallel Processing in the Vestibulo-Oculomotor System

The mechanisms of eye-movement control are among the best understood in motor neurophysiology. Detailed anatomical and physiological data have paved the way for theoretical models that have unified existing knowledge and suggested further experiments. These models have generally taken the form of black-box diagrams (for example, Robinson 1981) representing the flow of hypothetical signals between idealized signal-processing blocks. They approximate overall oculomotor behavior but indicate little about how real eye-movement signals would be carried and processed by real neural networks. Neurons that combine and transmit oculomotor signals, such as those in the vestibular nucleus (VN), actually do so in a diverse, seemingly random way that would be impossible to predict from a block diagram. The purpose of this study is to use a neural-network learning scheme (Rumelhart et al. 1986) to construct parallel, distributed models of the vestibulo-oculomotor system that simulate the diversity of responses recorded experimentally from VN neurons.