Neural Net System Research Articles

Dynamic slate recommendation with gated recurrent units and Thompson sampling

We consider the problem of recommending relevant content to users of an internet platform in the form of lists of items, called slates. We introduce a variational Bayesian Recurrent Neural Net recommender system that acts on time series of interactions between the internet platform and the user, and which scales to real world industrial situations. The recommender system is tested both online on real users, and on an offline dataset collected from a Norwegian web-based marketplace, FINN.no, that is made public for research. This is one of the first publicly available datasets which includes all the slates that are presented to users as well as which items (if any) in the slates were clicked on. Such a data set allows us to move beyond the common assumption that implicitly assumes that users are considering all possible items at each interaction. Instead we build our likelihood using the items that are actually in the slate, and evaluate the strengths and weaknesses of both approaches theoretically and in experiments. We also introduce a hierarchical prior for the item parameters based on group memberships. Both item parameters and user preferences are learned probabilistically. Furthermore, we combine our model with bandit strategies to ensure learning, and introduce ‘in-slate Thompson sampling’ which makes use of the slates to maximise explorative opportunities. We show experimentally that explorative recommender strategies perform on par or above their greedy counterparts. Even without making use of exploration to learn more effectively, click rates increase simply because of improved diversity in the recommended slates.

Time series prediction using deep echo state networks

Artificial neural networks have been used for time series modeling and forecasting in many domains. However, they are often limited in their handling of nonlinear and chaotic data. More recently, reservoir-based recurrent neural net systems, most notably echo state networks (ESN), have made substantial improvements for time series modeling. Their shallow nature lends themselves to an efficient training method, but has limitations on nonstationary, nonlinear chaotic time series, particularly large, multidimensional time series. In this paper, we propose a novel approach for forecasting time series data based on an additive decomposition (AD) applied to the time series as a preprocessor to a deep echo state network. We compare the performance of our method, AD-DeepESN, on popular neural net architectures used for time series prediction. Stationary and nonstationary data sets are used to evaluate the performance of the methods. Our results are compelling, demonstrating that AD-DeepESN has superior performance, particularly on the most challenging time series that exhibit non-stationarity and chaotic behavior compared to existing methods.

A combined method for designing operations using soft computing

The article considers the possibility and features of shared use of two approaches of modern artificial intelligence: neural networks and fuzzy logic. Taking into account the advantages and disadvantages of each of the methods considered, the method of sequential use of these systems is considered. Fuzzy systems can be used for the initial creation of an expert inference system with its subsequent adjustment by optimization methods. It is possible to use the formation of a fuzzy inference system based on a hybrid neural net inference system based on control points obtained analytically or experimentally, followed by the generation of dataset for training neural networks. The use of these methods together increases the speed of operation designing, reliability of output, flexibility of approach to the choice of parameters, expansion of capabilities in production environments characterized by a wide variety of operations, materials, processing methods. An example of using this approach for the cladding method is given.

Informing artificial intelligence generative techniques using cognitive theories of human creativity

The common view that our creativity is what makes us uniquely human suggests that incorporating research on human creativity into generative deep learning techniques might be a fruitful avenue for making their outputs more compelling and human-like. Using an original synthesis of DeepDream-based convolutional neural networks and cognitive based computational art rendering systems, we show how honing theory, intrinsic motivation, and the notion of a “seed incident” can be implemented computationally, and demonstrate their impact on the resulting generative art. Conversely, we discuss how explorations in deep learning convolutional neural net generative systems can inform our understanding of human creativity. We conclude with ideas for further cross-fertilization between AI based computational creativity and psychology of creativity.

Fuzzy-Neural Petri Net Distributed Control System Using Hybrid Wireless Sensor Network and CAN Fieldbus

The reluctance of industry to allow wireless paths to be incorporated in process control loops has limited the potential applications and benefits of wireless systems. The challenge is to maintain the performance of a control loop, which is degraded by slow data rates and delays in a wireless path. To overcome these challenges, this paper presents an application–level design for a wireless sensor/actuator network (WSAN) based on the “automated architecture”. The resulting WSAN system is used in the developing of a wireless distributed control system (WDCS). The implementation of our wireless system involves the building of a wireless sensor network (WSN) for data acquisition and controller area network (CAN) protocol fieldbus system for plant actuation. The sensor/actuator system is controlled by an intelligent digital control algorithm that involves a controller developed with velocity PID-like Fuzzy Neural Petri Net (FNPN) system. This control system satisfies two important real-time requirements: bumpless transfer and anti-windup, which are needed when manual/auto operating aspect is adopted in the system. The intelligent controller is learned by a learning algorithm based on back-propagation. The concept of petri net is used in the development of FNN to get a correlation between the error at the input of the controller and the number of rules of the fuzzy-neural controller leading to a reduction in the number of active rules. The resultant controller is called robust fuzzy neural petri net (RFNPN) controller which is created as a software model developed with MATLAB. The developed concepts were evaluated through simulations as well validated by real-time experiments that used a plant system with a water bath to satisfy a temperature control. The effect of disturbance is also studied to prove the system's robustness.

Exploring the non-linearity in empirical modelling of a steel system using statistical and neural network models

The relationship between the physical properties of metal is often very complex in nature with its chemistry and several other rolling parameters in operation. Non-linear regression models play a very important role in modelling the underlying mechanism, provided it is known. Artificial neural networks provide a wide class of general-purpose and flexible non-linear regression models. The most commonly used neural networks, called multi-layered perceptrons, can vary the complexity of the model from a simple parametric model to a highly flexible nonparametric model. In this particular work, an industry-based data set is used for learning and optimizing the neural network architecture using some well-known algorithms for prediction under neural-net systems. The outcome of the analysis is compared with the results achieved through empirical statistical modelling from its prediction error level and the knowledge of materials science.

A Comparative Study for Modeling of Hot-Rolled Steel Plate Classification Using a Statistical Approach and Neural-Net Systems

Classification of flat steel products from the point of view of reaching the target property is a common practice in industries. In most classification problems, standard statistical methods generally place constraints such as continuous, differentiable, otherwise well behaved, etc. However, Artificial Neural Networks (ANN) has an ability to learn and generalize any complex system without making any model assumptions. This work emphasizes on making performance evaluation of usual statistical techniques such as general clustering like K-means, partition around medoid (PAM), classification and regression tree (CART), linear discriminant analysis (LDA) vis-a-vis usage of multilayer perceptron (MLP) learning algorithm, radial basis function (RBF) family of methods and Kohonen networks. To recommend the utility of modeling, some real-life industrial databases are used. It can be observed from the results that learning of networks through back-propagation yielded minimum misclassification of two groups of heats including minimization of train-test error. The statistical techniques such as LDA and CART provide the same results of misclassification along with the results obtained from perceptron learning, RBF network algorithm and Kohonen learning with learning-vector quantization (LVQ) algorithm.

5140 人工知能を用いたハイポイドギヤ加工機設定の決定(S63-8 フェースギヤ・ハイポイドギヤ,S63 伝動装置の基礎と応用)

The tooth form of a hypoid gear depends on its cutting machine set-up, and has no theoretical one. Therefore, the pinion does not have the conjugate tooth form to the gear. Generally, the machine set-up is determined from the experience of a trained technician, and it is very difficult to evaluate the machine set-up theoretically. The purpose of the present study is to develop a neural net system that enables the machine set-up to be evaluated instead of trained technicians. For the first step of the investigations, for the input data to the developed neural net system, the topography of deviations from the generated tooth form from the conjugate one was calculated and its geometrical parameters was evaluated.

Generalized reduced-order synchronization of chaotic system based on fast slide mode

A new kind of generalized reduced-order synchronization of different chaotic systems is proposed in this paper. It is shown that dynamical evolution of third-order oscillator can be synchronized with the canonical projection of a fourth-order chaotic system generated through nonsingular states transformation from a cell neural net chaotic system. In this sense, it is said that generalized synchronization is achieved in reduced-order. The synchronization discussed here expands the scope of reduced-order synchronization studied in relevant literatures. In this way, we can achieve generalized reduced-order synchronization between many famous chaotic systems such as the second-order Duffing system and the third-order Lorenz system by designing a fast slide mode controller. Simulation results are provided to verify the operation of the designed synchronization.

Evolving in Real Time a Neural Net Controller of Robot‐Arm: Track and Evolve

Evolutionary Engineering (EE) is defined to be “the art of using evolutionary algorithms approach such as genetic algorithms to build complex systems”. This paper deals with a neural net based system. It analyses ability of genetically trained neural nets to control Simulated robot arm, witch tries to track a moving object. In difference from classical Approaches neural network learning is performed on line, i.e., in real time. Usually systems are built/evolved, i.e., genetically trained separately of their utilization. That is how it is commonly done. It's a fact that evolution process is heavy on time; that's why Real-Time approach is rarely taken into consideration. The results presented in this paper show that such approach (Real-Time EE) is possible. These successful results are essentially due to the “continuity” of the target's trajectory. In EE terms, we express this by the Neighbourhood Hypothesis (NH) concept.

PERSPECTIVES ON THE HISTORY OF THE CYBERNETICS MOVEMENT: THE PATH TO CURRENT RESEARCH THROUGH THE CONTRIBUTIONS OF NORBERT WIENER, WARREN McCULLOCH, AND JOHN VON NEUMANN

This is a historical and biographical article that traces the part and life of three of the main pillars of the emergent cybernetics science: Norbert Wiener, Warren McCulloch, and John von Neumann. Though different in their backgrounds, outlook, and scientific work, it is shown how these three main protagonists have jointly erected the cybernetics edifice which now houses robotics, artificial intelligence, automata theory, neural net theory, systems theory, biomedical cybernetics, informatics, and related fields.

A neural net compiler system for hierarchical organization

We present a language framework for handling arbitrarily complex neural computations. The software architecture - which we call an A rtificial N eural Network C ompiler for H ierarchical OR ganization ( ANCHOR ) - facilitates network hierarchy and simpler sub-mappings. We define a N et D efinition L anguage (NDL) which is implemented in object-oriented programming paradigm; a trained network is decompiled back into NDL. ANCHOR is configured around the concept of a Superneuron which is a generalized view of a neuron-processing element and designed using reuse of object-model. The indistinguishability between a superneuron and a neuron is employed in hierarchical nesting of superneurons, up to (theoretically) infinite depth within other superneurons as well as linear or tree-structured cascading. Hierarchical decomposition of simple boolean functions has been demonstrated as proof-of-concept.

Three-dimensional mapping of brainstem functional lesions

The human brainstem is a highly complex structure where even small lesions can give rise to a variety of symptoms and outward signs. Localising the area of dysfunction within the brainstem is often a difficult task. To make localisation easier, a neural net system has been developed which uses 72 clinical and neurophysiological data inputs to provide a display (using 5268 voxels) on a three-dimensional model of the human brainstem. The net was trained by means of a back-propagation algorithm, over a pool of 580 example cases. Assessed on 200 test cases, the net correctly localised 83.6% of the target voxels; furthermore the net correctly localised the lesions in 31 out of 37 patients. Because this computer-assisted method provides reliable and quantitative localisation of brainstem areas of dysfunction and can be used as a 3D interactive functional atlas, it is expected to prove useful as a diagnostic tool for assessing focal brainstem lesions.

Neural net and expert system diagnose transformer faults

Dissolved gas-in-oil analysis (DGA) is a common practice in transformer incipient fault diagnosis. The analysis techniques include the conventional key gas method, ratio methods, and artificial intelligence methods. Application of artificial intelligence (Al) techniques have shown very promising results. The methods include fuzzy logic, expert systems (EPS), evolutionary algorithms (EA), and artificial neural networks (ANN). A transformer incipient fault diagnosis system (ANNEPS) was developed over a period of 5 years at Virginia Tech, collaborating with Doble Engineering Company. The system can detect thermal faults (distinguishing overheating of oil from that of cellulose and between four overheating stages and overheating of oil), low-energy discharge (partial discharge), high-energy discharge (arcing), and cellulose degradation.

Use of neural net computing for statistical and kinetic modelling and simulation of supercritical fluid extractors

An empirical kinetic model was developed for the extraction of black cumin (Nigella Sativa L.) seed oil with supercritical carbon dioxide as solvent. Extraction conditions were up to 20MPa and 333K. Kinetic measurements provided by a neural-regressive hybrid prediction system allowed the simulation and design of a supercritical fluid extraction unit using differential laboratory data modelled with neural networks. Differential extraction yields were correlated with the factors that affected the rate (particle size, degree of milling, fluid flow rate, pressure, temperature, moisture, and time). Their influence on the extracted mass of oil (or system response) was determined by a statistical design of experiments. Response surface data were obtained by using non-parametric statistics. Thus, a neural net model was developed to predict extraction yields. It consisted in a feedforward multilayer neural network, conveniently trained with the backpropagation algorithm. Inputs to the neural net were: pressure, temperature and time. The system provided a single output (the amount extracted) as the only system response. The network architecture was kept to minimum complexity by carefully choosing the number of units of the hidden layer. The neural net computing program was intended as a design and simulation tool for SCF extractors. The bed was assimilated to a pile of layers, each corresponding to a neural net so that its extraction rate could be predicted by the associated neural network as a function of time. In the design stage, the neural-net system operates with numerical routines that carry out the integration of the fluid-phase mass balance equation. The intra-bed mass conservation equation takes into account solute accumulation, convective transport and transport by axial dispersion. Solution to the mass balance equation at each bed height and time was necessary for obtaining the breakthrough curve. By using certain scale-up rules, no fitting parameters were necessary to describe the actual breakthrough behaviour. A sensitivity analysis clearly pointed out the key process parameters, and how to use the model for design purposes. Predictions made with the present hybrid computing system (neural network in combination with pseudohomogeneous mass balance) successfully agreed with data.

Use of neural net computing for statistical and kinetic modelling and simulation of supercritical fluid extractors

An empirical kinetic model was developed for the extraction of black cumin ( Nigella sativa L.) seed oil with supercritical carbon dioxide as solvent. Extraction conditions were up to 20 MPa and 333 K. Kinetic measurements provided by a neural-regressive hybrid prediction system allowed the simulation and design of a supercritical fluid extraction unit using differential laboratory data modelled with neural networks. Differential extraction yields were correlated with the factors that affected the rate (particle size, degree of milling, fluid flow rate, pressure, temperature, moisture, and time). Their influence on the extracted mass of oil (or system response) was determined by a statistical design of experiments. Response surface data were obtained by using non-parametric statistics. Thus, a neural net model was developed to predict extraction yields. It consisted in a feedforward multilayer neural network, conveniently trained with the backpropagation algorithm. Inputs to the neural net were: pressure, temperature and time. The system provided a single output (the amount extracted) as the only system response. The network architecture was kept to minimum complexity by carefully choosing the number of units of the hidden layer. The neural net computing program was intended as a design and simulation tool for SCF extractors. The bed was assimilated to a pile of layers, each corresponding to a neural net so that its extraction rate could be predicted by the associated neural network as a function of time. In the design stage, the neural-net system operates with numerical routines that carry out the integration of the fluid-phase mass balance equation. The intra-bed mass conservation equation takes into account solute accumulation, convective transport and transport by axial dispersion. Solution to the mass balance equation at each bed height and time was necessary for obtaining the breakthrough curve. By using certain scale-up rules, no fitting parameters were necessary to describe the actual breakthrough behaviour. A sensitivity analysis clearly pointed out the key process parameters, and how to use the model for design purposes. Predictions made with the present hybrid computing system (neural network in combination with pseudohomogeneous mass balance) successfully agreed with data.

Adaptive-intelligent control by neural-net systems

Adaptive-intelligent control by neural-net systems is discussed. Actual adaptive-intelligent control is realized in a general system through the following two hierarchical steps: (1) choosing a hierarchical coordinate system (associated with the environment of the system) and constructing the hierarchical evaluation functions (specifying its control states) and (2) finding a set of the most appropriate hierarchical values for the control parameters (giving the minimum value to the evaluation function). Step 1 establishes intelligently self-controllable (thinking) algorithms with human-like intelligence for various events (concepts). Step 2 studies the intelligently self-controllable (thinking) algorithms for finding the most appropriate state. Adaptive-intelligent control by neural-net systems is realized by integrating both intelligently self-controllable (thinking) algorithms on the neural-net systems. Here step 2 is mainly discussed in the neural-net systems of Boltzmann type machines using the method of stochastic dynamics.

From CAD to computer aided welding

Within the context of integrated design, we propose a new approach for off-line programming of welding robots by interfacing a CAD modeller (geometric database) and an artificial intelligence system (welding database). The CAD system associated with our development, used to design the parts to be assembled, allows us to generate welding paths automatically and to extract the assembly features required to determine welding parameters. With these features, we propose a new approach to generate welding parameters automatically in the GMAW process with neural networks. We have chosen to use backpropagation neural networks as this approach integrates database and modelling aspects. Moreover, a neural net based system can easily be improved, it can enlarge its field of application using new experimental welding data. In this paper we present the system we have developed for the generation of paths and then an approach using neural networks to determine welding parameters. We show how CAD features can be used to determine the welding process, the welding wire and then to compute welding parameters.

Artificial neural networks for coordination and control: The portability of experiential representations

It is time to locate connectionist representation theory in the new wave of robotics research. The utility of representations developed in artificial neural networks (ANNs) during learning has been demonstrated in cognitive science research since the 1980s. The research reported here puts learned representations to work in a decentered control task, the disembodied arm problem, in which a mobile robot operates an arm fixed to a table to pick up objects. There is no physical linkage between the arm and the robot and so the robot's point of view must be decentered. This is done by developing a modular Artificial Neural Net system in three stages: (i) a classifier net is trained with laser scan data to output transformationally invariant position classes; (ii) an arm net is trained for picking up objects; (iii) an inter net is trained to communicate and coordinate the sensing and acting. The completed system is shown to create new nonsymbolic transformationally invariant representations in order to perform the effective generalization of decentered viewpoints.

Data-Defined Problems and Multiversion Neural-Net Systems

Data-Defined Problems and Multiversion Neural-Net Systems