Trainable Function Research Articles

Potential-Based Reward Shaping for Intrinsic Motivation (Student Abstract)

Recently there has been a proliferation of intrinsic motivation (IM) reward shaping methods to learn in complex and sparse-reward environments. These methods can often inadvertently change the set of optimal policies in an environment, leading to suboptimal behavior. Previous work on mitigating the risks of reward shaping, particularly through potential-based reward shaping (PBRS), has not been applicable to many IM methods, as they are often complex, trainable functions themselves, and therefore dependent on a wider set of variables than the traditional reward functions that PBRS was developed for. We present an extension to PBRS that we show preserves the set of optimal policies under a more general set of functions than has been previously demonstrated. We also present Potential-Based Intrinsic Motivation (PBIM), a method for converting IM rewards into a potential-based form that are useable without altering the set of optimal policies. Testing in the MiniGrid DoorKey environment, we demonstrate that PBIM successfully prevents the agent from converging to a suboptimal policy and can speed up training.

Functional Connectivity and Microstructural Network Correlates of Interindividual Variability in Distinct Executive Functions of Healthy Older Adults

Executive functions, essential for daily life, are known to be impaired in older age. Some executive functions, including working memory updating and value-based decision-making, are specifically sensitive to age-related deterioration. While their neural correlates in young adults are well-described, a comprehensive delineation of the underlying brain substrates in older populations, relevant to identify targets for modulation against cognitive decline, is missing. Here, we assessed letter updating and Markov decision-making task performance to operationalize these trainable functions in 48 older adults. Resting-state functional magnetic resonance imaging was acquired to quantify functional connectivity (FC) in task-relevant frontoparietal and default mode networks. Microstructure in white matter pathways mediating executive functions was assessed with diffusion tensor imaging and quantified by tract-based fractional anisotropy (FA). Superior letter updating performance correlated with higher FC between dorsolateral prefrontal cortex and left frontoparietal and hippocampal areas, while superior Markov decision-making performance correlated with decreased FC between basal ganglia and right angular gyrus. Furthermore, better working memory updating performance was related to higher FA in the cingulum bundle and the superior longitudinal fasciculus. Stepwise linear regression showed that cingulum bundle FA added significant incremental contribution to the variance explained by fronto-angular FC alone. Our findings provide a characterization of distinct functional and structural connectivity correlates associated with performance of specific executive functions. Thereby, this study contributes to the understanding of the neural correlates of updating and decision-making functions in older adults, paving the way for targeted modulation of specific networks by modulatory techniques such as behavioral interventions and non-invasive brain stimulation.

Information Bottleneck Signal Processing and Learning to Maximize Relevant Information for Communication Receivers

Digital communication receivers extract information about the transmitted data from the received signal in subsequent processing steps, such as synchronization, demodulation and channel decoding. Technically, the receiver-side signal processing for conducting these tasks is complex and hence causes bottleneck situations in terms of power, delay and chip area. Typically, many bits per sample are required to represent and process the received signal in the digital receiver hardware accurately. In addition, demanding arithmetical operations are required in the signal processing algorithms. A popular recent trend is designing entire receiver chains or some of their crucial building blocks from an information theoretical perspective. Signal processing blocks with very simple mathematical operations can be designed to directly maximize the relevant information that flows through them. At the same time, a strong quantization reduces the number of bits processed in the receiver to further lower the complexity. The described system design approach follows the principle of the information bottleneck method. Different authors proposed various ideas to design and implement mutual information-maximizing signal processing units. The first important aim of this article is to explain the fundamental similarities between the information bottleneck method and the functionalities of communication receivers. Based on that, we present and investigate new results on an entire receiver chain that is designed following the information bottleneck design principle. Afterwards, we give an overview of different techniques following the information bottleneck design paradigm from the literature, mainly dealing with channel decoding applications. We analyze the similarities of the different approaches for information bottleneck signal processing. This comparison leads to a general view on information bottleneck signal processing which goes back to the learning of parameters of trainable functions that maximize the relevant mutual information under compression.

Estimation of surface roughness and cutting speed in CNC WEDM by artificial neural network that employed trainable activation function

Activation functions are the most significant properties of artificial neural networks (ANN) because these functions are directly related with the ability of ANN in learning or modelling a system or a function. Furthermore, another reason for the significance of the fact that determination of optimal activation function in ANN is its relationship with success level. In this experimental study, the effects of different types of wire electrodes, cooling techniques and workpiece materials on surface roughness (Ra) and cutting speed (Vc) in wire electrical discharge machining (WEDM) were investigated by using trainable activation functions (AFt) and modelling them in ANNs. So far, a number of methods have been performed according to the data set in order to optimally predict Ra and Vc results. Among these methods, randomized ANN with AFt was found to be the best one for robust prediction according to RMSE values. While the value was 0.280 for Vc, it was 0.2104 for Ra. Optimum activation functions in Ra and Vc were found at first and third degree trainable functions, respectively.

Linearized sigmoidal activation: A novel activation function with tractable non-linear characteristics to boost representation capability

Activation functions play a crucial role in discriminative capabilities of the deep neural networks. They are also one of the main reasons for revival of the neural networks. Although, recent activation functions provide solution to vanishing and exploding gradient problems, but they do not have sufficient capacity to model non-linear data. This paper proposes a novel activation function, which imparts neural networks with capability to model non-linear dependencies in data. Behavior of proposed activation function remains non-saturating even with non-linear structure. The activation function provides distinct activation behaviors to different data range segments. The paper presents two different variants of proposed function. The first, “linear sigmoidal activation” function is a fixed structure activation function with function coefficients defined at the start of model design. Whereas second, “adaptive linear sigmoidal activation” function is a trainable function which can adapt itself according to the complexity of the given data. Both of the proposed models are tested against the state of the art activation functions on benchmark datasets (CIFAR-10, MNIST, SVHN, FER-2013). The proposed activation function is able to outperforms every known activation functions in all the tests.

Evolving Fuzzy Min–Max Neural Network Based Decision Trees for Data Stream Classification

Learning from data streams is a challenging task which demands a learning algorithm with several high quality features. In addition to space complexity and speed requirements needed for processing the huge volume of data which arrives at high speed, the learning algorithm must have a good balance between stability and plasticity. This paper presents a new approach to induce incremental decision trees on streaming data. In this approach, the internal nodes contain trainable split tests. In contrast with traditional decision trees in which a single attribute is selected as the split test, each internal node of the proposed approach contains a trainable function based on multiple attributes, which not only provides the flexibility needed in the stream context, but also improves stability. Based on this approach, we propose evolving fuzzy min–max decision tree (EFMMDT) learning algorithm in which each internal node of the decision tree contains an evolving fuzzy min–max neural network. EFMMDT splits the instance space non-linearly based on multiple attributes which results in much smaller and shallower decision trees. The extensive experiments reveal that the proposed algorithm achieves much better precision in comparison with the state-of-the-art decision tree learning algorithms on the benchmark data streams, especially in the presence of concept drift.

A trainable nonlinear function generator

Trainable threshold logic elements have been used in the adaptive pattern recognition field to design linear classifiers for patterns of binary variables. This paper shows how one of these trainable linear devices can be used to implement a trainable non-linear function generator. A trainable function generator is similar in function to the spatial function generators found in analog computers except that the coefficients of its function are set in an iterative "training" procedure instead of being predetermined in the design stage. The primary application of the trainable function generator, to date, has been in the real-time design of nonlinear pattern classification devices, therefore, the use of the trainable function generator as a nonlinear discriminant function in pattern classification is emphasized. It is demonstrated that in the implementation of the trainable function generator it is highly advantageous to use what has been defined as a linearly independent code to represent the analog pattern parameters as binary patterns for the threshold element. When a linearly independent code is used, a linear threshold element in a trainable function generator can be used to "learn" nonlinear discriminant functions just as it is used to learn linear discriminant functions.