Insight, limitations, criticism, and interpretability of the use of activation functions in deep learning artificial neural networks

Abstract

The existing neural networks activation functions, the Sigmoid among others is the only set uses across various applications of NNs such as microscopy, neuromorphic, optical, robotics, finance and transportation. Only one set applies to different areas of application. Also, these activation functions’ selections are based on trial and error, neither emanate from the AI and or training datasets, nor from the testing data. This formed NNs’ Black-box. Jamilu (2019) proposed that strong links between the AI and or training datasets and activation functions must be established. This is to replace the NNs’ Black-box models with the models rely much less on experts’ assumptions, and much more on input AI and or training datasets, time change and specific area of application. Thus, Jamilu (2019) proposed Criterion(s) for the rational selection of activation functions. The paper is to use superintelligent NNs for stock price predictions, portfolio optimization, and general application approaches to shed light on the paper’ title.