Abstract
Lightweight or mobile neural networks used for real-time computer vision tasks contain fewer parameters than normal networks, which lead to a constrained performance. Herein, a novel activation function named as Tanh Exponential Activation Function (TanhExp) is proposed which can improve the performance for these networks on image classification task significantly. The definition of TanhExp is f(x) = x tanh(ex). The simplicity, efficiency, and robustness of TanhExp on various datasets and network models is demonstrated and TanhExp outperforms its counterparts in both convergence speed and accuracy. Its behaviour also remains stable even with noise added and dataset altered. It is shown that without increasing the size of the network, the capacity of lightweight neural networks can be enhanced by TanhExp with only a few training epochs and no extra parameters added.
Highlights
Lightweight neural networks, known as mobile neural networks, are specially designed for realizing real-time visual information processing
We propose a Tanh Exponential Activation Function (TanhExp), which combines the advantages of activation functions similar to Rectified Linear Unit (ReLU) and other non-piecewise activation functions together
We introduce the Tanh Exponential Activation Function(TanhExp), which can be defined in Eq (3): f (x) = x tanh(ex) where tanh refers to the hyperbolic tangent function: ex − e−x tanh(x) = ex + e−x
Summary
Lightweight neural networks, known as mobile neural networks, are specially designed for realizing real-time visual information processing. They tune deep neural network architectures to strike an optimal balance between accuracy and performance, tailored for mobile and resource limitted environments [1]. These networks are necessary for computer vision tasks which require real-time computation [2,3,4,5]. Noticing that the powerful ability of a neural network to fit a non-linear function lays upon the activation function inside, we consider that an effective activation function can help boost the performance of these networks without sacrificing size and rapidity
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
