Abstract
In deep learning, deep neural network (DNN) hyperparameters can severely affect network performance. Currently, such hyperparameters are frequently optimized by several methods, such as Bayesian optimization and the covariance matrix adaptation evolution strategy. However, it is difficult for non-experts to employ these methods. In this paper, we adapted the simpler coordinate-search and Nelder-Mead methods to optimize hyperparameters. Several hyperparameter optimization methods were compared by configuring DNNs for character recognition and age/gender classification. Numerical results demonstrated that the Nelder-Mead method outperforms the other methods and achieves state-of-the-art accuracy for age/gender classification.
Highlights
The evolution of deep neural networks (DNNs) has dramatically improved the accuracy of character recognition [1], object recognition [2, 3], and other tasks
A hyperparameter optimization problem can be formulated as a stochastic black box optimization problem to minimize a noisy black box objective function f (x): Minimize f (x) (x ∈ χ)
The results demonstrated that Bayesian optimization outperforms manual search by a human expert and random search [7, 8]
Summary
The evolution of deep neural networks (DNNs) has dramatically improved the accuracy of character recognition [1], object recognition [2, 3], and other tasks. The their increasing complexity increases the number of hyperparameters, which makes tuning of hyperparameters an intractable task. Search space expands exponentially relative to the number of hyperparameters; such naive methods no longer work well. More sophisticated hyperparameter optimization methods are required. A hyperparameter optimization problem can be formulated as a stochastic black box optimization problem to minimize a noisy black box objective function f (x): Minimize f (x) (x ∈ χ)
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callMore From: IPSJ Transactions on Computer Vision and Applications
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
