Comparative Study on Image Classification Using Different Optimizer
Keywords:
AdaDelta, AdaGrad, Adam, CNN, CIFAR- 10, MNIST, RMSPropAbstract
The deep learning method is a promising computational technique, especially for image classification problems. One of them is the Convolutional Neural Network (CNN), which is the most popular neural network model used. Although CNN is highly accurate, overfitting is a problem that frequently occurs. The process of training a neural network to perform a task involves repeatedly testing it with example data and using the results to modify the parameters or weights within the model in such a way as to minimize errors occurring due to overfitting. The component that makes these changes to reduce overfitting is called the optimizer. Various optimizers such as Adagrad, RMSProp, AdaDelta and Adam can be used for model molding into its appropriate form by futzing with the weights. Aiming to overcome the problem in getting the optimized result, the research used various algorithms of weight optimization. This paper elaborates on convolutional network concept as well as the idea behind the use of optimizers. The goal of this paper is to conduct a performance evaluation of different optimizers and quantifying the speed and accuracy with which they perform an image classification task.
References
. Rathore H, Al-Ali AK, Mohamed A, Du X, Guizani M. A novel deep learning strategy for classifying different attack patterns for deep brain implants. IEEE Access. 2019 Feb 20;7:24154-64.
. Xin R, Zhang J, Shao Y. Complex network classification with convolutional neural network. Tsinghua Science and technology. 2020 Jan 13;25(4):447-57.
.Weng Y, Zhou T, Liu L, Xia C. Automatic convolutional neural architecture search for image classification under different scenes. IEEE Access. 2019 Mar 28;7:38495-506.
. Liang F, Shen C, Wu F. An iterative BP-CNN architecture for channel decoding. IEEE Journal of Selected Topics in Signal Processing. 2018 Jan 15;12(1):144-59.
. Wang Y, Liu J, Mišić J, Mišić VB, Lv S, Chang X. Assessing optimizer impact on dnn model sensitivity to adversarial examples. IEEE Access. 2019 Oct 21;7:152766-76.
. Zhao J, Gao Y, Yang Z, Li J, Feng Y, Qin Z, Bai Z. Truck traffic speed prediction under non- recurrent congestion: Based on optimized deep learning algorithms and GPS data. IEEE Access. 2019 Jan 1;7:9116-27.
. Mainprice J, Hayne R, Berenson D. Goal set inverse optimal control and iterative replanning for predicting human reaching motions in shared workspaces. IEEE Transactions on Robotics. 2016 Jul 27;32(4):897-908.
. Park SH, Bae YB, Fidan B, Ahn HS. Distance-based Mobile Node Localization of Fixed Beacons Using RMS Prop. In2019 58th Annual Conference of the Society of Instrument and Control Engineers of Japan (SICE) 2019 Sep 10 (pp. 376-381). IEEE.
. D. P. Kingma and J. Ba, “Adam: A method for stochastics optimization,” arXiv preprint arXiv:1706.08500, 2017.
S. J. Reddi, S. Kale, and S. Kumar. 2018. “On the convergence of Adam and beyond”. Proceedings of the International Conference on Learning Representations
.The apache software foundation, what is apachemahout. 2014.CIFAR Dataset
. Mohapatra RK, Majhi B, Jena SK. Classification performance analysis of mnist dataset utilizing a multi-resolution technique. In2015 International Conference on Computing, Communication and and Security (ICCCS) 2015 Dec 4 (pp. 1-5). IEEE
Hadgu AT, Nigam A, Diaz-Aviles E. Large-scale learning with AdaGrad on Spark. In2015 IEEE International Conference on Big Data (Big Data) 2015 Oct 1 (pp. 2828-2830). IEEE.
Liu Y, Huangfu W, Zhang H, Long K. An efficient stochastic gradient descent algorithm to maximize the coverage of cellular networks. IEEE Transactions on Wireless Communications. 2019 May7;18(7):3424-36.
Fleishman, G. M., & Thompson, P. M. (2017, April). Adaptive gradient descent optimization of initial momenta for geodesic shooting in diffeomorphisms. In 2017 IEEE 14th International Symposium on Biomedical Imaging (ISBI 2017) (pp. 868-872). IEEE.
