UDC 004.032.26
DOI: 10.36871/2618-9976.2021.11.004
Authors
Erokhin V.V.
Doctor of Technical Sciences, a Professor of the Department of Mathematical Methods and Business
Informatics, Moscow State Institute of International Relations (University), Ministry of Foreign Affairs
of the Russian Federation
Abstract
Layer-by-layer training is an alternative end-to-end backpropagation approach for training deep convolutional neural networks. Layer-by-layer training on specific architectures can yield very competitive results. In the ImageNet database (www.imagenet. org), layer-by-layer trained networks can perform comparable to many modern end-to-end trained networks. This article compares the performance gap between the two training procedures over a wide range of network architectures and further analyzes the potential limitations of layer-by-layer training. The results show that layer-by-layer learning quickly saturates after a certain critical level due to overfitting of early levels in neural networks. Several approaches that have been used to solve this problem are discussed, and a methodology for improving layer-by-layer learning in various neural network architectures is discussed. Fundamentally, this research highlights the need to open up the black box that represents modern deep neural networks and explore the layer-by-layer interactions between intermediate hidden layers within deep networks through the lens of layer-by-layer learning.
Keywords
Microfinance
Neural networks
Artificial intelligence
Convolutional neural networks
Deep learning
