UDC 004.032.26
1 National Technical University “Igor Sikorsky Kyiv Polytechnic Institute,” Kyiv, Ukraine
 lanashape@gmail.com
|
2 National Technical University “Igor Sikorsky Kyiv Polytechnic Institute,” Kyiv, Ukraine
yuramuv@gmail.com
|
INCREASING THE RECEPTIVE FIELD OF NEURONS OF CONVULSIONAL
NEURAL NETWORKS
Abstract. The convolutional neural network architectures for the classification of 1D- and 2D-signals are analyzed.
It has been determined that in the case of a large-dimensional input signal, sufficient classification accuracy can be ensured
by only using a large number of layers, which cannot be done under limited computing resources.
However, if the number of layers is limited, starting from some critical dimension value, the accuracy decreases. A method of modifying a convolutional neural network with a relatively small number of layers is proposed to solve this problem. Its effectiveness is proved experimentally.
Keywords: convolutional neural networks, ResNet, EfficientNet, WaveNet, receptive field.
full text
REFERENCES
- He K., Zhang X., Ren S., Sun J. Deep residual learning for image recognition. Proc. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (27-30 June 2016, Las Vegas, NV, USA). Las Vegas, 2016. P. 770–778. https://doi.org/10.48550/arXiv.1512.03385.
- Xie S., Girshick R., Dollїr P., Tu Z., He K. Aggregated residual transformations for deep neural networks. Proc. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (21-26 July 2017, Honolulu, HI, USA). Honolulu, 2017. P. 5987–5995. https://doi.org/10.48550/arXiv.1611.05431 .
- Hu J., Shen L., Sun G. Squeeze-and-excitation networks. Proc. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (18-23 June 2018, Salt Lake City, UT, USA). Salt Lake City, 2018. P. 7132–7141. https://doi.org/10.48550/arxiv.1709.01507 .
- Tan M., Le Q.V. EfficientNet: Rethinking model scaling for convolutional neural networks. Proc. 36th International Conference on Machine Learning (ICML 2019) (9-15 June 2019, Long Beach, CA, USA). Long Beach, 2019. P. 6105–6114. https://doi.org/10.48550/arxiv.1905.11946.
- Tan M., Le Q.V. EfficientNetV2: Smaller models and faster training. Proc. 38th International Conference on Machine Learning (ICML 2021) (18–24 July 2021, virtual event). P. 10096–10106. https://doi.org/10.48550/arxiv.2104.00298.
- Li S., Tan M., Pang R., Li A., Cheng L., Le Q., Jouppi N. Searching for fast model families on datacenter accelerators. Proc. 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2021) (20-25 June 2021, Nashville, TN, USA). Nashville, 2021. P. 8081–8091. https://doi.org/10.1109/CVPR46437.2021.00799 .
- Ridnik T., Lawen H., Noy A., Ben E., Sharir B.G., Friedman I. TResNet: High performance GPU-dedicated architecture. Proc. 2021 IEEE Winter Conference on Applications of Computer Vision (WACV) (03-08 January 2021, Waikoloa, HI, USA). Waikoloa, 2021. P. 1399–1408. https://doi.org/10.1109/WACV48630.2021.00144)
- Oord A., Dieleman S., Zen H., Simonyan K., Vinyals O., Graves A., Kalchbrenner N., Senior A., Kavukcuoglu K. Wavenet: A generative model for raw audio. 2016. P. 1–15. https://doi.org/10.48550/arxiv.1609.03499
- Shapovalova S., Moskalenko Y. Methods for increasing the classification accuracy based on modifications of the basic architecture of convolutional neural networks. ScienceRise. 2020. N 6 (71). P. 10–16. https://doi.org/10.21303/2313-8416.2020.001550 .
- Wavenet with SHIFTED-RFC Proba and CBR. Kaggle/Code. URL: https://www.kaggle.com/nxrprime/wavenet-with-shifted-rfc-proba-and-cbr .
- University of Liverpool — Ion Switching. Kaggle/Competitions. URL: https://www.kaggle.com/c/liverpool-ion-switching .
- Lin T.-Y., Goyal P., Girshick R., He K., Dollїr P. Focal loss for dense object detection. Proc. 2017 IEEE International Conference on Computer Vision (ICCV) (22-29 October 2017, Venice, Italy). Venice, 2017. P. 2999-3007. https://doi.org/10.1109/ICCV.2017.324.
- Salehi S.S.M., Erdogmus D., Gholipour A. Tversky loss function for image segmentation using 3D fully convolutional deep networks. Proc. MLMI 2017. In: Lecture Notes in Computer Science. Wang Q., Shi Y., Suk HI., Suzuki K. (Eds). 2017. Vol. 10541. P. 379–387. https://doi.org/10.1007/978-3-319-67389-9_44 .
- Sorensen T. A method of establishing groups of equal amplitude in plant sociology based on similarity of species content and its application to analyses of the vegetation on Danish commons. Biologiske Skrifter/Kongelige Danske Videnskabernes Selskab. 1948. Vol. V, N 4. P. 1–34.
- Quick, Draw! Doodle Recognition Challenge. Kaggle/Competitions. URL: https://www.kaggle.com/c/quickdraw-doodle-recognition .
