Abstract
Deep neural networks are widely used in data processing and analysis due to their high performance. However, a major drawback of such models is their large size, which imposes substantial memory requirements on storage, transmission, and computational systems. These demands can be significantly reduced through structural sparsification, including the approximation of convolutional kernels and the use of sparse neural layers. In this paper, we propose a unified approach that combines these techniques to compress a two-stage deep neural model composed of convolutional and fully connected networks, as commonly used in image classification tasks. The aim is to experimentally evaluate the extent to which this combination enables model compression while maintaining acceptable classification performance. Extensive experiments demonstrate that the proposed method can reduce the size of the model by up to 14 times, while improving the classification accuracy by approximately 1.2%, likely due to its regularization effect.
Paper Type
Short Paper
DOI
10.62036/ISD.2026.61
Efficient Compression of Deep Neural Networks via Structural Sparsification
Deep neural networks are widely used in data processing and analysis due to their high performance. However, a major drawback of such models is their large size, which imposes substantial memory requirements on storage, transmission, and computational systems. These demands can be significantly reduced through structural sparsification, including the approximation of convolutional kernels and the use of sparse neural layers. In this paper, we propose a unified approach that combines these techniques to compress a two-stage deep neural model composed of convolutional and fully connected networks, as commonly used in image classification tasks. The aim is to experimentally evaluate the extent to which this combination enables model compression while maintaining acceptable classification performance. Extensive experiments demonstrate that the proposed method can reduce the size of the model by up to 14 times, while improving the classification accuracy by approximately 1.2%, likely due to its regularization effect.
Recommended Citation
Cichońska, L., Puchala, D. & Stokfiszewski, K.(2026). Efficient Compression of Deep Neural Networks via Structural Sparsification. In M. Valenta, B. Mannová, R. Pergl, A. Przybylek, M. Lang, H. Linger, C. Schneider, N. Iivari, & E. Insfran (Eds.), Making ISD Sustainable: Reloaded with AI and Automation (ISD2026 Proceedings). Prague, Czech Republic: Czech Technical University in Prague. ISBN: 978-80-01-07585-2. https://doi.org/10.62036/ISD.2026.61