DNN compression approach based on Bayesian optimization tensor ring decomposition

Min Zhang; Jie Liu; Changhong Shi; Ning Zhang; Weirong Liu

doi:10.1117/12.3021187

25 March 2024 DNN compression approach based on Bayesian optimization tensor ring decomposition

Min Zhang, Jie Liu, Changhong Shi, Ning Zhang, Weirong Liu

Proceedings Volume 13089, Fifteenth International Conference on Graphics and Image Processing (ICGIP 2023); 130890N (2024) https://doi.org/10.1117/12.3021187
Event: Fifteenth International Conference on Graphics and Image Processing (ICGIP 2023), 2023, Suzhou, China

Abstract

Tensor ring (TR) decomposition is an effective method to achieve deep neural network (DNN) compression. However, there are two problems with TR decomposition: setting TR rank to equal in TR decomposition and selecting rank through an iterative process is time-consuming. To address the two problems, A TR network compression method by Bayesian optimization (TR-BO) is proposed. TR-BO involves selecting rank via Bayesian optimization, compressing the neural network layer via TR decomposition using rank obtained in the previous step, and, finally, further fine-tuning the compressed model to overcome some of the performance loss due to compression. Experimental results show that TR-BO achieves the best results in terms of Top-1 accuracy, parameter, and training time. For example, on the CIFAR-10 dataset Resnet20 network, TR-BO-1 achieves 87.67% accuracy with a compression ratio of 13.66 and a running time of only 2.4 hours. Furthermore, TR-BO has achieved state-of-the-art performance on the CIFAR-10/100 benchmark tests.

(2024) Published by SPIE. Downloading of the abstract is permitted for personal use only.

Citation Download Citation

Min Zhang, Jie Liu, Changhong Shi, Ning Zhang, and Weirong Liu "DNN compression approach based on Bayesian optimization tensor ring decomposition", Proc. SPIE 13089, Fifteenth International Conference on Graphics and Image Processing (ICGIP 2023), 130890N (25 March 2024); https://doi.org/10.1117/12.3021187

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available