Bayesian learning for neural network compression

Jen Tzung Chien, Su Ting Chang

研究成果: Conference contribution同行評審

摘要

Quantization on weight parameters in neural network training plays a key role for model compression in mobile devices. This paper presents a general M-ary adaptive quantization in construction of Bayesian neural networks. The trade-off between model capacity and memory cost is adjustable. The stochastic weight parameters are faithfully reflected. A compact model is trained to achieve robustness to model uncertainty due to heterogeneous data collection. To minimize the performance loss, the representation levels in quantized neural network are estimated by maximizing the variational lower bound of log likelihood conditioned on M-ary quantization. Bayesian learning is formulated by using the multi-spike-and- slab prior for quantization levels. An adaptive quantization is derived to implement a flexible parameter space for learning representation which is applied for object recognition. Experiments on image recognition show the merit of this Bayesian model compression for M-ary quantized neural networks.

原文English
主出版物標題2020 IEEE International Conference on Multimedia and Expo Workshops, ICMEW 2020
發行者Institute of Electrical and Electronics Engineers Inc.
ISBN(電子)9781728114859
DOIs
出版狀態Published - 七月 2020
事件2020 IEEE International Conference on Multimedia and Expo Workshops, ICMEW 2020 - London, United Kingdom
持續時間: 6 七月 202010 七月 2020

出版系列

名字2020 IEEE International Conference on Multimedia and Expo Workshops, ICMEW 2020

Conference

Conference2020 IEEE International Conference on Multimedia and Expo Workshops, ICMEW 2020
國家United Kingdom
城市London
期間6/07/2010/07/20

指紋 深入研究「Bayesian learning for neural network compression」主題。共同形成了獨特的指紋。

引用此