M-ARY quantized neural networks

Jen Tzung Chien, Su Ting Chang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Scopus citations

Abstract

Parameter quantization is crucial for model compression. This paper generalizes the binary and ternary quantizations to M-ary quantization for adaptive learning of the quantized neural networks. To compensate the performance loss, the representation values and the quantization partitions of model parameters are jointly trained to optimize the resolution of gradients for parameter updating where the non-differentiable function in back-propagation algorithm is tackled. An asymmetric quantization is implemented. The restriction in parameter quantization is sufficiently relaxed. The resulting M-ary quantization scheme is general and adaptive with different M. Training of the M-ary quantized neural network (MQNN) can be tuned to balance the tradeoff between system performance and memory storage. Experimental results show that MQNN is able to achieve comparable image classification performance with full-precision neural network (FPNN), but the memory storage can be far less than that in FPNN.

Original languageEnglish
Title of host publication2020 IEEE International Conference on Multimedia and Expo, ICME 2020
PublisherIEEE Computer Society
ISBN (Electronic)9781728113319
DOIs
StatePublished - Jul 2020
Event2020 IEEE International Conference on Multimedia and Expo, ICME 2020 - London, United Kingdom
Duration: 6 Jul 202010 Jul 2020

Publication series

NameProceedings - IEEE International Conference on Multimedia and Expo
Volume2020-July
ISSN (Print)1945-7871
ISSN (Electronic)1945-788X

Conference

Conference2020 IEEE International Conference on Multimedia and Expo, ICME 2020
CountryUnited Kingdom
CityLondon
Period6/07/2010/07/20

Keywords

  • Adaptive quantization
  • Binarized neural network
  • Model compression
  • Quantized neural network

Fingerprint Dive into the research topics of 'M-ARY quantized neural networks'. Together they form a unique fingerprint.

  • Cite this

    Chien, J. T., & Chang, S. T. (2020). M-ARY quantized neural networks. In 2020 IEEE International Conference on Multimedia and Expo, ICME 2020 [9102751] (Proceedings - IEEE International Conference on Multimedia and Expo; Vol. 2020-July). IEEE Computer Society. https://doi.org/10.1109/ICME46284.2020.9102751