Stochastic fusion for multi-stream neural network in video classification

Yu Min Huang*, Huan Hsin Tsengt, Jen-Tzung Chien

*Corresponding author for this work

研究成果: Conference contribution

摘要

Spatial image and optical flow provide complementary information for video representation and classification. Traditional methods separately encode two stream signals and then fuse them at the end of streams. This paper presents a new multi-stream recurrent neural network where streams are tightly coupled at each time step. Importantly, we propose a stochastic fusion mechanism for multiple streams of video data based on the Gumbel samples to increase the prediction power. A stochastic backpropagation algorithm is implemented to carry out a multi-stream neural network with stochastic fusion based on a joint optimization of convolutional encoder and recurrent decoder. Experiments on UCF101 dataset illustrate the merits of the proposed stochastic fusion in recurrent neural network in terms of interpretation and classification performance.

原文English
主出版物標題2019 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, APSIPA ASC 2019
發行者Institute of Electrical and Electronics Engineers Inc.
頁面69-74
頁數6
ISBN(電子)9781728132488
DOIs
出版狀態Published - 十一月 2019
事件2019 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, APSIPA ASC 2019 - Lanzhou, China
持續時間: 18 十一月 201921 十一月 2019

出版系列

名字2019 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, APSIPA ASC 2019

Conference

Conference2019 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, APSIPA ASC 2019
國家China
城市Lanzhou
期間18/11/1921/11/19

指紋 深入研究「Stochastic fusion for multi-stream neural network in video classification」主題。共同形成了獨特的指紋。

  • 引用此

    Huang, Y. M., Tsengt, H. H., & Chien, J-T. (2019). Stochastic fusion for multi-stream neural network in video classification. 於 2019 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, APSIPA ASC 2019 (頁 69-74). [9023327] (2019 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference, APSIPA ASC 2019). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/APSIPAASC47483.2019.9023327