Deep Reinforcement Learning for Video Prediction

Yung Han Ho, Chuan Yuan Cho, Wen Hsiao Peng

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

This paper introduces a hybrid video prediction scheme that combines the classic parametric overlapped block motion compensation (POBMC) technique with neural networks. Most learning-based video prediction methods rely on a black-box-like model for either direct generation of future video frames or estimation of a dense motion field. The model complexity often increases drastically with frame resolution. Departing from pure black-box approaches, this paper leverages the theoretically-grounded POBMC in a reinforcement learning framework to estimate a sparse motion field for future frame warping. Two neural networks are trained to identify critical points in the motion field for motion estimation. We train our model on 10k unlabeled frames in KITTI dataset and achieve the state-of-the-art SSIM score of 0.923 on CaltechPed and an average SSIM scroe of 0.856 on Common Intermediate Format (CIF) standard sequences.

Original languageEnglish
Title of host publication2019 IEEE International Conference on Image Processing, ICIP 2019 - Proceedings
PublisherIEEE Computer Society
Pages604-608
Number of pages5
ISBN (Electronic)9781538662496
DOIs
StatePublished - Sep 2019
Event26th IEEE International Conference on Image Processing, ICIP 2019 - Taipei, Taiwan
Duration: 22 Sep 201925 Sep 2019

Publication series

NameProceedings - International Conference on Image Processing, ICIP
Volume2019-September
ISSN (Print)1522-4880

Conference

Conference26th IEEE International Conference on Image Processing, ICIP 2019
CountryTaiwan
CityTaipei
Period22/09/1925/09/19

Keywords

  • deep video prediction
  • Reinforcement learning

Fingerprint Dive into the research topics of 'Deep Reinforcement Learning for Video Prediction'. Together they form a unique fingerprint.

  • Cite this

    Ho, Y. H., Cho, C. Y., & Peng, W. H. (2019). Deep Reinforcement Learning for Video Prediction. In 2019 IEEE International Conference on Image Processing, ICIP 2019 - Proceedings (pp. 604-608). [8803825] (Proceedings - International Conference on Image Processing, ICIP; Vol. 2019-September). IEEE Computer Society. https://doi.org/10.1109/ICIP.2019.8803825