Sequential learning and regularization in variational recurrent autoencoder

Jen-Tzung Chien, Chih Jung Tsai

研究成果: Conference contribution同行評審

摘要

Latent variable model based on variational autoencoder (VAE) is influential in machine learning for signal processing. VAE basically suffers from the issue of posterior collapse in sequential learning procedure where the variational posterior easily collapses to a prior as standard Gaussian. Latent semantics are then neglected in optimization process. The recurrent decoder therefore generates noninformative or repeated sequence data. To capture sufficient latent semantics from sequence data, this study simultaneously fulfills an amortized regularization for encoder, extends a Gaussian mixture prior for latent variable, and runs a skip connection for decoder. The noise robust prior, learned from the amortized encoder, is likely aware of temporal features. A variational prior based on the amortized mixture density is formulated in implementation of variational recurrent autoencoder for sequence reconstruction and representation. Owing to skip connection, the sequence samples are continuously predicted in decoder with contextual precision at each time step. Experiments on language model and sentiment classification show that the proposed method mitigates the issue of posterior collapse and learns the meaningful latent features to improve the inference and generation for semantic representation.

原文English
主出版物標題28th European Signal Processing Conference, EUSIPCO 2020 - Proceedings
發行者European Signal Processing Conference, EUSIPCO
頁面1613-1617
頁數5
ISBN(電子)9789082797053
DOIs
出版狀態Published - 24 一月 2021
事件28th European Signal Processing Conference, EUSIPCO 2020 - Amsterdam, Netherlands
持續時間: 24 八月 202028 八月 2020

出版系列

名字European Signal Processing Conference
2021-January
ISSN(列印)2219-5491

Conference

Conference28th European Signal Processing Conference, EUSIPCO 2020
國家Netherlands
城市Amsterdam
期間24/08/2028/08/20

指紋 深入研究「Sequential learning and regularization in variational recurrent autoencoder」主題。共同形成了獨特的指紋。

引用此