Amortized Mixture Prior for Variational Sequence Generation

Jen-Tzung Chien, Chih Jung Tsai

研究成果: Conference contribution同行評審

摘要

Variational autoencoder (VAE) is a popular latent variable model for data generation. However, in natural language applications, VAE suffers from the posterior collapse in optimization procedure where the model posterior likely collapses to a standard Gaussian prior which disregards latent semantics from sequence data. The recurrent decoder accordingly generates du-plicate or noninformative sequence data. To tackle this issue, this paper adopts the Gaussian mixture prior for latent variable, and simultaneously fulfills the amortized regularization in encoder and skip connection in decoder. The noise robust prior, learned from the amortized encoder, becomes semantically meaningful. The prediction of sequence samples, due to skip connection, becomes contextually precise at each time. The amortized mixture prior (AMP) is then formulated in construction of variational recurrent autoencoder (VRAE) for sequence generation. Experiments on different tasks show that AMP-VRAE can avoid the posterior collapse, learn the meaningful latent features and improve the inference and generation for semantic representation.

原文English
主出版物標題2020 International Joint Conference on Neural Networks, IJCNN 2020 - Proceedings
發行者Institute of Electrical and Electronics Engineers Inc.
ISBN(電子)9781728169262
DOIs
出版狀態Published - 七月 2020
事件2020 International Joint Conference on Neural Networks, IJCNN 2020 - Virtual, Glasgow, United Kingdom
持續時間: 19 七月 202024 七月 2020

出版系列

名字Proceedings of the International Joint Conference on Neural Networks

Conference

Conference2020 International Joint Conference on Neural Networks, IJCNN 2020
國家United Kingdom
城市Virtual, Glasgow
期間19/07/2024/07/20

指紋 深入研究「Amortized Mixture Prior for Variational Sequence Generation」主題。共同形成了獨特的指紋。

引用此