Multi-view and multi-objective semi-supervised learning for HMM-based automatic speech recognition

Xiaodong Cui*, Jing Huang, Jen-Tzung Chien

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

24 Scopus citations


Current hidden Markov acoustic modeling for large-vocabulary continuous speech recognition (LVCSR) heavily relies on the availability of abundant labeled transcriptions. Given that speech labeling is both expensive and time-consuming while there is a huge amount of unlabeled data easily available nowadays, the semi-supervised learning (SSL) from both labeled and unlabeled data aiming to reduce the development cost for LVCSR becomes more important than ever. In this paper, a new SSL approach is proposed which exploits the cross-view transfer learning for LVCSR through a committee machine consisting of multiple views learned from different acoustic features and randomized decision trees. In addition, a multi-objective learning scheme is developed in each view by maximizing a hybrid information-theoretic criterion which is established by the relative entropy between labeled data and their labels and the entropy of unlabeled data. The multi-objective scheme is then generalized to a unified SSL framework which can be interpreted into a variety of learning strategies under different weighting schemes. Experiments conducted on English Broadcast News using 50 hours of transcribed speech with 50 hours and 150 hours of untranscribed speech show the benefits of proposed approaches.

Original languageEnglish
Article number6175108
Pages (from-to)1923-1935
Number of pages13
JournalIEEE Transactions on Audio, Speech and Language Processing
Issue number7
StatePublished - 16 May 2012


  • Acoustic modeling
  • automatic speech recognition
  • multi-objective learning
  • multi-view committee machine
  • semi-supervised learning (SSL)

Fingerprint Dive into the research topics of 'Multi-view and multi-objective semi-supervised learning for HMM-based automatic speech recognition'. Together they form a unique fingerprint.

Cite this