Mining of association patterns for language modeling

Jen-Tzung Chien, Hung Ying Chen

Research output: Contribution to conferencePaperpeer-review

3 Scopus citations


Language modeling using n-gram is popular for speech recognition and many other applications. The conventional n-gram suffers from the insufficiencies of training data, domain knowledge and long distance language dependencies. This paper presents a new approach to mining long distance word associations and incorporating their mutual information into language models. We aim to discover the associations of multiple distant words from training corpus. An efficient algorithm is exploited to merge the frequent word subsets and construct the association patterns. The resulting association pattern n-gram is general with a special realization to trigger pair n-gram where only associations of two distant words are considered. To improve the modeling, we further compensate the weaknesses of sparse training data via parameter smoothing and domain mismatch via online adaptive learning. The proposed association pattern n-gram and several hybrid models are successfully applied for speech recognition. We also find that the incorporation of mutual information of association patterns can significantly reduce the perplexities of language models.

Original languageEnglish
Number of pages4
StatePublished - 1 Jan 2004
Event8th International Conference on Spoken Language Processing, ICSLP 2004 - Jeju, Jeju Island, Korea, Republic of
Duration: 4 Oct 20048 Oct 2004


Conference8th International Conference on Spoken Language Processing, ICSLP 2004
CountryKorea, Republic of
CityJeju, Jeju Island

Fingerprint Dive into the research topics of 'Mining of association patterns for language modeling'. Together they form a unique fingerprint.

Cite this