Minimum mean-squared error estimation of stochastic processes by mutual entropy

Bing-Fei Wu*

*Corresponding author for this work

Research output: Contribution to journalArticle


An upper bound on the correlation coefficients in terms of the mutual entropy is developed for a general estimation problem. This upper bound may be reached by means of a nonlinear transformation such that, after the transformation, the processes are jointly gaussian. In order to minimize globally the mean-squared estimation error, we use an approach based on the Calculus of Variations to find the vector nonlinear functions whose elements turn out to be the eigenfunctions of two vector integral operators that could be concurrently solved from two vector integral equations. The relationship between the minimum mean-squared error (MMSE) and the mutual entropy is discussed. Since the MMSE error is linear with the number of experimental data, we consider the mutual entropy rate, which is the average of the mutual entropy, to evaluate the average MMSE error. This rate is related to the average MMSE error and it is show that a mutual entropy rate of 0·5 is the critical threshold for the MMSE problem. Moreover, given a correlation coefficient, ergodic and jointly gaussian signals can be generated easily by computer. An approach to create these signals is also presented.

Original languageEnglish
Pages (from-to)1391-1402
Number of pages12
JournalInternational Journal of Systems Science
Issue number12
StatePublished - 1 Jan 1996

Fingerprint Dive into the research topics of 'Minimum mean-squared error estimation of stochastic processes by mutual entropy'. Together they form a unique fingerprint.

  • Cite this