|
|
| Line 1: |
Line 1: |
| The '''Wiener–Khinchin theorem''' (also known as the '''Wiener–Khintchine theorem''' and sometimes as the '''Wiener–Khinchin–Einstein theorem''' or the '''Khinchin–Kolmogorov theorem''') states that the [[autocorrelation]] function of a [[wide-sense-stationary random process]] has a spectral decomposition given by the [[power spectrum]] of that process.<ref>{{cite book | title = The Analysis of Time Series—An Introduction | author = C. Chatfield | edition = fourth | publisher = Chapman and Hall, London | year = 1989 | isbn=0-412-31820-2 | pages = 94–95}}</ref><ref>{{cite book | title = Time Series | author = Norbert Wiener | publisher = M.I.T. Press, Cambridge, Massachusetts | year = 1964 | page = 42}}</ref><ref>Hannan, E.J., "Stationary Time Series", in: John Eatwell, Murray Milgate, and Peter Newman, editors, ''The New Palgrave: A Dictionary of Economics. Time Series and Statistics'', Macmillan, London, 1990, p. 271.</ref><ref>{{cite book | title = Echo Signal Processing | author = Dennis Ward Ricker | publisher = Springer | year = 2003 | isbn = 1-4020-7395-X | url = http://books.google.com/books?id=NF2Tmty9nugC&pg=PA23&dq=%22power+spectral+density%22+%22energy+spectral+density%22&lr=&as_brr=3&ei=HZMvSPSWFZyStwPWsfyBAw&sig=1ZZcHwxXkErvNXtAHv21ijTXoP8#PPA23,M1 }}</ref><ref>{{cite book | title = Digital and Analog Communications Systems | author = Leon W. Couch II | edition = sixth | publisher = Prentice Hall, New Jersey | year = 2001 | isbn=0-13-522583-3| pages = 406–409}}</ref><ref>{{cite book | title = Wireless Technologies: Circuits, Systems, and Devices | author = Krzysztof Iniewski | publisher = CRC Press | year = 2007 | isbn = 0-8493-7996-2 | url = http://books.google.com/books?id=JJXrpazX9FkC&pg=PA390&dq=Wiener-Khinchin-Einstein&ei=1SxlSPGhB4jgsQPr5b3lDw&sig=ACfU3U2Phnk-zwJi57XrvNmdfosyg55FVA }}</ref><ref>{{cite book | title = Statistical Optics | author = Joseph W. Goodman | publisher = Wiley-Interscience | year = 1985 | isbn=0-471-01502-4}}</ref>
| | Andrew Simcox is the title his mothers and fathers gave him and he completely enjoys this name. Some time in the past he chose to reside in North Carolina and he doesn't plan on changing it. My live psychic reading ([http://netwk.hannam.ac.kr/xe/data_2/85669 http://netwk.hannam.ac.kr/xe/data_2/85669]) husband doesn't like it the way I do but what I truly like performing is caving but I don't have the time recently. Credit authorising is how she tends to make a living.<br><br>Feel free to surf [http://clothingcarearchworth.com/index.php?document_srl=441551&mid=customer_review telephone psychic] to my page - tarot readings ([http://c045.danah.co.kr/home/index.php?document_srl=1356970&mid=qna c045.danah.co.kr]) |
| | |
| ==History==
| |
| [[Norbert Wiener]] first published this [[theorem]] in 1930;<ref>{{cite journal|last=Wiener|first=Norbert|title=Generalized Harmonic Analysis|journal=Acta Mathematica|year=1930|volume=55|pages=117–258}}</ref> [[Aleksandr Khinchin]] independently<ref>{{cite book|last=Nahin|first=Paul J.|title=Dr. Euler's Fabulous Formula: Cures Many Mathematical Ills|year=2011|publisher=Princeton University Press|isbn=9780691150376|pages=225|url=http://books.google.com/books?id=GvSg5HQ7WPcC&pg=PA225}}</ref> discovered the result and published it in 1934.<ref>{{cite journal|last=Khintchine|first=A.|title=Korrelationstheorie der stationären stochastischen Prozesse |journal=[[Mathematische Annalen]]|year=1934 |volume=109 |issue=1 |pages=604–615 |doi=10.1007/BF01449156 }}</ref> [[Albert Einstein]] had probably anticipated the idea in a brief two-page memo in 1914.<ref>{{cite book|title=The Legacy of Norbert Wiener: A Centennial Symposium (Proceedings of Symposia in Pure Mathematics)|page=95|first1=David|last1=Jerison|first2=Isadore Manuel|last2=Singer|first3=Daniel W.|last3=Stroock|publisher=American Mathematical Society|year=1997|isbn=0-8218-0415-4}}</ref>
| |
| | |
| ==The case of a continuous time process==
| |
| For continuous time, the Wiener—Khinchin theorem <ref>Hannan, E.J., "Stationary Time Series", in: John Eatwell, Murray Milgate, and Peter Newman, editors, ''The New Palgrave: A Dictionary of Economics. Time Series and Statistics'', Macmillan, London, 1990, p. 271.</ref><ref>{{cite book | title = The Analysis of Time Series—An Introduction | author = C. Chatfield | edition = fourth | publisher = Chapman and Hall, London | year = 1989 | isbn=0-412-31820-2 | pages = 94–95}}</ref> says that if <math> x </math> is a wide-sense stationary process such that its [[autocorrelation function]] (sometimes called [[autocovariance]]) defined in terms of statistical [[expected value]] E,
| |
| <math>r_{xx}(\tau) = \operatorname{E}\big[\, x(t)x^*(t-\tau) \, \big] \ </math>
| |
| exists and is finite at every lag <math> \tau </math>, then there exists a monotone function <math> F(f) </math> in the frequency domain <math> -\infty < f < \infty </math> such that
| |
| | |
| <math>
| |
| r_{xx} (\tau) = \int_{-\infty}^\infty e^{2\pi i\tau f} dF(f)
| |
| </math>
| |
| | |
| where the integral is a [[Stieltjes integral]]. This is a kind of spectral decomposition of the auto-correlation function. F is called the power spectral distribution function, and is a statistical distribution function. It is sometimes called the integrated spectrum.
| |
| | |
| (The asterisk denotes complex conjugate, and of course it can be omitted if the random process is real-valued.)
| |
| | |
| Note that the Fourier transform of <math>x(t)\,</math> does not exist in general, because stationary random functions are not generally either [[square-integrable function|square integrable]] or absolutely integrable. Nor is <math> r_{xx} </math> assumed to be absolutely integrable, so it need not have a Fourier transform, either.
| |
| | |
| But if <math> F(f) </math> is absolutely continuous, for example if the process is purely indeterministic, then one can define the power [[spectral density]] of <math>x(t)\,</math> by taking the derivative of <math> F </math>, putting <math> S_{xx}(f) = F'(f) </math> almost everywhere,<ref>{{cite book | title = The Analysis of Time Series—An Introduction | author = C. Chatfield | edition = fourth | publisher = Chapman and Hall, London | year = 1989 | isbn=0-412-31820-2 | page = 96}}</ref> and the theorem simplifies to
| |
| | |
| <math>
| |
| r_{xx} (\tau) = \int_{-\infty}^\infty S_{xx}(f) e^{2\pi i\tau f} df.
| |
| </math>
| |
| | |
| If now one assumes that r and S satisfy the necessary conditions for Fourier inversion to be valid, the Wiener—Khinchin theorem takes the simple form of saying that r and S are a Fourier transform pair, and
| |
| | |
| <math>
| |
| S_{xx}(f) = \int_{-\infty}^\infty r_{xx} (\tau) e^{-2\pi if\tau} d\tau.
| |
| </math>
| |
| | |
| ==The case of a discrete time process==
| |
| For the discrete-time case, the power spectral density of the function with discrete values <math>x[n]\,</math> is
| |
| :<math> S_{xx}(f)=\sum_{k=-\infty}^\infty r_{xx}[k]e^{-i(2\pi f) k} </math>,
| |
| | |
| where
| |
| | |
| :<math>r_{xx}[k] = \operatorname{E}\big[ \, x[n] x^*[n-k] \, \big] \ </math> | |
| | |
| is the discrete autocorrelation function of <math>x[n]\,</math>, provided this is absolutely integrable. Being a sampled and discrete-time sequence, the spectral density is periodic in the frequency domain.
| |
| | |
| ==Application==
| |
| The theorem is useful for analyzing [[LTI system theory|linear time-invariant systems]], LTI systems, when the inputs and outputs are not square integrable, so their Fourier transforms do not exist. A corollary is that the Fourier transform of the autocorrelation function of the output of an LTI system is equal to the product of the Fourier transform of the autocorrelation function of the input of the system times the squared magnitude of the Fourier transform of the system impulse response.<ref>
| |
| {{cite book
| |
| | title = Random signals and noise: a mathematical introduction
| |
| | author = Shlomo Engelberg
| |
| | publisher = CRC Press
| |
| | year = 2007
| |
| | isbn = 978-0-8493-7554-5
| |
| | page = 130
| |
| | url = http://books.google.com/books?id=Zl51JGnoww4C&pg=PA130
| |
| }}</ref>
| |
| This works even when the Fourier transforms of the input and output signals do not exist because these signals are not square integrable, so the system inputs and outputs cannot be directly related by the Fourier transform of the impulse response.
| |
| | |
| Since the Fourier transform of the autocorrelation function of a signal is the power spectrum of the signal, this corollary is equivalent to saying that the power spectrum of the output is equal to the power spectrum of the input times the power [[transfer function]].
| |
| | |
| This corollary is used in the parametric method for power spectrum estimation.
| |
| | |
| ==Discrepancies in terminology==
| |
| | |
| In many textbooks and in much of the technical literature it is tacitly assumed that Fourier inversion of the [[autocorrelation]] function and the power spectral density is valid, and the Wiener—Khinchin theorem is stated, very simply, as if it said that the Fourier transform of the autocorrelation function was equal to the power [[spectral density]], ignoring all questions of convergence.<ref>{{cite book | title = The Analysis of Time Series—An Introduction | author = C. Chatfield | edition = fourth | publisher = Chapman and Hall, London | year = 1989 | isbn=0-412-31820-2 | page = 98}}</ref> (Einstein is an example.)
| |
| But the theorem (as stated here), was applied by [[Norbert Wiener]] and [[Aleksandr Khinchin]] to the sample functions (signals) of [[wide-sense-stationary random process]]es, signals whose Fourier transforms do not exist.
| |
| The whole point of Wiener's contribution was to make sense of the spectral decomposition of the autocorrelation function of a sample function of a [[wide-sense-stationary random process]] even when the integrals for the Fourier transform and Fourier inversion do not make sense.
| |
| | |
| Some authors refer to R as the autocovariance function. They then proceed to normalise it, by dividing by R(0), to obtain what they refer to as the autocorrelation function.
| |
| | |
| ==References==
| |
| {{Reflist}}
| |
| | |
| ==Further reading==
| |
| *{{cite book |last=Brockwell |first=Peter A. |last2=Davis |first2=Richard J. |title=Introduction to Times Series and Forecasting |edition=Second |publisher=Springer-Verlag |location=New York |year=2002 |isbn=038721657X }}
| |
| *{{cite book |last=Chatfield |first=C. |title=The Analysis of Time Series—An Introduction |edition=Fourth |publisher=Chapman and Hall |location=London |year=1989 |isbn=0412318202 }}
| |
| *{{cite book |last=Fuller |first=Wayne |title=Introduction to Statistical Time Series |series=Wiley Series in Probability and Statistics |edition=Second |publisher=Wiley |location=New York |year=1996 |isbn=0471552399 }}
| |
| *{{cite paper |last=Wiener |first=Norbert |title=Extrapolation, Interpolation, and Smoothing of Stationary Time Series |publisher=Technology Press and Johns Hopkins Univ. Press |location=Cambridge, Massachusetts |year=1949 }} (a classified document written for the Dept. of War in 1943).
| |
| *{{cite book |last=Yaglom |first=A. M. |title=An Introduction to the Theory of Stationary Random Functions |publisher=Prentice-Hall |location=Englewood Cliffs, New Jersey |year=1962 }}
| |
| | |
| {{DEFAULTSORT:Wiener-Khinchin theorem}}
| |
| [[Category:Fourier analysis]]
| |
| [[Category:Signal processing]]
| |
| [[Category:Probability theorems]]
| |
Andrew Simcox is the title his mothers and fathers gave him and he completely enjoys this name. Some time in the past he chose to reside in North Carolina and he doesn't plan on changing it. My live psychic reading (http://netwk.hannam.ac.kr/xe/data_2/85669) husband doesn't like it the way I do but what I truly like performing is caving but I don't have the time recently. Credit authorising is how she tends to make a living.
Feel free to surf telephone psychic to my page - tarot readings (c045.danah.co.kr)