en>Marianne.rooman: /* Software */

2015-01-05T13:09:22Z

Software

en>Monkbot: Fix CS1 deprecated date parameter errors

2014-01-28T22:12:11Z

Fix CS1 deprecated date parameter errors

New page

In [[statistics]] a '''uniformly minimum-variance unbiased estimator''' or '''minimum-variance unbiased estimator (UMVUE''' or '''MVUE)''' is an unbiased estimator that has ''lower variance'' than ''any other'' unbiased estimator for ''all possible'' values of the parameter.

For practical statistics problems, it is important to determine the UMVUE if one exists, since less-than-optimal procedures would naturally be avoided, other things being equal. This has led to substantial development of statistical theory related to the problem of optimal estimation. While the particular specification of "optimal" here — requiring [[unbiasedness]] and measuring "goodness" using the [[variance]] — may not always be what is wanted for any given practical situation, it is one where useful and generally applicable results can be found.

==Definition==

Consider estimation of <math>g(\theta)</math> based on data <math>X_1, X_2, \ldots, X_n</math> i.i.d. from some member of a family of densities <math> p_\theta, \theta \in \Omega</math>, where <math>\Omega</math> is the parameter space. An unbiased estimator <math>\delta(X_1, X_2, \ldots, X_n)</math> of <math> g(\theta) </math> is ''UMVUE'' if <math> \forall \theta \in \Omega</math>,

:<math> \mathrm{var}(\delta(X_1, X_2, \ldots, X_n)) \leq \mathrm{var}(\tilde{\delta}(X_1, X_2, \ldots, X_n)) </math>

for any other unbiased estimator <math> \tilde{\delta}. </math>

If an unbiased estimator of <math> g(\theta) </math> exists, then one can prove there is an essentially unique MVUE. Using the [[Rao–Blackwell theorem]] one can also prove that determining the MVUE is simply a matter of finding a [[complete statistic|complete]] [[sufficient statistic|sufficient]] statistic for the family <math>p_\theta, \theta \in \Omega </math> and conditioning ''any'' unbiased estimator on it.

Further, by the [[Lehmann–Scheffé theorem]], an unbiased estimator that is a function of a complete, sufficient statistic is the UMVUE estimator.

Put formally, suppose <math>\delta(X_1, X_2, \ldots, X_n)</math> is unbiased for <math>g(\theta)</math>, and that <math>T</math> is a complete sufficient statistic for the family of densities. Then

:<math> \eta(X_1, X_2, \ldots, X_n) = \mathrm{E}(\delta(X_1, X_2, \ldots, X_n)|T)\,</math>

is the MVUE for <math>g(\theta). </math>

A [[Bayesian statistics|Bayesian]] analog is a [[Bayes estimator]], particularly with [[minimum mean square error]] (MMSE).

==Estimator selection==
An [[efficient estimator]] need not exist, but if it does and if it is unbiased,
it is the MVUE. Since the [[mean squared error]] (MSE) of an estimator ''δ'' is

:<math> \operatorname{MSE}(\delta) = \mathrm{var}(\delta) +[ \mathrm{bias}(\delta)]^{2}\ </math>

the MVUE minimizes MSE ''among unbiased estimators''. In some cases biased estimators have lower MSE because they have a smaller variance than does any unbiased estimator; see [[estimator bias]].

==Example==
Consider the data to be a single observation from an [[Absolutely continuous random variable|absolutely continuous distribution]] on <math>\mathbb{R} </math>
with density

:<math> p_\theta(x) = \frac{ \theta e^{-x} }{(1 + e^{-x})^{\theta + 1} } </math>

and we wish to find the UMVU estimator of

:<math> g(\theta) = \frac{1}{\theta^{2}} </math>

First we recognize that the density can be written as

:<math> \frac{ e^{-x} } { 1 + e^{-x} } \exp( -\theta \log(1 + e^{-x}) + \log(\theta)) </math>

Which is an exponential family with sufficient statistic <math>T = \mathrm{log}(1 + e^{-x})</math>. In
fact this is a full rank exponential family, and therefore <math> T </math> is complete sufficient. See [[exponential family]]
for a derivation which shows

: <math> \mathrm{E}(T) = -\frac{1}{\theta},\quad \mathrm{var}(T) = \frac{1}{\theta^{2}} </math>

Therefore

:<math> \mathrm{E}(T^2) = \frac{2}{\theta^{2}} </math>

Clearly <math> \delta(X) = \frac{T^2}{2} </math> is unbiased, thus the UMVU estimator is

:<math> \eta(X) = \mathrm{E}(\delta(X) | T) = \mathrm{E} \left( \left. \frac{T^2}{2} \,\right|\, T \right) = \frac{T^{2}}{2} = \frac{\log(1 + e^{-X})^{2}}{2}</math>

This example illustrates that an unbiased function of the complete sufficient statistic will be UMVU.

== Other examples ==
* For a normal distribution with unknown mean and variance, the [[sample mean]] and (unbiased) [[sample variance]] are the MVUEs for the population mean and population variance.
*:However, the [[sample standard deviation]] is not unbiased for the population standard deviation – see [[unbiased estimation of standard deviation]].
*:Further, for other distributions the sample mean and sample variance are not in general MVUEs – for a [[Uniform distribution (continuous)|uniform distribution]] with unknown upper and lower bounds, the [[mid-range]] is the MVUE for the population mean.
* If ''k'' exemplars are chosen (without replacement) from a [[discrete uniform distribution]] over the set {1, 2, ..., ''N''} with unknown upper bound ''N'', the MVUE for ''N'' is
:: <math>\frac{k+1}{k} m - 1,</math>
:where ''m'' is the [[sample maximum]]. This is a scaled and shifted (so unbiased) transform of the sample maximum, which is a sufficient and complete statistic. See [[German tank problem]] for details.

== See also ==
* [[Best linear unbiased estimator]] (BLUE)
* [[Lehmann–Scheffé theorem]]
* [[U-statistic]]

=== Bayesian analogs ===
* [[Bayes estimator]]
* [[Minimum mean square error]] (MMSE)

== References ==
* {{cite book
| last = Keener
| first = Robert W.
| title = Statistical Theory: Notes for a Course in Theoretical Statistics
| publisher = Springer
| date = 2006
| pages = 47–48, 57–58 }}
* {{cite book
| last = Voinov V. G.,
| first = Nikulin M.S.
| title = Unbiased estimators and their applications, Vol.1: Univariate case
| publisher = Kluwer Academic Publishers
| date = 1993
| pages = 521p }}

[[Category:Estimation theory]]

Protein design - Revision history

en>Marianne.rooman: /* Software */

en>Monkbot: Fix CS1 deprecated date parameter errors