en>Mild Bill Hiccup: spelling

2012-06-03T19:02:26Z

spelling

New page

In [[statistics]], the '''Chapman–Robbins bound''' or '''Hammersley–Chapman–Robbins bound''' is a lower bound on the [[variance]] of [[estimator]]s of a deterministic parameter. It is a generalization of the [[Cramér–Rao bound]]; compared to the Cramér–Rao bound, it is both tighter and applicable to a wider range of problems. However, it is usually more difficult to compute.

The bound was independently discovered by [[John Hammersley]] in 1950,<ref>{{Citation
| last = Hammersley | first = J. M. |authorlink=John Hammersley
| title = On estimating restricted parameters
| journal = [[Journal of the Royal Statistical Society]], Series B
| volume = 12 | issue = 2 | pages = 192–240 | year = 1950
| mr = 40631
| jstor = 2983981
}}</ref> and by Douglas Chapman and [[Herbert Robbins]] in 1951.<ref>{{Citation
| last = Chapman | first = D. G.
| last2 = Robbins | first2 = H. | author2-link = Herbert Robbins
| title = Minimum variance estimation without regularity assumptions
| journal = [[Annals of Mathematical Statistics]]
| volume = 22 | issue =4 | pages =581–586 | year =1951
| doi = 10.1214/aoms/1177729548
| mr = 44084
| jstor = 2236927
}}</ref>

== Statement ==
Let {{nowrap|''θ'' ∈ '''R'''''n''}} be an unknown, deterministic parameter, and let {{nowrap|''X'' ∈ '''R'''''k''}} be a random variable, interpreted as a measurement of ''θ''. Suppose the [[probability density function]] of ''X'' is given by ''p''(''x''; ''θ''). It is assumed that ''p''(''x''; ''θ'') is well-defined and that {{nowrap|''p''(''x''; ''θ'') > 0}} for all values of ''x'' and ''θ''.

Suppose ''δ''(''X'') is an [[bias (statistics)|unbiased]] estimate of an arbitrary scalar function {{nowrap|''g'': '''R'''''n'' → '''R'''}} of ''θ'', i.e.,

:<math>E\{\delta(X)\} = g(\theta)\text{ for all }\theta.\,</math>

The Chapman–Robbins bound then states that

:<math>\mathrm{Var}(\delta(X)) \ge \sup_\Delta \frac{\left[ g(\theta+\Delta) - g(\theta) \right]^2}{E_{\theta} \left[ \tfrac{p(X;\theta+\Delta)}{p(X;\theta)} - 1 \right]^2}.</math>

Note that the denominator in the lower bound above is exactly the [[F-divergence#Instances_of_f-divergences|<math> \chi^2</math>-divergence]] of <math> p(\cdot; \theta+\Delta)</math> with respect to <math> p(\cdot; \theta)</math>.

== Relation to Cramér–Rao bound ==
The Chapman–Robbins bound converges to the [[Cramér–Rao bound]] when {{nowrap|Δ → 0}}, assuming the regularity conditions of the Cramér–Rao bound hold. This implies that, when both bounds exist, the Chapman–Robbins version is always at least as tight as the Cramér–Rao bound; in many cases, it is substantially tighter.

The Chapman–Robbins bound also holds under much weaker regularity conditions. For example, no assumption is made regarding differentiability of the probability density function ''p''(''x''; ''θ''). When ''p''(''x''; ''θ'') is non-differentiable, the [[Fisher information]] is not defined, and hence the Cramér–Rao bound does not exist.

== See also ==
* [[Cramér–Rao bound]]
* [[Estimation theory]]

== References ==
{{reflist}}

== Further reading ==
* {{Citation
| last = Lehmann
| first = E. L.
| coauthors = Casella, G.
| title = Theory of Point Estimation
| year = 1998
| publisher = Springer
| isbn = 0-387-98502-6
| edition = 2nd
| pages = 113–114 }}

{{DEFAULTSORT:Chapman-Robbins bound}}
[[Category:Statistical inequalities]]
[[Category:Estimation theory]]

Complementary monopoly - Revision history

en>Mild Bill Hiccup: spelling