Helicoid

In statistics, the Neyman–Pearson lemma, named after Jerzy Neyman and Egon Pearson, states that when performing a hypothesis test between two point hypotheses H₀: θ = θ₀ and H₁: θ = θ₁, then the likelihood-ratio test which rejects H₀ in favour of H₁ when

$\mathrm{\Lambda }(x)=\frac{L({\theta }_0\mid x)}{L({\theta }_1\mid x)}\le \eta$

where

$P(\mathrm{\Lambda }(X)\le \eta \mid H_0)=\alpha$

is the most powerful test of size α for a threshold η. If the test is most powerful for all ${\theta }_1\in {\mathrm{\Theta }}_1$, it is said to be uniformly most powerful (UMP) for alternatives in the set ${\mathrm{\Theta }}_1$.

In practice, the likelihood ratio is often used directly to construct tests — see Likelihood-ratio test. However it can also be used to suggest particular test-statistics that might be of interest or to suggest simplified tests — for this one considers algebraic manipulation of the ratio to see if there are key statistics in it related to the size of the ratio (i.e. whether a large statistic corresponds to a small ratio or to a large one).

Proof

Define the rejection region of the null hypothesis for the NP test as

$R_{NP}=\{x:\frac{L({\theta }_0|x)}{L({\theta }_1|x)}\le \eta \}.$

Any other test will have a different rejection region that we define as $R_A$. Furthermore, define the probability of the data falling in region R, given parameter $\theta$ as

$P(R,\theta )=\underset{R}{\int }L(\theta |x)dx,$

For both tests to have size $\alpha$, it must be true that

$\alpha =P(R_{NP},{\theta }_0)=P(R_A,{\theta }_0).$

It will be useful to break these down into integrals over distinct regions:

$P(R_{NP},\theta )=P(R_{NP}\cap R_A,\theta )+P(R_{NP}\cap R_A^c,\theta ),$

and

$P(R_A,\theta )=P(R_{NP}\cap R_A,\theta )+P(R_{NP}^c\cap R_A,\theta ).$

Setting $\theta ={\theta }_0$ and equating the above two expression yields that

$P(R_{NP}\cap R_A^c,{\theta }_0)=P(R_{NP}^c\cap R_A,{\theta }_0).$

Comparing the powers of the two tests, $P(R_{NP},{\theta }_1)$ and $P(R_A,{\theta }_1)$, one can see that

$P(R_{NP},{\theta }_1)\ge P(R_A,{\theta }_1)⟺P(R_{NP}\cap R_A^c,{\theta }_1)\ge P(R_{NP}^c\cap R_A,{\theta }_1).$

Now by the definition of $R_{NP}$,

$P(R_{NP}\cap R_A^c,{\theta }_1)=\underset{R_{NP}\cap R_A^c}{\int }L({\theta }_1|x)dx\ge \frac{1}{\eta }\underset{R_{NP}\cap R_A^c}{\int }L({\theta }_0|x)dx=\frac{1}{\eta }P(R_{NP}\cap R_A^c,{\theta }_0)$

$=\frac{1}{\eta }P(R_{NP}^c\cap R_A,{\theta }_0)=\frac{1}{\eta }\underset{R_{NP}^c\cap R_A}{\int }L({\theta }_0|x)dx\ge \underset{R_{NP}^c\cap R_A}{\int }L({\theta }_1|x)dx=P(R_{NP}^c\cap R_A,{\theta }_1).$

Hence the inequality holds.

Example

Let $X_1,\dots ,X_n$ be a random sample from the $\mathcal{𝒩}(\mu ,{\sigma }^2)$ distribution where the mean $\mu$ is known, and suppose that we wish to test for $H_0:{\sigma }^2={\sigma }_0^2$ against $H_1:{\sigma }^2={\sigma }_1^2$. The likelihood for this set of normally distributed data is

$L({\sigma }^2;\mathbf{x})\propto {\left({\sigma }^2\right)}^{-n/2}\exp \{-\frac{\sum _{i=1}^n{(x_i-\mu )}^2}{2{\sigma }^2}\}.$

We can compute the likelihood ratio to find the key statistic in this test and its effect on the test's outcome:

$\mathrm{\Lambda }(\mathbf{x})=\frac{L({\sigma }_0^2;\mathbf{x})}{L({\sigma }_1^2;\mathbf{x})}={\left(\frac{{\sigma }_0^2}{{\sigma }_1^2}\right)}^{-n/2}\exp \{-\frac{1}{2}({\sigma }_0^{-2}-{\sigma }_1^{-2})\sum _{i=1}^n{(x_i-\mu )}^2\}.$

This ratio only depends on the data through $\sum _{i=1}^n{(x_i-\mu )}^2$. Therefore, by the Neyman–Pearson lemma, the most powerful test of this type of hypothesis for this data will depend only on $\sum _{i=1}^n{(x_i-\mu )}^2$. Also, by inspection, we can see that if ${\sigma }_1^2>{\sigma }_0^2$, then $\mathrm{\Lambda }(\mathbf{x})$ is a decreasing function of $\sum _{i=1}^n{(x_i-\mu )}^2$. So we should reject $H_0$ if $\sum _{i=1}^n{(x_i-\mu )}^2$ is sufficiently large. The rejection threshold depends on the size of the test. In this example, the test statistic can be shown to be a scaled Chi-square distributed random variable and an exact critical value can be obtained.

See also

• Statistical power

References

• Template:Cite doi
• cnx.org: Neyman–Pearson criterion

External links

• Cosma Shalizi, a professor of statistics at Carnegie Mellon University, gives an intuitive derivation of the Neyman–Pearson Lemma using ideas from economics