|
|
| Line 1: |
Line 1: |
| {{Probability fundamentals}}
| | Hello, I'm Anya, a 21 year old from Coonamble, Australia.<br>My hobbies include (but are not limited to) Cricket, Rugby league football and watching Sons of Anarchy.<br><br>Check out my weblog - [http://bbqmovil.com/?p=47/RK=0 how to get free fifa 15 coins] |
| | |
| In [[probability theory]], to say that two [[event (probability theory)|event]]s are '''independent''' (alternatively called '''statistically independent''' or '''stochastically independent''' )<ref name="Artificial Intelligence">{{cite book | last = Russell| first =Stuart| last2 = Norvig | first2 = Peter | title = Artificial Intelligence: A Modern Approach | page = 478 | publisher = [[Prentice Hall]] | year = 2002 | isbn = 0-13-790395-2}}</ref> means that the occurrence of one does not affect the probability of the other. Similarly, two [[random variable]]s are independent if the realization of one does not affect the probability distribution of the other.
| |
| | |
| The concept of independence extends to dealing with collections of more than two events or random variables.
| |
| | |
| == Definition ==
| |
| === For events ===
| |
| ==== Two events ====
| |
| Two events ''A'' and ''B'' are '''independent''' [[if and only if]] their [[joint probability]] equals the product of their probabilities:
| |
| | |
| :<math>\mathrm{P}(A \cap B) = \mathrm{P}(A)\mathrm{P}(B)</math>.
| |
| | |
| Why this defines independence is made clear by rewriting with [[Conditional probability|conditional probabilities]]:
| |
| | |
| :<math>\begin{align}
| |
| \mathrm{P}(A \cap B) = \mathrm{P}(A)\mathrm{P}(B) &\Leftrightarrow \mathrm{P}(A) = \frac{\mathrm{P}(A \cap B)}{\mathrm{P}(B)} \\
| |
| &\Leftrightarrow \mathrm{P}(A) = \mathrm{P}(A\mid B)
| |
| \end{align}</math>
| |
| | |
| and similarly
| |
| | |
| :<math>\mathrm{P}(A \cap B) = \mathrm{P}(A)\mathrm{P}(B) \Leftrightarrow \mathrm{P}(B) = \mathrm{P}(B\mid A)</math>.
| |
| | |
| Thus, the occurrence of ''B'' does not affect the probability of ''A'', and vice versa. Although the derived expressions may seem more intuitive, they are not the preferred definition, as the conditional probabilities may be undefined if ''P''(''A'') or ''P''(''B'') are 0. Furthermore, the preferred definition makes clear by symmetry that when ''A'' is independent of ''B'', ''B'' is also independent of ''A''.
| |
| | |
| ==== More than two events ====
| |
| A finite set of events {''A<sub>i</sub>''} is '''pairwise independent''' [[if and only if|iff]] every pair of events is independent.<ref name ="Feller">{{cite book | last = Feller | first = W | year = 1971 | title = An Introduction to Probability Theory and Its Applications | publisher = [[John Wiley & Sons|Wiley]] | chapter = Stochastic Independence}}</ref> That is, if and only if for all distinct pairs of indices ''m'', ''n''
| |
| | |
| :<math>\mathrm{P}(A_m \cap A_n) = \mathrm{P}(A_m)\mathrm{P}(A_n)</math>.
| |
| | |
| A finite set of events is '''mutually independent''' [[if and only if]] every event is independent of any intersection of the other events.<ref name="Feller" /> That is, iff for every subset {''A<sub>n</sub>''}
| |
| | |
| :<math>\mathrm{P}\left(\bigcap_{i=1}^n A_i\right)=\prod_{i=1}^n \mathrm{P}(A_i).</math>
| |
| | |
| This is called the ''multiplication rule'' for independent events.
| |
| | |
| For more than two events, a mutually independent set of events is (by definition) pairwise independent, but the converse is not necessarily true.
| |
| | |
| === For random variables ===
| |
| ==== Two random variables ====
| |
| Two random variables ''X'' and ''Y'' are '''independent''' [[if and only if|iff]] the elements of the [[Pi system|π-system]] generated by them are independent; that is to say, for every ''a'' and ''b'', the events {''X'' ≤ ''a''} and {''Y'' ≤ ''b''} are independent events (as defined above). That is, ''X'' and ''Y'' with [[cumulative distribution function]]s <math>F_X(x)</math> and <math>F_Y(y)</math>, and [[probability density function|probability densities]] <math>f_X(x)</math> and <math>f_Y(y)</math>, are independent [[if and only if|iff]] the combined random variable (''X'', ''Y'') has a [[joint distribution|joint]] cumulative distribution function
| |
| | |
| :<math>F_{X,Y}(x,y) = F_X(x) F_Y(y),</math> | |
| | |
| or equivalently, a joint density
| |
| | |
| :<math>f_{X,Y}(x,y) = f_X(x) f_Y(y).</math>
| |
| | |
| ==== More than two random variables ====
| |
| A set of random variables is '''pairwise independent''' [[if and only if|iff]] every pair of random variables is independent.
| |
| | |
| A set of random variables is '''mutually independent''' [[if and only if|iff]] for any finite subset <math>X_1, \ldots, X_n</math> and any finite sequence of numbers <math>a_1, \ldots, a_n</math>, the events <math>\{X_1 \le a_1\}, \ldots, \{X_n \le a_n\}</math> are mutually independent events (as defined above).
| |
| | |
| The measure-theoretically inclined may prefer to substitute events {''X'' ∈ ''A''} for events {''X'' ≤ ''a''} in the above definition, where ''A'' is any [[Borel algebra|Borel set]]. That definition is exactly equivalent to the one above when the values of the random variables are [[real number]]s. It has the advantage of working also for complex-valued random variables or for random variables taking values in any [[measurable space]] (which includes [[topological space]]s endowed by appropriate σ-algebras).
| |
| | |
| ==== Conditional independence ====
| |
| {{main|Conditional independence}}
| |
| | |
| Intuitively, two random variables ''X'' and ''Y'' are conditionally independent given ''Z'' if, once ''Z'' is known, the value of ''Y'' does not add any additional information about ''X''. For instance, two measurements ''X'' and ''Y'' of the same underlying quantity ''Z'' are not independent, but they are '''conditionally independent given ''Z''''' (unless the errors in the two measurements are somehow connected).
| |
| | |
| The formal definition of conditional independence is based on the idea of [[conditional distribution]]s. If ''X'', ''Y'', and ''Z'' are [[discrete random variable]]s, then we define ''X'' and ''Y'' to be ''conditionally independent given'' ''Z'' if
| |
| | |
| :<math>\mathrm{P}(X \le x, Y \le y\;|\;Z = z) = \mathrm{P}(X \le x\;|\;Z = z) \cdot \mathrm{P}(Y \le y\;|\;Z = z)</math>
| |
| | |
| for all ''x'', ''y'' and ''z'' such that P(''Z'' = ''z'') > 0. On the other hand, if the random variables are [[continuous random variable|continuous]] and have a joint [[probability density function]] ''p'', then ''X'' and ''Y'' are [[conditionally independent]] given ''Z'' if
| |
| | |
| :<math>p_{XY|Z}(x, y | z) = p_{X|Z}(x | z) \cdot p_{Y|Z}(y | z)</math>
| |
| | |
| for all real numbers ''x'', ''y'' and ''z'' such that ''p''<sub>''Z''</sub>(''z'') > 0.
| |
| | |
| If ''X'' and ''Y'' are conditionally independent given ''Z'', then
| |
| | |
| :<math>\mathrm{P}(X = x | Y = y, Z = z) = \mathrm{P}(X = x | Z = z)</math>
| |
| | |
| for any ''x'', ''y'' and ''z'' with P(''Z'' = ''z'') > 0. That is, the conditional distribution for ''X'' given ''Y'' and ''Z'' is the same as that given ''Z'' alone. A similar equation holds for the conditional probability density functions in the continuous case.
| |
| | |
| Independence can be seen as a special kind of conditional independence, since probability can be seen as a kind of conditional probability given no events.
| |
| | |
| === Independent σ-algebras ===
| |
| The definitions above are both generalized by the following definition of independence for [[sigma algebra|σ-algebras]]. Let (Ω, Σ, Pr) be a probability space and let '''A''' and '''B''' be two sub-σ-algebras of Σ. '''A''' and '''B''' are said to be '''independent''' if, whenever ''A'' ∈ '''A''' and ''B'' ∈ '''B''',
| |
| | |
| :<math>\mathrm{P}(A \cap B) = \mathrm{P}(A) \mathrm{P}(B).</math>
| |
| | |
| Likewise, a finite family of σ-algebras <math>(\tau_i)_{i\in I}</math> is said to be independent if and only if for all
| |
| | |
| :<math>\forall \left(A_i\right)_{i\in I} \in \prod\nolimits_{i\in I}\tau_i \ : \ \mathrm{P}\left(\bigcap\nolimits_{i\in I}A_i\right) = \prod\nolimits_{i\in I}\mathrm{P}\left(A_i\right)</math>
| |
| | |
| and an infinite family of σ-algebras is said to be independent if all its finite subfamilies are independent.
| |
| | |
| The new definition relates to the previous ones very directly:
| |
| * Two events are independent (in the old sense) [[if and only if]] the σ-algebras that they generate are independent (in the new sense). The σ-algebra generated by an event ''E'' ∈ Σ is, by definition,
| |
| ::<math>\sigma(E) = \{ \emptyset, E, \Omega \setminus E, \Omega \}.</math>
| |
| * Two random variables ''X'' and ''Y'' defined over Ω are independent (in the old sense) if and only if the σ-algebras that they generate are independent (in the new sense). The σ-algebra generated by a random variable ''X'' taking values in some [[measurable space]] ''S'' consists, by definition, of all subsets of Ω of the form ''X''<sup>−1</sup>(''U''), where ''U'' is any measurable subset of ''S''.
| |
| | |
| Using this definition, it is easy to show that if ''X'' and ''Y'' are random variables and ''Y'' is constant, then ''X'' and ''Y'' are independent, since the σ-algebra generated by a constant random variable is the trivial σ-algebra {∅, Ω}. Probability zero events cannot affect independence so independence also holds if ''Y'' is only Pr-[[almost surely]] constant.
| |
| | |
| == Properties ==
| |
| === Self-dependence ===
| |
| Note that an event is independent of itself [[if and only if|iff]]
| |
| | |
| :<math>\mathrm{P}(A) = \mathrm{P}(A \cap A) = \mathrm{P}(A) \cdot \mathrm{P}(A) \Rightarrow \mathrm{P}(A) = 0 \text{ or } 1</math>.
| |
| | |
| Thus if an event or its [[Complement (set theory)|complement]] [[almost surely]] occurs, it is independent of itself. For example, if ''A'' is choosing any number but 0.5 from a [[uniform distribution (continuous)|uniform distribution]] on the [[unit interval]], ''A'' is independent of itself, even though, [[tautology (logic)|tautologically]], ''A'' fully determines ''A''.
| |
| | |
| === Expectation and covariance ===
| |
| If ''X'' and ''Y'' are independent, then the [[expected value|expectation operator]] ''E'' has the property
| |
| | |
| :<math>E[X Y] = E[X] E[Y],</math>
| |
| | |
| and for the [[covariance]] since we have
| |
| | |
| :<math>\text{cov}[X, Y] = E[X Y] - E[X] E[Y],</math>
| |
| | |
| the [[covariance]] cov(''X'', ''Y'') is zero. (The converse of these, i.e. the proposition that if two random variables have a covariance of 0 they must be independent, is not true. See [[uncorrelated]].)
| |
| | |
| === Characteristic function ===
| |
| Two random variables ''X'' and ''Y'' are independent if and only if the [[characteristic function (probability theory)|characteristic function]] of the random vector
| |
| (''X'', ''Y'') satisfies
| |
| :<math>\varphi_{(X,Y)}(t,s) = \varphi_{X}(t)\cdot \varphi_{Y}(s).</math>
| |
| | |
| In particular the characteristic function of their sum is the product of their marginal characteristic functions:
| |
| :<math>\varphi_{X+Y}(t) = \varphi_X(t)\cdot\varphi_Y(t),</math>
| |
| | |
| though the reverse implication is not true. Random variables that satisfy the latter condition are called [[subindependence | subindependent]].
| |
| | |
| == Examples ==
| |
| === Rolling a die ===
| |
| The event of getting a 6 the first time a die is rolled and the event of getting a 6 the second time are ''independent''. By contrast, the event of getting a 6 the first time a die is rolled and the event that the sum of the numbers seen on the first and second trials is 8 are ''not'' independent.
| |
| | |
| === Drawing cards ===
| |
| If two cards are drawn ''with'' replacement from a deck of cards, the event of drawing a red card on the first trial and that of drawing a red card on the second trial are ''independent''. By contrast, if two cards are drawn ''without'' replacement from a deck of cards, the event of drawing a red card on the first trial and that of drawing a red card on the second trial are again ''not'' independent.
| |
| | |
| === Pairwise and mutual independence ===
| |
| [[File:Pairwise independent.svg|thumb|Pairwise independent, but not mutually independent, events.]]
| |
| [[File:Mutually independent.svg|thumb|Mutually independent events.]]
| |
| | |
| Consider the two probability spaces shown. In both cases, ''P''(''A'') = ''P''(''B'') = 1/2 and ''P''(''C'') = 1/4 The first space is pairwise independent but not mutually independent. The second space is mutually independent. To illustrate the difference, consider conditioning on two events. In the pairwise independent case, although, for example, ''A'' is independent of both ''B'' and ''C'', it is not independent of ''B'' ∩ ''C'':
| |
| | |
| :<math>\mathrm{P}(A|BC) = \frac{\frac{4}{40}}{\frac{4}{40} + \frac{1}{40}} = \tfrac{4}{5} \ne \mathrm{P}(A)</math>
| |
| :<math>\mathrm{P}(B|AC) = \frac{\frac{4}{40}}{\frac{4}{40} + \frac{1}{40}} = \tfrac{4}{5} \ne \mathrm{P}(B)</math>
| |
| :<math>\mathrm{P}(C|AB) = \frac{\frac{4}{40}}{\frac{4}{40} + \frac{6}{40}} = \tfrac{2}{5} \ne \mathrm{P}(C)</math>
| |
| | |
| In the mutually independent case however:
| |
| :<math>\mathrm{P}(A|BC) = \frac{\frac{1}{16}}{\frac{1}{16} + \frac{1}{16}} = \tfrac{1}{2} = \mathrm{P}(A)</math>
| |
| :<math>\mathrm{P}(B|AC) = \frac{\frac{1}{16}}{\frac{1}{16} + \frac{1}{16}} = \tfrac{1}{2} = \mathrm{P}(B)</math>
| |
| :<math>\mathrm{P}(C|AB) = \frac{\frac{1}{16}}{\frac{1}{16} + \frac{3}{16}} = \tfrac{1}{4} = \mathrm{P}(C)</math>
| |
| | |
| See also<ref>George, Glyn, "Testing for the independence of three events," ''Mathematical Gazette'' 88, November 2004, 568. [http://www.engr.mun.ca/~ggeorge/MathGaz04.pdf PDF]</ref> for a three-event example in which
| |
| | |
| :<math>\mathrm{P}(A \cap B \cap C) = \mathrm{P}(A)\mathrm{P}(B)\mathrm{P}(C),</math>
| |
| | |
| and yet no two of the three events are pairwise independent.
| |
| | |
| == See also ==
| |
| * [[Copula (statistics)]]
| |
| * [[Independent and identically distributed random variables]]
| |
| * [[Mutually exclusive events]]
| |
| * [[Subindependence]]
| |
| * [[Linear_independence#Linear_dependence_between_random_variables|Linear dependence between random variables]]
| |
| * [[Conditional independence]]
| |
| * [[Normally distributed and uncorrelated does not imply independent]]
| |
| | |
| ==References==
| |
| {{Reflist}}
| |
| | |
| {{DEFAULTSORT:Independence (Probability Theory)}}
| |
| [[Category:Probability theory]]
| |
| [[Category:Statistical dependence]]
| |