|
|
| Line 1: |
Line 1: |
| {{Regression bar}}
| | Hi there, I am Sophia. Alaska is the only location I've been residing in but now I'm contemplating other choices. For years she's been operating as a journey agent. What me and my family members love is to climb but I'm considering on starting some thing new.<br><br>Here is my homepage ... free online tarot card readings ([http://alles-herunterladen.de/excellent-advice-for-picking-the-ideal-hobby/ http://alles-herunterladen.de/]) |
| In [[statistics]], '''Poisson regression''' is a form of [[regression analysis]] used to model [[count data]] and [[contingency table]]s. Poisson regression assumes the response variable ''Y'' has a [[Poisson distribution]], and assumes the [[logarithm]] of its [[expected value]] can be modeled by a linear combination of unknown [[parameter]]s. A Poisson regression model is sometimes known as a [[log-linear model]], especially when used to model contingency tables.
| |
| | |
| Poisson regression models are [[generalized linear model]]s with the logarithm as the (canonical) [[link function]], and the [[Poisson distribution|Poisson]] distribution function.
| |
| | |
| ==Regression models==
| |
| If <math>\mathbf{x} \in \mathbb{R}^n</math> is a vector of [[independent variables]], then the model takes the form
| |
| | |
| <math>\log (\operatorname{E}(Y|\mathbf{x}))=\mathbf{a}' \mathbf{x} + b,</math>
| |
| | |
| where <math>\mathbf{a} \in \mathbb{R}^n</math> and <math>b \in \mathbb{R}</math>. Sometimes this is written more compactly as
| |
| | |
| :<math>\log (\operatorname{E}(Y|\mathbf{x}))=\boldsymbol{\theta}' \mathbf{x},\,</math>
| |
| | |
| where '''x''' is now an (''n'' + 1)-dimensional vector consisting of ''n'' independent variables concatenated to some constant, usually 1. Here '''''θ''''' is simply '''a''' concatenated to ''b''.
| |
| | |
| Thus, when given a Poisson regression model '''''θ''''' and an input vector '''x''', the predicted mean of the associated Poisson distribution is given by
| |
| | |
| :<math>\operatorname{E}(Y|\mathbf{x})=e^{\boldsymbol{\theta}' \mathbf{x}}.\,</math>
| |
| | |
| If ''Y''<sub>''i''</sub> are [[statistical independence|independent]] observations with corresponding values '''x'''<sub>''i''</sub> of the predictor variables, then '''''θ''''' can be estimated by [[maximum likelihood]]. The maximum-likelihood estimates lack a [[closed-form expression]] and must be found by numerical methods. The probability surface for maximum-likelihood Poisson regression is always convex, making Newton–Raphson or other gradient-based methods appropriate estimation techniques.
| |
| | |
| ==Maximum likelihood-based parameter estimation==
| |
| | |
| Given a set of parameters ''θ'' and an input vector ''x'', the mean of the predicted [[Poisson distribution]], as stated above, is given by
| |
| | |
| :<math>\operatorname{E}(Y|x)=e^{\theta' x}\,</math>,
| |
| | |
| and thus, the Poisson distribution's [[probability mass function]] is given by
| |
| | |
| :<math>p(y|x;\theta) = \frac{[\operatorname{E}(Y|x)]^y \times e^{-\operatorname{E}(Y|x)}}{y!} = \frac{e^{y \theta' x} e^{-e^{\theta' x}}}{y!}</math>
| |
| | |
| Now suppose we are given a data set consisting of ''m'' vectors <math>x_i \in \mathbb{R}^{n+1}, \, i = 1,...,m</math>, along with a set of ''m'' values <math>y_1,\ldots,y_m \in \mathbb{R}</math>. Then, for a given set of parameters ''θ'', the probability of attaining this particular set of data is given by
| |
| | |
| :<math>p(y_1,\ldots,y_m|x_1,\ldots,x_m;\theta) = \prod_{i=1}^m \frac{e^{y_i \theta' x_i} e^{-e^{\theta' x_i}}}{y_i!}.</math>
| |
| | |
| By the method of [[maximum likelihood]], we wish to find the set of parameters ''θ'' that makes this probability as large as possible. To do this, the equation is first rewritten as a [[likelihood function]] in terms of θ:
| |
| | |
| :<math>L(\theta|X,Y) = \prod_{i=1}^m \frac{e^{y_i \theta' x_i} e^{-e^{\theta' x_i}}}{y_i!}</math>.
| |
| | |
| Note that the expression on the [[Sides of an equation|right hand side]] has not actually changed. A formula in this form is typically difficult to work with; instead, one uses the ''log-likelihood'':
| |
| | |
| :<math>\ell(\theta|X,Y) = \log L(\theta|X,Y) = \sum_{i=1}^m \left( y_i \theta' x_i - e^{\theta' x_i} - \log(y_i!)\right)</math>.
| |
| | |
| Notice that the parameters ''θ'' only appear in the first two terms of each term in the summation. Therefore, given that we are only interested in finding the best value for ''θ'' we may drop the ''y''<sub>''i''</sub><nowiki>!</nowiki> and simply write
| |
| | |
| :<math>\ell(\theta|X,Y) = \sum_{i=1}^m \left( y_i \theta' x_i - e^{\theta' x_i}\right)</math>.
| |
| | |
| To find a maximum, we need to solve an equation <math>\frac{\partial \ell(\theta|X,Y)}{\partial \theta} = 0 </math> which has no closed-form solution. However, the negative log-likelihood, <math>-\ell(\theta|X,Y)</math>, is a convex function, and so standard [[convex optimization]] techniques such as [[gradient descent]] can be applied to find the optimal value of ''θ''.
| |
| | |
| ==Poisson regression in practice==
| |
| | |
| Poisson regression may be appropriate when the dependent variable is a count, for instance of [[Poisson distribution#Occurrence|events]] such as the arrival of a telephone call at a call centre.<ref>{{cite book |last=Greene |first=William H. |title=Econometric Analysis |edition=Fifth |publisher=Prentice-Hall |year=2003 |pages=740-752 |isbn=0130661899 }}</ref> The events must be independent in the sense that the arrival of one call will not make another more or less likely, but the probability per unit time of events is understood to be related to covariates such as time of day.
| |
| | |
| ==="Exposure" and offset===
| |
| Poisson regression may also be appropriate for rate data, where the rate is a count of events occurring to a particular unit of observation, divided by some measure of that unit's ''exposure''. For example, biologists may count the number of tree species in a forest, and the rate would be the number of species per square kilometre. Demographers may model death rates in geographic areas as the count of deaths divided by person−years. More generally, event rates can be calculated as events per unit time, which allows the observation window to vary for each unit. In these examples, exposure is respectively unit area, person−years and unit time. In Poisson regression this is handled as an '''offset''', where the exposure variable enters on the right-hand side of the equation, but with a parameter estimate (for log(exposure)) constrained to 1.
| |
| | |
| :<math>\log{(\operatorname{E}(Y|x))} = \log{(\text{exposure})} + \theta' x</math>
| |
| which implies
| |
| :<math>\log{(\operatorname{E}(Y|x))} - \log{(\text{exposure})} =
| |
| \log{\left(\frac{\operatorname{E}(Y|x)}{\text{exposure}}\right)} = \theta' x</math>
| |
| | |
| Offset in the case of a [[Generalized linear model|GLM]] in [[R (programming language)|R]] can be achieved using the offset() function:
| |
| | |
| <source lang="rsplus">
| |
| glm.fit <- glm(y ~ offset(log(exposure)) + x, family=poisson(link=log) )
| |
| </source>
| |
| | |
| ===Overdispersion===
| |
| A characteristic of the [[Poisson distribution]] is that its mean is equal to its variance. In certain circumstances, it will be found that the observed [[variance]] is greater than the mean; this is known as [[overdispersion]] and indicates that the model is not appropriate. A common reason is the omission of relevant explanatory variables, or dependent observations. Under some circumstances, the problem of overdispersion can be solved by using [[quasi-likelihood]] estimation or a [[negative binomial distribution]] instead.<ref>{{cite journal|author=Paternoster R, Brame R|year=1997|title=Multiple routes to delinquency? A test of developmental and general theories of crime|journal=Criminology|volume=35|pages=45–84|doi=10.1111/j.1745-9125.1997.tb00870.x}}</ref><ref>{{cite journal|author=Berk R, MacDonald J|title=Overdispersion and Poisson regression|journal=Journal of Quantitative Criminology|volume=24|pages=269–284|year=2008|url=http://www.crim.upenn.edu/faculty/papers/berk/regression.pdf|doi=10.1007/s10940-008-9048-4|issue=3}}</ref>
| |
| | |
| Another common problem with Poisson regression is excess zeros: if there are two processes at work, one determining whether there are zero events or any events, and a Poisson process determining how many events there are, there will be more zeros than a Poisson regression would predict. An example would be the distribution of cigarettes smoked in an hour by members of a group where some individuals are non-smokers.
| |
| | |
| Other [[generalized linear model]]s such as the [[negative binomial distribution|negative binomial]] model or [[zero-inflated model]] may function better in these cases.
| |
| | |
| ===Use in survival analysis===
| |
| | |
| Poisson regression creates proportional hazards models, one class of [[survival analysis]]: see [[proportional hazards models]] for descriptions of Cox models.
| |
| | |
| ==Extensions==
| |
| ===Regularized Poisson regression===
| |
| | |
| When estimating the parameters for Poisson regression, one typically tries to find values for ''θ'' that maximize the likelihood of an expression of the form
| |
| | |
| :<math>\sum_{i=1}^m \log(p(y_i;e^{\theta' x})),</math>
| |
| | |
| where ''m'' is the number of examples in the data set, and <math>p(y_i;e^{\theta' x})</math> is the [[probability mass function]] of the [[Poisson distribution]] with the mean set to <math>e^{\theta' x}</math>. Regularization can be added to this optimization problem by instead maximizing
| |
| | |
| :<math>\sum_{i=1}^m \log(p(y_i;e^{\theta' x})) - \lambda \left\|\theta\right\|_2^2</math>,
| |
| | |
| for some positive constant <math>\lambda</math>. This technique, similar to [[ridge regression]], can reduce [[overfitting]].
| |
| | |
| ==Implementations==
| |
| Some [[statistics packages]] include implementations of Poisson regression.
| |
| * [[GenStat]]: Poisson regression is a standard option of the regression section, using the "MODEL", "FIT" and associated commands; it is also available in the "Stats > Regression Analysis > Generalized Linear Models" menu.
| |
| * [[MATLAB]] Statistics Toolbox: Poisson regression can be performed using the "glmfit" and "glmval" functions.<ref>http://www.mathworks.com/help/toolbox/stats/glmfit.html</ref>
| |
| * [[Microsoft Excel]]: Excel is not capable of doing Poisson regression by default. One of the Excel Add-ins for Poisson regression is [http://www.indiana.edu/~jslsoc/files_research/xpost/xpost.pdf XPost]
| |
| * [[mPlus]]: mPlus allows for Poisson regression using the command COUNT IS when specifying the data
| |
| * [[R (programming language)|R]]: The function for fitting a generalized linear model in R is glm(), and can be used for Poisson Regression
| |
| * [[SAS System|SAS]]: Poisson regression in SAS is done by using [[GENMOD]] and [[HPGENSELECT]]
| |
| * [[SPSS]]: In SPSS, Poisson regression is done by using the [[GENLIN]] command
| |
| * [[Stata]]: Stata has a procedure for Poisson regression named "poisson", and for panel data "xtpoisson"
| |
| * [[CrimeStat]]: CrimeStat has Poisson, Poisson NB1, Poisson-Gamma(negative binomial), and Poisson-Lognormal regression models.
| |
| | |
| ==See also==
| |
| {{div col|colwidth=20em}}
| |
| * [[Zero-inflated model]]
| |
| * [[Poisson distribution]]
| |
| {{div col end}}
| |
| | |
| ==References==
| |
| {{reflist}}
| |
| | |
| ==Further reading==
| |
| * {{cite book |last=Cameron |first=A. C. |first2=P. K. |last2=Trivedi |year=1998 |title=Regression analysis of count data |publisher=Cambridge University Press |isbn=0-521-63201-3 }}
| |
| * {{cite book | last=Christensen | first=Ronald | title=Log-linear models and logistic regression | edition=Second | series=Springer Texts in Statistics | publisher=Springer-Verlag | location=New York | year=1997 | pages=xvi+483 | isbn=0-387-98247-7 | mr=1633357 }}
| |
| * {{cite book |last=Hilbe |first=J. M. |year=2007 |title=Negative Binomial Regression |publisher=Cambridge University Press |isbn=978-0-521-85772-7 }}
| |
| | |
| {{Statistics|correlation}}
| |
| {{least squares and regression analysis}}
| |
| | |
| [[Category:Regression analysis]]
| |
| [[Category:Generalized linear models]]
| |
| [[Category:Categorical data]]
| |
| [[Category:Econometrics]]
| |