|
|
| Line 1: |
Line 1: |
| The '''deviance information criterion''' ('''DIC''') is a hierarchical modeling generalization of the AIC ([[Akaike information criterion]]) and BIC ([[Schwarz criterion|Bayesian information criterion]], also known as the Schwarz criterion). It is particularly useful in [[Bayesian inference|Bayesian]] [[model selection]] problems where the [[posterior distribution]]s of the [[statistical model|model]]s have been obtained by [[Markov chain Monte Carlo]] (MCMC) simulation. Like AIC and BIC it is an asymptotic approximation as the sample size becomes large. It is only valid when the posterior distribution is approximately [[multivariate normal distribution|multivariate normal]].
| | Marvella is what you can contact her but it's not the most female title out there. What I adore performing is to gather badges but I've been taking on new things lately. North Dakota is our beginning place. Managing people is what I do and the salary has been really fulfilling.<br><br>Feel free to visit my web site ... [http://www.crmidol.com/discussion/19701/how-tell-if-woman-interested-or-sexually-captivated-you at home std test] |
| | |
| Define the [[deviance (statistics)|deviance]] as <math> D(\theta)=-2 \log(p(y|\theta))+C\, </math>, where <math>y\,</math> are the data, <math>\theta\,</math> are the unknown parameters of the model and <math> p(y|\theta)\, </math> is the [[likelihood function]]. <math>C\,</math> is a constant that cancels out in all calculations that compare different models, and which therefore does not need to be known.
| |
| | |
| The [[expected value|expectation]] <math>\bar{D}=\mathbf{E}^\theta[D(\theta)]</math> is a measure of how well the model fits the data; the larger this is, the worse the fit.
| |
| | |
| The effective number of parameters of the model is computed as <math>p_D=\bar{D}-D(\bar{\theta})</math>, where <math>\bar{\theta}</math> is the expectation of <math>\theta\,</math>. The larger this is, the ''easier'' it is for the model to fit the data.
| |
| | |
| The deviance information criterion is calculated as
| |
| | |
| :<math>\mathit{DIC} = p_D+\bar{D},</math>
| |
| | |
| or equivalently as
| |
| | |
| :<math>\mathit{DIC} = D(\bar{\theta})+2 p_D.</math>
| |
| | |
| From this later form, the connection with Akaike's information criterion is evident.
| |
| | |
| The idea is that models with smaller DIC should be preferred to models with larger DIC. Models are penalized both by the value of <math>\bar{D}</math>, which favors a good fit, but also (in common with AIC and BIC) by the effective number of parameters <math>p_D\,</math>. Since <math> \bar D </math> will decrease as the number of parameters in a model increases, the <math>p_D\,</math> term compensates for this effect by favoring models with a smaller number of parameters.
| |
| | |
| The advantage of DIC over other criteria in the case of Bayesian model selection is that the DIC is easily calculated from the samples generated by a Markov chain Monte Carlo simulation. AIC and BIC require calculating the likelihood at its maximum over <math>\theta\,</math>, which is not readily available from the MCMC simulation. But to calculate DIC, simply compute <math>\bar{D}</math> as the average of <math>D(\theta)\,</math> over the samples of <math>\theta\,</math>, and <math>D(\bar{\theta})</math> as the value of <math>D\,</math> evaluated at the average of the samples of <math> \theta\, </math>. Then the DIC follows directly from these approximations. Claeskens and Hjort (2008, Ch. 3.5) show that the DIC is [[asymptotic distribution|large-sample]] equivalent to the natural model-robust version of the AIC.
| |
| | |
| In the derivation of DIC, it is assumed that the specified parametric family of probability distributions that generate future observations encompasses the true model. This assumption does not always hold, and it is desirable to consider model assessment procedures in that scenario.
| |
| Also, the observed data are used both to construct the posterior distribution and to evaluate the estimated models.
| |
| Therefore, DIC tends to select over-fitted models.
| |
| Recently, these issues are resolved by Ando (2007), Bayesian predictive information criterion, BPIC.
| |
| | |
| To avoid the over-fitting problems of DIC, Ando (2011) developed Bayesian model selection criteria from a predictive view point.
| |
| The criterion is calculated as
| |
| | |
| :<math>\mathit{IC} = -2\mathbf{E}^\theta[ \log(p(y|\theta))]+2p_D.</math> | |
| | |
| The first term is a measure of how well the model fits the data, while the second term is a penalty on the model complexity.
| |
| | |
| ==See also==
| |
| * [[Akaike information criterion]] (AIC)
| |
| * [[Bayesian information criterion]] (BIC)
| |
| * [[Bayesian predictive information criterion]] (BPIC)
| |
| * [[Focused information criterion]] (FIC)
| |
| * [[Kullback–Leibler divergence]]
| |
| * [[Jensen–Shannon divergence]]
| |
| | |
| ==References==
| |
| {{refbegin}}
| |
| *{{cite journal
| |
| | first = Tomohiro | last = Ando
| |
| | year = 2007
| |
| | title = Bayesian predictive information criterion for the evaluation of hierarchical Bayesian and empirical Bayes models
| |
| | journal = [[Biometrika]]
| |
| | volume = 94 | pages = 443–458
| |
| | doi = 10.1093/biomet/asm017
| |
| | issue = 2
| |
| }}
| |
| *{{cite journal
| |
| | first = Tomohiro | last = Ando
| |
| | year = 2011
| |
| | title = Predictive Bayesian model selection
| |
| | journal = [[American Journal of Mathematical and Management Sciences]]
| |
| | volume = 31 | pages = 13–38
| |
| }}
| |
| * Claeskens, G, and [[Nils Lid Hjort|Hjort, N.L.]] (2008). ''Model Selection and Model Averaging'', Cambridge. Section 3.5.
| |
| *{{cite book
| |
| | first = Andrew | last = Gelman
| |
| | first2 = John B. |last2=Carlin
| |
| | first3=Hal. S. |last3=Stern
| |
| | first4= Donald B. |last4=Rubin |authorlink4=Donald Rubin
| |
| | year = 2004
| |
| | title = Bayesian Data Analysis
| |
| | edition = 2nd
| |
| | pages = 182–184
| |
| | publisher = Chapman & Hall/CRC
| |
| | location = Boca Raton, Florida
| |
| | isbn = 1-58488-388-X
| |
| | id = {{LCC|QA279.5.B386|2004}} | mr = 2027492
| |
| }}
| |
| * van der Linde, A. (2005). "DIC in variable selection", ''Statistica Neerlandica'', 59: 45-56. doi:[http://dx.doi.org/10.1111/j.1467-9574.2005.00278.x 10.1111/j.1467-9574.2005.00278.x]
| |
| *{{cite journal
| |
| | first = David J. | last = Spiegelhalter | authorlink = David Spiegelhalter
| |
| | first2 = Nicola G. |last2=Best
| |
| | first3=Bradley P. |last3=Carlin
| |
| |first4= Angelika |last4=van der Linde
| |
| |date=October 2002
| |
| | title = Bayesian measures of model complexity and fit (with discussion)
| |
| | journal = [[Journal of the Royal Statistical Society, Series B]]
| |
| | volume = 64 | issue = 4 | pages = 583–639
| |
| | doi = 10.1111/1467-9868.00353 |mr=1979380 | jstor = 3088806
| |
| }}
| |
| {{refend}}
| |
| | |
| [[Category:Bayesian statistics]]
| |
| [[Category:Regression variable selection]]
| |
Marvella is what you can contact her but it's not the most female title out there. What I adore performing is to gather badges but I've been taking on new things lately. North Dakota is our beginning place. Managing people is what I do and the salary has been really fulfilling.
Feel free to visit my web site ... at home std test