Stein's unbiased risk estimate: Difference between revisions
en>Mygskr Corrected expression for SURE (pending search for source verifying more general version with arbitrary variance.) See talk page. |
en>Mygskr →Proof: Corrected error (typo) |
||
Line 1: | Line 1: | ||
In [[probability theory]] and [[statistics]], the '''index of dispersion''',<ref>Cox &Lewis (1966)</ref> '''dispersion index,''' '''coefficient of dispersion,''' or '''variance-to-mean ratio (VMR)''', like the [[coefficient of variation]], is a [[Normalization (statistics)|normalized]] measure of the [[statistical dispersion|dispersion]] of a [[probability distribution]]: it is a measure used to quantify whether a set of observed occurrences are clustered or dispersed compared to a standard statistical model. | |||
It is defined as the ratio of the [[variance]] <math>\sigma^2</math> to the [[mean]] <math>\mu</math>,'' | |||
:<math>D = {\sigma^2 \over \mu }.</math> | |||
It is also known as the '''[[Fano factor]],''' though this term is sometimes reserved for ''windowed'' data (the mean and variance are computed over a subpopulation), where the index of dispersion is the special case where the window is infinite. Windowing data is frequently done: the VMR is frequently computed over various intervals in time or small regions in space, which may be called "windows", and the resulting statistic called the Fano factor. | |||
It is only defined when the mean <math>\mu</math> is non-zero, and is generally only used for positive statistics, such as [[count data]] or time between events, or where the underlying distribution is assumed to be the [[exponential distribution]] or [[Poisson distribution]]. | |||
== Terminology == | |||
In this context, the observed dataset may consist of the times of occurrence of predefined events, such as earthquakes in a given region over a given magnitude, or of the locations in geographical space of plants of a given species. Details of such occurrences are first converted into counts of the numbers of events or occurrences in each of a set of equal-sized time- or space-regions. | |||
The above defines a ''dispersion index for counts''.<ref>Cox & Lewis (1966), p72</ref> A different definition applies for a ''dispersion index for intervals'',<ref>Cox & Lewis (1966), p71</ref> where the quantities treated are the lengths of the time-intervals between the events, and where the index is equivalent to the square of the [[coefficient of variation]] of the interval lengths. Common usage is that "index of dispersion" means the dispersion index for counts. | |||
== Interpretation == | |||
Some distributions, most notably the [[Poisson distribution]], have equal variance and mean, giving them a VMR = 1. The [[geometric distribution]] and the [[negative binomial distribution]] have VMR > 1, while the [[binomial distribution]] has VMR < 1, and the [[constant random variable]] has VMR = 0. This yields the following table: | |||
{| | |||
! Distribution !! VMR !! | |||
|- | |||
| [[constant random variable]] || VMR = 0 || not dispersed | |||
|- | |||
| [[binomial distribution]] || 0 < VMR < 1 || under-dispersed | |||
|- | |||
| [[Poisson distribution]] || VMR = 1 || | |||
|- | |||
| [[negative binomial distribution]] || VMR > 1 || over-dispersed | |||
|} | |||
This can be considered analogous to the classification of [[conic sections]] by [[Eccentricity (mathematics)|eccentricity]]; see [[Cumulant#Cumulants of some discrete probability distributions|Cumulants of particular probability distributions]] for details. | |||
When the coefficient of dispersion is less than 1, a dataset is said to be "under-dispersed": this condition can relate to patterns of occurrence that are more regular than the randomness associated with a Poisson process. For instance, points spread uniformly in space or regular, periodic events will be under-dispersed. | |||
If the index of dispersion is larger than 1, a dataset is said to be [[overdispersion|over-dispersed]]: this can correspond to the existence of clusters of occurrences. Clumped, concentrated data is over-dispersed. | |||
In terms of the interval-counts, over-dispersion corresponds to there being more intervals with low counts and more intervals with high counts, compared to a Poisson distribution: in contrast, under-dispersion is characterised by there being more intervals having counts close to the mean count, compared to a Poisson distribution. | |||
The relevance of the index of dispersion is that it has a value of one when the probability distribution of the number of occurrences in an interval is a [[Poisson distribution]]. Thus the measure can be used to assess whether observed data can be modeled using a [[Poisson process]]. | |||
A sample-based estimate of the dispersion index can be used to construct a formal [[statistical hypothesis testing| statistical hypothesis test]] for the adequacy of the model that a series of counts follow a Poisson distribution.<ref>Cox & Lewis (1966), p158</ref><ref>Upton&Cook(2006), under index of dispersion</ref> | |||
The VMR is a good measure of the degree of randomness of a given phenomenon. This technique is also commonly used in currency management. | |||
== Example == | |||
For randomly diffusing particles ([[Brownian motion]]), the distribution of the number of particle inside a given volume is poissonian, i.e. VMR=1. Therefore, to assess if a given spatial pattern (assuming you have a way to measure it) is due purely to diffusion or if some particle-particle interaction is involved : divide the space into patches, Quadrats or Sample Units (SU), count the number of individuals in each patch or SU, and compute the VMR. VMRs significantly higher than 1 denote a clustered distribution, where random walk is not enough to smother the attractive inter-particle potential. | |||
== Statistics == | |||
The first to discuss the use of a test to detect deviations from a Poisson or binomial distribution appears to have been Lexis in 1877. One of the tests he developed was the [[Lexis ratio]]. | |||
This index was first used in botany by [[Arthur Roy Clapham|Clapham]] in 1936. | |||
If the variates are Poisson distributed then the index of dispersion is distributed as a χ<sup>2</sup> statistic with ''n'' - 1 degrees of freedom when ''n'' is large and is ''μ'' > 3.<ref name=Frome1982>{{cite journal |last=Frome |first=E. L. |year=1982 |title=Algorithm AS 171: Fisher's Exact Variance Test for the Poisson Distribution |journal=[[Journal of the Royal Statistical Society, Series C]]|volume=31 |issue=1 |pages=67–71 |jstor=2347079 }}</ref> For many cases of interest this approximation is accurate and Fisher in 1950 derived an exact test for it. | |||
[[Paul G. Hoel|Hoel]] studied the first four moments of its distribution.<ref name=Hoel1943>{{cite journal |last=Hoel |first=P. G. |year=1943 |title=On Indices of Dispersion |journal=[[Annals of Mathematical Statistics]] |volume=14 |issue=2 |pages=155–162 |doi= |jstor=2235818 }}</ref> He found that the approximation to the χ<sup>2</sup> statistic is reasonable if ''μ'' > 5. | |||
== See also == | |||
* [[Count data]] | |||
* [[Harmonic mean]] | |||
=== Similar ratios === | |||
* [[Coefficient of variation]], <math>\sigma/\mu</math> | |||
* [[Standardized moment]], <math>\mu_k/\sigma^k</math> | |||
* [[Fano factor]], <math>\sigma^2_W/\mu_W</math> (windowed VMR) | |||
* [[Signal to noise ratio]], <math>\mu/\sigma</math> (in [[signal processing]]) | |||
** [[Signal to noise ratio (image processing)]] | |||
== Notes == | |||
{{Reflist}} | |||
== References == | |||
*{{cite book |last=Cox |first=D. R. |last2=Lewis |first2=P. A. W. |year=1966 |title=The Statistical Analysis of Series of Events |publisher=Methuen |location=London }} | |||
*{{cite book |last=Upton |first=G. |last2=Cook |first2=I. |year=2006 |title=Oxford Dictionary of Statistics |edition=2nd |publisher=Oxford University Press |isbn=978-0-19-954145-4 }} | |||
{{Statistics|descriptive}} | |||
[[Category:Statistical deviation and dispersion]] | |||
[[Category:Statistical ratios]] | |||
[[Category:Data analysis]] | |||
[[Category:Statistical randomness]] | |||
[[Category:Point processes]] | |||
[[Category:Statistical terminology]] |
Revision as of 15:46, 29 January 2014
In probability theory and statistics, the index of dispersion,[1] dispersion index, coefficient of dispersion, or variance-to-mean ratio (VMR), like the coefficient of variation, is a normalized measure of the dispersion of a probability distribution: it is a measure used to quantify whether a set of observed occurrences are clustered or dispersed compared to a standard statistical model.
It is defined as the ratio of the variance to the mean ,
It is also known as the Fano factor, though this term is sometimes reserved for windowed data (the mean and variance are computed over a subpopulation), where the index of dispersion is the special case where the window is infinite. Windowing data is frequently done: the VMR is frequently computed over various intervals in time or small regions in space, which may be called "windows", and the resulting statistic called the Fano factor.
It is only defined when the mean is non-zero, and is generally only used for positive statistics, such as count data or time between events, or where the underlying distribution is assumed to be the exponential distribution or Poisson distribution.
Terminology
In this context, the observed dataset may consist of the times of occurrence of predefined events, such as earthquakes in a given region over a given magnitude, or of the locations in geographical space of plants of a given species. Details of such occurrences are first converted into counts of the numbers of events or occurrences in each of a set of equal-sized time- or space-regions.
The above defines a dispersion index for counts.[2] A different definition applies for a dispersion index for intervals,[3] where the quantities treated are the lengths of the time-intervals between the events, and where the index is equivalent to the square of the coefficient of variation of the interval lengths. Common usage is that "index of dispersion" means the dispersion index for counts.
Interpretation
Some distributions, most notably the Poisson distribution, have equal variance and mean, giving them a VMR = 1. The geometric distribution and the negative binomial distribution have VMR > 1, while the binomial distribution has VMR < 1, and the constant random variable has VMR = 0. This yields the following table:
Distribution | VMR | |
---|---|---|
constant random variable | VMR = 0 | not dispersed |
binomial distribution | 0 < VMR < 1 | under-dispersed |
Poisson distribution | VMR = 1 | |
negative binomial distribution | VMR > 1 | over-dispersed |
This can be considered analogous to the classification of conic sections by eccentricity; see Cumulants of particular probability distributions for details.
When the coefficient of dispersion is less than 1, a dataset is said to be "under-dispersed": this condition can relate to patterns of occurrence that are more regular than the randomness associated with a Poisson process. For instance, points spread uniformly in space or regular, periodic events will be under-dispersed.
If the index of dispersion is larger than 1, a dataset is said to be over-dispersed: this can correspond to the existence of clusters of occurrences. Clumped, concentrated data is over-dispersed.
In terms of the interval-counts, over-dispersion corresponds to there being more intervals with low counts and more intervals with high counts, compared to a Poisson distribution: in contrast, under-dispersion is characterised by there being more intervals having counts close to the mean count, compared to a Poisson distribution.
The relevance of the index of dispersion is that it has a value of one when the probability distribution of the number of occurrences in an interval is a Poisson distribution. Thus the measure can be used to assess whether observed data can be modeled using a Poisson process.
A sample-based estimate of the dispersion index can be used to construct a formal statistical hypothesis test for the adequacy of the model that a series of counts follow a Poisson distribution.[4][5]
The VMR is a good measure of the degree of randomness of a given phenomenon. This technique is also commonly used in currency management.
Example
For randomly diffusing particles (Brownian motion), the distribution of the number of particle inside a given volume is poissonian, i.e. VMR=1. Therefore, to assess if a given spatial pattern (assuming you have a way to measure it) is due purely to diffusion or if some particle-particle interaction is involved : divide the space into patches, Quadrats or Sample Units (SU), count the number of individuals in each patch or SU, and compute the VMR. VMRs significantly higher than 1 denote a clustered distribution, where random walk is not enough to smother the attractive inter-particle potential.
Statistics
The first to discuss the use of a test to detect deviations from a Poisson or binomial distribution appears to have been Lexis in 1877. One of the tests he developed was the Lexis ratio.
This index was first used in botany by Clapham in 1936.
If the variates are Poisson distributed then the index of dispersion is distributed as a χ2 statistic with n - 1 degrees of freedom when n is large and is μ > 3.[6] For many cases of interest this approximation is accurate and Fisher in 1950 derived an exact test for it.
Hoel studied the first four moments of its distribution.[7] He found that the approximation to the χ2 statistic is reasonable if μ > 5.
See also
Similar ratios
- Coefficient of variation,
- Standardized moment,
- Fano factor, (windowed VMR)
- Signal to noise ratio, (in signal processing)
Notes
43 year old Petroleum Engineer Harry from Deep River, usually spends time with hobbies and interests like renting movies, property developers in singapore new condominium and vehicle racing. Constantly enjoys going to destinations like Camino Real de Tierra Adentro.
References
- 20 year-old Real Estate Agent Rusty from Saint-Paul, has hobbies and interests which includes monopoly, property developers in singapore and poker. Will soon undertake a contiki trip that may include going to the Lower Valley of the Omo.
My blog: http://www.primaboinca.com/view_profile.php?userid=5889534 - 20 year-old Real Estate Agent Rusty from Saint-Paul, has hobbies and interests which includes monopoly, property developers in singapore and poker. Will soon undertake a contiki trip that may include going to the Lower Valley of the Omo.
My blog: http://www.primaboinca.com/view_profile.php?userid=5889534
- ↑ Cox &Lewis (1966)
- ↑ Cox & Lewis (1966), p72
- ↑ Cox & Lewis (1966), p71
- ↑ Cox & Lewis (1966), p158
- ↑ Upton&Cook(2006), under index of dispersion
- ↑ One of the biggest reasons investing in a Singapore new launch is an effective things is as a result of it is doable to be lent massive quantities of money at very low interest rates that you should utilize to purchase it. Then, if property values continue to go up, then you'll get a really high return on funding (ROI). Simply make sure you purchase one of the higher properties, reminiscent of the ones at Fernvale the Riverbank or any Singapore landed property Get Earnings by means of Renting
In its statement, the singapore property listing - website link, government claimed that the majority citizens buying their first residence won't be hurt by the new measures. Some concessions can even be prolonged to chose teams of consumers, similar to married couples with a minimum of one Singaporean partner who are purchasing their second property so long as they intend to promote their first residential property. Lower the LTV limit on housing loans granted by monetary establishments regulated by MAS from 70% to 60% for property purchasers who are individuals with a number of outstanding housing loans on the time of the brand new housing purchase. Singapore Property Measures - 30 August 2010 The most popular seek for the number of bedrooms in Singapore is 4, followed by 2 and three. Lush Acres EC @ Sengkang
Discover out more about real estate funding in the area, together with info on international funding incentives and property possession. Many Singaporeans have been investing in property across the causeway in recent years, attracted by comparatively low prices. However, those who need to exit their investments quickly are likely to face significant challenges when trying to sell their property – and could finally be stuck with a property they can't sell. Career improvement programmes, in-house valuation, auctions and administrative help, venture advertising and marketing, skilled talks and traisning are continuously planned for the sales associates to help them obtain better outcomes for his or her shoppers while at Knight Frank Singapore. No change Present Rules
Extending the tax exemption would help. The exemption, which may be as a lot as $2 million per family, covers individuals who negotiate a principal reduction on their existing mortgage, sell their house short (i.e., for lower than the excellent loans), or take part in a foreclosure course of. An extension of theexemption would seem like a common-sense means to assist stabilize the housing market, but the political turmoil around the fiscal-cliff negotiations means widespread sense could not win out. Home Minority Chief Nancy Pelosi (D-Calif.) believes that the mortgage relief provision will be on the table during the grand-cut price talks, in response to communications director Nadeam Elshami. Buying or promoting of blue mild bulbs is unlawful.
A vendor's stamp duty has been launched on industrial property for the primary time, at rates ranging from 5 per cent to 15 per cent. The Authorities might be trying to reassure the market that they aren't in opposition to foreigners and PRs investing in Singapore's property market. They imposed these measures because of extenuating components available in the market." The sale of new dual-key EC models will even be restricted to multi-generational households only. The models have two separate entrances, permitting grandparents, for example, to dwell separately. The vendor's stamp obligation takes effect right this moment and applies to industrial property and plots which might be offered inside three years of the date of buy. JLL named Best Performing Property Brand for second year running
The data offered is for normal info purposes only and isn't supposed to be personalised investment or monetary advice. Motley Fool Singapore contributor Stanley Lim would not personal shares in any corporations talked about. Singapore private home costs increased by 1.eight% within the fourth quarter of 2012, up from 0.6% within the earlier quarter. Resale prices of government-built HDB residences which are usually bought by Singaporeans, elevated by 2.5%, quarter on quarter, the quickest acquire in five quarters. And industrial property, prices are actually double the levels of three years ago. No withholding tax in the event you sell your property. All your local information regarding vital HDB policies, condominium launches, land growth, commercial property and more
There are various methods to go about discovering the precise property. Some local newspapers (together with the Straits Instances ) have categorised property sections and many local property brokers have websites. Now there are some specifics to consider when buying a 'new launch' rental. Intended use of the unit Every sale begins with 10 p.c low cost for finish of season sale; changes to 20 % discount storewide; follows by additional reduction of fiftyand ends with last discount of 70 % or extra. Typically there is even a warehouse sale or transferring out sale with huge mark-down of costs for stock clearance. Deborah Regulation from Expat Realtor shares her property market update, plus prime rental residences and houses at the moment available to lease Esparina EC @ Sengkang - ↑ One of the biggest reasons investing in a Singapore new launch is an effective things is as a result of it is doable to be lent massive quantities of money at very low interest rates that you should utilize to purchase it. Then, if property values continue to go up, then you'll get a really high return on funding (ROI). Simply make sure you purchase one of the higher properties, reminiscent of the ones at Fernvale the Riverbank or any Singapore landed property Get Earnings by means of Renting
In its statement, the singapore property listing - website link, government claimed that the majority citizens buying their first residence won't be hurt by the new measures. Some concessions can even be prolonged to chose teams of consumers, similar to married couples with a minimum of one Singaporean partner who are purchasing their second property so long as they intend to promote their first residential property. Lower the LTV limit on housing loans granted by monetary establishments regulated by MAS from 70% to 60% for property purchasers who are individuals with a number of outstanding housing loans on the time of the brand new housing purchase. Singapore Property Measures - 30 August 2010 The most popular seek for the number of bedrooms in Singapore is 4, followed by 2 and three. Lush Acres EC @ Sengkang
Discover out more about real estate funding in the area, together with info on international funding incentives and property possession. Many Singaporeans have been investing in property across the causeway in recent years, attracted by comparatively low prices. However, those who need to exit their investments quickly are likely to face significant challenges when trying to sell their property – and could finally be stuck with a property they can't sell. Career improvement programmes, in-house valuation, auctions and administrative help, venture advertising and marketing, skilled talks and traisning are continuously planned for the sales associates to help them obtain better outcomes for his or her shoppers while at Knight Frank Singapore. No change Present Rules
Extending the tax exemption would help. The exemption, which may be as a lot as $2 million per family, covers individuals who negotiate a principal reduction on their existing mortgage, sell their house short (i.e., for lower than the excellent loans), or take part in a foreclosure course of. An extension of theexemption would seem like a common-sense means to assist stabilize the housing market, but the political turmoil around the fiscal-cliff negotiations means widespread sense could not win out. Home Minority Chief Nancy Pelosi (D-Calif.) believes that the mortgage relief provision will be on the table during the grand-cut price talks, in response to communications director Nadeam Elshami. Buying or promoting of blue mild bulbs is unlawful.
A vendor's stamp duty has been launched on industrial property for the primary time, at rates ranging from 5 per cent to 15 per cent. The Authorities might be trying to reassure the market that they aren't in opposition to foreigners and PRs investing in Singapore's property market. They imposed these measures because of extenuating components available in the market." The sale of new dual-key EC models will even be restricted to multi-generational households only. The models have two separate entrances, permitting grandparents, for example, to dwell separately. The vendor's stamp obligation takes effect right this moment and applies to industrial property and plots which might be offered inside three years of the date of buy. JLL named Best Performing Property Brand for second year running
The data offered is for normal info purposes only and isn't supposed to be personalised investment or monetary advice. Motley Fool Singapore contributor Stanley Lim would not personal shares in any corporations talked about. Singapore private home costs increased by 1.eight% within the fourth quarter of 2012, up from 0.6% within the earlier quarter. Resale prices of government-built HDB residences which are usually bought by Singaporeans, elevated by 2.5%, quarter on quarter, the quickest acquire in five quarters. And industrial property, prices are actually double the levels of three years ago. No withholding tax in the event you sell your property. All your local information regarding vital HDB policies, condominium launches, land growth, commercial property and more
There are various methods to go about discovering the precise property. Some local newspapers (together with the Straits Instances ) have categorised property sections and many local property brokers have websites. Now there are some specifics to consider when buying a 'new launch' rental. Intended use of the unit Every sale begins with 10 p.c low cost for finish of season sale; changes to 20 % discount storewide; follows by additional reduction of fiftyand ends with last discount of 70 % or extra. Typically there is even a warehouse sale or transferring out sale with huge mark-down of costs for stock clearance. Deborah Regulation from Expat Realtor shares her property market update, plus prime rental residences and houses at the moment available to lease Esparina EC @ Sengkang