Topos: Difference between revisions

From formulasearchengine
Jump to navigation Jump to search
← Older edit
en>Kephir
en>TakuyaMurata
 
Line 1: Line 1:
In [[information theory]] and [[statistics]], '''Kullback's inequality''' is a lower bound on the [[Kullback–Leibler divergence]] expressed in terms of the [[large deviations theory|large deviations]] [[rate function]].<ref>Aimé Fuchs and Giorgio Letta, ''L'inégalité de Kullback. Application à la théorie de l'estimation.'' Séminaire de probabilités (Strasbourg), vol. 4, pp. 108-131, 1970.  http://www.numdam.org/item?id=SPS_1970__4__108_0</ref>  If ''P'' and ''Q'' are [[probability distribution]]s on the real line, such that ''P'' is '''absolutely continuous''' with respect to ''Q'', i.e. ''P''<<''Q'', and whose first moments exist, then
The main advantage of using the blog is that anyone can use the Word - Press blog and customize the elements in the theme regardless to limited knowledge about internet and website development. Thus, it is important to keep pace with this highly advanced age and have a regular interaction with your audience to keep a strong hold in the business market. A pinch of tablet centric strategy can get your Word - Press site miles ahead of your competitors, so here are few strategies that will give your Wordpress websites and blogs an edge over your competitors:. s and intelligently including a substantial amount of key words in the title tags, image links, etc. This particular wordpress plugin is essential for not only having the capability where you improve your position, but to enhance your organic searches for your website. <br><br>
:<math>D_{KL}(P\|Q) \ge \Psi_Q^*(\mu'_1(P)),</math>
where <math>\Psi_Q^*</math> is the rate function, i.e. the [[convex conjugate]] of the [[cumulant]]-generating function, of <math>Q</math>, and <math>\mu'_1(P)</math> is the first [[Moment (mathematics)|moment]] of <math>P.</math>


The [[Cramér–Rao bound]] is a corollary of this result.
Word - Press is known as the most popular blogging platform all over the web and is used by millions of blog enthusiasts worldwide. Some of the Wordpress development services offered by us are:. Our Daily Deal Software plugin brings the simplicity of setting up a Word - Press blog to the daily deal space. Being able to help with your customers can make a change in how a great deal work, repeat online business, and referrals you'll be given. As soon as you start developing your Word - Press MLM website you'll see how straightforward and simple it is to create an online presence for you and the products and services you offer. <br><br>You can down load it here at this link:  and utilize your FTP software program to upload it to your Word - Press Plugin folder. It was also the very first year that the category of Martial Arts was included in the Parents - Connect nationwide online poll, allowing parents to vote for their favorite San Antonio Martial Arts Academy. Those who cannot conceive with donor eggs due to some problems can also opt for surrogacy option using the services of surrogate mother. You or your web designer can customize it as per your specific needs. Socrates: (link to  ) Originally developed for affiliate marketers, I've used this theme again and again to develop full-fledged web sites that include static pages, squeeze pages, and a blog. <br><br>Word - Press installation is very easy and hassle free. But the Joomla was created as the CMS over years of hard work. Next you'll go by way of to your simple Word - Press site. So, we have to add our social media sharing buttons in website. If your site does well you can get paid professional designer to create a unique Word - Press theme. <br><br>Website security has become a major concern among individuals all over the world. If you liked this posting and you would like to obtain far more data regarding [http://clubciclistatorrejon.com/?attachment_id=36/RK=0 wordpress dropbox backup] kindly pay a visit to our own web-page. Automated deal feed integration option to populate your blog with relevant deals. It's not a secret that a lion share of activity on the internet is takes place on the Facebook. If this is not possible you still have the choice of the default theme that is Word - Press 3. Get started today so that people searching for your type of business will be directed to you.
 
==Proof==
Let ''P'' and ''Q'' be [[probability distribution]]s (measures) on the real line, whose first moments exist, and such that [[Absolutely_continuous#Absolute_continuity_of_measures|''P''<<''Q'']]. Consider the '''[[natural exponential family]]''' of ''Q'' given by
:<math>Q_\theta(A) = \frac{\int_A e^{\theta x}Q(dx)}{\int_{-\infty}^\infty e^{\theta x}Q(dx)}
  = \frac{1}{M_Q(\theta)} \int_A e^{\theta x}Q(dx)</math>
for every measurable set ''A'', where <math>M_Q</math> is the '''[[moment-generating function]]''' of ''Q''. (Note that ''Q''<sub>0</sub>=''Q''.)  Then
:<math>D_{KL}(P\|Q) = D_{KL}(P\|Q_\theta)
  + \int_{\mathrm{supp}P}\left(\log\frac{\mathrm dQ_\theta}{\mathrm dQ}\right)\mathrm dP.</math>
By [[Gibbs' inequality]] we have <math>D_{KL}(P\|Q_\theta) \ge 0</math> so that
:<math>D_{KL}(P\|Q) \ge
  \int_{\mathrm{supp}P}\left(\log\frac{\mathrm dQ_\theta}{\mathrm dQ}\right)\mathrm dP
= \int_{\mathrm{supp}P}\left(\log\frac{e^{\theta x}}{M_Q(\theta)}\right) P(dx)</math>
Simplifying the right side, we have, for every real θ where <math>M_Q(\theta) < \infty:</math>
:<math>D_{KL}(P\|Q) \ge \mu'_1(P) \theta - \Psi_Q(\theta),</math>
where <math>\mu'_1(P)</math> is the first moment, or mean, of ''P'', and <math>\Psi_Q = \log M_Q</math> is called the '''[[cumulant|cumulant-generating function]]'''. Taking the supremum completes the process of [[convex conjugate|convex conjugation]] and yields the [[rate function]]:
:<math>D_{KL}(P\|Q) \ge \sup_\theta \left\{ \mu'_1(P) \theta - \Psi_Q(\theta) \right\}
  = \Psi_Q^*(\mu'_1(P)).</math>
 
==Corollary: the Cramér–Rao bound==
{{main|Cramér–Rao bound}}
===Start with Kullback's inequality===
Let ''X''<sub>θ</sub> be a family of probability distributions on the real line indexed by the real parameter θ, and satisfying certain [[Cramér–Rao_bound#Regularity_conditions|regularity conditions]]. Then
:<math> \lim_{h\rightarrow 0} \frac {D_{KL}(X_{\theta+h}\|X_\theta)} {h^2}
    \ge \lim_{h\rightarrow 0} \frac {\Psi^*_\theta (\mu_{\theta+h})}{h^2},
</math>
 
where <math>\Psi^*_\theta</math> is the [[convex conjugate]] of the [[Cumulant|cumulant-generating function]] of <math>X_\theta</math> and <math>\mu_{\theta+h}</math> is the first moment of <math>X_{\theta+h}.</math>
 
===Left side===
The left side of this inequality can be simplified as follows:
:<math>\lim_{h\rightarrow 0}
      \frac {D_{KL}(X_{\theta+h}\|X_\theta)} {h^2}
      =\lim_{h\rightarrow 0}
      \frac 1 {h^2}
      \int_{-\infty}^\infty \left( \log\frac{\mathrm dX_{\theta+h}}{\mathrm dX_\theta} \right)
      \mathrm dX_{\theta+h}
</math>
 
:<math>  = \lim_{h\rightarrow 0} \frac 1 {h^2} \int_{-\infty}^\infty \left[
            \left( 1 - \frac{\mathrm dX_\theta}{\mathrm dX_{\theta+h}} \right)
+\frac 1 2 \left( 1 - \frac{\mathrm dX_\theta}{\mathrm dX_{\theta+h}} \right) ^ 2
+ o \left( \left( 1 - \frac{\mathrm dX_\theta}{\mathrm dX_{\theta+h}} \right) ^ 2 \right)
          \right]\mathrm dX_{\theta+h},
</math>
::where we have expanded the logarithm <math>\log x</math> in a [[Taylor series]] in <math>1-1/x</math>,
:<math>  = \lim_{h\rightarrow 0} \frac 1 {h^2} \int_{-\infty}^\infty \left[
  \frac 1 2 \left( 1 - \frac{\mathrm dX_\theta}{\mathrm dX_{\theta+h}} \right) ^ 2
          \right]\mathrm dX_{\theta+h}
</math>
:<math>
        = \lim_{h\rightarrow 0} \frac 1 {h^2} \int_{-\infty}^\infty \left[
  \frac 1 2 \left( \frac{\mathrm dX_{\theta+h} - \mathrm dX_\theta}{\mathrm dX_{\theta+h}} \right) ^ 2
          \right]\mathrm dX_{\theta+h}
= \frac 1 2 \mathcal I_X(\theta),</math>
which is half the [[Fisher information]] of the parameter θ.
 
===Right side===
The right side of the inequality can be developed as follows:
:<math>
  \lim_{h\rightarrow 0} \frac {\Psi^*_\theta (\mu_{\theta+h})}{h^2}
= \lim_{h\rightarrow 0} \frac 1 {h^2} {\sup_t \{\mu_{\theta+h}t - \Psi_\theta(t)\} }.
</math>
This supremum is attained at a value of ''t''=τ where the first derivative of the cumulant-generating function is <math>\Psi'_\theta(\tau) = \mu_{\theta+h},</math> but we have <math>\Psi'_\theta(0) = \mu_\theta,</math> so that
:<math>\Psi''_\theta(0) = \frac{d\mu_\theta}{d\theta} \lim_{h \rightarrow 0} \frac h \tau.</math>
Moreover,
:<math>\lim_{h\rightarrow 0} \frac {\Psi^*_\theta (\mu_{\theta+h})}{h^2}
  = \frac 1 {2\Psi''_\theta(0)}\left(\frac {d\mu_\theta}{d\theta}\right)^2
  = \frac 1 {2\mathrm{Var}(X_\theta)}\left(\frac {d\mu_\theta}{d\theta}\right)^2.</math>
===Putting both sides back together===
We have:
:<math>\frac 1 2 \mathcal I_X(\theta)
  \ge \frac 1 {2\mathrm{Var}(X_\theta)}\left(\frac {d\mu_\theta}{d\theta}\right)^2,</math>
which can be rearranged as:
:<math>\mathrm{Var}(X_\theta) \ge \frac{(d\mu_\theta / d\theta)^2} {\mathcal I_X(\theta)}.</math>
 
==See also==
* [[Kullback–Leibler divergence]]
* [[Cramér–Rao bound]]
* [[Fisher information]]
* [[Large deviations theory]]
* [[Convex conjugate]]
* [[Rate function]]
* [[Moment-generating function]]
 
==Notes and references==
<references/>
 
{{DEFAULTSORT:Kullback's Inequality}}
[[Category:Information theory]]
[[Category:Statistical inequalities]]
[[Category:Estimation theory]]

Latest revision as of 21:45, 28 November 2014

The main advantage of using the blog is that anyone can use the Word - Press blog and customize the elements in the theme regardless to limited knowledge about internet and website development. Thus, it is important to keep pace with this highly advanced age and have a regular interaction with your audience to keep a strong hold in the business market. A pinch of tablet centric strategy can get your Word - Press site miles ahead of your competitors, so here are few strategies that will give your Wordpress websites and blogs an edge over your competitors:. s and intelligently including a substantial amount of key words in the title tags, image links, etc. This particular wordpress plugin is essential for not only having the capability where you improve your position, but to enhance your organic searches for your website.

Word - Press is known as the most popular blogging platform all over the web and is used by millions of blog enthusiasts worldwide. Some of the Wordpress development services offered by us are:. Our Daily Deal Software plugin brings the simplicity of setting up a Word - Press blog to the daily deal space. Being able to help with your customers can make a change in how a great deal work, repeat online business, and referrals you'll be given. As soon as you start developing your Word - Press MLM website you'll see how straightforward and simple it is to create an online presence for you and the products and services you offer.

You can down load it here at this link: and utilize your FTP software program to upload it to your Word - Press Plugin folder. It was also the very first year that the category of Martial Arts was included in the Parents - Connect nationwide online poll, allowing parents to vote for their favorite San Antonio Martial Arts Academy. Those who cannot conceive with donor eggs due to some problems can also opt for surrogacy option using the services of surrogate mother. You or your web designer can customize it as per your specific needs. Socrates: (link to ) Originally developed for affiliate marketers, I've used this theme again and again to develop full-fledged web sites that include static pages, squeeze pages, and a blog.

Word - Press installation is very easy and hassle free. But the Joomla was created as the CMS over years of hard work. Next you'll go by way of to your simple Word - Press site. So, we have to add our social media sharing buttons in website. If your site does well you can get paid professional designer to create a unique Word - Press theme.

Website security has become a major concern among individuals all over the world. If you liked this posting and you would like to obtain far more data regarding wordpress dropbox backup kindly pay a visit to our own web-page. Automated deal feed integration option to populate your blog with relevant deals. It's not a secret that a lion share of activity on the internet is takes place on the Facebook. If this is not possible you still have the choice of the default theme that is Word - Press 3. Get started today so that people searching for your type of business will be directed to you.

Retrieved from "https://en.formulasearchengine.com/index.php?title=Topos&oldid=263382"