Curse of dimensionality

In mathematics, concentration of measure (about a median) is a principle that is applied in measure theory, probability and combinatorics, and has consequences for other fields such as Banach space theory. Informally, it states that "A random variable that depends in a Lipschitz way on many independent variables (but not too much on any of them) is essentially constant". ^[1]

The c.o.m. phenomenon was put forth in the early 1970s by Vitali Milman in his works on the local theory of Banach spaces, extending an idea going back to the work of Paul Lévy.^[2]^[3] It was further developed in the works of Milman and Gromov, Maurey, Pisier, Shechtman, Talagrand, Ledoux, and others.

The general setting

Let $(X, d, μ)$ be a metric measure space, $μ (X) = 1$ . Let

α (ϵ) = \sup {μ (X ∖ A_{ϵ}) | μ (A) \geq 1 / 2},

where

A_{ϵ} = {x | d (x, A) < ϵ}

is the $ϵ$ -extension of a set $A$ .

The function $α (\cdot)$ is called the concentration rate of the space $X$ . The following equivalent definition has many applications:

α (ϵ) = \sup {μ ({F \geq M + ϵ})},

where the supremum is over all 1-Lipschitz functions $F : X \to ℝ$ , and the median (or Levy mean) $M = M e d F$ is defined by the inequalities

μ {F \geq M} \geq 1 / 2, μ {F \leq M} \geq 1 / 2 .

Informally, the space $X$ exhibits a concentration phenomenon if $α (ϵ)$ decays very fast as $ϵ$ grows. More formally, a family of metric measure spaces $(X_{n}, d_{n}, μ_{n})$ is called a Lévy family if the corresponding concentration rates $α_{n}$ satisfy

\forall ϵ > 0 α_{n} (ϵ) \to 0 a s n \to \infty,

and a normal Lévy family if

\forall ϵ > 0 α_{n} (ϵ) \leq C \exp (- c n ϵ^{2})

for some constants $c, C > 0$ . For examples see below.

Concentration on the sphere

The first example goes back to Paul Lévy. According to the spherical isoperimetric inequality, among all subsets $A$ of the sphere $S^{n}$ with prescribed spherical measure $σ_{n} (A)$ , the spherical cap

{x \in S^{n} | d i s t (x, x_{0}) \leq R}

has the smallest $ϵ$ -extension $A_{ϵ}$ (for any $ϵ > 0$ ).

Applying this to sets of measure $σ_{n} (A) = 1 / 2$ (where $σ_{n} (S^{n}) = 1$ ), one can deduce the following concentration inequality:

σ_{n} (A_{ϵ}) \geq 1 - C \exp (- c n ϵ^{2})

,

where $C, c$ are universal constants.

Therefore $(S^{n})_{n}$ form a normal Lévy family.

Vitali Milman applied this fact to several problems in the local theory of Banach spaces, in particular, to give a new proof of Dvoretzky's theorem.

Other examples

Footnotes

↑ Michel Talagand, A New Look at Independence, The Annals of Probability, 1996, Vol. 24, No.1, 1-34
↑ "The concentration of $f_{*} (μ)$ , ubiquitous in the probability theory and statistical mechanics, was brought to geometry (starting from Banach spaces) by Vitali Milman, following the earlier work by Paul Lévy" - M. Gromov, Spaces and questions, GAFA 2000 (Tel Aviv, 1999), Geom. Funct. Anal. 2000, Special Volume, Part I, 118–161.
↑ "The idea of concentration of measure (which was discovered by V.Milman) is arguably one of the great ideas of analysis in our times. While its impact on Probability is only a small part of the whole picture, this impact should not be ignored." - M. Talagrand, A new look at independence, Ann. Probab. 24 (1996), no. 1, 1–34.

Curse of dimensionality

Contents

The general setting

Concentration on the sphere

Other examples

Footnotes

Further reading

Navigation menu

Curse of dimensionality

The general setting

Concentration on the sphere

Other examples

Footnotes

Further reading

Navigation menu

Search