Baux score

2013-12-11T15:36:19Z

120.144.0.179: Fixed formatting of mathematical expression.

[[File:Sauer–Shelah lemma.svg|thumb|300px|Pajor's formulation of the Sauer–Shelah lemma: for every finite family of sets (green) there is another family of equally many sets (blue outlines) such that each set in the second family is shattered by the first family]]
In [[combinatorics|combinatorial mathematics]] and [[extremal set theory]], the '''Sauer–Shelah lemma''' states that every [[family of sets]] with small [[VC dimension]] consists of a small number of sets. It is named after [[Norbert Sauer]] and [[Saharon Shelah]], who published it independently of each other in 1972.<ref>{{citation
| last = Sauer | first = N.
| journal = [[Journal of Combinatorial Theory]]
| mr = 0307902
| pages = 145–147
| series = Series A
| title = On the density of families of sets
| volume = 13
| year = 1972}}.</ref><ref>{{citation
| last = Shelah | first = Saharon
| journal = Pacific Journal of Mathematics
| mr = 0307903
| pages = 247–261
| title = A combinatorial problem; stability and order for models and theories in infinitary languages
| url = http://projecteuclid.org/getRecord?id=euclid.pjm/1102968432
| volume = 41
| year = 1972}}.</ref> The same result was also published slightly earlier and again independently, by [[Vladimir Vapnik]] and [[Alexey Chervonenkis]], after whom the VC dimension is named.<ref>{{citation
| last1 = Vapnik | first1 = V. N. | author1-link = Vladimir Vapnik
| last2 = Červonenkis | first2 = A. Ja. | author2-link = Alexey Chervonenkis
| journal = Akademija Nauk SSSR
| mr = 0288823
| pages = 264–279
| title = The uniform convergence of frequencies of the appearance of events to their probabilities
| volume = 16
| year = 1971}}.</ref> In his paper containing the lemma, Shelah gives credit also to [[Micha Perles]], and for this reason the lemma has also been called the '''Perles–Sauer–Shelah lemma'''.<ref name="brr">{{citation
| last1 = Buzaglo | first1 = Sarit
| last2 = Pinchasi | first2 = Rom
| last3 = Rote | first3 = Günter
| editor-last = Pach | editor-first = János | editor-link = János Pach
| contribution = Topological hypergraphs
| doi = 10.1007/978-1-4614-0110-0_6
| pages = 71–81
| publisher = Springer
| title = Thirty Essays on Geometric Graph Theory
| year = 2013}}.</ref>

Buzaglo et al. call this lemma "one of the most fundamental results on VC-dimension",<ref name="brr"/> and it has applications in many areas. Sauer's motivation was in the [[combinatorics]] of set systems, while Shelah's was in [[model theory]] and that of Vapnik and Chervonenkis was in [[statistics]]. It has also been applied in [[discrete geometry]]<ref name="pa95">{{citation
| last1 = Pach | first1 = János | author1-link = János Pach
| last2 = Agarwal | first2 = Pankaj K. | author2-link = Pankaj K. Agarwal
| doi = 10.1002/9781118033203
| isbn = 0-471-58890-3
| location = New York
| mr = 1354145
| page = 247
| publisher = John Wiley & Sons Inc.
| series = Wiley-Interscience Series in Discrete Mathematics and Optimization
| title = Combinatorial geometry
| year = 1995}}.</ref> and [[graph theory]].<ref name="km13">{{citation
| last1 = Kozma | first1 = László
| last2 = Moran | first2 = Shay
| arxiv = 1211.1319
| journal = [[Electronic Journal of Combinatorics]]
| volume = 20 | issue = 3 | at = P44
| title = Shattering, Graph Orientations, and Connectivity
| url = http://www.combinatorics.org/ojs/index.php/eljc/article/view/v20i3p44
| year = 2013}}.</ref>

==Definitions and statement==
If <math>\mathcal{F}=\{S_1,S_2,\dots\}</math> is a family of sets, and <math>T</math> is another set, then <math>T</math> is said to be [[Shattered set|shattered]] by <math>\mathcal{F}</math> if every subset of <math>T</math> (including the [[empty set]] and <math>T</math> itself) can be obtained as an intersection <math>T\cap S_i</math> between <math>T</math> and a set in the family. The VC dimension of <math>\mathcal{F}</math> is the largest [[cardinality]] of a set shattered by <math>\mathcal{F}</math>.

In terms of these definitions, the Sauer–Shelah lemma states that if <math>\mathcal{F}</math> is a family of sets with <math>n</math> distinct elements such that
<math> |\mathcal{F}| > \sum_{i=0}^{k-1} {\binom{n}{i}} </math>, then <math>\mathcal{F}</math> shatters a set of size <math>k</math>. Equivalently, if the VC dimension of <math>\mathcal{F}</math> is <math>k</math>, then <math>\mathcal{F}</math> can consist of at most <math>\sum_{i=0}^{k} {\binom{n}{i}} =O(n^k)</math> sets.

The bound of the lemma is tight: there exists a family <math>\mathcal{F}</math> with <math> |\mathcal{F}| = \sum_{i=0}^{k-1} {\binom{n}{i}} </math> that does not shatter any set of size <math>k</math>. Namely, let <math>\mathcal{F}</math> be the family of all subsets of <math>\{1,2,\dots n\}</math> that have cardinality less than <math>k</math>.<ref name="gowers">{{citation|title=Dimension arguments in combinatorics|contribution=Example 3|first=Timothy|last=Gowers|authorlink=Timothy Gowers|work=Gowers's Weblog: Mathematics related discussions|url=http://gowers.wordpress.com/2008/07/31/dimension-arguments-in-combinatorics/|date=July 31, 2008}}.</ref>

==The number of shattered sets==
A strengthening of the Sauer–Shelah lemma, due to {{harvtxt|Pajor|1985}}, states that every finite set family <math>\mathcal{F}</math>shatters at least <math>|\mathcal{F}|</math> sets.<ref>{{citation
| last = Pajor | first = Alain
| isbn = 2-7056-6021-6
| location = Paris
| mr = 903247
| publisher = Hermann
| series = Travaux en Cours [Works in Progress]
| title = Sous-espaces <math>l^n_1</math> des espaces de Banach
| volume = 16
| year = 1985}}. As cited by {{harvtxt|Anstee|Rónyai|Sali|2002}}.</ref> This immediately implies the Sauer–Shelah lemma, because only <math>\sum_{i=0}^{k-1} {\tbinom{n}{i}} </math> of the subsets of an <math>n</math>-item universe have cardinality less than <math>k</math>. Thus, when <math>|\mathcal{F}|>\sum_{i=0}^{k-1} {\tbinom{n}{i}}</math>, there are not enough small sets to be shattered, so one of the shattered sets must have cardinality at least <math>k</math>.

For a restricted type of shattered set, called an order-shattered set, the number of shattered sets always equals the cardinality of the set family.<ref name="ars02"/>

==Proof==
Pajor's variant of the Sauer–Shelah lemma may be proved by [[mathematical induction]]; the proof has variously been credited to [[Noga Alon]]<ref>{{citation|url=http://gilkalai.wordpress.com/2008/09/28/extremal-combinatorics-iii-some-basic-theorems/|first=Gil|last=Kalai|authorlink=Gil Kalai|title=Extremal Combinatorics III: Some Basic Theorems|work=Combinatorics and More|date=September 28, 2008}}.</ref> or to [[Ron Aharoni]] and Ron Holzman.<ref name="ars02">{{citation
| last1 = Anstee | first1 = R. P.
| last2 = Rónyai | first2 = Lajos
| last3 = Sali | first3 = Attila
| doi = 10.1007/s003730200003
| issue = 1
| journal = Graphs and Combinatorics
| mr = 1892434
| pages = 59–73
| title = Shattering news
| volume = 18
| year = 2002}}.</ref> As a base case to the induction, every family of only one set shatters the empty set. To see that every finite family <math>\mathcal{F}</math> of two or more sets shatters at least <math>|\mathcal{F}|</math> sets, let <math>x</math> be an element that belongs to some but not all of the sets in <math>\mathcal{F}</math>. Split <math>\mathcal{F}</math> into two subfamilies, of the sets that contain <math>x</math> and the sets that do not contain <math>x</math>. By induction, these two subfamilies shatter two collections of sets whose sizes add to at least <math>|\mathcal{F}|</math>. None of these shattered sets contain <math>x</math>, but some of them may be shattered by both subfamilies. When a set <math>S</math> is shattered by only one of the two subfamilies, it contributes one unit both to the number of shattered sets of the subfamily and to the number of shattered sets of <math>\mathcal{F}</math>. When a set <math>S</math> is shattered by both subfamilies, then both <math>S</math> and <math>S\cup\{x\}</math> are shattered by <math>\mathcal{F}</math>, and <math>S</math> contributes two units to the number of shattered sets of the subfamilies and of <math>\mathcal{F}</math>. Therefore, the number of shattered sets of <math>\mathcal{F}</math> is at least equal to the number shattered by the two subfamilies of <math>\mathcal{F}</math>, which is at least <math>|\mathcal{F}|</math>.

A different proof of the Sauer–Shelah lemma in its original form, by [[Péter Frankl]] and [[János Pach]], is based on [[linear algebra]] and the [[inclusion–exclusion principle]].<ref name="pa95"/><ref name="gowers"/>

==Applications==
The original application of the lemma, by Vapnik and Chervonenkis, was in showing that every probability distribution can be approximated (with respect to a family of events of a given VC dimension) by a finite set of sample points whose [[cardinality]] depends only on the VC dimension of the family of events. In this context, there are two important notions of approximation, both parameterized by a number ε: a set ''S'' of samples, and a probability distribution on ''S'', is said to be an ε-approximation of the original distribution if the probability of each event with respect to ''S'' differs from its original probability by at most ε. A set ''S'' of (unweighted) samples is said to be an [[ε-net (computational geometry)|ε-net]] if every event with probability at least ε includes at least one point of ''S''. An ε-approximation must also be an ε-net but not necessarily vice versa.

Vapnik and Chervonenkis used the lemma to show that set systems of VC dimension ''d'' always have ε-approximations of cardinality <math>O(\tfrac{d}{\epsilon^2}\log\tfrac{d}{\epsilon})</math>. Later authors including {{harvtxt|Haussler|Welzl|1987}}<ref name="hw87">{{citation
| last1 = Haussler | first1 = David | author1-link = David Haussler
| last2 = Welzl | first2 = Emo | author2-link = Emo Welzl
| doi = 10.1007/BF02187876
| issue = 2
| journal = [[Discrete and Computational Geometry]]
| mr = 884223
| pages = 127–151
| title = ε-nets and simplex range queries
| volume = 2
| year = 1987}}.</ref> and {{harvtxt|Komlós|Pach|Woeginger|1992}}<ref>{{citation
| last1 = Komlós | first1 = János | author1-link = János Komlós (mathematician)
| last2 = Pach | first2 = János | author2-link = János Pach
| last3 = Woeginger | first3 = Gerhard
| doi = 10.1007/BF02187833
| issue = 2
| journal = [[Discrete and Computational Geometry]]
| mr = 1139078
| pages = 163–173
| title = Almost tight bounds for ε-nets
| volume = 7
| year = 1992}}.</ref> similarly showed that there always exist ε-nets of cardinality <math>O(\tfrac{d}{\epsilon}\log\tfrac{1}{\epsilon})</math>, and more precisely of cardinality at most <math>\tfrac{d}{\epsilon}\ln\tfrac{1}{\epsilon}+\tfrac{2d}{\epsilon}\ln\ln\tfrac{1}{\epsilon}+\tfrac{6d}{\epsilon}</math>.<ref name="pa95"/> The main idea of the proof of the existence of small ε-nets is to choose a random sample ''x'' of cardinality <math>O(\tfrac{d}{\epsilon}\log\tfrac{1}{\epsilon})</math> and a second independent random sample ''y'' of cardinality <math>O(\tfrac{d}{\epsilon}\log^2\tfrac{1}{\epsilon})</math>, and to bound the probability that ''x'' is missed by some large event ''E'' by the probability that ''x'' is missed and simultaneously the intersection of ''y'' with ''E'' is larger than its median value. For any particular ''E'', the probability that ''x'' is missed while ''y'' is larger than its median is very small,
and the Sauer–Shelah lemma (applied to <math>x\cup y</math>) shows that only a small number of distinct events ''E'' need to be considered, so by the [[Boole's inequality|union bound]], with nonzero probability, ''x'' is an ε-net.<ref name="pa95"/>

In turn, ε-nets and ε-approximations, and the likelihood that a random sample of large enough cardinality has these properties, have important applications in [[machine learning]], in the area of [[probably approximately correct learning]].<ref>{{citation
| last1 = Blumer | first1 = Anselm
| last2 = Ehrenfeucht | first2 = Andrzej
| last3 = Haussler | first3 = David | author3-link = David Haussler
| last4 = Warmuth | first4 = Manfred K.
| doi = 10.1145/76359.76371
| issue = 4
| journal = [[Journal of the ACM]]
| mr = 1072253
| pages = 929–965
| title = Learnability and the Vapnik–Chervonenkis dimension
| volume = 36
| year = 1989}}.</ref> In [[computational geometry]], they have been applied to [[range searching]],<ref name="hw87"/> [[Randomized algorithm#Derandomization|derandomization]],<ref>{{citation
| last1 = Chazelle | first1 = B. | author1-link = Bernard Chazelle
| last2 = Friedman | first2 = J.
| doi = 10.1007/BF02122778
| issue = 3
| journal = Combinatorica
| mr = 1092541
| pages = 229–249
| title = A deterministic view of random sampling and its use in geometry
| volume = 10
| year = 1990}}.</ref> and [[approximation algorithm]]s.<ref>{{citation
| last1 = Brönnimann | first1 = H.
| last2 = Goodrich | first2 = M. T. | author2-link = Michael T. Goodrich
| doi = 10.1007/BF02570718
| issue = 4
| journal = [[Discrete and Computational Geometry]]
| mr = 1360948
| pages = 463–479
| title = Almost optimal set covers in finite VC-dimension
| volume = 14
| year = 1995}}.</ref><ref>{{citation
| last = Har-Peled | first = Sariel
| contribution = On complexity, sampling, and ε-nets and ε-samples
| isbn = 978-0-8218-4911-8
| location = Providence, RI
| mr = 2760023
| pages = 61–85
| publisher = American Mathematical Society
| series = Mathematical Surveys and Monographs
| title = Geometric approximation algorithms
| volume = 173
| year = 2011}}.</ref>

{{harvtxt|Kozma|Moran|2013}} use generalizations of the Sauer–Shelah lemma to prove results in [[graph theory]] such as that the number of [[strong orientation]]s of a given graph is sandwiched between its numbers of [[connected graph|connected]] and [[Bridge (graph theory)#Bridgeless graphs|2-edge-connected]] subgraphs.<ref name="km13"/>

== References ==
{{reflist}}

{{DEFAULTSORT:Sauer-Shelah lemma}}
[[Category:Set families]]

formulasearchengine - User contributions [en]

Baux score