|
|
| Line 1: |
Line 1: |
| The '''exponential mechanism''' is a technique for designing differentially private algorithms developed by [http://research.microsoft.com/en-us/people/mcsherry/ Frank McSherry] and [http://research.microsoft.com/en-us/people/kunal/default.aspx Kunal Talwar]. [[Differential privacy]] is a technique for releasing statistical information about a database without revealing information about its individual entries. <br>
| | If you present photography effectively, it helps you look much more properly at the globe around you. This means you can setup your mailing list and auto-responder on your wordpress site and then you can add your subscription form to any other blog, splash page, capture page or any other site you like. Should you go with simple HTML or use a platform like Wordpress. They found out all the possible information about bringing up your baby and save money at the same time. You can customize the appearance with PSD to Word - Press conversion ''. <br><br>These folders as well as files have to copied and the saved. If you wish to sell your services or products via internet using your website, you have to put together on the website the facility for trouble-free payment transfer between customers and the company. We also help to integrate various plug-ins to expand the functionalities of the web application. This is identical to doing a research as in depth above, nevertheless you can see various statistical details like the number of downloads and when the template was not long ago updated. This can be done by using a popular layout format and your unique Word - Press design can be achieved in other elements of the blog. <br><br>The least difficult and very best way to do this is by acquiring a Word - Press site. By using Word - Press MLM websites or blogs, you and your companies presence on the internet can be created swiftly and simply. If you loved this short article and you wish to receive much more information regarding [http://l4.vc/wordpress_backup_142636 wordpress dropbox backup] i implore you to visit our internet site. all the necessary planning and steps of conversion is carried out in this phase, such as splitting, slicing, CSS code, adding images, header footer etc. Thousands of plugins are available in Word - Press plugin's library which makes the task of selecting right set of plugins for your website a very tedious task. " Thus working with a Word - Press powered web application, making any changes in the website design or website content is really easy and self explanatory. <br><br>Digg Digg Social Sharing - This plugin that is accountable for the floating social icon located at the left aspect corner of just about every submit. Quttera - Quttera describes itself as a 'Saa - S [Software as a Service] web-malware monitoring and alerting solution for websites of any size and complexity. re creating a Word - Press design yourself, the good news is there are tons of Word - Press themes to choose from. IVF ,fertility,infertility expert,surrogacy specialist in India at Rotundaivf. It does take time to come up having a website that gives you the much needed results hence the web developer must be ready to help you along the route. <br><br>Millions of individuals and organizations are now successfully using this tool throughout the world. If you operate a website that's been built on HTML then you might have to witness traffic losses because such a site isn't competent enough in grabbing the attention of potential consumers. You can select color of your choice, graphics of your favorite, skins, photos, pages, etc. You should stay away from plugins that are full of flaws and bugs. You can check out the statistics of page of views for your web pages using free tools that are available on the internet. |
| Most of the initial research in the field of [[differential privacy]] revolved around real valued functions which have relatively low [[Differential privacy|sensitivity]] to change in the data of a single individual and whose usefulness is not hampered by small additive perturbations. A natural question is what happens in the situation when one wants to preserve more general sets of properties. The Exponential Mechanism helps to extend the notion of differential privacy to address these issues. Moreover, it describes a class of mechanisms that includes all possible differentially private mechanisms.
| |
| | |
| == The exponential mechanism <ref>[http://research.microsoft.com/pubs/65075/mdviadp.pdf F.McSherry and K.Talwar. Mechasim Design via Differential Privacy. Proceedings of the 48th Annual Symposium of Foundations of Computer Science, 2007.]</ref> ==
| |
| | |
| === Algorithm ===
| |
| In very generic terms a privacy mechanism maps a set of <math>n\,\!</math> inputs from domain <math>\mathcal{D}\,\!</math>, to a range <math>\mathcal{R}\,\!</math>. The map may be randomized, in which case each element of the domain <math>D\,\!</math> corresponds to the probability distribution over the range <math>R\,\!</math>. The privacy mechanism we are going to design makes no assumption about the nature of <math>\mathcal{D}\,\!</math> and <math>\mathcal{R}\,\!</math> apart from a base [[Measure (mathematics)|measure]] <math>\mu\,\!</math> on <math>\mathcal{R}\,\!</math>. Let us define a function <math>q:\mathcal{D}^n\times\mathcal{R}\rightarrow\mathbb{R}\,\!</math>. Intuitively this function assigns score to the pair <math>(d,r)\,\!</math>, where <math>d\in\mathcal{D}^n\,\!</math> and <math>r\in\mathcal{R}\,\!</math>. The score reflects how appealing is the pair <math>(d,r)\,\!</math>, i.e. the higher the score, the more appealing the pair is.
| |
| Once we are given the input <math>d\in\mathcal{D}^n\,\!</math>, the mechanism's objective is to return an <math>r\in\mathcal{R}\,\!</math> such that the function <math>q(d,r)\,\!</math> is approximately maximized. To achieve this, we set up the mechanism <math>\mathcal{E}_{q}^{\epsilon}(d)\,\!</math> as follows: <br>
| |
| '''Definition:''' For any function <math>q:(\mathcal{D}^{n}\times\mathcal{R})\rightarrow\mathbb{R}\,\!</math>, and a base measure <math>\mu\,\!</math> over <math>\mathcal{R}\,\!</math>, we define:
| |
| :<math>\mathcal{E}_{q}^{\epsilon}(d):=\,\!</math> Choose <math>r\,\!</math> with probability proportional to <math>e^{\epsilon q(d,r)}\times\mu(r)\,\!</math>, where <math>d\in\mathcal{D}^n,r\in R\,\!</math>.
| |
| This definition implies the fact that the probability of returning an <math>r\,\!</math> increases exponentially with the increase in the value of <math>q(d,r)\,\!</math>. For now if we ignore the base measure <math>\mu\,\!</math> then the value <math>r\,\!</math> which maximizes <math>q(d,r)\,\!</math> has the highest probability. Moreover we claim that this mechanism is differentially private. We will prove this claim shortly. One technicality that should be kept in mind is that in order to properly define <math>\mathcal{E}_{q}^{\epsilon}(d)\,\!</math> the <math>\int_{r}e^{\epsilon q(d,r)}\times\mu(r)\,\!</math> should be finite.
| |
| | |
| '''Theorem (Differential Privacy):''' <math>\mathcal{E}_{q}^{\epsilon}(d)\,\!</math> gives <math>(2\epsilon\Delta q)\,\!</math>-differential privacy.
| |
| | |
| Proof: The probability density of <math>\mathcal{E}_{q}^{\epsilon}(d)\,\!</math> at <math>r\,\!</math> equals
| |
| :<math>\frac{e^{\epsilon q(d,r)}\mu(r)}{\int e^{\epsilon q(d,r)}\mu(r)dr}\,\!</math>.
| |
| Now, if a single change in <math>d\,\!</math> changes <math>q\,\!</math> by at most <math>\Delta q\,\!</math> then the numerator can change at most by a factor of <math>e^{\epsilon\Delta q}\,\!</math> and the denominator minimum by a factor of <math>e^{-\epsilon\Delta q}\,\!</math>. Thus, the ratio of the new probability density (i.e. with new <math>d\,\!</math>) and the earlier one is at most <math>\exp(2\epsilon\Delta q)\,\!</math>.
| |
| | |
| === Accuracy ===
| |
| | |
| We would ideally want the random draws of <math>r\,\!</math> from the mechanism <math>\mathcal{E}_{q}^{\epsilon}(d)\,\!</math> to nearly maximize <math>q(d,r)\,\!</math>. If we consider <math>\max_rq(d,r)\,\!</math> to be <math>OPT\,\!</math> then we can show that the probability of the mechanism deviating from <math>OPT\,\!</math>is low, as long as there is a sufficient mass (in terms of <math>\mu</math>) of values <math>r\,\!</math> with value <math>q\,\!</math> close to the optimum.
| |
| | |
| '''Lemma:''' Let <math>S_{t}=\{r:q(d,r)>OPT-t\}\,\!</math> and <math>\bar{S}_{2t}=\{r:q(d,r)\leq OPT-2t\}\,\!</math>, we have <math>p(\bar{S}_{2t})\,\!</math> is at most <math>\exp(-\epsilon t)/\mu(S_{t})\,\!</math>. The probability is taken over <math>R\,\!</math>.
| |
| | |
| Proof: The probability <math>p(\bar{S}_{2t})\,\!</math> is at most <math>p(\bar{S}_{2t})/p(S_t)\,\!</math>, as the denominator can be at most one. Since both the probabilities have the same normalizing term so,
| |
| | |
| :<math>\frac{p(\bar{S}_{2t})}{p(S_{t})} = \frac{\int_{\bar{S}_{2t}}\exp(\epsilon q(d,r))\mu(r) \, dr}{\int_{S_{t}} \exp(\epsilon q(d,r))\mu(r) \, dr} \leq \exp(-\epsilon t) \frac{\mu(\bar{S}_{2t})}{\mu(S_t)}. </math>
| |
| | |
| The value of <math>\mu(\bar{S}_{2t})\,\!</math> is at most one, and so this bound implies the lemma statement. | |
| | |
| '''Theorem (Accuracy):''' For those values of <math>t\geq \ln(\frac{OPT}{t\mu(S_{t})})/\epsilon\,\!</math>, we have <math> E[q(d,\mathcal{E}_{q}^{\epsilon}(d))]\geq OPT-3t\,\!</math>.
| |
| | |
| Proof: It follows from the previous lemma that the probability of the score being at least <math>OPT-2t\,\!</math> is <math>1-\exp(-\epsilon t)/\mu(S_{t})\,\!</math>. By Hypothesis, <math>t\geq \ln(\frac{OPT}{t\mu(S_{t})})/\epsilon\,\!</math>. Substituting the value of <math>t\,\!</math> we get this probability to be at least <math>1-t/OPT\,\!</math>. Multiplying with <math>OPT-2t\,\!</math> yields the desired bound.
| |
| | |
| We can assume <math>\mu(A)\,\!</math> for <math>A\subseteq \mathcal{R}\,\!</math> to be less than or equal to one in all the computations, because we can always normalize with <math>\mu(\mathcal{R})\,\!</math> .
| |
| | |
| == Example application of the exponential mechanism <ref>[http://www.cs.cmu.edu/~alroth/Papers/dataprivacy.pdf Avrim Blum,Katrina Ligett,Aaron Roth. A Learning Theory Approach to Non-Iteractive Database Privacy.In Proceedings of the 40th annual ACM symposium on Theory of computing, 2008]</ref>==
| |
| | |
| Before we get into the details of the example let us define some terms which we will be using extensively throughout our discussion.
| |
| | |
| '''Definition (global sensitivity):''' The global sensitivity of a query <math>Q\,\!</math> is its maximum difference when evaluated on two neighbouring datasets <math>D_1,D_2\in\mathcal{D}^n\,\!</math>:
| |
| :<math>GS_{Q}=\max_{D_1,D_2:d(D_1,D_2)=1}|(Q(D_1)-Q(D_2))|\,\!</math>.
| |
| | |
| '''Definition:''' A predicate query <math>Q_{\varphi}\,\!</math> for any predicate <math>\varphi\,\!</math> is defined to be
| |
| :<math>Q_{\varphi}=\frac{|\{x\in D:\varphi(x)\}|}{|D|}\,\!</math>.
| |
| | |
| Note that <math>GS_{Q_{\varphi}}\leq 1/n\,\!</math> for any predicate <math>\varphi\,\!</math>.
| |
| | |
| === Release mechanism ===
| |
| | |
| The following is due to [http://www.cs.cmu.edu/~avrim/ Avrim Blum], [http://www.cs.cmu.edu/~katrina/ Katrina Ligett] and [http://www.cs.cmu.edu/~alroth/ Aaron Roth].
| |
| | |
| '''Definition (Usefulness):''' A [http://cryptowiki.cse.psu.edu/mediawiki/index.php/CSE546-Spring-2009/Differential-Privacy mechanism] <math>\mathcal{A}\,\!</math> is <math>(\alpha,\delta)\,\!</math>-useful for queries in class <math>H\,\!</math> with probability <math>1-\delta\,\!</math>, if <math>\forall h\in H\,\!</math> and every dataset <math>D\,\!</math>, for <math>\widehat{D}=\mathcal{A}(D)\,\!</math>, <math>|Q_h(\widehat{D})-Q_h(D)|\leq \alpha\,\!</math>.
| |
| | |
| Informally, it means that with high probability the query <math>Q_{h}\,\!</math> will behave in a similar way on the original dataset <math>D\,\!</math> and on the synthetic dataset <math>\widehat{D}\,\!</math>. <br>
| |
| Let us consider a common problem in Data Mining. Assume there is a database <math>D\,\!</math> with <math>n\,\!</math> entries. Each entry consist of <math>k\,\!</math>-tuples of the form <math>(x_1,x_2,\dots,x_k)\,\!</math> where <math>x_{i}\in\{0,1\}\,\!</math>. Now, a user wants to learn a [[Half-space (geometry)|linear halfspace]] of the form <math>\pi_1 x_1 + \pi_2 x_2+\cdots+\pi_{k-1}x_{k-1}\geq x_{k}\,\!</math>. In essence the user wants to figure out the values of <math>\pi_1,\pi_2,\dots,\pi_{k-1}\,\!</math> such that maximum number of tuples in the database satisfy the inequality. The algorithm we describe below can generate a synthetic database <math>\widehat{D}\,\!</math> which will allow the user to learn (approximately) the same linear half-space while querying on this synthetic database. The motivation for such an algorithm being that the new database will be generated in a differentially private manner and thus asssure privacy to the individual records in the database <math>D\,\!</math>.
| |
| | |
| In this section we show that it is possible to release a dataset which is useful for concepts from a polynomial [[VC dimension|VC-Dimension]] class and at the same time adhere to <math>\epsilon\,\!</math>-differential privacy as long as the size of the original dataset is at least polynomial on the [[VC dimension|VC-Dimension]] of the concept class. To state formally:
| |
| | |
| '''Theorem:''' For any class of functions <math>H\,\!</math> and any dataset <math>D\subset \{0,1\}^{k}\,\!</math> such that
| |
| :<math>|D|\geq O\left(\frac{k\cdot VCDIM(H)\log(1/\alpha)}{\alpha^{3}\epsilon}+\frac{\log(1/\delta)}{\alpha\epsilon}\right)\,\!</math>
| |
| we can output an <math>(\alpha,\delta)\,\!</math>-useful dataset <math>\widehat{D}\,\!</math> that preserves <math>\epsilon\,\!</math>-differential privacy. As we had mentioned earlier the algorithm need not be efficient.
| |
| | |
| One interesting fact is that the algorithm which we are going to develop generates a synthetic dataset whose size is independent of the original dataset; in fact, it only depends on the [[VC dimension|VC-dimension]] of the concept class and the parameter <math>\alpha\,\!</math>. The algorithm outputs a dataset of size <math>\tilde{O}(VCDIM(H)/\alpha^{2})\,\!</math>
| |
| | |
| We borrow the [[Uniform convergence (combinatorics)|Uniform Convergence Theorem]] from [[combinatorics]] and state a corollary of it which aligns to our need.
| |
| | |
| '''Lemma:''' Given any dataset <math>D\,\!</math> there exists a dataset <math>\widehat{D}\,\!</math> of size <math>=O(VCDIM(H)\log(1/\alpha))/\alpha^{2}\,\!</math> such that <math>\max_{h\in H}|Q_{h}(D)-Q_{h}(\widehat{D})|\leq \alpha/2\,\!</math>.
| |
| | |
| Proof:
| |
| | |
| We know from the uniform convergence theorem that,
| |
| | |
| :<math>\Pr[|Q_{h}(D)-Q_{h}(\widehat{D})|\geq \alpha/2\,\!</math> for some <math>h\in H]\leq 2(\frac{em}{VCDIM(H)})^{VCDIM(H)}\cdot e^{-\frac{\alpha^{2}m}{8}}\,\!</math>,
| |
| where probability is over the distribution of the dataset.
| |
| Thus, if the RHS is less than one then we know for sure that the data set <math>\widehat{D}\,\!</math> exists. To bound the RHS to less than one we need <math>m\geq\lambda(VCDIM(H)\log(m/VCDIM(H))/\alpha^{2})\,\!</math>, where <math>\lambda\,\!</math> is some positive constant. Since we stated earlier that we will output a dataset of size <math>\tilde{O}(VCDIM(H)/\alpha^{2})\,\!</math>, so using this bound on <math>m\,\!</math> we get <math>m\geq\lambda(VCDIM(H)\log(1/\alpha)/\alpha^{2})\,\!</math>. Hence the lemma.
| |
| | |
| Now we invoke the Exponential Mechanism.
| |
| | |
| '''Definition:''' For any function <math>q:((\{0,1\}^k)^n \times(\{0,1\}^k)^m)\rightarrow\mathbb{R}\,\!</math> and input dataset <math>D\,\!</math>, the Exponential mechanism outputs each dataset <math>\widehat{D}\,\!</math> with probability proportional to <math>e^{q(D,\widehat{D})\epsilon n/2}\,\!</math>.
| |
| | |
| From the Exponential Mechanism we know this preserves <math>(\epsilon nGS_{q})\,\!</math>-differential privacy.
| |
| Lets get back to the proof of the Theorem.
| |
| | |
| We define <math>(q(D),q(\widehat{D}))=-\max_{h\in H}|Q_{h}(D)-Q_{h}(\widehat{D})|\,\!</math>. <br>
| |
| To show that the mechanism satisfies the <math>(\alpha,\delta)\,\!</math>-usefulness, we should show that it outputs some dataset <math>\widehat{D}\,\!</math> with <math>q(D,\widehat{D})\geq -\alpha\,\!</math> with probability <math>1-\delta\,\!</math>.
| |
| There are at most <math>2^{km}\,\!</math> output datasets and the probability that <math>q(D,\widehat{D})\leq -\alpha\,\!</math> is at most proportional to <math>e^{-\epsilon\alpha n/2}\,\!</math>. Thus by union bound, the probability of outputting any such dataset <math>\widehat{D}\,\!</math> is at most proportional to <math>2^{km}e^{-\epsilon\alpha n/2}\,\!</math>.
| |
| Again, we know that there exists some dataset <math>\widehat{D}\in(\{0,1\}^{k})^{m}\,\!</math> for which <math>q(D,\widehat{D})\geq -\alpha/2\,\!</math>. Therefore, such a dataset is output with probability at least proportional to <math>e^{-\alpha\epsilon n/4}\,\!</math>. <br>
| |
| Let, <math>A:=\,\!</math> the event that the Exponential mechanism outputs some dataset <math>\widehat{D}\,\!</math> such that <math>q(D,\widehat{D})\geq-\alpha/2\,\!</math>. <br>
| |
| <math>B:=\,\!</math> the event that the Exponential mechanism outputs some dataset <math>\widehat{D}\,\!</math> such that <math>q(D,\widehat{D})\leq-\alpha\,\!</math>.
| |
| :<math>\therefore \frac{\Pr[A]}{\Pr[B]}\geq \frac{e^{-\alpha\epsilon n/4}}{2^{km}e^{-\alpha\epsilon n/2}}=\frac{e^{\alpha\epsilon n/4}}{2^{km}}.\,\!</math>
| |
| Now setting this quantity to be at least <math>1/\delta\geq(1-\delta)/\delta\,\!</math>, we find that it suffices to have
| |
| :<math>n\geq\frac{4}{\epsilon\alpha}\left(km+ln\frac{1}{\delta}\right)\geq O\left(\frac{d\cdot VCDIM(H)\log(1/\alpha)}{\alpha^{3}\epsilon}+\frac{\log(1/\delta)}{\alpha\epsilon}\right)\,\!</math>.
| |
| And hence we prove the theorem.
| |
| | |
| == The Exponential Mechanism in other domains ==
| |
| | |
| We just showed an example of the usage of Exponential Mechanism where one can output a synthetic dataset in a differentially private manner and can use the dataset to answer queries with good accuracy. Apart from these kinds of setting, the Exponential Mechanism has also been studied in the context of [[auction theory]] and [[Statistical classification|classification algorithms]].<ref>[http://arxiv.org/abs/0803.0924v2 Shiva Prasad Kasiviswanathan, Homin K. Lee, Kobbi Nissim,Sofya Raskhodnikova, Adam Smith. What Can We Learn Privately? Proceedings of the 2008 49th Annual IEEE Symposium on Foundations of Computer Science.]</ref> In the case of auctions the Exponential Mechanism helps to achieve a ''truthful'' auction setting.
| |
| | |
| ==References==
| |
| {{Reflist}}
| |
| | |
| {{DEFAULTSORT:Exponential Mechanism (Differential Privacy)}}
| |
| [[Category:Data privacy]]
| |
| [[Category:Theory of cryptography]]
| |
| [[Category:Applied probability]]
| |