|
|
| Line 1: |
Line 1: |
| CoBoost is a semi-supervised training algorithm proposed by Collins and Singer in 1999. The original application for the algorithm was the task of [[Named-entity recognition|Named Entity Classification]] using very weak learners.<ref name="Collins99">Michael Collins and Yoram Singer, Unsupervised Models for Named Entity Classification. Proceedings of the 1999 Joint SIGDAT Conference on Empirical Methods in Natural Language Processing and Very Large Corpora, pp. 100-110, 1999.</ref> It can be used for performing semi-supervised learning in cases in which there exist redundancy in features.
| | It depends on the quality of the Wordpress theme but even if it's not a professional one you will be able to average 50-60$ EACH link. You can either install Word - Press yourself or use free services offered on the web today. This is a service where people write articles using a specific keyword you have given them. If you're using Wordpress and want to make your blog a "dofollow" blog, meaning that links from your blog pass on the benefits of Google pagerank, you can install one of the many dofollow plugins available. Understanding how Word - Press works can be a challenge, but it is not too difficult when you learn more about it. <br><br>Word - Press is known as the most popular blogging platform all over the web and is used by millions of blog enthusiasts worldwide. Wordpress have every reason with it which promote wordpress development. Our Daily Deal Software plugin brings the simplicity of setting up a Word - Press blog to the daily deal space. t need to use the back button or the URL to get to your home page. Word - Press makes it possible to successfully and manage your website. <br><br>Here are a few reasons as to why people prefer Word - Press over other software's. The following piece of content is meant to make your choice easier and reassure you that the decision to go ahead with this conversion is requited with rich benefits:. Possibly the most downloaded Word - Press plugin, the Google XML Sitemaps plugin but not only automatically creates a site map linking to everyone your pages and posts, it also notifies Google, Bing, Yahoo, and Ask. You can allow visitors to post comments, or you can even allow your visitors to register and create their own personal blogs. Purchase these from our site, or bring your own, it doesn't matter, we will still give you free installation and configuration. <br><br>Word - Press has plenty of SEO benefits over Joomla and Drupal. And, that is all the opposition events with nationalistic agenda in favor of the individuals of Pakistan marching collectively in the battle in opposition to radicalism. Specialty about our themes are that they are easy to load, compatible with latest wordpress version and are also SEO friendly. The company gains commission from the customers' payment. This includes enriching the content with proper key words, tactfully defining the tags and URL. <br><br>More it extends numerous opportunities where your firm is at comfort and rest assured of no risks & errors. Automated deal feed integration option to populate your blog with relevant deals. When you have any kind of concerns about in which as well as the best way to employ [http://snipitfor.me/wordpress_backup_plugin_731616 wordpress dropbox backup], you'll be able to email us at the web-site. However, you must also manually approve or reject comments so that your website does not promote parasitic behavior. Page speed is an important factor in ranking, especially with Google. For your information, it is an open source web content management system. |
| | |
| It may be seen as a combination of [[co-training]] and [[Boosting (meta-algorithm)|boosting]]. Each example is available in two views (subsections of the feature set), and boosting is applied iteratively in alternation with each view using predicted labels produced in the alternate view on the previous iteration. CoBoosting is not a valid boosting algorithm in the PAC learning sense.
| |
| | |
| ==Motivation==
| |
| CoBoosting was an attempt by Collins and Singer to improve on previous attempts to leverage redundancy in features for training classifiers in a semi-supervised fashion. CoTraining, a seminal work by Blum and Mitchell, was shown to be a powerful framework for learning classifiers given a small number of seed examples by iteratively inducing rules in a decision list. The advantage of CoBoosting to CoTraining is that it generalizes the CoTraining pattern so that it could be used with any classifier. CoBoosting accomplishes this feat by borrowing concepts from [[AdaBoost]].
| |
| | |
| In both CoTrain and CoBoost the training and testing example sets must follow two properties. The first is that the feature space of the examples can separated into two feature spaces (or views) such that each view is sufficiently expressive for classification.
| |
| Formally, there exist two functions <math>f_1(x_1)</math> and <math>f_2(x_2)</math> such that for all examples <math>x=(x_1,x_2)</math>, <math>f_1(x_1)=f_2(x_2)=f(x)</math>. While ideal, this constraint is in fact too strong due to noise and other factors, and both algorithms instead seek to maximize the agreement between the two functions. The second property is that the two views must not be highly correlated.
| |
| | |
| ==Algorithm==
| |
| Input: <math>\{(x_{1,i},x_{2,i})\}_{i=1}^n</math>, <math>\{y_i\}_{i=1}^m</math>
| |
| | |
| Initialize: <math>\forall i,j : g_j^0(\boldsymbol{x_i})=0</math>.
| |
| | |
| For <math>t=1,...,T</math> and for <math>j=1,2</math>:
| |
| | |
| Set pseudo-labels:
| |
|
| |
| <math>
| |
| \hat{y_i} = \left\{
| |
| \begin{array}{ll}
| |
| y_i 1 \le i \le m \\
| |
| sign(g_{3-j}^{t-1}(\boldsymbol{x_{3-j,i}})) m < i \le n
| |
| \end{array}
| |
| \right.
| |
| </math>
| |
| | |
| Set virtual distribution:
| |
| <math>
| |
| D_t^j(i) = \frac{1}{Z_t^j}e^{-\hat{y_i}g_j^{t-1}(\boldsymbol{x_{j,i}})}
| |
| </math>
| |
|
| |
| where <math>Z_t^j = \sum_{i=1}^ne^{-\hat{y_i}g_j^{t-1}(\boldsymbol{x_{j,i}})}</math>
| |
|
| |
| Find the weak hypothesis <math>h_t^j</math> that minimizes expanded training error.
| |
|
| |
| Choose value for <math>\alpha_t</math> that minimizes expanded training error.
| |
| | |
| Update the value for current strong non-thresholded classfier:
| |
| | |
| <math>
| |
| \forall i:g_j^t(\boldsymbol{x_{j,i}}) = g_j^{t-1}(\boldsymbol{x_{j,i}})+\alpha_th_t^j(\boldsymbol{x_{j,i}})
| |
| </math>
| |
| | |
| The final strong classifier output is
| |
| | |
| <math>
| |
| f(\boldsymbol{x}) = sign\left(\sum_{j=1}^2g_j^T(\boldsymbol{x_j})\right)
| |
| </math>
| |
| | |
| ==Setting up AdaBoost==
| |
| CoBoosting builds on the [[AdaBoost]] algorithm, which gives CoBoosting its generalization ability since AdaBoost can be used in conjunction with many other learning algorithms. This build up assumes a two class classification task, although it can be adapted to multiple class classification. In the AdaBoost framework, weak classifiers are generated in series as well as a distribution over examples in the training set. Each weak classifier is given a weight and the final strong classifier is defined as the sign of the sum of the weak classifiers weighted by their assigned weight. (See [[AdaBoost]] Wikipedia page for notation). In the AdaBoost framework Schapire and Singer have shown that the training error is bounded by the following equation:
| |
| | |
| <math>
| |
| \frac{1}{m}\sum_{i=1}^m e^{\left(-y_i\left(\sum_{t=1}^T\alpha_t h_t(\boldsymbol{x_i})\right)\right)}=\prod_tZ_t
| |
| </math>
| |
| | |
| Where <math>Z_t</math> is the normalizing factor for the distribution <math>D_{t+1}</math>. Solving for <math>Z_t</math> in the equation for <math>D_t(i)</math> we get:
| |
| | |
| <math>
| |
| Z_t = \sum_{i:x_t\notin x_i}D_t(i) + \sum_{i:x_t\in x_i}D_t(i)e^{-y_i\alpha_i h_t(\boldsymbol{x_i})}
| |
| </math>
| |
| | |
| Where <math>x_t</math> is the feature selected in the current weak hypothesis. Three equations are defined describing the sum of the distributions for in which the current hypothesis has selected either correct or incorrect label. Note that it is possible for the classifier to abstain from selecting a label for an example, in which the label provided is 0. The two labels are selected to be either -1 or 1.
| |
| | |
| <math>
| |
| W_0 = \sum_{i:h_t(x_i)=0} D_t(i)
| |
| </math>
| |
| | |
| <math>
| |
| W_+ = \sum_{i:h_t(x_i)=y_i} D_t(i)
| |
| </math>
| |
| | |
| <math>
| |
| W_- = \sum_{i:h_t(x_i)=-y_i} D_t(i)
| |
| </math>
| |
| | |
| Schapire and Singer have shown that the value <math>Z_t</math> can be minimized (and thus the training error) by selecting <math>\alpha_t</math> to be as follows:
| |
| | |
| <math>
| |
| \alpha_t = \frac{1}{2}\ln\left(\frac{W_+}{W_-}\right)
| |
| </math>
| |
| | |
| Providing confidence values for the current hypothesized classifier based on the number of correctly classified vs. the number of incorrectly classified examples weighted by the distribution over examples. This equation can be smoothed to compensate for cases in which <math>W_-</math> is too small. Deriving <math>Z_t</math> from this equation we get:
| |
| | |
| <math> | |
| Z_t = W_0 + 2\sqrt{W_+W_-}
| |
| </math> | |
| | |
| The training error thus is minimized by selecting the weak hypothesis at every iteration that minimizes the previous equation.
| |
| | |
| ==AdaBoost with two views==
| |
| CoBoosting extends this framework in the case where one has a labeled training set (examples from <math>1...m</math>) and an unlabeled training set (from <math>m_1...n</math>), as well as satisfy the conditions of redundancy in features in the form of <math>x_i=(x_{1,i},x_{2,i})</math>. The algorithm trains two classifiers in the same fashion as [[AdaBoost]] that agree on the labeled training sets correct labels and maximizes the agreement between the two classifiers on the unlabeled training set. The final classifier is the sign of the sum of the two strong classifiers. The bounded training error on CoBoost is extended as follows, where <math>Z_{CO}</math> is the extension of <math>Z_t</math>:
| |
| | |
| <math>
| |
| Z_{CO} = \sum_{i=1}^m e^{-y_ig_1(\boldsymbol{x_{1,i}})}
| |
| + \sum_{i=1}^m e^{-y_ig_2(\boldsymbol{x_{2,i}})}
| |
| + \sum_{i=m+1}^n e^{-f_2(\boldsymbol{x_{2,i}})g_1(\boldsymbol{x_{1,i}})}
| |
| + \sum_{i=m+1}^n e^{-f_1(\boldsymbol{x_{1,i}})g_2(\boldsymbol{x_{2,i}})}
| |
| </math>
| |
| | |
| Where <math>g_j</math> is the summation of hypotheses weight by their confidence values for the <math>j^{th}</math> view (j = 1 or 2). <math>f_j</math> is the sign of <math>g_j</math>. At each iteration of CoBoost both classifiers are updated iteratively. If <math>g_j^{t-1}</math> is the strong classifier output for the <math>j^{th}</math> view up to the <math>t-1</math> iteration we can set the pseudo-labels for the ''j''th update to be:
| |
| | |
| <math>
| |
| \hat{y_i} = \left\{
| |
| \begin{array}{ll}
| |
| y_i 1 \le i \le m \\
| |
| sign(g_{3-j}^{t-1}(\boldsymbol{x_{3-j,i}})) m < i \le n
| |
| \end{array}
| |
| \right.
| |
| </math>
| |
| | |
| In which <math>3-j</math> selects the other view to the one currently being updated. <math>Z_{CO}</math> is split into two such that <math>Z_{CO} = Z_{CO}^1+Z_{CO}^2</math>. Where
| |
| | |
| <math>
| |
| Z_{CO}^j = \sum_{i=1}^n e^{-\hat{y_i}(g_j^{t-1}(\boldsymbol{x_i})+\alpha_t^jg_t^j(\boldsymbol{x_{j,i}}))}
| |
| </math>
| |
| | |
| The distribution over examples for each view <math>j</math> at iteration <math>t</math> is defined as follows:
| |
| | |
| <math>
| |
| D_t^j(i) = \frac{1}{Z_t^j}e^{-\hat{y_i}g_j^{t-1}(\boldsymbol{x_{j,i}})}
| |
| </math>
| |
| | |
| At which point <math>Z_{CO}^j</math> can be rewritten as
| |
| | |
| <math>
| |
| Z_{CO}^j = \sum_{i=1}^n D_t^j e^{-\hat{y_i}\alpha_t^jg_t^j(\boldsymbol{x_{j,i}})}
| |
| </math>
| |
| | |
| Which is identical to the equation in AdaBoost. Thus the same process can be used to update the values of <math>\alpha_t^j</math> as in AdaBoost using <math>\hat{y_i}</math> and <math>D_t^j</math>. By alternating this, the minimization of <math>Z_{CO}^1</math> and <math>Z_{CO}^2</math> in this fashion <math>Z_{CO}</math> is minimized in a greedy fashion.
| |
| | |
| == References ==
| |
| | |
| ===Footnotes===
| |
| {{Reflist}}
| |
| | |
| {{DEFAULTSORT:Coboosting}}
| |
| [[Category:Classification algorithms]]
| |
It depends on the quality of the Wordpress theme but even if it's not a professional one you will be able to average 50-60$ EACH link. You can either install Word - Press yourself or use free services offered on the web today. This is a service where people write articles using a specific keyword you have given them. If you're using Wordpress and want to make your blog a "dofollow" blog, meaning that links from your blog pass on the benefits of Google pagerank, you can install one of the many dofollow plugins available. Understanding how Word - Press works can be a challenge, but it is not too difficult when you learn more about it.
Word - Press is known as the most popular blogging platform all over the web and is used by millions of blog enthusiasts worldwide. Wordpress have every reason with it which promote wordpress development. Our Daily Deal Software plugin brings the simplicity of setting up a Word - Press blog to the daily deal space. t need to use the back button or the URL to get to your home page. Word - Press makes it possible to successfully and manage your website.
Here are a few reasons as to why people prefer Word - Press over other software's. The following piece of content is meant to make your choice easier and reassure you that the decision to go ahead with this conversion is requited with rich benefits:. Possibly the most downloaded Word - Press plugin, the Google XML Sitemaps plugin but not only automatically creates a site map linking to everyone your pages and posts, it also notifies Google, Bing, Yahoo, and Ask. You can allow visitors to post comments, or you can even allow your visitors to register and create their own personal blogs. Purchase these from our site, or bring your own, it doesn't matter, we will still give you free installation and configuration.
Word - Press has plenty of SEO benefits over Joomla and Drupal. And, that is all the opposition events with nationalistic agenda in favor of the individuals of Pakistan marching collectively in the battle in opposition to radicalism. Specialty about our themes are that they are easy to load, compatible with latest wordpress version and are also SEO friendly. The company gains commission from the customers' payment. This includes enriching the content with proper key words, tactfully defining the tags and URL.
More it extends numerous opportunities where your firm is at comfort and rest assured of no risks & errors. Automated deal feed integration option to populate your blog with relevant deals. When you have any kind of concerns about in which as well as the best way to employ wordpress dropbox backup, you'll be able to email us at the web-site. However, you must also manually approve or reject comments so that your website does not promote parasitic behavior. Page speed is an important factor in ranking, especially with Google. For your information, it is an open source web content management system.