en>Bibcode Bot: Adding 0 arxiv eprint(s), 1 bibcode(s) and 0 doi(s). Did it miss something? Report bugs, errors, and suggestions at User talk:Bibcode Bot

2012-09-04T18:46:09Z

Adding 0 arxiv eprint(s), 1 bibcode(s) and 0 doi(s). Did it miss something? Report bugs, errors, and suggestions at User talk:Bibcode Bot

← Older revision		Revision as of 20:46, 4 September 2012
Line 1:		Line 1:
	~~They contact me Emilia~~. ~~North Dakota~~ is ~~her beginning place but she will have~~ to ~~move 1 working day or~~ an ~~additional~~. ~~For many years he's been working~~ as a ~~meter reader~~ and it'~~s some thing he really appreciate~~. ~~What I love performing~~ is ~~taking part in baseball but I haven't made~~ a ~~dime with it~~.<br><br>~~my blog~~; [http://~~wixothek~~.~~com~~/~~user~~/~~MBuckmast at home std test~~]		In the unconstrained [[optimization (mathematics)\|minimization]] problem, the '''Wolfe conditions''' are a set of inequalities for performing '''inexact''' [[line search]], especially in [[quasi-Newton methods]], first published by Philip Wolfe in 1969.<ref>{{cite doi\|10.1137/1011036}}</ref><ref>{{cite doi\|10.1137/1013035}}</ref>

			In these methods the idea is to find

			::<math>\min_x f(\mathbf{x})</math>

			for some [[smooth function\|smooth]] <math>f:\mathbb R^n\to\mathbb R</math>. Each step often involves approximately solving the subproblem

			::<math>\min_{\alpha} f(\mathbf{x}_k + \alpha \mathbf{p}_k)</math>

			where <math>\mathbf{x}_k</math> is the current best guess, <math>\mathbf{p}_k \in \mathbb R^n</math> is a search direction, and <math>\alpha \in \mathbb R</math> is the step length.

			The inexact line searches provide an efficient way of computing an acceptable step length <math>\alpha</math> that reduces the [[objective function]] 'sufficiently', rather than minimizing the objective function over <math>\alpha\in\mathbb R^+</math> exactly. A line search algorithm can use Wolfe conditions as a requirement for any guessed <math>\alpha</math>, before finding a new search direction <math>\mathbf{p}_k</math>.

			==Armijo rule and curvature==
			Denote a univariate function <math>\phi</math> restricted to the direction <math>\mathbf{p}_k</math> as <math>\phi(\alpha)=f(\mathbf{x}_k+\alpha\mathbf{p}_k)</math>. A step length <math>\alpha_k</math> is said to satisfy the ''Wolfe conditions'' if the following two inequalities hold:

			:i) <math>f(\mathbf{x}_k+\alpha_k\mathbf{p}_k)\leq f(\mathbf{x}_k)+c_1\alpha_k\mathbf{p}_k^{\mathrm T}\nabla f(\mathbf{x}_k)</math>,
			:ii) <math>\mathbf{p}_k^{\mathrm T}\nabla f(\mathbf{x}_k+\alpha_k\mathbf{p}_k) \geq c_2\mathbf{p}_k^{\mathrm T}\nabla f(\mathbf{x}_k)</math>,

			with <math>0<c_1<c_2<1</math>. (In examining condition (ii), recall that to ensure that <math>\mathbf{p}_k</math> is a descent direction, we have <math>\mathbf{p}_k^{\mathrm T}\nabla f(\mathbf{x}_k) < 0 </math>.)

			<math>c_1</math> is usually chosen to be quite small while <math>c_2</math> is much larger; Nocedal gives example values of <math>c_1=10^{-4}</math><ref>{{cite book \| title = Numerical Optimization \| last1=Nocedal \|first1=Jorge \| last2=Wright \|first2=Stephen \| url = http://books.google.com.tw/books?id=epc5fX0lqRIC&lpg=PP1&pg=PA38#v=onepage&q \| year=1999}}</ref>
			and <math>c_2=0.9</math> for Newton or quasi-Newton methods and <math>c_2=0.1</math> for the nonlinear [[conjugate gradient method]]. Inequality i) is known as the '''Armijo rule''' <ref>{{cite journal \| last = Armijo \| first = Larry \| year = 1966 \| title = Minimization of functions having Lipschitz continuous first partial derivatives \| journal = Pacific J. Math. \| volume = 16 \| issue = 1 \| pages = 1–3 \| url = http://projecteuclid.org/euclid.pjm/1102995080}}</ref> and ii) as the '''curvature condition'''; i) ensures that the step length <math>\alpha_k</math> decreases <math>f</math> 'sufficiently', and ii) ensures that the slope has been reduced sufficiently.

			==Strong Wolfe condition on curvature==
			The Wolfe conditions, however, can result in a value for the step length that is not close to a minimizer of <math>\phi</math>. If we modify the curvature condition to the following,

			:iia) <math>\big\|\mathbf{p}_k^{\mathrm T}\nabla f(\mathbf{x}_k+\alpha_k\mathbf{p}_k)\big\|\leq c_2\big\|\mathbf{p}_k^{\mathrm T}\nabla f(\mathbf{x}_k)\big\|</math>

			then i) and iia) together form the so-called '''strong Wolfe conditions''', and force <math>\alpha_k</math> to lie close to a [[critical point (mathematics)\|critical point]] of <math>\phi</math>.

			The principal reason for imposing the Wolfe conditions in an optimization algorithm where <math> \mathbf{x}_{k+1} = \mathbf{x}_k + \alpha \mathbf{p}_k </math> is to ensure convergence of the gradient to zero. In particular, if the cosine of the angle between <math>\mathbf{p}_k</math> and the gradient,

			:: <math> \cos \theta_k = \frac {\nabla f(\mathbf{x}_k)^{\mathrm T}\mathbf{p}_k }{\\| \nabla f(\mathbf{x}_k)\\| \\|\mathbf{p}_k\\| } </math>

			is bounded away from zero and the i) and ii) hold, then <math> \nabla f(\mathbf{x}_k) \rightarrow 0 </math>.

			An additional motivation, in the case of a [[quasi-Newton method]] is that if <math> \mathbf{p}_k = -B_k^{-1} \nabla f(\mathbf{x}_k) </math>, where the matrix <math> B_k </math> is updated by the [[BFGS]] or [[Davidon-Fletcher-Powell_formula\|DFP]] formula, then if <math> B_k </math> is positive definite ii) implies <math> B_{k+1} </math> is also positive definite.

			==References==
			* {{cite doi\|10.1007/978-0-387-40065-5_3}}
			* {{cite doi\|10.1007/978-0-387-40065-5_6}}
			<references />

			{{Optimization algorithms}}
			[[Category:Mathematical optimization]]

en>Helpful Pixie Bot: ISBNs (Build KE)

2012-05-09T23:42:40Z

ISBNs (Build KE)

New page

They contact me Emilia. North Dakota is her beginning place but she will have to move 1 working day or an additional. For many years he's been working as a meter reader and it's some thing he really appreciate. What I love performing is taking part in baseball but I haven't made a dime with it.<br><br>my blog; [http://wixothek.com/user/MBuckmast at home std test]

Unbeatable strategy - Revision history

en>Bibcode Bot: Adding 0 arxiv eprint(s), 1 bibcode(s) and 0 doi(s). Did it miss something? Report bugs, errors, and suggestions at User talk:Bibcode Bot

en>Helpful Pixie Bot: ISBNs (Build KE)