|
|
| Line 1: |
Line 1: |
| In [[numerical linear algebra]], the method of '''successive over-relaxation''' ('''SOR''') is a variant of the [[Gauss–Seidel method]] for solving a [[linear system of equations]], resulting in faster convergence. A similar method can be used for any slowly converging [[iterative method|iterative process]].
| | The writer is known as Irwin. Since [http://www.blaze16.com/blog/257588 blaze16.com] she was [http://gcjcteam.org/index.php?mid=etc_video&document_srl=655020&sort_index=regdate&order_type=desc at home std test] 18 she's been operating as at home std testing a meter reader but she's always needed her own business. Puerto Rico is exactly where he's always been living but she [http://Www.ppsev.org/services/warts.html requirements] to move simply because of her family. His spouse doesn't like it the way he does but what he truly likes doing is to do aerobics and he's been [http://browardchd.org/healths_sexually.aspx performing] it for quite a whilst.<br><br>Have a look at std home test my web-site over the counter std test ([http://vei.cuaed.unam.mx/es/node/4882 relevant web site]) |
| | |
| It was devised simultaneously by [[David M. Young, Jr.]] and by H. Frankel in 1950 for the purpose of automatically solving linear systems on digital computers. Over-relaxation methods had been used before the work of Young and Frankel. For instance, the method of [[Lewis Fry Richardson]], and the methods developed by [[Richard V. Southwell|R. V. Southwell]]. However, these methods were designed for computation by human calculators, and they required some expertise to ensure convergence to the solution which made them inapplicable for programming on digital computers. These aspects are discussed in the thesis of David M. Young, Jr.<ref>{{citation
| |
| |last=Young |first=David M. |authorlink=David M. Young
| |
| |title=Iterative methods for solving partial difference equations of elliptical type
| |
| |url=http://www.ma.utexas.edu/CNA/DMY/david_young_thesis.pdf
| |
| |date=May 1, 1950 | |
| |series=PhD thesis, Harvard University
| |
| |accessdate=2009-06-15
| |
| }}</ref>
| |
| | |
| ==Formulation==
| |
| Given a square system of ''n'' linear equations with unknown '''x''':
| |
| | |
| :<math>A\mathbf x = \mathbf b</math>
| |
| | |
| where:
| |
| | |
| :<math>A=\begin{bmatrix} a_{11} & a_{12} & \cdots & a_{1n} \\ a_{21} & a_{22} & \cdots & a_{2n} \\ \vdots & \vdots & \ddots & \vdots \\a_{n1} & a_{n2} & \cdots & a_{nn} \end{bmatrix}, \qquad \mathbf{x} = \begin{bmatrix} x_{1} \\ x_2 \\ \vdots \\ x_n \end{bmatrix} , \qquad \mathbf{b} = \begin{bmatrix} b_{1} \\ b_2 \\ \vdots \\ b_n \end{bmatrix}.</math>
| |
| | |
| Then ''A'' can be decomposed into a [[diagonal matrix|diagonal]] component ''D'', and [[triangular matrix#Strictly triangular matrix|strictly lower and upper triangular]] components ''L'' and ''U'':
| |
| | |
| :<math>A=D+L+U, </math>
| |
| where
| |
| :<math>D = \begin{bmatrix} a_{11} & 0 & \cdots & 0 \\ 0 & a_{22} & \cdots & 0 \\ \vdots & \vdots & \ddots & \vdots \\0 & 0 & \cdots & a_{nn} \end{bmatrix}, \quad L = \begin{bmatrix} 0 & 0 & \cdots & 0 \\ a_{21} & 0 & \cdots & 0 \\ \vdots & \vdots & \ddots & \vdots \\a_{n1} & a_{n2} & \cdots & 0 \end{bmatrix}, \quad U = \begin{bmatrix} 0 & a_{12} & \cdots & a_{1n} \\ 0 & 0 & \cdots & a_{2n} \\ \vdots & \vdots & \ddots & \vdots \\0 & 0 & \cdots & 0 \end{bmatrix}. </math>
| |
| | |
| The system of linear equations may be rewritten as:
| |
| | |
| :<math>(D+\omega L) \mathbf{x} = \omega \mathbf{b} - [\omega U + (\omega-1) D ] \mathbf{x} </math>
| |
| | |
| for a constant ''ω'' > 1, called the ''relaxation factor''.
| |
| | |
| The method of successive over-relaxation is an [[Iterative method|iterative technique]] that solves the left hand side of this expression for '''x''', using previous value for '''x''' on the right hand side. Analytically, this may be written as:
| |
| | |
| :<math> \mathbf{x}^{(k+1)} = (D+\omega L)^{-1} \big(\omega \mathbf{b} - [\omega U + (\omega-1) D ] \mathbf{x}^{(k)}\big)=L_w \mathbf{x}^{(k)}+\mathbf{c}. </math> | |
| | |
| However, by taking advantage of the triangular form of (''D''+''ωL''), the elements of '''x'''<sup>(''k''+1)</sup> can be computed sequentially using [[forward substitution]]:
| |
| | |
| :<math> x^{(k+1)}_i = (1-\omega)x^{(k)}_i + \frac{\omega}{a_{ii}} \left(b_i - \sum_{j<i} a_{ij}x^{(k+1)}_j - \sum_{j>i} a_{ij}x^{(k)}_j \right),\quad i=1,2,\ldots,n. </math>
| |
| | |
| The choice of relaxation factor ''ω'' is not necessarily easy, and depends upon the properties of the coefficient matrix. In 1947, Ostrowski proved that if <math>A</math> is [[Symmetric matrix|symmetric]] and [[Positive-definite matrix|positive-definite]] then <math>\rho(L_\omega)<1</math> for <math>0<\omega<2 </math>. Thus convergence of the iteration process follows, but we are generally interested in faster convergence rather than just convergence.
| |
| | |
| == Algorithm ==
| |
| | |
| Since elements can be overwritten as they are computed in this algorithm, only one storage vector is needed, and vector indexing is omitted. The algorithm goes as follows:
| |
| | |
| Inputs: {{var|A}}, {{var|b}}, {{var|ω}} <br>
| |
| Output: <math>\phi</math>
| |
| | |
| Choose an initial guess <math>\phi</math> to the solution <br>
| |
| '''repeat''' until convergence
| |
| :'''for''' {{var|i}} '''from''' 1 '''until''' {{var|n}} '''do'''
| |
| ::<math>\sigma \leftarrow 0</math>
| |
| ::'''for''' {{var|j}} '''from''' 1 '''until''' {{var|n}} '''do'''
| |
| :::'''if''' {{var|j}} ≠ {{var|i}} '''then'''
| |
| ::::<math> \sigma \leftarrow \sigma + a_{ij} \phi_j </math>
| |
| :::'''end if'''
| |
| ::'''end''' ({{var|j}}-loop)
| |
| ::<math> \phi_i \leftarrow (1-\omega)\phi_i + \frac{\omega}{a_{ii}} (b_i - \sigma)</math>
| |
| :'''end''' ({{var|i}}-loop)
| |
| :check if convergence is reached
| |
| '''end''' (repeat)
| |
| | |
| '''Note:'''<br /> <math>(1-\omega)\phi_i + \frac{\omega}{a_{ii}} (b_i - \sigma)</math> can also be written <math>\phi_i + \omega \left( \frac{b_i - \sigma}{a_{ii}} - \phi_i\right)</math>, thus saving one multiplication in each iteration of the outer ''for''-loop.
| |
| | |
| ==Symmetric successive over-relaxation==
| |
| | |
| The version for symmetric matrices ''A'', in which
| |
| | |
| :<math> U=L^T,\,</math>
| |
| | |
| is referred to as '''Symmetric Successive Over-Relaxation''', or ('''SSOR'''), in which
| |
| | |
| :<math>P=\left(\frac{D}{\omega}+L\right)\frac{\omega}{2-\omega}D^{-1}\left(\frac{D}{\omega}+U\right),</math>
| |
| | |
| and the iterative method is
| |
| | |
| :<math>\mathbf{x}^{k+1}=\mathbf{x}^k-\gamma^k P^{-1}(A\mathbf{x}^k-\mathbf{b}),\ k \ge 0.</math>
| |
| | |
| The SOR and SSOR methods are credited to [[David M. Young, Jr.]].
| |
| | |
| ==Other applications of the method==
| |
| {{main|Richardson extrapolation}}
| |
| A similar technique can be used for any iterative method. If the original iteration had the form
| |
| | |
| :<math>x_{n+1}=f(x_n)</math>
| |
| | |
| then the modified version would use
| |
| | |
| :<math>x^\mathrm{SOR}_{n+1}=(1-\omega)x^{\mathrm{SOR}}_n+\omega f(x^\mathrm{SOR}_n).</math>
| |
| | |
| Note however that the formulation presented above, used for solving systems of linear equations, is not a special case of this formulation if {{mvar|x}} is considered to be the complete vector. If this formulation is used instead, the equation for calculating the next vector will look like
| |
| | |
| :<math> \mathbf{x}^{(k+1)} = (1-\omega)\mathbf{x}^{(k)} + \omega L_*^{-1} (\mathbf{b} - U\mathbf{x}^{(k)}). </math>
| |
| | |
| Values of <math>\omega>1</math> are used to speedup convergence of a slow-converging process, while values of <math>\omega<1</math> are often used to help establish convergence of a diverging iterative process or speed up the convergence of an [[Newton's method#Overshoot|overshooting]] process.
| |
| | |
| There are various methods that adaptively set the relaxation parameter <math>\omega</math> based on the observed behavior of the converging process. Usually they help to reach a super-linear convergence for some problems but fail for the others.
| |
| | |
| ==See also==
| |
| *[[Jacobi method]]
| |
| *[[Belief propagation#Gaussian belief propagation .28GaBP.29|Gaussian Belief Propagation]]
| |
| *[[Matrix splitting]]
| |
| | |
| ==Notes==
| |
| {{reflist}}
| |
| | |
| ==References==
| |
| *{{CFDWiki|name=Successive_over-relaxation_method_-_SOR}}
| |
| | |
| * Abraham Berman, Robert J. Plemmons, ''Nonnegative Matrices in the Mathematical Sciences'', 1994, SIAM. ISBN 0-89871-321-8.
| |
| *{{MathWorld|urlname=SuccessiveOverrelaxationMethod|title=Successive Overrelaxation Method|author=Black, Noel and Moore, Shirley}}
| |
| *[[Yousef Saad]], ''[http://www-users.cs.umn.edu/%7Esaad/books.html Iterative Methods for Sparse Linear Systems]'', 1st edition, PWS, 1996.
| |
| *[http://www.netlib.org/utk/papers/templates/node11.html Netlib]'s copy of "Templates for the Solution of Linear Systems", by Barrett et al.
| |
| * [[Richard S. Varga]] 2002 ''Matrix Iterative Analysis'', Second ed. (of 1962 Prentice Hall edition), Springer-Verlag.
| |
| * [[David M. Young, Jr.]] ''Iterative Solution of Large Linear Systems'', Academic Press, 1971. (reprinted by Dover, 2003)
| |
| | |
| ==External links==
| |
| *[http://math.fullerton.edu/mathews/n2003/SORmethodMod.html Module for the SOR Method]
| |
| * [http://michal.is/projects/tridiagonal-system-solver-sor-c/ Tridiagonal linear system solver] based on SOR, in C++
| |
| | |
| {{Numerical linear algebra}}
| |
| | |
| [[Category:Numerical linear algebra]]
| |
| [[Category:Articles with example pseudocode]]
| |
| [[Category:Relaxation (iterative methods)]]
| |