Carlson's theorem

In numerical linear algebra, the method of successive over-relaxation (SOR) is a variant of the Gauss–Seidel method for solving a linear system of equations, resulting in faster convergence. A similar method can be used for any slowly converging iterative process.

It was devised simultaneously by David M. Young, Jr. and by H. Frankel in 1950 for the purpose of automatically solving linear systems on digital computers. Over-relaxation methods had been used before the work of Young and Frankel. For instance, the method of Lewis Fry Richardson, and the methods developed by R. V. Southwell. However, these methods were designed for computation by human calculators, and they required some expertise to ensure convergence to the solution which made them inapplicable for programming on digital computers. These aspects are discussed in the thesis of David M. Young, Jr.

Formulation

Given a square system of n linear equations with unknown x:

$A\mathbf{x}=\mathbf{b}$

where:

$A=\left[\begin{array}{cccc}{a}_{11}& a_{12}& \cdots & a_{1n}\\ a_{21}& a_{22}& \cdots & a_{2n}\\ ⋮& ⋮& \ddots & ⋮\\ a_{n1}& a_{n2}& \cdots & a_{nn}\end{array}\right],\mathbf{x}=\left[\begin{array}{c}{x}_1\\ x_2\\ ⋮\\ x_n\end{array}\right],\mathbf{b}=\left[\begin{array}{c}{b}_1\\ b_2\\ ⋮\\ b_n\end{array}\right].$

Then A can be decomposed into a diagonal component D, and strictly lower and upper triangular components L and U:

$A=D+L+U,$

where

$D=\left[\begin{array}{cccc}{a}_{11}& 0& \cdots & 0\\ 0& a_{22}& \cdots & 0\\ ⋮& ⋮& \ddots & ⋮\\ 0& 0& \cdots & a_{nn}\end{array}\right],L=\left[\begin{array}{cccc}0& 0& \cdots & 0\\ a_{21}& 0& \cdots & 0\\ ⋮& ⋮& \ddots & ⋮\\ a_{n1}& a_{n2}& \cdots & 0\end{array}\right],U=\left[\begin{array}{cccc}0& a_{12}& \cdots & a_{1n}\\ 0& 0& \cdots & a_{2n}\\ ⋮& ⋮& \ddots & ⋮\\ 0& 0& \cdots & 0\end{array}\right].$

The system of linear equations may be rewritten as:

$(D+\omega L)\mathbf{x}=\omega \mathbf{b}-[\omega U+(\omega -1)D]\mathbf{x}$

for a constant ω > 1, called the relaxation factor.

The method of successive over-relaxation is an iterative technique that solves the left hand side of this expression for x, using previous value for x on the right hand side. Analytically, this may be written as:

${\mathbf{x}}^{(k+1)}=(D+\omega L{)}^{-1}(\omega \mathbf{b}-[\omega U+(\omega -1)D]{\mathbf{x}}^{(k)})=L_w{\mathbf{x}}^{(k)}+\mathbf{c}.$

However, by taking advantage of the triangular form of (D+ωL), the elements of x^(k+1) can be computed sequentially using forward substitution:

$x_i^{(k+1)}=(1-\omega )x_i^{(k)}+\frac{\omega }{a_{ii}}(b_i-\sum _{j<i}{a}_{ij}{x}_j^{(k+1)}-\sum _{j>i}{a}_{ij}{x}_j^{(k)}),i=1,2,\dots ,n.$

The choice of relaxation factor ω is not necessarily easy, and depends upon the properties of the coefficient matrix. In 1947, Ostrowski proved that if $A$ is symmetric and positive-definite then $\rho (L_{\omega })<1$ for $0<\omega <2$. Thus convergence of the iteration process follows, but we are generally interested in faster convergence rather than just convergence.

Algorithm

Since elements can be overwritten as they are computed in this algorithm, only one storage vector is needed, and vector indexing is omitted. The algorithm goes as follows:

Inputs: Template:Var, Template:Var, Template:Var
Output: $\varphi$

Choose an initial guess $\varphi$ to the solution
repeat until convergence

for Template:Var from 1 until Template:Var do

$\sigma \leftarrow 0$

for Template:Var from 1 until Template:Var do

if Template:Var ≠ Template:Var then

$\sigma \leftarrow \sigma +a_{ij}{\varphi }_j$

end if

end (Template:Var-loop)

${\varphi }_i\leftarrow (1-\omega ){\varphi }_i+\frac{\omega }{a_{ii}}(b_i-\sigma )$

end (Template:Var-loop)

check if convergence is reached

end (repeat)

Note:
$(1-\omega ){\varphi }_i+\frac{\omega }{a_{ii}}(b_i-\sigma )$ can also be written ${\varphi }_i+\omega (\frac{b_i-\sigma }{a_{ii}}-{\varphi }_i)$, thus saving one multiplication in each iteration of the outer for-loop.

Symmetric successive over-relaxation

The version for symmetric matrices A, in which

$U=L^T,$

is referred to as Symmetric Successive Over-Relaxation, or (SSOR), in which

$P=(\frac{D}{\omega }+L)\frac{\omega }{2-\omega }{D}^{-1}(\frac{D}{\omega }+U),$

and the iterative method is

${\mathbf{x}}^{k+1}={\mathbf{x}}^k-{\gamma }^k{P}^{-1}(A{\mathbf{x}}^k-\mathbf{b}),k\ge 0.$

The SOR and SSOR methods are credited to David M. Young, Jr..

Other applications of the method

Mining Engineer (Excluding Oil ) Truman from Alma, loves to spend time knotting, largest property developers in singapore developers in singapore and stamp collecting. Recently had a family visit to Urnes Stave Church. A similar technique can be used for any iterative method. If the original iteration had the form

$x_{n+1}=f(x_n)$

then the modified version would use

$x_{n+1}^{\mathrm{S}\mathrm{O}\mathrm{R}}=(1-\omega )x_n^{\mathrm{S}\mathrm{O}\mathrm{R}}+\omega f(x_n^{\mathrm{S}\mathrm{O}\mathrm{R}}).$

Note however that the formulation presented above, used for solving systems of linear equations, is not a special case of this formulation if Template:Mvar is considered to be the complete vector. If this formulation is used instead, the equation for calculating the next vector will look like

${\mathbf{x}}^{(k+1)}=(1-\omega ){\mathbf{x}}^{(k)}+\omega L_{*}^{-1}(\mathbf{b}-U{\mathbf{x}}^{(k)}).$

Values of $\omega >1$ are used to speedup convergence of a slow-converging process, while values of $\omega <1$ are often used to help establish convergence of a diverging iterative process or speed up the convergence of an overshooting process.

There are various methods that adaptively set the relaxation parameter $\omega$ based on the observed behavior of the converging process. Usually they help to reach a super-linear convergence for some problems but fail for the others.

See also

• Jacobi method
• Gaussian Belief Propagation
• Matrix splitting

Notes

43 year old Petroleum Engineer Harry from Deep River, usually spends time with hobbies and interests like renting movies, property developers in singapore new condominium and vehicle racing. Constantly enjoys going to destinations like Camino Real de Tierra Adentro.

References

• Template:CFDWiki

• Abraham Berman, Robert J. Plemmons, Nonnegative Matrices in the Mathematical Sciences, 1994, SIAM. ISBN 0-89871-321-8.
• 
  
  I had like 17 domains hosted on single account, and never had any special troubles. If you are not happy with the service you will get your money back with in 45 days, that's guaranteed. But the Search Engine utility inside the Hostgator account furnished an instant score for my launched website. Fantastico is unable to install WordPress in a directory which already have any file i.e to install WordPress using Fantastico the destination directory must be empty and it should not have any previous installation files. When you share great information, others will take note. Once your hosting is purchased, you will need to setup your domain name to point to your hosting. Money Back: All accounts of Hostgator come with a 45 day money back guarantee. If you have any queries relating to where by and how to use Hostgator Discount Coupon, you can make contact with us at our site. If you are starting up a website or don't have too much website traffic coming your way, a shared plan is more than enough. Condition you want to take advantage of the worldwide web you prerequisite a HostGator web page, -1 of the most trusted and unfailing web suppliers on the world wide web today. Since, single server is shared by 700 to 800 websites, you cannot expect much speed.
  
  
  
  Hostgator tutorials on how to install Wordpress need not be complicated, especially when you will be dealing with a web hosting service that is friendly for novice webmasters and a blogging platform that is as intuitive as riding a bike. After that you can get Hostgator to host your domain and use the wordpress to do the blogging. Once you start site flipping, trust me you will not be able to stop. I cut my webmaster teeth on Control Panel many years ago, but since had left for other hosting companies with more commercial (cough, cough) interfaces. If you don't like it, you can chalk it up to experience and go on. First, find a good starter template design. When I signed up, I did a search for current "HostGator codes" on the web, which enabled me to receive a one-word entry for a discount. Your posts, comments, and pictures will all be imported into your new WordPress blog.
• Yousef Saad, Iterative Methods for Sparse Linear Systems, 1st edition, PWS, 1996.
• Netlib's copy of "Templates for the Solution of Linear Systems", by Barrett et al.
• Richard S. Varga 2002 Matrix Iterative Analysis, Second ed. (of 1962 Prentice Hall edition), Springer-Verlag.
• David M. Young, Jr. Iterative Solution of Large Linear Systems, Academic Press, 1971. (reprinted by Dover, 2003)

External links

• Module for the SOR Method
• Tridiagonal linear system solver based on SOR, in C++