|
|
| Line 1: |
Line 1: |
| {{Expert-subject|statistics|date=November 2008}}
| | Let me initial start by introducing myself. My title is Boyd Butts although it is not the name on my birth certificate. Bookkeeping is her working day job now. He is really fond of performing ceramics but he is having difficulties to discover time for it. North Dakota is our beginning location.<br><br>Feel free to visit my web-site ... [http://www.fanproof.com/index.php?do=/profile-115513/info/ home std test kit] |
| | |
| [[File:Matching pursuit.png|thumb|right|350px|Signal reconstruction with matching pursuit algorithm.]]'''Matching pursuit''' is a type of numerical technique which involves finding the "best matching" projections of multidimensional data onto an over-complete dictionary <math>D</math>. The basic idea is to represent a signal <math>f</math> from [[Hilbert space]] <math>H</math> as a weighted sum of functions <math>g_{\gamma_n}</math> (called atoms) taken from <math>D</math>:
| |
| | |
| : <math> f(t) = \sum_{n=0}^{+\infty} a_n g_{\gamma_n}(t)</math>
| |
| | |
| where <math>n</math> indexes the atoms that have been chosen, and <math>a_n</math> a weighting factor (an amplitude) for each atom. Given a fixed dictionary, matching pursuit will first find the one atom that has the biggest inner product with the signal, then subtract the contribution due to that atom, and repeat the process until the signal is satisfactorily decomposed.
| |
| | |
| For comparison, consider the [[Fourier series]] representation of a signal - this can be described in the terms given above, where the dictionary is built from sinusoidal basis functions (the smallest possible complete dictionary).
| |
| The main disadvantage of Fourier analysis in signal processing is that it extracts only global features of signals and does not adapt to analysed signals <math>f</math>.
| |
| By taking an extremely redundant dictionary we can look in it for functions that best match a signal <math>f</math>. Finding a representation where most of the coefficients in the sum are close to 0 ([[Sparse approximation|sparse representation]]) is desirable for signal coding and compression.
| |
| | |
| == The algorithm ==
| |
| Searching over an extremely large dictionary for the best matches is computationally unacceptable for practical applications.
| |
| In 1993 [[Stéphane Mallat|Mallat]] and Zhang<ref name="Mallat1993" >S. G. Mallat and Z. Zhang, {{doi-inline|10.1109/78.258082|Matching Pursuits with Time-Frequency Dictionaries}}, IEEE Transactions on Signal Processing, December 1993, pp. 3397–3415.</ref> proposed a [[Greedy algorithm|greedy]] solution that is known from that time as Matching Pursuit.
| |
| The algorithm iteratively generates for any signal <math> f </math> and any dictionary <math>D</math> a sorted list of indexes and scalars which are the sub-optimal solution to the problem of sparse signal representation. The residual after calculating <math>\gamma_n</math> and <math>a_n</math> is denoted by <math>R_{n+1}</math>.
| |
| | |
| {{algorithm-begin|name=Matching Pursuit}}
| |
| Input: Signal: <math>f(t)</math>, dictionary <math>D</math>.
| |
| Output: List of coefficients: <math> \left( a_n, g_{\gamma_n}\right) </math>.
| |
| Initialization:
| |
| <math>R_1\,\leftarrow\,f(t)</math>;
| |
| <math>n\,\leftarrow\,1</math>;
| |
| Repeat:
| |
| find <math>g_{\gamma_n} \in D</math> with maximum inner product <math> | \langle R_n, g_{\gamma_n} \rangle | </math>;
| |
| <math> a_n\,\leftarrow\,\langle R_n, g_{\gamma_n}\rangle </math>;
| |
| <math> R_{n+1}\,\leftarrow\,R_n - a_n g_{\gamma_n}</math>;
| |
| <math> n\,\leftarrow\,n + 1</math>;
| |
| Until stop condition (for example: <math>\|R_n\| < \mathrm{threshold} </math>)
| |
| {{algorithm-end}}
| |
| | |
| The concept of matching pursuit in signal processing is related to statistical [[projection pursuit]], in which "interesting" projections were found; ones that deviate more from a [[normal distribution]] are considered to be more interesting.
| |
| | |
| == Properties ==
| |
| * The algorithm converges for any <math>f</math> in the space spanned by the dictionary.
| |
| * For all <math>m</math> the energy conservation equation is satisfied:
| |
| {|
| |
| <math>\|f\|^2 = \sum_{n=0}^{m-1}{|a_n|^2} + \|R_m\|^2</math>
| |
| |}
| |
| * The error <math>\|R_n\|</math> decreases monotonically and its decay is exponential.
| |
| | |
| == Applications ==
| |
| Matching pursuit has been applied to signal, image and video coding,<ref name="Mallat1995" >
| |
| F. Bergeaud and S. Mallat. {{doi-inline|10.1109/ICIP.1995.529037|Matching pursuit of images}}, In Proc. International Conference on Image Processing, volume 1, pages 53–56 vol.1, 1995.</ref><ref name="Neff1997" >
| |
| R. Neff and A. Zakhor. {{doi-inline|10.1109/76.554427|Very low bit-rate video coding based on matching pursuits}}, IEEE Transactions on Circuits and Systems for Video Technology,
| |
| 7(1):158–171, 1997.
| |
| </ref> shape representation and recognition,<ref name="Mendels2006" >
| |
| F. Mendels, P. Vandergheynst, and J.P. Thiran. Matching pursuit-based shape representation and recognition using scale-space,
| |
| 2006.
| |
| </ref> 3D objects coding,<ref name="Frossard2005" >
| |
| Tosic, I.; Frossard, P. & Vandergheynst, P. Progressive coding of 3D objects based on over-complete decompositions, 2005.
| |
| </ref> and in interdisciplinary applications like structural health monitoring.<ref name="debejyo2009">Debejyo Chakraborty, Narayan Kovvali, Jun Wei, Antonia Papandreou-Suppappola, Douglas Cochran, and Aditi Chattopadhyay, {{doi-inline|doi:10.1177/1045389X08100044| Damage Classification Structural Health Monitoring in Bolted Structures Using Time-frequency Techniques}}, Journal of Intelligent Material Systems and Structures, special issue on Structural Health Monitoring, Vol. 20(11), pp 1289-1305, July, 2009.
| |
| </ref> It has been shown that it performs better than [[Discrete cosine transform|DCT]] based coding for low bit rates in both efficiency of coding and quality of image.<ref name="Perrinet2002" >
| |
| L. U. Perrinet, M. Samuelides and S. Thorpe [http://www.incm.cnrs-mrs.fr/LaurentPerrinet/Publications/Perrinet02sparse "Sparse spike coding in an asynchronous feed-forward multi-layer neural network using Matching Pursuit."] Neurocomputing, Vol. 57C, 2002, pp. 125–34
| |
| </ref>
| |
| The main problem with Matching Pursuit is the computational complexity of the encoder. In the basic version of an algorithm the large dictionary has to be searched at each iteration. Improvements include the use of approximate dictionary representations and suboptimal ways of choosing the best match at each iteration (atom extraction).<ref name="Lin2007" >
| |
| Jian-Liang Lin, Wen-Liang Hwang, and Soo-Chang Pei. Fast matching pursuit video coding by combining dictionary approximation and atom extraction. IEEE Transactions on Circuits and Systems for Video Technology, 17(12):1679–1689, 2007.
| |
| </ref>
| |
| | |
| The Matching Pursuit algorithm is also used in [[dictionary learning]].<ref>
| |
| "The K-SVD: An Algorithm for Designing of Overcomplete Dictionaries for Sparse Representation", M. Aharon, M. Elad, and A.M. Bruckstein, IEEE Trans. on Signal Processing, 2006
| |
| </ref> In this algorithm, atoms are learned from a database and not chosen among generic dictionaries.
| |
| | |
| The Matching Pursuit algorithm is used in MP/SOFT, a method of simulating quantum dynamics. <ref>
| |
| "Matching-pursuit for simulations of quantum processes", Yinghua Wu and Victor S. Batista, Chem. Phys., Vol. 118, No. 15, 15 April 2003
| |
| </ref>
| |
| | |
| == Extensions ==
| |
| A popular extension of Matching Pursuit (MP) is its orthogonal version: Orthogonal Matching Pursuit<ref>"Orthogonal Matching Pursuit : recursive function approximation with application to wavelet decomposition", Y. Pati, R. Rezaiifar, P. Krishnaprasad, in Asilomar Conf. on Signals, Systems and Comput., 1993</ref><ref>"Adaptive time-frequency decompositions with matching pursuits", G. Davis, S. Mallat, Z. Zhang, Optical Engineering, 1994</ref> (OMP). The main difference from MP is that after every step, ''all'' the coefficients extracted so far are updated, by computing the orthogonal projection of the signal onto the set of atoms selected so far. This can lead to better results than standard MP, but requires more computation.
| |
| | |
| In fact, this algorithm approximates the sparse problem{{citation needed|date=November 2011}} :
| |
| | |
| : <math> \min_x \| f - D x \|_2^2 \ \text{ subject to } \ \|x\|_0 \le N, </math>
| |
| | |
| with <math> \|x\|_0 </math> the <math>L_0</math> pseudo-norm (i.e. the number of nonzero elements of <math>x</math>).
| |
| | |
| Extensions such as Multichannel MP<ref>"Piecewise linear source separation", R. Gribonval, Proc. SPIE '03, 2003</ref> and Multichannel OMP<ref>"Algorithms for simultaneous sparse approximations ; Part I : Greedy pursuit", J. Tropp, A. Gilbert, M. Strauss , Signal Proc. – Sparse approximations in signal and image processing, vol.86, pp 572–588, 2006</ref> allow to process multicomponents signals.
| |
| | |
| Matching pursuit is related to the field of [[compressed sensing]] and has been extended by researchers in that community. Notable extensions are Orthogonal Matching Pursuit (OMP),<ref>[http://www.acm.caltech.edu/~jtropp/papers/TG07-Signal-Recovery.pdf Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit], Joel A. Tropp and Anna C. Gilbert, IEEE Transactions on Information Theory, Vol. 53, NO. 12, December 2007</ref> Stagewise OMP (StOMP),<ref>"Sparse solution of underdetermined linear equations by stagewise orthogonal matching pursuit", David L. Donoho , Yaakov Tsaig , Iddo Drori , Jean-luc Starck, 2006</ref> and compressive sampling matching pursuit (CoSaMP).<ref>"CoSaMP: Iterative signal recovery from incomplete and inaccurate samples", D. Needell and J.A. Tropp, Applied and Computational Harmonic Analysis, 2009</ref>
| |
| | |
| == References ==
| |
| <references/>
| |
| {{Reflist}}
| |
| | |
| == See also ==
| |
| | |
| * [[Principal component analysis|Principal component analysis (PCA)]]
| |
| * [[Projection pursuit]]
| |
| * [[Image processing]]
| |
| * [[Signal processing]]
| |
| * [[Sparse approximation]]
| |
| | |
| [[Category:Multivariate statistics]]
| |
| [[Category:Signal processing]]
| |