|
|
| Line 1: |
Line 1: |
| '''Compressed sensing''' (also known as '''compressive sensing''', '''compressive sampling''', or '''sparse sampling''') is a [[signal processing]] technique for efficiently acquiring and reconstructing a [[Signal (electronics)|signal]], by finding solutions to [[Underdetermined system|underdetermined linear systems]]. <ref>For most large underdetermined systems of linear equations the minimal 𝓁1-norm solution is also the sparsest solution; See Donoho, David L, Communications on pure and applied mathematics, 59, 797 (2006) http://dx.doi.org/10.1002/cpa.20132</ref> <ref>[http://www.brainshark.com/brainshark/brainshark.net/portal/title.aspx?pid=zCdz10BfTRz0z0 M. Davenport, "The Fundamentals of Compressive Sensing", IEEE Signal Processing Society Online Tutorial Library, April 12, 2013.]</ref> This takes advantage of the signal's [[Sparse matrix|sparseness]] or [[data compression|compressibility]] in some domain, allowing the entire signal to be determined from relatively few measurements.<ref>[http://nuit-blanche.blogspot.com/2009/09/cs.html CS: Compressed Genotyping, DNA Sudoku - Harnessing high throughput sequencing for multiplexed specimen analysis]</ref> [[Magnetic resonance imaging|MRI]] is a prominent application. <ref>Sparse MRI: The application of compressed sensing for rapid MR imaging; See Lustig, Michael and Donoho, David and Pauly, John M, Magnetic resonance in medicine, 58(6), 1182-1195 (2007) http://dx.doi.org/10.1002/mrm.21391</ref><ref>Compressed Sensing MRI; See Lustig, M.; Donoho, D.L.; Santos, J.M. ; Pauly, J.M., Signal Processing Magazine, IEEE, 25(2),72-82 (2008) http://dx.doi.org/10.1109/MSP.2007.914728</ref><ref>[http://www.theengineer.co.uk/news/news-analysis/picture-of-health/1001485.article Compressive sampling makes medical imaging safer]</ref>
| | The author's name is Andera and she thinks it seems quite good. Some time ago she selected to live in Alaska and her mothers and fathers reside close by. My husband doesn't like it the way I do but what I really like doing is caving but I don't have the time recently. Invoicing is what I do.<br><br>my blog; love psychic readings, [http://cartoonkorea.com/ce002/1093612 click through the up coming website page], |
| | |
| == Overview ==
| |
| A common goal of [[signal processing]] is to reconstruct a signal from a series of sampling measurements. In general, this task is impossible because there is no way to reconstruct a signal during the times that the signal isn’t measured. Nevertheless, with prior knowledge or assumptions about the signal, it turns out to be possible to perfectly reconstruct a signal from a series of measurements. Over time, mathematicians have improved their understanding of which assumptions are practical and how they can be generalized.
| |
| | |
| An early breakthrough in signal processing was the [[Nyquist-Shannon sampling theorem]]. It proved that a signal can be perfectly reconstructed from sampling if the signal’s highest frequency is half (or less) of the sampling rate. The main idea is that if you have prior knowledge about the signal’s frequencies, you need fewer samples to reconstruct the signal.
| |
| | |
| Around 2004, [[Emmanuel Candès]], [[Terence Tao]], and [[David Donoho]] proved that given knowledge about a signal's [[sparsity]], the signal may be reconstructed with fewer samples than the Nyquist-Shannon theorem requires.<ref>{{Cite journal|doi=10.1002/cpa.20124|url=http://www-stat.stanford.edu/~candes/papers/StableRecovery.pdf|title=Stable signal recovery from incomplete and inaccurate measurements|year=2006|last1=Candès|first1=Emmanuel J.|last2=Romberg|first2=Justin K.|last3=Tao|first3=Terence|journal=Communications on Pure and Applied Mathematics|volume=59|issue=8|pages=1207}}</ref><ref name=Donoho>{{Cite journal|doi=10.1109/TIT.2006.871582|title=Compressed sensing|year=2006|last1=Donoho|first1=D.L.|journal=IEEE Transactions on Information Theory|volume=52|issue=4|pages=1289}}</ref> This idea is the basis of compressed sensing.
| |
| | |
| ==History==
| |
| Compressed sensing relies on [[Lp space|L1]] techniques, which several other scientific fields have used historically.<ref>[http://2.bp.blogspot.com/_0ZCyAOBrUtA/TTwqLEeLvdI/AAAAAAAAEXI/7S0_SnWoC0E/s1600/l1-minimization.JPG List of L1 regularization ideas] from Vivek Goyal, Alyson Fletcher, Sundeep Rangan, [http://www.math.uiuc.edu/%7Elaugesen/imaha10/goyal_talk.pdf The Optimistic Bayesian: Replica Method Analysis of Compressed Sensing]</ref> In statistics, the [[least-squares method]] was complemented by the [[Lp norm|<math>L^1</math>-norm]], which was introduced by [[Laplace]]. Following the introduction of [[linear programming]] and [[George B. Dantzig|Dantzig]]'s [[simplex algorithm]], the <math>L^1</math>-norm was used in [[computational statistics]]. In statistical theory, the <math>L^1</math>-norm was used by [[George W. Brown]] and later writers on [[median-unbiased estimator]]s. It was used by [[Peter Huber]] and others working on [[robust statistics]]. The <math>L^1</math>-norm was also used in signal processing, for example, in the 1970s, when seismologists constructed images of reflective layers within the earth based on data that did not seem to satisfy the [[Nyquist–Shannon sampling theorem|Nyquist–Shannon criterion]].<ref>{{Cite journal|doi=10.1511/2009.79.276|title=The Best Bits|year=2009|last1=Hayes|first1=Brian|journal=American Scientist|volume=97|issue=4|pages=276}}</ref> It was used in [[matching pursuit]] in 1993, the [[Lasso regression|LASSO estimator]] by [[Robert Tibshirani]] in 1996<ref>{{Cite journal|url=http://www-stat.stanford.edu/~tibs/lasso.html The Lasso page|first= Robert |last=Tibshirani|title=Regression shrinkage and selection via the lasso|journal=[[Journal of the Royal Statistical Society, Series B]]|volume= 58|issue=1| pages= 267–288}}</ref> and [[basis pursuit]] in 1998.<ref>"Atomic decomposition by basis pursuit", by Scott Shaobing Chen, David L. Donoho, Michael, A. Saunders. SIAM Journal on Scientific Computing</ref> There were theoretical results describing when these algorithms recovered sparse solutions, but the required type and number of measurements were sub-optimal and subsequently greatly improved by compressed sensing.{{citation needed|date=May 2013}}
| |
| | |
| At first glance, compressed sensing might seem to violate [[Nyquist–Shannon sampling theorem|the sampling theorem]], because compressed sensing depends on the [[Sparse matrix|sparsity]] of the signal in question and not its highest frequency. This is a misconception, because the sampling theorem guarantees perfect reconstruction given sufficient, not necessary, conditions. A sampling method different from the classical fixed-rate sampling therefore can not "violate" the sampling theorem. Sparse signals with high frequency components can be highly under-sampled using compressed sensing compared to classical fixed-rate sampling.<ref>{{Cite journal|url=http://www-stat.stanford.edu/~candes/papers/ExactRecovery.pdf|title=Robust Uncertainty Principles: Exact Signal Reconstruction from Highly Incomplete Fourier Information|year=2006|last1=Candès|first1=Emmanuel J.|last2=Romberg|first2=Justin K.|last3=Tao|first3=Terence|journal=IEEE Trans. Inf. Theory|volume=52|issue=8|pages=489–509}}</ref>
| |
| | |
| ==Method==
| |
| | |
| ===Underdetermined linear system===
| |
| An [[underdetermined system]] of linear equations has more unknowns than equations and generally has an infinite number of solutions. In order to choose a solution to such a system, one must impose extra constraints or beliefs (such as smoothness) as appropriate.
| |
| | |
| In compressed sensing, one adds the constraint of sparsity, allowing only solutions which have a small number of nonzero coefficients. Not all underdetermined systems of linear equations have a sparse solution. However, if there is a unique sparse solution to the underdetermined system, then the Compressed Sensing framework allows the recovery of that solution.
| |
| | |
| ===Solution / reconstruction method===
| |
| Compressed sensing takes advantage of the redundancy in many interesting signals—they are not pure noise. In particular, many signals are [[sparse matrix|sparse]], that is, they contain many coefficients close to or equal to zero, when represented in some domain.<ref>Candès, E.J., & Wakin, M.B., ''An Introduction To Compressive Sampling'', IEEE Signal Processing Magazine, V.21, March 2008 [http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=4472240&isnumber=4472102]</ref> This is the same insight used in many forms of [[lossy compression]].
| |
| | |
| Compressed sensing typically starts with taking a weighted linear combination of samples also called compressive measurements in a [[Basis (linear algebra)|basis]] different from the basis in which the signal is known to be sparse. The results found by [[Emmanuel Candès]], [[Justin Romberg]], [[Terence Tao]] and [[David Donoho]], showed that the number of these compressive measurements can be small and still contain nearly all the useful information. Therefore, the task of converting the image back into the intended domain involves solving an [[Underdetermined system|underdetermined]] [[matrix equation]] since the number of compressive measurements taken is smaller than the number of pixels in the full image. However, adding the constraint that the initial signal is sparse enables one to solve this [[Underdetermined system|underdetermined]] [[system of linear equations]].
| |
| | |
| The least-squares solution to such problems is to minimize the [[L2 norm|<math>L^2</math> norm]]—that is, minimize the amount of energy in the system. This is usually simple mathematically (involving only a [[matrix multiplication]] by the [[pseudo-inverse]] of the basis sampled in). However, this leads to poor results for many practical applications, for which the unknown coefficients have nonzero energy.
| |
| | |
| To enforce the sparsity constraint when solving for the [[Underdetermined system|underdetermined system of linear equations]], one can minimize the number of nonzero components of the solution.
| |
| | |
| The function counting the number of non-zero components of a vector was called the [[L0 norm|<math>L^0</math> "norm"]] by David Donoho. The quotation marks served two warnings. First, the number-of-nonzeros <math>L^0</math>-"norm" is not a proper [[F-space|F-norm]], because it is not continuous in its scalar argument: ''nnzs''(α''x'') is constant as α approaches zero. Unfortunately, authors now neglect the quotation marks and [[abuse of terminology|abused terminology]]—clashing with the established use of the <math>L^0</math> norm for the space of measurable functions (equipped with an appropriate metric) or for the [[F-space|space]] of sequences with [[F-space|F–norm]] <math>(x_n) \mapsto \sum_n{2^{-n} x_n/(1+x_n)}</math>.<ref>Stefan Rolewicz. ''Metric Linear Spaces''.</ref>
| |
| | |
| [[Emmanuel Candès|Candès]]. et al., proved that for many problems it is probable that the [[L1 norm|<math>L^1</math> norm]] is equivalent to the [[L0 norm|<math>L^0</math> norm]], in a technical sense: This equivalence result allows one to solve the <math>L^1</math> problem, which is easier than the <math>L^0</math> problem. Finding the candidate with the smallest <math>L^1</math> norm can be expressed relatively easily as a [[linear program]], for which efficient solution methods already exist.<ref>[http://www.acm.caltech.edu/l1magic/ L1-MAGIC is a collection of MATLAB routines]</ref> When measurements may contain a finite amount of noise, [[basis pursuit denoising]] is preferred over linear programming, since it preserves sparsity in the face of noise and can be solved faster than an exact linear program.
| |
| | |
| ==Applications==
| |
| The field of compressive sensing is related to other topics in signal processing and computational mathematics, such as to [[underdetermined system|underdetermined linear-system]]s, [[group testing]], heavy hitters, [[sparse coding]], [[multiplexing]], sparse sampling, and finite rate of innovation. Imaging techniques having a strong affinity with compressive sensing include [[coded aperture]] and [[computational photography]]. Implementations of compressive sensing in hardware at different [[technology readiness level]] is available.<ref>Compressive Sensing Hardware, http://sites.google.com/site/igorcarron2/compressedsensinghardware</ref>
| |
| | |
| ===Photography===
| |
| Compressed sensing is used in a mobile phone camera sensor. The approach allows a reduction in image acquisition energy per image by as much as a factor of 15 at the cost of complex decompression algorithms; the computation may require an off-device implementation.<ref>{{cite journal|title=New Camera Chip Captures Only What It Needs|author=David Schneider|journal=IEEE Spectrum|date=March 2013|url=http://spectrum.ieee.org/semiconductors/optoelectronics/camera-chip-makes-alreadycompressed-images|accessdate=2013-03-20}}</ref>
| |
| | |
| Compressed sensing is used in single-pixel cameras from [[Rice University]].<ref name=cscamera>{{cite web|url=http://dsp.rice.edu/cscamera |title=Compressive Imaging: A New Single-Pixel Camera | Rice DSP |publisher=Dsp.rice.edu |date= |accessdate=2013-06-04}}</ref> [[Bell Labs]] employed the technique in a lensless single-pixel camera that takes stills using repeated snapshots of randomly chosen apertures from a grid. Image quality improves with the number of snapshots, and generally requires a small fraction of the data of conventional imaging, while eliminating lens/focus-related aberrations.<ref>{{cite web|author=The Physics arXiv Blog June 3, 2013 |url=http://www.technologyreview.com/view/515651/bell-labs-invents-lensless-camera/ |title=Bell Labs Invents Lensless Camera | MIT Technology Review |publisher=Technologyreview.com |date=2013-05-25 |accessdate=2013-06-04}}</ref><ref>{{cite journal|author1=Gang Huang|author2=Hong Jiang|author3=Kim Matthews|author4=Paul Wilford|title=Lensless Imaging by Compressive Sensing|year=2393|volume=2013|journal=IEEE International Conference on Image Processing, ICIP , Paper #|arxiv=1305.7181}}</ref>
| |
| | |
| ===Holography===
| |
| Compressed sensing can be used to improve image reconstruction in holography by increasing the number of voxels one can infer from a single hologram.<ref>David Brady, Kerkil Choi, Daniel Marks, Ryoichi Horisaki, and Sehoon Lim. Compressive holography. Optics Express, 17:13040–13049, 2009</ref><ref>Rivenson, Y., Stern, A., & Javidi, B. (2010). Compressive fresnel holography. Display Technology, Journal of, 6(10), 506-509.</ref><ref>Loic Denis, Dirk Lorenz, Eric Thibaut, Corinne Fournier, and Dennis Trede. Inline hologram reconstruction with sparsity constraints. Opt. Lett., 34(22):3475–3477, 2009.</ref> It is also used for image retrieval from undersampled measurements in optical <ref>Marim, M., Angelini, E., Olivo-Marin, J. C., & Atlan, M. (2011). Off-axis compressed holographic microscopy in low-light conditions. Optics Letters, 36(1), 79-81. http://arxiv.org/abs/1101.1735</ref><ref>Marim, M. M., Atlan, M., Angelini, E., & Olivo-Marin, J. C. (2010). Compressed sensing with off-axis frequency-shifting holography. Optics letters, 35(6), 871-873. http://arxiv.org/abs/1004.5305</ref> and millimeter-wave <ref>Christy Fernandez Cull, David A. Wikner, Joseph N. Mait, Michael Mattheiss, and David J. Brady. Millimeter-wave compressive holography. Appl. Opt., 49(19):E67–E82, 2010.</ref> holography.
| |
| | |
| ===Shortwave Infrared Cameras===
| |
| InView Technology Corporation<ref>{{cite web|title=InView web site|publisher=http://www.inviewcorp.com/products}}</ref> has developed commercially-available shortwave infrared cameras based upon Compressed Sensing. These cameras have light sensitivity from 0.9 [[µm]] to 1.7 µm, which are wavelengths invisible to the human eye. InView is building upon the Rice University single-pixel camera results.
| |
| | |
| ===Optical System Research===
| |
| InView Technology Corporation<ref>{{cite web|title=InView web site|publisher=http://www.inviewcorp.com/technology/compressive-sensing/}}</ref> has developed optical Compressive Sensing Workstations that allow optical-system researchers to develop and test novel modulation and reconstruction algorithms. These Workstations include a spatial light modulation subsystem, and a light acquisition sub-system.
| |
| | |
| ===Facial recognition===
| |
| Compressed sensing is being used in facial recognition applications.<ref>[http://www.wired.com/science/discoveries/news/2008/03/new_face_recognition Engineers Test Highly Accurate Face Recognition]</ref>
| |
| | |
| ===MRI===
| |
| Compressed sensing has been used <ref>Sparse MRI: The application of compressed sensing for rapid MR imaging; See Lustig, Michael and Donoho, David and Pauly, John M, Magnetic resonance in medicine, 58(6), 1182-1195 (2007) http://dx.doi.org/10.1002/mrm.21391</ref><ref>Compressed Sensing MRI; See Lustig, M.; Donoho, D.L.; Santos, J.M. ; Pauly, J.M., Signal Processing Magazine, IEEE, 25(2),72-82 (2008) http://dx.doi.org/10.1109/MSP.2007.914728</ref> to shorten MRI scanning sessions on conventional hardware.<ref>{{cite web|author=By Jordan EllenbergEmail Author |url=http://www.wired.com/magazine/2010/02/ff_algorithm/all/1 |title=Fill in the Blanks: Using Math to Turn Lo-Res Datasets Into Hi-Res Samples | Wired Magazine |publisher=Wired.com |date=2010-03-04 |accessdate=2013-06-04}}</ref><ref>[http://nuit-blanche.blogspot.com/2010/03/why-compressed-sensing-is-not-csi.html Why Compressed Sensing is NOT a CSI "Enhance" technology ... yet !]</ref><ref>[http://nuit-blanche.blogspot.com/2010/03/surely-you-must-be-joking-mr.html Surely You Must Be Joking Mr. Screenwriter]</ref>
| |
| | |
| ==See also==
| |
| *[[Noiselet]]
| |
| *[[Sparse approximation]]
| |
| | |
| ==References==
| |
| {{reflist|30em}}
| |
| | |
| ==Further reading==
| |
| * "The Fundamentals of Compressive Sensing" [http://www.brainshark.com/brainshark/brainshark.net/portal/title.aspx?pid=zCdz10BfTRz0z0 Part 1], [http://www.brainshark.com/brainshark/brainshark.net/portal/title.aspx?pid=zCgzXgcEKz0z0 Part 2] and [http://www.brainshark.com/brainshark/brainshark.net/portal/title.aspx?pid=zAvz9F41cz0z0 Part 3]: video tutorial by Mark Davenport, Georgia Tech. at [http://www.brainshark.com/sps IEEE Signal Processing Society Online Tutorial Library].
| |
| * [http://www.wired.com/magazine/2010/02/ff_algorithm/all/1 Using Math to Turn Lo-Res Datasets Into Hi-Res Samples] Wired Magazine article
| |
| * [http://dsp.rice.edu/cs Compressive Sensing Resources] at [[Rice University]].
| |
| * [http://igorcarron.googlepages.com/cs Compressed Sensing: The Big Picture]
| |
| * [http://igorcarron.googlepages.com/compressedsensinghardware A list of different hardware implementation of Compressive Sensing]
| |
| * [http://compressedsensing.googlepages.com/home Compressed Sensing 2.0 ]
| |
| * [http://www.ams.org/happening-series/hap7-pixel.pdf Compressed Sensing Makes Every Pixel Count] – article in the AMS ''What's Happening in the Mathematical Sciences'' series
| |
| * [http://nuit-blanche.blogspot.com/search/label/CS Nuit Blanche] A blog on Compressive Sensing featuring the most recent information on the subject (preprints, presentations, Q/As)
| |
| * [http://igorcarron.googlepages.com/csvideos Online Talks focused on Compressive Sensing]
| |
| * [http://ugcs.caltech.edu/~srbecker/wiki/Main_Page Wiki on sparse reconstruction]
| |
| | |
| {{DEFAULTSORT:Compressed Sensing}}
| |
| [[Category:Information theory]]
| |
| [[Category:Signal processing]]
| |
| [[Category:Linear algebra]]
| |
| [[Category:Regression analysis]]
| |
| [[Category:Mathematical optimization]]
| |