|
|
| Line 1: |
Line 1: |
| '''Exponential smoothing''' is a technique that can be applied to [[time series]] data, either to produce smoothed data for presentation, or to make forecasts. The time series data themselves are a sequence of observations. The observed phenomenon may be an essentially [[stochastic process|random process]], or it may be an orderly, but [[statistical noise|noisy]], process. Whereas in the [[simple moving average]] the past observations are weighted equally, exponential smoothing assigns exponentially decreasing weights over time.
| | Hello, I'm Carrol, a 19 year old from Wingellina, Australia.<br>My hobbies include (but are not limited to) Crocheting, Radio-Controlled Car Racing and watching Arrested Development.<br><br>Here is my page - [http://Drunvalo.net/forum//profile.php?id=117903 How To Get Free Fifa 15 Coins] |
| | |
| Exponential smoothing is commonly applied to financial market and economic data, but it can be used with any discrete set of repeated measurements. The raw data sequence is often represented by {''x''<sub>''t''</sub>}, and the output of the exponential smoothing algorithm is commonly written as {''s''<sub>''t''</sub>}, which may be regarded as a best estimate of what the next value of ''x'' will be. When the sequence of observations begins at time ''t'' = 0, the simplest form of exponential smoothing is given by the formulae:<ref name=NIST />
| |
| | |
| <math>
| |
| \begin{align}
| |
| s_0& = x_0\\
| |
| s_{t}& = \alpha x_{t-1} + (1-\alpha)s_{t-1},\ t>0
| |
| \end{align}
| |
| </math>
| |
| | |
| where α is the ''smoothing factor'', and 0 < α < 1.
| |
| | |
| ==Background==
| |
| | |
| ===The simple moving average===
| |
| | |
| Intuitively, the simplest way to smooth a time series is to calculate a simple, or unweighted, moving average. The smoothed statistic ''s''<sub>''t''</sub> is then just the [[mean#Arithmetic mean|mean]] of the last ''k'' observations:
| |
| | |
| :<math>
| |
| s_t = \frac{1}{k} \, \sum_{n=0}^{k-1} x_{t-n}
| |
| = \frac{x_t + x_{t-1} + x_{t-2} + \cdots + x_{t-k+1}}{k} = s_{t-1} + \frac{x_t - x_{t-k}}{k},
| |
| </math>
| |
| | |
| where the choice of an integer ''k'' > 1 is arbitrary. A small value of ''k'' will have less of a smoothing effect and be more responsive to recent changes in the data, while a larger ''k'' will have a greater smoothing effect, and produce a more pronounced [[autocorrelation#Statistics|lag]] in the smoothed sequence. One disadvantage of this technique is that it cannot be used on the first ''k'' −1 terms of the time series without the addition of values created by some other means. This means effectively [[extrapolating]] outside the existing data, and the validity of this section would therefore be questionable and not a direct representation of the data.
| |
| | |
| It also introduces a [[phase shift]] into the data of half the window length. For example if the data were all the same except for one high data point, the peak in the "smoothed" data would appear half a window length later than when it actually occurred. Where the phase of the result is important, this can be simply corrected by shifting the resulting series back by half the window length.
| |
| | |
| A major drawback with the SMA is that it lets through a significant amount of the signal shorter than the window length. Worse, it actually inverts it. This can lead to unexpected artifacts, such as peaks in the "smoothed" result appearing where there were troughs in the data. It also leads to the result being less "smooth" than expected since some of the higher frequencies are not properly removed.
| |
| | |
| See [[moving average]] for more detail.
| |
| | |
| ===The weighted moving average===
| |
| | |
| A slightly more intricate method for smoothing a raw time series {''x''<sub>''t''</sub>} is to calculate a weighted moving average by first choosing a set of weighting factors
| |
| | |
| :<math>
| |
| \lbrace w_1, w_2,\dots,w_k \rbrace </math> such that <math> \sum_{n=1}^k w_n = 1
| |
| </math>
| |
| | |
| and then using these weights to calculate the smoothed statistics {''s''<sub>''t''</sub>}:
| |
| | |
| :<math>
| |
| s_t = \sum_{n=1}^k w_n x_{t+1-n} = w_1x_t + w_2x_{t-1} + \cdots + w_kx_{t-k+1}.
| |
| </math>
| |
| | |
| In practice the weighting factors are often chosen to give more weight to the most recent terms in the time series and less weight to older data. Notice that this technique has the same disadvantage as the simple moving average technique (i.e., it cannot be used until at least ''k'' observations have been made), and that it entails a more complicated calculation at each step of the smoothing procedure. In addition to this disadvantage, if the data from each stage of the averaging is not available for analysis, it may be difficult if not impossible to reconstruct a changing signal accurately (because older samples may be given less weight). If the number of stages missed is known however, the weighting of values in the average can be adjusted to give equal weight to all missed samples to avoid this issue.
| |
| | |
| ==The exponential moving average==
| |
| | |
| Exponential smoothing was first suggested by [[Robert Goodell Brown]] in 1956,<ref>{{cite book|last=Brown|first=Robert G.|title=Exponential Smoothing for Predicting Demand|year=1956|publisher=Arthur D. Little Inc|location=Cambridge, Massachusetts|pages=15|url=http://legacy.library.ucsf.edu/tid/dae94e00;jsessionid=104A0CEFFA31ADC2FA5E0558F69B3E1D.tobacco03}}</ref> and then expanded by [[Charles C. Holt]] in 1957.<ref>{{cite journal|title=Forecasting Trends and Seasonal by Exponentially Weighted Averages|first=Charles C.|last=Holt|authorlink=Charles C. Holt|journal=Office of Naval Research Memorandum|volume=52|year=1957}} reprinted in {{cite journal|title=Forecasting Trends and Seasonal by Exponentially Weighted Averages|first=Charles C.|last=Holt|authorlink=Charles C. Holt|journal=[[International Journal of Forecasting]] |volume=20 |issue=1 |date=January–March 2004|pages=5–10|url=http://www.sciencedirect.com/science?_ob=ArticleURL&_udi=B6V92-4BJVV07-3&_user=1535420&_coverDate=03%2F31%2F2004&_rdoc=1&_fmt=high&_orig=gateway&_origin=gateway&_sort=d&_docanchor=&view=c&_searchStrId=1755927165&_rerunOrigin=google&_acct=C000053610&_version=1&_urlVersion=0&_userid=1535420&md5=0e881c06b0512cbf976a92268d48edc4&searchtype=a|doi= 10.1016/j.ijforecast.2003.09.015}}</ref> The formulation below, which is the one commonly used, is attributed to Brown and is known as "Brown's simple exponential smoothing".<ref>{{cite book|title=Smoothing Forecasting and Prediction of Discrete Time Series |last=Brown |first=Robert Goodell |year=1963 |publisher=Prentice-Hall|location=Englewood Cliffs, NJ}}</ref>
| |
| | |
| The simplest form of exponential smoothing is given by the formulae:
| |
| | |
| :<math>
| |
| \begin{align}
| |
| s_1& = x_0\\
| |
| s_t& = \alpha x_{t-1} + (1-\alpha)s_{t-1} = s_{t-1} + \alpha (x_{t-1} - s_{t-1}), t>1 \,
| |
| \end{align}
| |
| </math>
| |
| | |
| where α is the ''smoothing factor'', and 0 < α < 1. In other words, the smoothed statistic ''s''<sub>''t''</sub> is a simple weighted average of the previous observation ''x''<sub>''t''-1</sub> and the previous smoothed statistic ''s''<sub>''t''−1</sub>. The term ''smoothing factor'' applied to α here is something of a misnomer, as larger values of α actually reduce the level of smoothing, and in the limiting case with α = 1 the output series is just the same as the original series (with lag of one time unit).
| |
| Simple exponential smoothing is easily applied, and it produces a smoothed statistic as soon as two observations are available.
| |
| | |
| Values of α close to one have less of a smoothing effect and give greater weight to recent changes in the data, while values of α closer to zero have a greater smoothing effect and are less responsive to recent changes. There is no formally correct procedure for choosing α. Sometimes the statistician's judgment is used to choose an appropriate factor. Alternatively, a statistical technique may be used to ''optimize'' the value of α. For example, the [[least squares|method of least squares]] might be used to determine the value of α for which the sum of the quantities (''s''<sub>''n''-1</sub> − ''x''<sub>''n''-1</sub>)<sup>2</sup> is minimized.
| |
| | |
| Unlike some other smoothing methods, this technique does not require any minimum number of observations to be made before it begins to produce results. In practice, however, a "good average" will not be achieved until several samples have been averaged together; for example, a constant signal will take approximately ''3''/''α'' stages to reach 95% of the actual value. To accurately reconstruct the original signal without information loss all stages of the exponential moving average must also be available, because older samples decay in weight exponentially. This is in contrast to a simple moving average, in which some samples can be skipped without as much loss of information due to the constant weighting of samples within the average. If a known number of samples will be missed, one can adjust a weighted average for this as well, by giving equal weight to the new sample and all those to be skipped.
| |
| | |
| This simple form of exponential smoothing is also known as an [[Moving average#Exponential moving average|exponentially weighted moving average]] (EWMA). Technically it can also be classified as an [[Autoregressive integrated moving average]] (ARIMA) (0,1,1) model with no constant term.<ref>{{cite web|url=http://www.duke.edu/~rnau/411avg.htm|title=Averaging and Exponential Smoothing Models|accessdate=26 July 2010}}</ref>
| |
| | |
| ===Choosing the initial smoothed value===
| |
| Note that in the above definition ''s''<sub>''1''</sub> is being initialized to ''x''<sub>''0''</sub>. There are many other ways of setting this initial value, such as averaging the first 4 values of ''x'', but it is important to note that the smaller the value of ''α'', the more sensitive your forecast will be on the selection of this initial smoother value ''s''<sub>''1''</sub>.
| |
| | |
| ===Why is it "exponential"?===
| |
| | |
| By direct substitution of the defining equation for simple exponential smoothing back into itself we find that
| |
| | |
| :<math>
| |
| \begin{align}
| |
| s_t& = \alpha x_{t-1} + (1-\alpha)s_{t-1}\\[3pt]
| |
| & = \alpha x_{t-1} + \alpha (1-\alpha)x_{t-2} + (1 - \alpha)^2 s_{t-2}\\[3pt]
| |
| & = \alpha \left[x_{t-1} + (1-\alpha)x_{t-2} + (1-\alpha)^2 x_{t-3} + (1-\alpha)^3 x_{t-4} + \cdots \right]
| |
| + (1-\alpha)^{t} s_0.
| |
| \end{align}
| |
| </math>
| |
| | |
| In other words, as time passes the smoothed statistic ''s''<sub>''t''</sub> becomes the weighted average of a greater and greater number of the past observations ''x''<sub>''t−n''</sub>, and the weights assigned to previous observations are in general proportional to the terms of the geometric progression {1, (1 − α), (1 − α)<sup>2</sup>, (1 − α)<sup>3</sup>, ...}. A [[geometric progression]] is the discrete version of an [[exponential function]], so this is where the name for this smoothing method originated.
| |
| | |
| ===Comparison with moving average===
| |
| | |
| Exponential smoothing and moving average have similar defects of introducing a lag relative to the input data. While this can be corrected by shifting the result by half the window length for a symmetrical kernel, such as a moving average or gaussian, it is unclear how appropriate this would be for exponential smoothing. They also both have roughly the same distribution of forecast error when ''α = 2/(k+1)''. They differ in that exponential smoothing takes into account all past data, whereas moving average only takes into account ''k'' past data points. Computationally speaking, they also differ in that moving average requires that the past ''k'' data points be kept, whereas exponential smoothing only needs the most recent forecast value to be kept.<ref>{{cite book|last=Nahmias|first=Steven|title=Production and Operations Analysis|edition=6th edition|ISBN=0-07-337785-6}}{{Page needed|date=September 2011}}</ref>
| |
| | |
| ==Double exponential smoothing==
| |
| Simple exponential smoothing does not do well when there is a [[Trend estimation|trend]] in the data.<ref name=NIST>{{cite web|url=http://www.itl.nist.gov/div898/handbook/|title=NIST/SEMATECH e-Handbook of Statistical Methods|accessdate=2010-05-23|publisher=NIST}}</ref> In such situations, several methods were devised under the name "double exponential smoothing" or "second-order exponential smoothing."<ref>{{cite web |url=http://help.sap.com/saphelp_45b/helpdata/en/7d/c27a14454011d182b40000e829fbfe/content.htm |title=Model: Second-Order Exponential Smoothing |author=<!--Staff writer(s); no by-line.--> |publisher=[[SAP AG]] |accessdate=23 January 2013}}</ref>
| |
| | |
| One method, sometimes referred to as "Holt-Winters double exponential smoothing"<ref>{{cite web|url=http://www.it.iitb.ac.in/~praj/acads/seminar/04329008_ExponentialSmoothing.pdf|title=Time series Forecasting using Holt-Winters Exponential Smoothing|format=PDF|author=Prajakta S. Kalekar}}</ref> works as follows:<ref>{{cite web |url= http://www.itl.nist.gov/div898/handbook/pmc/section4/pmc433.htm |title=6.4.3.3. Double Exponential Smoothing |work=itl.nist.gov |accessdate=25 September 2011}}</ref>
| |
| | |
| Again, the raw data sequence of observations is represented by {''x''<sub>''t''</sub>}, beginning at time ''t'' = 0. We use {''s''<sub>''t''</sub>} to represent the smoothed value for time ''t'', and {''b''<sub>''t''</sub>} is our best estimate of the trend at time ''t''. The output of the algorithm is now written as ''F''<sub>''t+m''</sub>, an estimate of the value of ''x'' at time ''t+m, m>0'' based on the raw data up to time ''t''. Double exponential smoothing is given by the formulas
| |
| | |
| :<math>
| |
| \begin{align}
| |
| s_1& = x_1\\
| |
| b_1& = x_1 - x_0\\
| |
| \end{align}
| |
| | |
| </math>
| |
| | |
| And for t > 1 by
| |
| | |
| :<math>
| |
| \begin{align}
| |
| s_{t}& = \alpha x_{t} + (1-\alpha)(s_{t-1} + b_{t-1})\\
| |
| b_{t}& = \beta (s_t - s_{t-1}) + (1-\beta)b_{t-1}\\
| |
| \end{align}
| |
| | |
| </math>
| |
| where α is the ''data smoothing factor'', 0 < α < 1, and β is the ''trend smoothing factor'', 0 < β < 1.
| |
| | |
| To forecast beyond x_t | |
| :<math>
| |
| \begin{align}
| |
| F_{t+m}& = s_t + mb_t
| |
| \end{align}
| |
| </math>
| |
| | |
| Setting the initial value ''b''<sub>''0''</sub> is a matter of preference. An option other than the one listed above is ''(x<sub>''n''</sub> - x<sub>''0''</sub>)/n'' for some ''n > 1''.
| |
| | |
| Note that ''F''<sub>0</sub> is undefined (there is no estimation for time 0), and according to the definition ''F''<sub>1</sub>=''s''<sub>0</sub>+''b''<sub>0</sub>, which is well defined, thus further values can be evaluated.
| |
| | |
| A second method, referred to as either Brown's linear exponential smoothing (LES) or Brown's double exponential smoothing works as follows.<ref>{{cite web |url= http://www.duke.edu/~rnau/411avg.htm |title=Averaging and Exponential Smoothing Models |work=duke.edu |accessdate=25 September 2011}}</ref>
| |
| | |
| :<math>
| |
| \begin{align}
| |
| s'_0& = x_0\\
| |
| s''_0& = x_0\\
| |
| s'_{t}& = \alpha x_{t} + (1-\alpha)s'_{t-1}\\
| |
| s''_{t}& = \alpha s'_{t} + (1-\alpha)s''_{t-1}\\
| |
| F_{t+m}& = a_t + mb_t,
| |
| \end{align}
| |
| </math>
| |
| | |
| where ''a''<sub>''t''</sub>, the estimated level at time ''t'' and ''b''<sub>''t''</sub>, the estimated trend at time ''t'' are:
| |
| | |
| :<math>
| |
| \begin{align}
| |
| a_t& = 2s'_t - s''_t\\
| |
| b_t& = \frac \alpha {1-\alpha} (s'_t - s''_t).
| |
| \end{align}
| |
| </math>
| |
| | |
| ==Triple exponential smoothing==
| |
| Triple exponential smoothing takes into account seasonal changes as well as trends. It was first suggested by Holt's student, Peter Winters, in 1960.<ref>{{cite journal|first=P. R.|last=Winters|title=Forecasting Sales by Exponentially Weighted Moving Averages|journal=[[Management Science: A Journal of the Institute for Operations Research and the Management Sciences|Management Science]]|volume=6|issue=3|date=April 1960|pages=324–342|doi=10.1287/mnsc.6.3.324|url=http://mansci.journal.informs.org/cgi/content/abstract/6/3/324}}</ref>
| |
| Suppose we have a sequence of observations {''x''<sub>''t''</sub>}, beginning at time ''t'' = 0 with a cycle of seasonal change of length ''L''.
| |
| | |
| The method calculates a trend line for the data as well as seasonal indices that weight the values in the trend line based on where that time point falls in the cycle of length ''L''.
| |
| | |
| {''s''<sub>''t''</sub>} represents the smoothed value of the constant part for time ''t''. {''b''<sub>''t''</sub>} represents the sequence of best estimates of the linear trend that are superimposed on the seasonal changes. {''c''<sub>''t''</sub>} is the sequence of seasonal correction factors. ''c''<sub>''t''</sub> is the expected proportion of the predicted trend at any time ''t mod L'' in the cycle that the observations take on. To initialize the seasonal indices ''c''<sub>''t-L''</sub> there must be at least one complete cycle in the data.
| |
| | |
| The output of the algorithm is again written as ''F''<sub>''t+m''</sub>, an estimate of the value of ''x'' at time ''t+m, m>0'' based on the raw data up to time ''t''. Triple exponential smoothing is given by the formulas<ref name=NIST />
| |
| | |
| :<math>
| |
| \begin{align}
| |
| s_0& = x_0\\
| |
| s_{t}& = \alpha \frac{x_{t}}{c_{t-L}} + (1-\alpha)(s_{t-1} + b_{t-1})\\
| |
| b_{t}& = \beta (s_t - s_{t-1}) + (1-\beta)b_{t-1}\\
| |
| c_{t}& = \gamma \frac{x_{t}}{s_{t}}+(1-\gamma)c_{t-L}\\
| |
| F_{t+m}& = (s_t + mb_t)c_{t-L+(m\mod L)},
| |
| \end{align}
| |
| </math>
| |
| | |
| where α is the ''data smoothing factor'', 0 < α < 1, β is the ''trend smoothing factor'', 0 < β < 1, and γ is the ''seasonal change smoothing factor'', 0 < γ < 1.
| |
| | |
| The general formula for the initial trend estimate ''b''<sub>''0''</sub> is:
| |
| :<math>
| |
| \begin{align}
| |
| b_0& = \frac{1}{L} \left(\frac{x_{L+1}-x_1}{L} + \frac{x_{L+2}-x_2}{L} + \ldots + \frac{x_{L+L}-x_L}{L}\right)
| |
| \end{align}
| |
| </math>
| |
| | |
| Setting the initial estimates for the seasonal indices ''c''<sub>''i''</sub> for ''i'' = 1,2,...,L is a bit more involved. If ''N'' is the number of complete cycles present in your data, then:
| |
| :<math>
| |
| \begin{align}
| |
| \\
| |
| c_i& = \frac{1}{N} \sum_{j=1}^{N} \frac{x_{L(j-1)+i}}{A_j} \quad \forall i& = 1,2,\ldots,L \\
| |
| \end{align}
| |
| </math>
| |
| where
| |
| :<math>
| |
| \begin{align}
| |
| A_j& = \frac{\sum_{i=1}^{L} x_{L(j-1)+i}}{L} \quad \forall j& = 1,2,\ldots,N
| |
| \end{align}
| |
| </math>
| |
| Note that ''A''<sub>''j''</sub> is the average value of ''x'' in the ''j''th cycle of your data.
| |
| | |
| ==See also==
| |
| *[[Moving average]]
| |
| *[[Autoregressive moving average model]] (ARMA)
| |
| *[[Autoregressive integrated moving average]] (ARIMA)
| |
| *[[Errors and residuals in statistics]]
| |
| | |
| ==Notes==
| |
| {{Reflist}}
| |
| | |
| ==External links==
| |
| *[http://www.duke.edu/~rnau/411avg.htm Notes for a statistics class (''Decision 411'') at Duke University]
| |
| * [http://demonstrations.wolfram.com/DataSmoothing/ Data Smoothing] by Jon McLoone, [[The Wolfram Demonstrations Project]].
| |
| * [http://forecasters.org/pdfs/foresight/free/Issue19_goodwin.pdf The Holt-Winters Approach to Exponential Smoothing: 50 Years Old and Going Strong] by Paul Goodwin (2010) [[Foresight: The International Journal of Applied Forecasting]]
| |
| {{Use dmy dates|date=September 2011}}
| |
| | |
| {{Statistics|analysis}}
| |
| | |
| [[Category:Time series analysis]]
| |
| [[Category:Data analysis]]
| |