|
|
| Line 1: |
Line 1: |
| {{Regression bar}}
| | Ugg Boots are created using traditional wool and sheep sales. When not cleaned properly, not so nice to see and [http://tinyurl.com/nqbkz6z ugg boots usa] boots need cleaning from time to time. Thomas Sabo OnlineshopThere are some cleaning techniques and methods that are practical and useful for making clean uggs and glitter. Combining these techniques will give you clean ugg boots and a longer life for your uggs.<br><br>When you think of UGG boots will get dirty due to wet weather, it is best to use alternatives when heavy rain shoes and where the snow is soft and moist. When wearing your [http://tinyurl.com/nqbkz6z ugg boots usa] boots, not running on dirt, dry quickly and is [http://Mondediplo.com/spip.php?page=recherche&recherche=difficult difficult] to stand during cleaning. You should keep away from wetlands, streams, ponds and slough. Another thing, do not wear your Ugg boots at places such as walking, can be scratched. [http://tinyurl.com/nqbkz6z ugg boots usa] boots are so soft you could leave a permanent scratch easily.<br><br>Where to buy your UGG boots, ask for any spray water repellent and stain resistant spray. Consider spraying your boots [http://tinyurl.com/nqbkz6z ugg boots usa] boots with these easy solutions, so they can at least keep you away from many stains and dirt. To do this, spray evenly boots and leave for a period of 24 hours of drying. Consistent application of these solutions can provide additional protective boots.<br><br>Washing is done by hand. Most manufacturers will suggest and advise uggs to wash by hand. It is recommended to use the hand cleaner or vinegar to 1 part water. Wetting of the outside of the pocket using a sponge or clean cloth, do not pour water directly on your boots, Ugg Cardy Boots, not wet, it is essential, because they might lose their shape. It is easy to apply to areas with dirt or stains.<br><br>Let your [http://tinyurl.com/nqbkz6z ugg boots usa] boots to dry away from direct heat. It is dry on fire or heat. You hang your boots upside down coat hanger using his boot to keep Sabo Bracelet shape.Thomas let dry for 24 hours. After drying, use a soft suede brush or comb, soft brush uggs its natural feel your uniform. spray to make your selling ugg boots and a clean look shine again. |
| In [[statistics]], '''errors-in-variables models''' or '''measurement errors models''' are [[regression model]]s that account for [[measurement errors]] in the [[independent variables]]. In contrast, standard regression models assume that those regressors have been measured exactly, or observed without error; as such, those models account only for errors in the [[dependent variables]], or responses.
| |
| | |
| In the case when some regressors have been measured with errors, estimation based on the standard assumption leads to [[Consistent estimator|inconsistent]] estimates, meaning that the parameter estimates do not tend to the true values even in very large samples. For [[simple linear regression]] the effect is an underestimate of the coefficient, known as the ''[[attenuation bias]]''. In non-linear models the direction of the bias is likely to be more complicated.<ref>{{harvnb|Griliches|Ringstad|1970}}, {{harvnb|Chesher|1991}}</ref>
| |
| | |
| == Motivational example ==
| |
| Consider a simple linear regression model of the form
| |
| : <math> | |
| y_t = \alpha + \beta x_t^* + \varepsilon_t\,, \quad t=1,\ldots,T,
| |
| </math>
| |
| where ''x*'' denotes the ''true'' but unobserved value of the regressor. Instead we observe this value with an error:
| |
| : <math>
| |
| x_t = x^*_t + \eta_t\,,
| |
| </math>
| |
| where the measurement error ''η<sub>t</sub>'' is assumed to be independent from the true value ''x*<sub style="position:relative;left:-.4em">t</sub>''.
| |
| | |
| If the ''y<sub>t</sub>''′s are simply regressed on the ''x<sub>t</sub>''′s (see [[simple linear regression]]), then the estimator for the slope coefficient is
| |
| : <math>
| |
| \hat\beta = \frac{\tfrac{1}{T}\sum_{t=1}^T(x_t-\bar{x})(y_t-\bar{y})}
| |
| {\tfrac{1}{T}\sum_{t=1}^T(x_t-\bar{x})^2}\,,
| |
| </math>
| |
| which converges as the sample size ''T'' increases without bound:
| |
| : <math>
| |
| \hat\beta\ \xrightarrow{p}\
| |
| \frac{\operatorname{Cov}[\,x_t,y_t\,]}{\operatorname{Var}[\,x_t\,]}
| |
| = \frac{\beta \sigma^2_{x^*}} {\sigma_{x^*}^2 + \sigma_\eta^2}
| |
| = \frac{\beta} {1 + \sigma_\eta^2/\sigma_{x^*}^2}\,.
| |
| </math>
| |
| Variances are non-negative, so that in the limit the estimate is smaller in magnitude than the true value of ''β'', an effect which statisticians call ''attenuation'' or [[regression dilution]].<ref>{{harvnb|Greene|2003|loc=Chapter 5.6.1}}</ref> Thus the "naїve" least squares estimator is [[consistent estimator|inconsistent]] in this setting. However, the estimator is a [[consistent estimator]] of the parameter required for a best linear predictor of ''y'' given ''x'': in some applications this may be what is required, rather than an estimate of the "true" regression coefficient, although that would assume that the variance of the errors in observing ''x*'' remains fixed. This follows directly from the result quoted immediately above, and the fact that the regression coefficient relating the ''y<sub>t</sub>''′s to the actually observed ''x<sub>t</sub>''′s, in a simple linear regression, is given by
| |
| : <math>
| |
| \beta_x = \frac{\operatorname{Cov}[\,x_t,y_t\,]}{\operatorname{Var}[\,x_t\,]} .
| |
| </math>
| |
| It is this coefficient, rather than ''β'', that would be required for constructing a predictor of ''y'' based on an observed ''x'' which is subject to noise.
| |
| | |
| It can be argued that almost all existing data sets contain errors of different nature and magnitude, so that attenuation bias is extremely frequent (although in multivariate regression the direction of bias is ambiguous.<ref>{{harvnb|Wansbeek and Meijer|2000}}</ref> [[Jerry Hausman]] sees this as an ''iron law of econometrics'': "The magnitude of the estimate is usually smaller than expected."<ref>{{harvnb|Hausman|2001|p=58}}</ref>
| |
| | |
| == Specification ==
| |
| Usually measurement error models are described using the [[latent variable model|latent variables]] approach. If ''y'' is the response variable and ''x'' are observed values of the regressors, then we assume there exist some [[latent variable]]s ''y*'' and ''x*'' which follow the model's "true" functional relationship ''g'', and such that the observed quantities are their noisy observations:
| |
| | |
| : <math>\begin{cases}
| |
| x = x^* + \eta, \\
| |
| y = y^* + \varepsilon, \\
| |
| y^* = g(x^*\!,w\,|\,\theta),
| |
| \end{cases}</math>
| |
| | |
| where ''θ'' is the model's parameter and ''w'' are those regressors which are assumed to be error-free (for example when linear regression contains an intercept, the regressor which corresponds to the constant certainly has no "measurement errors"). Depending on the specification these error-free regressors may or may not be treated separately; in the latter case it is simply assumed that corresponding entries in the variance matrix of ''η'''s are zero.
| |
| | |
| The variables ''y'', ''x'', ''w'' are all ''observed'', meaning that the statistician possesses a [[data set]] of ''n'' [[statistical unit]]s {{nowrap|{''y<sub>i</sub>, x<sub>i</sub>, w<sub>i</sub>''}<sub>''i'' {{=}} 1, ..., ''n''</sub>}} which follow the [[data collection|data generating process]] described above; the latent variables ''x*'', ''y*'', ''ε'', and ''η'' are not observed however.
| |
| | |
| This specification does not encompass all the existing EiV models. For example in some of them function ''g'' may be non-parametric or semi-parametric. Other approaches model the relationship between ''y*'' and ''x*'' as distributional instead of functional, that is they assume that ''y*'' conditionally on ''x*'' follows a certain (usually parametric) distribution.
| |
| | |
| === Terminology and assumptions ===
| |
| * The observed variable ''x'' may be called the ''manifest'', ''indicator'', or ''proxy'' variable.
| |
| * The unobserved variable ''x*'' may be called the ''latent'' or ''true'' variable. It may be regarded either as an unknown constant (in which case the model is called a ''functional model''), or as a random variable (correspondingly a ''structural model'').<ref>{{harvnb|Fuller|1987|p=2}}</ref>
| |
| * The relationship between the measurement error ''η'' and the latent variable ''x*'' can be modeled in different ways:
| |
| ** ''Classical errors'': <math>\scriptstyle\eta\,\perp\,x^*,</math> the errors are independent from the latent variable. This is the most common assumption, it implies that the errors are introduced by the measuring device and their magnitude does not depend on the value being measured.
| |
| ** ''Mean-independence'': <math>\scriptstyle\operatorname{E}[\eta|x^*]\,=\,0,</math> the errors are mean-zero for every value of the latent regressor. This is a less restrictive assumption than the classical one, as it allows for the presence of [[heteroscedasticity]] or other effects in the measurement errors.
| |
| ** [[Berkson error model|''Berkson's errors'']]: <math>\scriptstyle\eta\,\perp\,x,</math> the errors are independent from the ''observed'' regressor ''x''. This assumption has very limited applicability. One example is round-off errors: for example if a person's <span style="font-variant:small-caps">age*</span> is a continuous random variable, whereas the observed <span style="font-variant:small-caps">age</span> is truncated to the next smallest integer, then the truncation error is approximately independent from the observed <span style="font-variant:small-caps">age</span>. Another possibility is with the fixed design experiment: for example if a scientist decides to make a measurement at a certain predetermined moment of time ''x'', say at ''x'' = 10 s, then the real measurement may occur at some other value of ''x*'' (for example due to her finite reaction time) and such measurement error will be generally independent from the "observed" value of the regressor.
| |
| ** ''Misclassification errors'': special case used for the [[dummy variable (statistics)|dummy regressors]]. If ''x*'' is an indicator of a certain event or condition (such as person is male/female, some medical treatment given/not, etc.), then the measurement error in such regressor will correspond to the incorrect classification similar to [[type I and type II errors]] in statistical testing. In this case the error ''η'' may take only 3 possible values, and its distribution conditional on ''x*'' is modeled with two parameters: ''α'' = Pr[''η''=−1 | ''x*''=1], and ''β'' = Pr[''η''=1 | ''x*''=0]. The necessary condition for identification is that ''α+β''<1, that is misclassification should not happen "too often". (This idea can be generalized to discrete variables with more than two possible values.)
| |
| | |
| == Linear model == | |
| Linear errors-in-variables models were studied first, probably because [[linear model]]s were so widely used and they are easier than non-linear ones. Unlike standard [[ordinary least squares|least squares]] regression (OLS), extending errors in variables regression (EiV) from the simple to the multivariate case is not straightforward.
| |
| | |
| ===Simple linear model===
| |
| The simple linear errors-in-variables model was already presented in the "motivation" section:
| |
| : <math>\begin{cases}
| |
| y_t = \alpha + \beta x_t^* + \varepsilon_t, \\
| |
| x_t = x_t^* + \eta_t,
| |
| \end{cases}</math>
| |
| where all variables are scalar. Here ''α'' and ''β'' are the parameters of interest, whereas ''σ<sub>ε</sub>'' and ''σ<sub>η</sub>'' — standard deviations of the error terms — are the [[nuisance parameter]]s. The "true" regressor ''x*'' is treated as a random variable (''structural'' model), independent from the measurement error ''η'' (''classic'' assumption).
| |
| | |
| This model is [[identifiable]] in two cases: (1) either the latent regressor ''x*'' is ''not'' [[Normal distribution|normally distributed]], (2) or ''x*'' has normal distribution, but neither ''ε<sub>t</sub>'' nor ''η<sub>t</sub>'' are divisible by a normal distribution.<ref>{{harvnb|Reiersøl|1950|p=383}}. A somewhat more restrictive result was established earlier by R. C. Geary in "Inherent relations between random variables", ''Proceedings of Royal Irish Academy'', vol.47 (1950). He showed that under the additional assumption that (''ε, η'') are jointly normal, the model is not identified if and only if ''x*''s are normal.</ref> That is, the parameters ''α'', ''β'' can be consistently estimated from the data set <math>\scriptstyle(x_t,\,y_t)_{t=1}^T</math> without any additional information, provided the latent regressor is not Gaussian.
| |
| | |
| Before this identifiability result was established, statisticians attempted to apply the [[maximum likelihood]] technique by assuming that all variables are normal, and then concluded that the model is not identified. The suggested remedy was to ''assume'' that some of the parameters of the model are known or can be estimated from the outside source. Such estimation methods include:<ref>{{harvnb|Fuller|1987|loc=ch. 1}}</ref>
| |
| | |
| * [[Deming regression]] — assumes that the ratio ''δ'' = ''σ²<sub style="position:relative;left:-.4em">ε</sub>''/''σ²<sub style="position:relative;left:-.4em">η</sub>'' is known. This could be appropriate for example when errors in ''y'' and ''x'' are both caused by measurements, and the accuracy of measuring devices or procedures are known. The case when ''δ'' = 1 is also known as the [[orthogonal regression]].
| |
| * Regression with known [[Reliability (statistics)|reliability ratio]] ''λ'' = ''σ²''<sub style="position:relative;left:-.6em">∗</sub>/ ( ''σ²<sub style="position:relative;left:-.4em">η</sub>'' + ''σ²''<sub style="position:relative;left:-.6em">∗</sub>), where ''σ²''<sub style="position:relative;left:-.6em">∗</sub> is the variance of the latent regressor. Such approach may be applicable for example when repeating measurements of the same unit are available, or when the reliability ratio has been known from the independent study. In this case the consistent estimate of slope is equal to the least-squares estimate divided by ''λ''.
| |
| * Regression with known ''σ²<sub style="position:relative;left:-.4em">η</sub>'' may occur when the source of the errors in ''x'''s is known and their variance can be calculated. This could include rounding errors, or errors introduced by the measuring device. When ''σ²<sub style="position:relative;left:-.4em">η</sub>'' is known we can compute the reliability ratio as ''λ'' = ( ''σ²<sub style="position:relative;left:-.4em">x</sub>'' − ''σ²<sub style="position:relative;left:-.4em">η</sub>'') / ''σ²<sub style="position:relative;left:-.4em">x</sub>'' and reduce the problem to the previous case.
| |
| | |
| Newer estimation methods that do not assume knowledge of some of the parameters of the model, include:
| |
| {{unordered list
| |
| |1= Method of moments <!-- A link to [[Method of moments (statistics)]] seems inappropriate since that article does not explain how to estimate EiV models--> — the [[Generalized method of moments|GMM]] estimator based on the third- (or higher-) order joint [[cumulant]]s of observable variables. The slope coefficient can be estimated from <ref>{{harvnb|Pal|1980}}, §6</ref>
| |
| : <math> | |
| \hat\beta = \frac{\hat{K}(n_1,n_2+1)}{\hat{K}(n_1+1,n_2)}, \quad n_1,n_2>0,
| |
| </math>
| |
| where (''n''<sub>1</sub>,''n''<sub>2</sub>) are such that ''K''(''n''<sub>1</sub>+1,''n''<sub>2</sub>) — the joint [[cumulant]] of (''x'',''y'') — is not zero. In the case when the third central moment of the latent regressor ''x*'' is non-zero, the formula reduces to
| |
| : <math>
| |
| \hat\beta = \frac{\tfrac{1}{T}\sum_{t=1}^T (x_t-\bar x)(y_t-\bar y)^2}
| |
| {\tfrac{1}{T}\sum_{t=1}^T (x_t-\bar x)^2(y_t-\bar y)}\ .
| |
| </math>
| |
| | |
| |2= [[Instrumental variables]] — a regression which requires that certain additional data variables ''z'', called ''instruments'', were available. These variables should be uncorrelated with the errors in the equation for the dependent variable, and they should also be correlated (''relevant'') with the true regressors ''x*''. If such variables can be found then the estimator takes form
| |
| : <math>\hat\beta = \frac{\tfrac{1}{T}\sum_{t=1}^T (z_t-\bar z)(y_t-\bar y)}
| |
| {\tfrac{1}{T}\sum_{t=1}^T (z_t-\bar z)(x_t-\bar x)}\ .</math>
| |
| }}
| |
| | |
| ===Multivariate linear model===
| |
| Multivariate model looks exactly like the linear model, only this time ''β'', ''η''<sub>''t''</sub>, ''x''<sub>''t''</sub> and ''x*''<sub style="position:relative;left:-.4em">''t''</sub> are ''k×''1 vectors.
| |
| : <math>\begin{cases}
| |
| y_t = \alpha + \beta'x_t^* + \varepsilon_t, \\
| |
| x_t = x_t^* + \eta_t.
| |
| \end{cases}</math>
| |
| | |
| The general [[identifiability condition]] for this model remains an open question. It is known however that in the case when (''ε'',''η'') are independent and jointly normal, the parameter ''β'' is identified if and only if it is impossible to find a non-singular ''k×k'' block matrix [''a A''] (where ''a'' is a ''k×''1 vector) such that ''a′x*'' is distributed normally and independently from ''A′x*''.<ref>{{harvnb|Bekker|1986}}. An earlier proof by Y. Willassen in "Extension of some results by Reiersøl to multivariate models", ''Scand. J. Statistics'', '''6'''(2) (1979) contained errors.</ref>
| |
| | |
| Some of the estimation methods for multivariate linear models are:
| |
| {{unordered list
| |
| |1= [[Total least squares]] is an extension of [[Deming regression]] to the multivariate setting. When all the ''k''+1 components of the vector (''ε'',''η'') have equal variances and are independent, this is equivalent to running the orthogonal regression of ''y'' on the vector ''x'' — that is, the regression which minimizes the sum of squared distances between points (''y<sub>t</sub>'',''x<sub>t</sub>'') and the ''k''-dimensional hyperplane of "best fit".
| |
| | |
| |2= The [[Generalized method of moments|method of moments]] estimator <ref>{{harvnb|Dagenais|Dagenais|1997}}. In the earlier paper {{harv|Pal|1980}} considered a simpler case when all components in vector (''ε'', ''η'') are independent and symmetrically distributed.</ref> can be constructed based on the moment conditions E[''z<sub>t</sub>''·(''y<sub>t</sub>'' − ''α'' − ''β'x<sub>t</sub>'')] = 0, where the (5''k''+3)-dimensional vector of instruments ''z<sub>t</sub>'' is defined as
| |
| : <math>\begin{align}
| |
| & z_t = \left( 1\ z_{t1}'\ z_{t2}'\ z_{t3}'\ z_{t4}'\ z_{t5}'\ z_{t6}'\ z_{t7}' \right)', \quad \text{where} \\
| |
| & z_{t1} = x_t \ast x_t \\
| |
| & z_{t2} = x_t y_t \\
| |
| & z_{t3} = y_t^2 \\
| |
| & z_{t4} = x_t \ast x_t \ast x_t - 3\big(\operatorname{E}[x_tx_t']\ast I_k\big)x_t \\
| |
| & z_{t5} = x_t \ast x_t y_t - 2\big(\operatorname{E}[y_tx_t']\ast I_k\big)x_t - y_t\big(\operatorname{E}[x_tx_t']\ast I_k\big)\iota_k \\
| |
| & z_{t6} = x_t y_t^2 - \operatorname{E}[y_t^2]x_t - 2y_t\operatorname{E}[x_ty_t] \\
| |
| & z_{t7} = y_t^3 - 3y_t\operatorname{E}[y_t^2]
| |
| \end{align}</math>
| |
| where * designates the [[Hadamard matrix product|Hadamard product]] of matrices, and variables ''x<sub>t</sub>'', ''y<sub>t</sub>'' have been preliminarily de-meaned. The authors of the method suggest to use Fuller's modified IV estimator.<ref>{{harvnb|Fuller|1987|page=184}}</ref><br>
| |
| This method can be extended to use moments higher than the third order, if necessary, and to accommodate variables measured without error.<ref>{{harvnb|Erickson|Whited|2002}}</ref>
| |
| | |
| |3= The [[instrumental variables]] approach requires to find additional data variables ''z<sub>t</sub>'' which would serve as ''instruments'' for the mismeasured regressors ''x<sub>t</sub>''. This method is the simplest from the implementation point of view, however its disadvantage is that it requires to collect additional data, which may be costly or even impossible. When the instruments can be found, the estimator takes standard form
| |
| : <math>
| |
| \hat\beta = \big(X'Z(Z'Z)^{-1}Z'X\big)^{-1}X'Z(Z'Z)^{-1}Z'y.
| |
| </math>
| |
| | |
| }}
| |
| | |
| == Non-linear models ==
| |
| A generic non-linear measurement error model takes form
| |
| : <math>\begin{cases}
| |
| y_t = g(x^*_t) + \varepsilon_t, \\
| |
| x_t = x^*_t + \eta_t.
| |
| \end{cases}</math>
| |
| Here function ''g'' can be either parametric or non-parametric. When function ''g'' is parametric it will be written as ''g(x*, β)''.
| |
| | |
| For a general vector-valued regressor ''x*'' the conditions for model [[identifiability]] are not known. However in the case of scalar ''x*'' the model is identified unless the function ''g'' is of the "log-exponential" form <ref>{{harvnb|Schennach|Hu|Lewbel|2007}}</ref>
| |
| : <math>g(x^*) = a + b \ln\big(e^{cx^*} + d\big)</math>
| |
| and the latent regressor ''x*'' has density
| |
| : <math>
| |
| f_{x^*}(x) = \begin{cases}
| |
| A e^{-Be^{Cx}+CDx}(e^{Cx}+E)^{-F}, & \text{if}\ d>0 \\
| |
| A e^{-Bx^2 + Cx} & \text{if}\ d=0
| |
| \end{cases}
| |
| </math>
| |
| where constants ''A,B,C,D,E,F'' may depend on ''a,b,c,d''.
| |
| | |
| Despite this optimistic result, as of now no methods exist for estimating non-linear errors-in-variables models without any extraneous information. However there are several techniques which make use of some additional data: either the instrumental variables, or repeated observations.
| |
| | |
| ===Instrumental variables methods===
| |
| {{unordered list
| |
| |1=
| |
| '''Newey's simulated moments method''' <ref>{{harvnb|Newey|2001}}</ref> for parametric models — requires that there is an additional set of observed ''predictor variabels'' ''z<sub>t</sub>'', such that the true regressor can be expressed as
| |
| : <math>x^*_t = \pi_0'z_t + \sigma_0 \zeta_t,</math>
| |
| where ''π''<sub>0</sub> and ''σ''<sub>0</sub> are (unknown) constant matrices, and ''ζ<sub>t</sub>'' ⊥ ''z<sub>t</sub>''. The coefficient ''π''<sub>0</sub> can be estimated using standard [[Ordinary least squares|least squares]] regression of ''x'' on ''z''. The distribution of ''ζ<sub>t</sub>'' is unknown, however we can model it as belonging to a flexible parametric family — the [[Edgeworth series]]:
| |
| : <math>f_\zeta(v;\,\gamma) = \phi(v)\,\textstyle\sum_{j=1}^J \!\gamma_j v^j</math>
| |
| where ''ϕ'' is the [[standard normal]] distribution.
| |
| | |
| Simulated moments can be computed using the [[importance sampling]] algorithm: first we generate several random variables {''v<sub>ts</sub>'' ~ ''ϕ'', ''s'' = 1,…,''S'', ''t'' = 1,…,''T''} from the standard normal distribution, then we compute the moments at ''t''-th observation as
| |
| : <math>m_t(\theta) = A(z_t) \frac{1}{S}\sum_{s=1}^S H(x_t,y_t,z_t,v_{ts};\theta) \sum_{j=1}^J\!\gamma_j v_{ts}^j,</math>
| |
| where ''θ'' = (''β'', ''σ'', ''γ''), ''A'' is just some function of the instrumental variables ''z'', and ''H'' is a two-component vector of moments
| |
| : <math>\begin{align}
| |
| & H_1(x_t,y_t,z_t,v_{ts};\theta) = y_t - g(\hat\pi'z_t + \sigma v_{ts}, \beta), \\
| |
| & H_2(x_t,y_t,z_t,v_{ts};\theta) = z_t y_t - (\hat\pi'z_t + \sigma v_{ts}) g(\hat\pi'z_t + \sigma v_{ts}, \beta)
| |
| \end{align}</math>
| |
| With moment functions ''m<sub>t</sub>'' one can apply standard [[Generalized method of moments|GMM]] technique to estimate the unknown parameter ''θ''.
| |
| }}
| |
| | |
| ===Repeated observations===
| |
| In this approach two (or maybe more) repeated observations of the regressor ''x*'' are available. Both observations contain their own measurement errors, however those errors are required to be independent:
| |
| : <math>\begin{cases}
| |
| x_{1t} = x^*_t + \eta_{1t}, \\
| |
| x_{2t} = x^*_t + \eta_{2t},
| |
| \end{cases}</math>
| |
| where ''x*'' ⊥ ''η''<sub>1</sub> ⊥ ''η''<sub>2</sub>. Variables ''η''<sub>1</sub>, ''η''<sub>2</sub> need not be identically distributed (although if they are efficiency of the estimator can be slightly improved). With only these two observations it is possible to consistently estimate the density function of ''x*'' using Kotlarski's [[deconvolution]] technique.<ref>{{harvnb|Li|Vuong|1998}}</ref>
| |
| {{unordered list
| |
| |1= '''Li's conditional density method'''<ref>{{harvnb|Li|2002}}</ref> for parametric models. The regression equation can be written in terms of the observable variables as
| |
| : <math>
| |
| \operatorname{E}[\,y_t|x_t\,] = \int g(x^*_t,\beta) f_{x^*|x}(x^*_t|x_t)dx^*_t ,
| |
| </math>
| |
| where it would be possible to compute the integral if we knew the conditional density function ''ƒ<sub>x*{{!}}x</sub>''. If this function could be known or estimated, then the problem turns into standard non-linear regression, which can be estimated for example using the [[NLLS]] method.<br>
| |
| Assuming for simplicity that ''η''<sub>1</sub>, ''η''<sub>2</sub> are identically distributed, this conditional density can be computed as
| |
| : <math>
| |
| \hat f_{x^*|x}(x^*|x) = \frac{\hat f_{x^*}(x^*)}{\hat f_{x}(x)} \prod_{j=1}^k \hat f_{\eta_{j}}\big( x_{j} - x^*_{j} \big),
| |
| </math>
| |
| where with slight abuse of notation ''x<sub>j</sub>'' denotes the ''j''-th component of a vector.<br>
| |
| All densities in this formula can be estimated using inversion of the empirical [[Characteristic function (probability theory)|characteristic functions]]. In particular,
| |
| : <math>\begin{align} | |
| & \hat \varphi_{\eta_j}(v) = \frac{\hat\varphi_{x_j}(v,0)}{\hat\varphi_{x^*_j}(v)}, \quad \text{where }
| |
| \hat\varphi_{x_j}(v_1,v_2) = \frac{1}{T}\sum_{t=1}^T e^{iv_1x_{1tj}+iv_2x_{2tj}}, \ \
| |
| \hat\varphi_{x^*_j}(v) = \exp \int_0^v \frac{\partial\hat\varphi_{x_j}(0,v_2)/\partial v_1}{\hat\varphi_{x_j}(0,v_2)}dv_2, \\
| |
| & \hat \varphi_x(u) = \frac{1}{2T}\sum_{t=1}^T \Big( e^{iu'x_{1t}} + e^{iu'x_{2t}} \Big), \quad
| |
| \hat \varphi_{x^*}(u) = \frac{\hat\varphi_x(u)}{\prod_{j=1}^k \hat\varphi_{\eta_j}(u_j)}.
| |
| \end{align}</math>
| |
| | |
| In order to invert these characteristic function one has to apply the inverse Fourier transform, with a trimming parameter ''C'' needed to ensure the numerical stability. For example:
| |
| : <math>\hat f_x(x) = \frac{1}{(2\pi)^k} \int_{-C}^{C}\cdots\int_{-C}^C e^{-iu'x} \hat\varphi_x(u) du.</math>
| |
| | |
| |2= '''Schennach's estimator'''<ref>{{harvnb|Schennach|2004a}}</ref> for a parametric linear-in-parameters nonlinear-in-variables model. This is a model of the form
| |
| : <math>\begin{cases}
| |
| y_t = \textstyle \sum_{j=1}^k \beta_j g_j(x^*_t) + \sum_{j=1}^\ell \beta_{k+j}w_{jt} + \varepsilon_t, \\
| |
| x_{1t} = x^*_t + \eta_{1t}, \\
| |
| x_{2t} = x^*_t + \eta_{2t},
| |
| \end{cases}</math>
| |
| where ''w<sub>t</sub>'' represents variables measured without errors. The regressor ''x*'' here is scalar (the method can be extended to the case of vector ''x*'' as well).<br>
| |
| If not for the measurement errors, this would have been a standard [[linear model]] with the estimator
| |
| : <math>
| |
| \hat{\beta} = \big(\hat{\operatorname{E}}[\,\xi_t\xi_t'\,]\big)^{-1} \hat{\operatorname{E}}[\,\xi_t y_t\,],
| |
| </math>
| |
| where
| |
| | |
| :<math> \xi_t'= (g_1(x^*_t), \cdots ,g_k(x^*_t), w_{1,t}, \cdots , w_{l,t}).</math>
| |
| | |
| It turns out that all the expected values in this formula are estimable using the same deconvolution trick. In particular, for a generic observable ''w<sub>t</sub>'' (which could be 1, ''w''<sub>1''t''</sub>, …, ''w''<sub>''ℓ t''</sub>, or ''y<sub>t</sub>'') and some function ''h'' (which could represent any ''g<sub>j</sub>'' or ''g<sub>i</sub>g<sub>j</sub>'') we have
| |
| : <math>
| |
| \operatorname{E}[\,w_th(x^*_t)\,] = \frac{1}{2\pi} \int_{-\infty}^\infty \varphi_h(-u)\psi_w(u)du,
| |
| </math>
| |
| where ''φ<sub>h</sub>'' is the [[Fourier transform]] of ''h''(''x*''), but using the same convention as for the [[characteristic function (probability theory)|characteristic functions]],
| |
| : <math> \varphi_h(u)=\int e^{iux}h(x)dx</math>,
| |
| | |
| and
| |
| : <math>
| |
| \psi_w(u) = \operatorname{E}[\,w_te^{iux^*}\,]
| |
| = \frac{\operatorname{E}[w_te^{iux_{1t}}]}{\operatorname{E}[e^{iux_{1t}}]}
| |
| \exp \int_0^u i\frac{\operatorname{E}[x_{2t}e^{ivx_{1t}}]}{\operatorname{E}[e^{ivx_{1t}}]}dv
| |
| </math>
| |
| The resulting estimator <math>\scriptstyle\hat\beta</math> is consistent and asymptotically normal.
| |
| | |
| |3= '''Schennach's estimator''' <ref>{{harvnb|Schennach|2004b}}</ref> for a nonparametric model. The standard [[Nadaraya–Watson estimator]] for a nonparametric model takes form
| |
| : <math>
| |
| \hat{g}(x) = \frac{\hat{\operatorname{E}}[\,y_tK_h(x^*_t - x)\,]}{\hat{\operatorname{E}}[\,K_h(x^*_t - x)\,]},
| |
| </math>
| |
| for a suitable choice of the kernel ''K'' and the bandwidth ''h''. Both expectations here can be estimated using the same technique as in the previous method.
| |
| }}
| |
| | |
| == Notes ==
| |
| {{Reflist|30em}}
| |
| | |
| == References ==
| |
| | |
| {{Refbegin}}
| |
| * {{Citation
| |
| | last = Bekker
| |
| | first = Paul A.
| |
| | year = 1986
| |
| | title = Comment on identification in the linear errors in variables model
| |
| | journal = Econometrica
| |
| | volume = 54
| |
| | issue = 1
| |
| | pages = 215–217
| |
| | jstor = 1914166
| |
| | doi = 10.2307/1914166
| |
| }}
| |
| * {{Citation
| |
| | last = Chesher
| |
| | first = Andrew
| |
| | year = 1991
| |
| | title = The effect of measurement error
| |
| | journal = Biometrika
| |
| | volume = 78
| |
| | issue = 3
| |
| | pages = 451–462
| |
| | jstor = 2337015
| |
| | doi = 10.1093/biomet/78.3.451
| |
| }}
| |
| * {{Citation
| |
| | last = Dagenais
| |
| | first = Marcel G.
| |
| | coauthors = Dagenais, Denyse L.
| |
| | year = 1997
| |
| | title = Higher moment estimators for linear regression models with errors in the variables
| |
| | journal = Journal of Econometrics
| |
| | volume = 76
| |
| | pages = 193–221
| |
| | doi = 10.1016/0304-4076(95)01789-5
| |
| | ref = CITEREFDagenaisDagenais1997
| |
| }}
| |
| * {{Citation
| |
| | last = Erickson
| |
| | first = Timothy
| |
| | coauthors = Whited, Toni M.
| |
| | year = 2002
| |
| | title = Two-step GMM estimation of the errors-in-variables model using high-order moments
| |
| | journal = Econometric Theory
| |
| | volume = 18
| |
| | issue = 3
| |
| | pages = 776–799
| |
| | jstor = 3533649
| |
| | ref = CITEREFEricksonWhited2002
| |
| }}
| |
| * {{Citation
| |
| | last = Fuller
| |
| | first = Wayne A.
| |
| | year = 1987
| |
| | title = Measurement error models
| |
| | publisher = John Wiley & Sons, Inc
| |
| | isbn = 0-471-86187-1
| |
| }}
| |
| * {{Citation
| |
| | last = Greene
| |
| | first = William H.
| |
| | year = 2003
| |
| | title = Econometric analysis
| |
| | edition = 5th
| |
| | publisher = Prentice Hall
| |
| | location = New Jersey
| |
| | isbn = 0-13-066189-9
| |
| | lccn = HB139.G74 2002
| |
| }}
| |
| * {{Citation
| |
| | last = Griliches
| |
| | first = Zvi
| |
| | coauthors = Ringstad, Vidar
| |
| | year = 1970
| |
| | title = Errors-in-the-variables bias in nonlinear contexts
| |
| | journal = Econometrica
| |
| | volume = 38
| |
| | issue = 2
| |
| | pages = 368–370
| |
| | ref = CITEREFGrilichesRingstad1970
| |
| | jstor = 1913020
| |
| | doi = 10.2307/1913020
| |
| }}
| |
| * {{Citation
| |
| | doi = 10.1257/jep.15.4.57
| |
| | last = Hausman
| |
| | first = Jerry A.
| |
| | year = 2001
| |
| | title = Mismeasured variables in econometric analysis: problems from the right and problems from the left
| |
| | journal = The Journal of Economic Perspectives
| |
| | volume = 15
| |
| | issue = 4
| |
| | pages = 57–67
| |
| | jstor = 2696516
| |
| }}
| |
| *Jung, Kang-Mo (2007) "Least Trimmed Squares Estimator in the Errors-in-Variables Model", ''Journal of Applied Statistics'', 34 (3), 331–338. {{DOI| 10.1080/02664760601004973}}
| |
| * {{Citation
| |
| | last = Kummell
| |
| | first = C. H.
| |
| | year = 1879
| |
| | title = Reduction of observation equations which contain more than one observed quantity
| |
| | journal = The Analyst
| |
| | volume = 6
| |
| | issue = 4
| |
| | pages = 97–105
| |
| | jstor = 2635646
| |
| | doi = 10.2307/2635646
| |
| }}
| |
| * {{Citation
| |
| | last = Li
| |
| | first = Tong
| |
| | year = 2002
| |
| | title = Robust and consistent estimation of nonlinear errors-in-variables models
| |
| | journal = Journal of Econometrics
| |
| | volume = 110
| |
| | issue = 1
| |
| | pages = 1–26
| |
| | doi = 10.1016/S0304-4076(02)00120-3
| |
| }}
| |
| * {{Citation
| |
| | last = Li
| |
| | first = Tong
| |
| | coauthors = Vuong, Quang
| |
| | year = 1998
| |
| | title = Nonparametric estimation of the measurement error model using multiple indicators
| |
| | journal = Journal of Multivariate Analysis
| |
| | volume = 65
| |
| | issue = 2
| |
| | pages = 139–165
| |
| | doi = 10.1006/jmva.1998.1741
| |
| | ref = CITEREFLiVuong1998
| |
| }}
| |
| * {{Citation
| |
| | last = Newey
| |
| | first = Whitney K.
| |
| | year = 2001
| |
| | title = Flexible simulated moment estimation of nonlinear errors-in-variables model
| |
| | journal = The Review of Economics and Statistics
| |
| | volume = 83
| |
| | issue = 4
| |
| | pages = 616–627
| |
| | jstor = 3211757
| |
| | doi = 10.1162/003465301753237704
| |
| }}
| |
| * {{Citation
| |
| | last = Pal
| |
| | first = Manoranjan
| |
| | year = 1980
| |
| | title = Consistent moment estimators of regression coefficients in the presence of errors in variables
| |
| | journal = Journal of Econometrics
| |
| | volume = 14
| |
| | issue = 3
| |
| | pages = 349–364
| |
| | doi = 10.1016/0304-4076(80)90032-9
| |
| }}
| |
| * {{Citation
| |
| | last = Reiersøl
| |
| | first = Olav
| |
| | year = 1950
| |
| | title = Identifiability of a linear relation between variables which are subject to error
| |
| | journal = Econometrica
| |
| | volume = 18
| |
| | issue = 4
| |
| | pages = 375–389
| |
| | jstor = 1907835
| |
| | doi = 10.2307/1907835
| |
| }}
| |
| * {{Citation
| |
| | last = Schennach
| |
| | first = Susanne M.
| |
| | year = 2004
| |
| | title = Estimation of nonlinear models with measurement error
| |
| | journal = Econometrica
| |
| | volume = 72
| |
| | issue = 1
| |
| | pages = 33–75
| |
| | jstor = 3598849
| |
| | ref = CITEREFSchennach2004a
| |
| | doi = 10.1111/j.1468-0262.2004.00477.x
| |
| | postscript = .
| |
| }}
| |
| * {{Citation
| |
| | last = Schennach
| |
| | first = Susanne M.
| |
| | year = 2004
| |
| | title = Nonparametric regression in the presence of measurement error
| |
| | journal = Econometric Theory
| |
| | volume = 20
| |
| | issue = 6
| |
| | pages = 1046–1093
| |
| | doi = 10.1017/S0266466604206028
| |
| | ref = CITEREFSchennach2004b
| |
| | postscript = .
| |
| }}
| |
| * {{Citation
| |
| | year = 2007
| |
| | title = Nonparametric identification of the classical errors-in-variables model without side information
| |
| | version = Working paper
| |
| | url = http://escholarship.bc.edu/cgi/viewcontent.cgi?article=1433&context=econ_papers
| |
| | ref = CITEREFSchennachHuLewbel2007
| |
| | author = Schennach S., Hu Y., Lewbel A.
| |
| | postscript = .
| |
| }}
| |
| * {{Citation
| |
| | year = 2000
| |
| | title = Measurement error and latent variables in econometrics
| |
| | version = Book
| |
| | url = http://www.blackwellreference.com/public/tocnode?id=g9781405106764_chunk_g978140510676413
| |
| | ref = CITEREFWansbeek_and_Meijer2000
| |
| | author = Wansbeek T., Meijer E.
| |
| | postscript = .
| |
| }}
| |
| * {{Citation
| |
| | last = Amiri-Simkooei
| |
| | first = et al.
| |
| | year = 2014
| |
| | title = Estimation of straight line parameters with fully correlated
| |
| | journal = Measurement
| |
| | volume = 48
| |
| | pages = 378–386
| |
| | doi = 10.1016/j.measurement.2013.11.005
| |
| | url = http://www.sciencedirect.com/science/article/pii/S0263224113005484
| |
| }}
| |
| {{Refend}}
| |
| | |
| ==Further reading==
| |
| | |
| * [http://www.cardiff.ac.uk/maths/resources/Gillard_Tech_Report.pdf An Historical Overview of Linear Regression with Errors in both Variables], J.W. Gillard 2006
| |
| * {{Citation
| |
| | last = Söderström
| |
| | first = Torsten
| |
| | year = 2007
| |
| | title = Errors-in-variables methods in system identification
| |
| | journal = Automatica
| |
| | volume = 43
| |
| | issue = 6
| |
| | pages = 939–958
| |
| | doi = 10.1016/j.automatica.2006.11.025
| |
| | ref = CITEREFSoderstrom2007
| |
| }}
| |
| * A. R. Amiri-Simkooei and S. Jazaeri ''Weighted total least squares formulated by standard least squares theory'',in Journal of Geodetic Science, 2 (2): 113-124, 2012 [http://engold.ui.ac.ir/~amiri/JGS_Amiri_Jazaeri_2012.pdf].
| |
| | |
| * {{cite book|title=Total Least Squares and Errors-in-Variables Modeling: Analysis, Algorithms and Applications|year=2002|publisher=Springer Netherlands|location=Dordrecht|isbn=978-90-481-5957-4|editor=Van Huffel, Sabine }}
| |
| | |
| | |
| | |
| | |
| {{DEFAULTSORT:Errors-In-Variables Models}}
| |
| [[Category:Regression analysis]]
| |
| [[Category:Statistical models]]
| |
| [[Category:Econometrics]]
| |