View source for Degree Lintner

'''Uncertainty quantification (UQ)''' is the science of quantitative characterization and reduction of [[Uncertainty|uncertainties]] in applications. It tries to determine how likely certain outcomes are if some aspects of the system are not exactly known. An example would be to predict the acceleration of a human body in a head-on crash with another car: even if we exactly knew the speed, small differences in the manufacturing of individual cars, how tightly every bolt has been tightened, etc., will lead to different results that can only be predicted in a statistical sense.

Many problems in the natural sciences and engineering are also rife with sources of uncertainty. [[Computer experiment]]s on [[computer model|computer simulation]]s are the most common approach to study problems in uncertainty quantification.<ref>Jerome Sacks, William J. Welch, Toby J. Mitchell and Henry P. Wynn, ''Design and Analysis of Computer Experiments'', Statistical Science, Vol. 4, No. 4 (Nov., 1989), pp. 409-423</ref><ref>Ronald L. Iman, Jon C. Helton, ''An Investigation of Uncertainty and Sensitivity Analysis Techniques for Computer Models'', Risk Analysis, Volume 8, Issue 1, pages 71–90, March 1988, DOI: 10.1111/j.1539-6924.1988.tb01155.x</ref><ref>W.E. Walker, P. Harremoës, J. Rotmans, J.P. van der Sluijs, M.B.A. van Asselt, P. Janssen and M.P. Krayer von Krauss, ''Defining Uncertainty: A Conceptual Basis for Uncertainty Management in Model-Based Decision Support'', Integrated Assessment, Volume 4, Issue 1, 2003, DOI: 10.1076/iaij.4.1.5.16466</ref>

==Sources of uncertainty==
Uncertainty can enter [[mathematical model]]s and experimental measurements in various contexts. One way to categorize the sources of uncertainty is to consider:<ref name="KennedyOHagan">Marc C. Kennedy, Anthony O'Hagan, ''Bayesian calibration of computer models'', Journal of the Royal Statistical Society, Series B Volume 63, Issue 3, pages 425–464, 2001</ref>

* '''Parameter uncertainty''', which comes from the model parameters that are inputs to the computer model (mathematical model) but whose exact values are unknown to experimentalists and cannot be controlled in physical experiments. Examples are the local [[free fall|free-fall]] acceleration in a falling object experiment, and various material properties in a finite element analysis for engineering.
* '''Structural uncertainty''', aka model inadequacy, model bias, or model discrepancy, which comes from the lack of knowledge of the underlying true physics. It depends on how accurately a mathematical model describes the true system for a real-life situation, considering the fact that models are almost always only approximations to reality. One example is when modeling the process of a falling object using the free-fall model; the model itself is inaccurate since there always exists air friction. In this case, even if there is no unknown parameter in the model, a discrepancy is still expected between the model and true physics.
* '''Algorithmic uncertainty''', aka numerical uncertainty, which comes from numerical errors and numerical approximations per implementation of the computer model. Most models are too complicated to solve exactly. For example the [[finite element method]] or [[finite difference|finite difference method]] may be used to approximate the solution of a [[partial differential equation]], which, however, introduces numerical errors. Other examples are numerical integration and infinite sum truncation that are necessary approximations in numerical implementation.
* '''Parametric variability''', which comes from the variability of input variables of the model. For example, the dimensions of a work piece in a process of manufacture may not be exactly as designed and instructed, which would cause variability in its performance.
* '''Experimental uncertainty''', aka observation error, which comes from the variability of experimental measurements. The experimental uncertainty is inevitable and can be noticed by repeating a measurement for many times using exactly the same settings for all inputs/variables.
* '''Interpolation uncertainty''', which comes a lack of available data collected from computer model simulations and/or experimental measurements. For other input settings that don't have simulation data or experimental measurements, one must interpolate or extrapolate in order to predict the corresponding responses.
<p id="aleatoric_epistemic">Another way of categorization is to classify uncertainty into two categories:<ref>Armen Der Kiureghiana, Ove Ditlevsen, ''Aleatory or epistemic? Does it matter?'', Structural Safety, Volume 31, Issue 2, March 2009, Pages 105–112</ref><ref>Hermann G. Matthies, ''Quantifying uncertainty: modern computational representation of probability and applications'', Extreme Man-Made and Natural Hazards in Dynamics of Structures, NATO Security through Science Series, 2007, 105-135, DOI: 10.1007/978-1-4020-5656-7_4</ref>

* '''Aleatoric uncertainty''', aka statistical uncertainty, which is representative of unknowns that differ each time we run the same experiment. For an example of simulating the take-off of an airplane, even if we could exactly control the wind speeds along the run way, if we let 10 planes of the same make start, their trajectories would still differ due to fabrication differences. Similarly, if all we knew is that the average wind speed is the same, letting the same plane start 10 times would still yield different trajectories because we do not know the exact wind speed at every point of the runway, only its average. Aleatoric uncertainties are therefore something an experimenter cannot do anything about: they exist, and they cannot be suppressed by more accurate measurements.
* '''Epistemic uncertainty''', aka systematic uncertainty, which is due to things we could in principle know but don't in practice. This may be because we have not measured a quantity sufficiently accurately, or because our model neglects certain effects, or because particular data are deliberately hidden. An example of a source of this uncertainty would be the [[Drag (physics)|drag]] of a feather in an experiment designed to measure the acceleration of gravity near the earth's surface. The commonly used gravitational acceleration of 9.8&nbsp;m/s^2 ignores the effects of air resistance, but the air resistance for the feather could be measured and incorporated into the experiment to reduce the systematic uncertainty in the calculation of the gravitational acceleration. The latter activities would then leave only aleatoric (statistical) uncertainties (wind velocity, feather orientation, etc) present in the experiment.

In real life applications, both kinds of uncertainties are often present. Uncertainty quantification intends to work toward reducing epistemic uncertainties to aleatoric uncertainties. The quantification for the aleatoric uncertainties is relatively straightforward to perform. Techniques such as [[Monte Carlo method]] are frequently used. A probability distribution can be represented by its [[Moment (mathematics)|moments]] (in the [[Normal distribution|Gaussian]] case, the [[mean]] and [[covariance]] suffice, although it should be noted that, in general, even knowledge of all moments to arbitrarily high order still does not specify the distribution function uniquely), or more recently, by techniques such as [[Karhunen–Loève theorem|Karhunen–Loève]] and [[polynomial chaos]] expansions. To evaluate epistemic uncertainties, the efforts are made to gain better knowledge of the system, process or mechanism. Methods such as [[fuzzy logic]] or evidence theory ([[Dempster–Shafer theory]] – a generalization of the Bayesian theory of subjective probability) are used.

==Two types of uncertainty quantification problems==
There are two major types of problems in uncertainty quantification: one is the '''forward''' propagation of uncertainty and the other is the '''inverse''' assessment of model uncertainty and parameter uncertainty. There has been a proliferation of research on the former problem and a majority of uncertainty analysis techniques were developed for it. On the other hand, the latter problem is drawing increasing attention in the engineering design community, since uncertainty quantification of a model and the subsequence predictions of the true system response(s) are of great interest in both robust design and engineering design making.

===Forward uncertainty propagation===
{{See also|Propagation of uncertainty}}
Uncertainty propagation is the quantification of uncertainties in system output(s) propagated from uncertain inputs. It focuses on the influence on the outputs from the ''parametric variability'' listed in the sources of uncertainty. The targets of uncertainty propagation analysis can be:
* To evaluate low-order moments of the outputs, i.e. [[mean]] and [[variance]].
* To evaluate the reliability of the outputs. This is especially useful in [[reliability engineering]] where outputs of a system are usually closely related to the performance of the system.
* To assess the complete probability distribution of the outputs. This is useful in the scenario of [[utility]] optimization where the complete distribution is used to calculate the utility.

===Inverse uncertainty quantification===
Inverse uncertainty quantification is that, given some experimental measurements of a system and some computer simulation results from its mathematical model, to estimate the discrepancy between the experiment and the mathematical model (which is called '''bias correction'''), and to estimate the values of unknown parameters in the model if there is any (which is called '''parameter calibration''' or simply '''calibration'''). Generally it is a much more difficult problem than uncertainty propagation; however it is of great importance since it is typically implemented in a model updating process. There are several scenarios in inverse uncertainty quantification:
[[File:Bias Correction.png|thumb|right|300px|The outcome of bias correction, including an updated model (prediction mean) and prediction confidence interval. Drawn and granted permission to use by Dr. Paul D. Arendt from [[Northwestern University]], IL, USA]]

====Bias correction only====
It considers an inaccurate model but without any unknown parameter. The target is to assess the ''model inadequacy''. The general model updating formulation for bias correction is:
:<math> y^e(\mathbf{x})=y^m(\mathbf{x})+\delta(\mathbf{x})+\varepsilon </math>
where <math> y^e(\mathbf{x}) </math> denotes the experimental measurements as a function of several input variables <math> \mathbf{x} </math>, <math> y^m(\mathbf{x}) </math> denotes the computer model (mathematical model) response, <math> \delta(\mathbf{x}) </math> denotes the additive discrepancy function (aka bias function), and <math> \varepsilon </math> denotes the experimental uncertainty. The objective is to estimate the discrepancy function <math> \delta(\mathbf{x}) </math>, and as a by-product, the resulting updated model is <math> y^m(\mathbf{x})+\delta(\mathbf{x}) </math>. A prediction confidence interval is provided with the updated model as the quantification of the uncertainty.

====Parameter calibration only====
It considers a model that fully describes the underlying physics but with one or more unknown parameters. The general model updating formulation for calibration is:
:<math> y^e(\mathbf{x})=y^m(\mathbf{x},\boldsymbol{\theta}^*)+\varepsilon </math>
where <math> y^m(\mathbf{x},\boldsymbol{\theta}) </math> denotes the computer model response that depends on several unknown model paramters <math> \boldsymbol{\theta} </math>, and <math> \boldsymbol{\theta}^* </math> denotes the true values of the unknown parameters in the course of experiments. The objective is to either estimate <math> \boldsymbol{\theta}^* </math>, or to come up with a probability distribution of <math> \boldsymbol{\theta}^* </math> that encompasses the best knowledge of the true parameter values.

====Bias correction and parameter calibration====
It considers an inaccurate model with one or more unknown parameters, and its model updating formulation combines the two together:
:<math> y^e(\mathbf{x})=y^m(\mathbf{x},\boldsymbol{\theta}^*)+\delta(\mathbf{x})+\varepsilon </math>
It is the most comprehensive model updating formulation that includes all possible sources of uncertainty, and it requires the most effort to solve.

==Selective methodologies for uncertainty quantification==
Much research has been done to solve uncertainty quantification problems, though a majority of them deal with uncertainty propagation. During the past one to two decades, a number of approaches for inverse uncertainty quantification problems have also been developed and have proved to be useful for most small- to medium-scale problems.

===Methodologies for forward uncertainty propagation===
Existing uncertainty propagation approaches include probabilistic approaches and non-probabilistic approaches. There are basically five categories of probabilistic approaches for uncertainty propagation:<ref>S. H. Lee and W. Chen, ''A comparative study of uncertainty propagation methods for black-box-type problems'', Structural and Multidisciplinary Optimization
Volume 37, Number 3 (2009), 239-253, DOI: 10.1007/s00158-008-0234-7</ref>
* Simulation-based methods: [[Monte Carlo method|Monte Carlo simulations]], [[importance sampling]], adaptive sampling, etc.
* Local expansion-based methods: Taylor series, [[Perturbation theory|perturbation method]], etc. These methods have advantages when dealing with relatively small input variability and outputs that don't express high nonlinearity. These linear or linearized methods are detailed in the article [[Uncertainty propagation]].
* Functional expansion-based methods: Neumann expansion, orthogonal or Karhunen-Loeve expansions (KLE), with [[Polynomial chaos|polynomial chaos expansion (PCE)]] and wavelet expansions as special cases.
* Most probable point (MPP)-based methods: first-order reliability method (FORM) and second-order reliability method (SORM).
* Numerical integration-based methods: Full factorial numerical integration (FFNI) and dimension reduction (DR).
For non-probabilistic approaches, [[Interval arithmetic|interval analysis]]
<ref>
{{cite book|last1=Jaulin|first1=L.|last2=Kieffer|first2=M.|last3=Didrit|first3=O.|last4=Walter|first4=E.|
title=Applied Interval Analysis|year=2001|publisher=Springer|location=Berlin|isbn=1-85233-219-0}}
</ref>
, [[Fuzzy logic|Fuzzy theory]], possibility theory and evidence theory are among the most widely used.

The probabilistic approach is considered as the most rigorous approach to uncertainty analysis in engineering design due to its consistency with the theory of decision analysis. Its cornerstone is the calculation of probability density functions for sampling statistics.<ref>[http://publications.npl.co.uk/npl_web/pdf/tqe2.pdf Arnaut, L. R. Measurement uncertainty in reverberation chambers - I. Sample statistics. Technical report TQE 2, 2nd. ed., sec. 3.1, National Physical Laboratory, 2008.]
</ref> This can be performed rigorously for random variables that are obtainable as transformations of Gaussian variables, leading to exact confidence intervals.

===Methodologies for inverse uncertainty quantification===
Existing methodologies for inverse uncertainty quantification are mostly under the [[Bayesian inference|Bayesian framework]]. The most intriguing direction is to aim at solving problems with both bias correction and parameter calibration. The challenges of such problems include not only the influences from model inadequacy and parameter uncertainty, but also the lack of data from both computer simulations and experiments. A common situation is that the input settings are not the same over experiments and simulations.

====Modular Bayesian approach====
An approach to inverse uncertainty quantification is the modular Bayesian approach.<ref name="KennedyOHagan" /><ref>Marc C. Kennedy, Anthony O'Hagan, ''Supplementary Details on Bayesian Calibration of Computer Models'', Sheffield, University of Sheffield: 1-13, 2000</ref> The modular Bayesian approach derives its name from its four-module procedure. Apart from the current available data, a [[prior distribution]] of unknown parameters should be assigned.

;'''Module 1:''' Gaussian process modeling for the computer model

To address the issue from lack of simulation results, the computer model is replaced with a [[Gaussian process|Gaussian Process (GP)]] model
:<math> y^m(\mathbf{x},\boldsymbol{\theta})\sim\mathcal{GP}\big(\mathbf{h}^m(\cdot)^T\boldsymbol{\beta}^m,\sigma_m^2R^m(\cdot,\cdot)\big) </math>
where
:<math> R^m\big((\mathbf{x},\boldsymbol{\theta}),(\mathbf{x}',\boldsymbol{\theta}')\big)=\exp\left\{-\sum_{k=1}^d \omega_k^m(x_k-x_k')^2\right\}\exp\left\{-\sum_{k=1}^r \omega_{d+k}^m(\theta_k-\theta_k')^2 \right\}. </math>
<math> d </math> is the dimension of input variables, and <math> r </math> is the dimension of unknown parameters.While <math> \mathbf{h}^m(\cdot) </math> is pre-defined, <math> \left\{\boldsymbol{\beta}^m, \sigma_m, \omega_k^m, k=1,\ldots,d+r\right\} </math>, known as ''[[hyperparameter]]s'' of the GP model, need to be estimated via [[Maximum likelihood|maximum likelihood estimation (MLE)]]. This module can be considered as a generalized [[Kriging]] method.

;'''Module 2:''' Gaussian process modeling for the discrepancy function
Similarly with the first module, the discrepancy function is replaced with a GP model
:<math> \delta(\mathbf{x})\sim\mathcal{GP}\big(\mathbf{h}^\delta(\cdot)^T\boldsymbol{\beta}^\delta,\sigma_\delta^2R^\delta(\cdot,\cdot)\big) </math>
where
:<math> R^\delta(\mathbf{x},\mathbf{x}')=\exp\left\{-\sum_{k=1}^d \omega_k^\delta(x_k-x_k')^2 \right\}. </math>
Together with the prior distribution of unknown parameters, and data from both computer models and experiments, one can derive the maximum likelihood estimates for <math> \left\{\boldsymbol{\beta}^\delta, \sigma_\delta, \omega_k^\delta, k=1,\ldots,d\right\} </math>. At the same time, <math> \boldsymbol{\beta}^m </math> from Module 1 gets updated as well.

;'''Module 3:''' Posterior distribution of unknown parameters
[[Bayes' theorem]] is applied to calculate the [[posterior distribution]] of the unknown parameters:
:<math> p(\boldsymbol{\theta}|\rm{data},\boldsymbol{\phi})\propto p(\rm{data}|\boldsymbol{\theta},\boldsymbol{\phi})p(\boldsymbol{\theta}) </math>
where <math> \boldsymbol{\phi} </math> includes all the fixed hyperparameters in previous modules.

;'''Module 4:''' Prediction of the experimental response and discrepancy function

====Fully Bayesian approach====
Fully Bayesian approach requires that not only the priors for unknown parameters <math> \boldsymbol{\theta} </math> but also the priors for the other hyperparameters <math> \boldsymbol{\phi} </math> should be assigned. It follows the following steps:<ref name="Liu">F. Liu, M.J. Bayarriy and J.O.Bergerz, ''Modularization in Bayesian Analysis, with Emphasis on Analysis of Computer Models'', Bayesian Analysis (2009) 4, Number 1, pp. 119-150, DOI:10.1214/09-BA404</ref>

# Derive the posterior distribution <math> p(\boldsymbol{\theta},\boldsymbol{\phi}|\rm{data}) </math>;
# Integrate <math> \boldsymbol{\phi} </math> out and obtain <math> p(\boldsymbol{\theta}|\rm{data}) </math>. This single step accomplishes the calibration;
# Prediction of the experimental response and discrepancy function.

However, the approach has significant drawbacks:
* For most cases, <math> p(\boldsymbol{\theta},\boldsymbol{\phi}|\rm{data}) </math> is a highly intractable function of <math> \boldsymbol{\phi} </math>. Hence the integration becomes very troublesome. Moreover, if priors for the other hyperparameters <math> \boldsymbol{\phi} </math> are not carefully chosen, the complexity in numerical integration increases even more.
* In the prediction stage, the prediction (which should at least include the expected value of system responses) also requires numerical integration. [[Markov chain Monte Carlo|Markov chain Monte Carlo (MCMC)]] is often used for integration; however it is computationally expensive.

The fully Bayesian approach requires a huge amount of calculations and may not yet be practical for dealing with the most complicated modelling situations.<ref name="Liu" />

==Known issues==
The theories and methodologies for uncertainty propagation are much better established, compared with inverse uncertainty quantification. For the latter, several difficulties remain unsolved:
# Dimensionality issue: The computational cost increases dramatically with the dimensionality of the problem, i.e. the number of input variables and/or the number of unknown parameters.
# Identifiability issue:<ref>Arendt, P., W. Chen, and D. Apley, ''Improving Identifiability in Model Calibration Using Multiple Responses'', DETC2011-48623, ASME International Design Engineering Technical Conferences, August 28–31, Washington, D.C., 2011</ref> Multiple combinations of unknown parameters and discrepancy function can yield the same experimental prediction. Hence different values of parameters cannot be distinguished/identified.

==See also==
* [[Computer experiment]]
* [[Quantification of margins and uncertainties]]

==References==
{{reflist}}

==Further reading==
* da Silva, R.B., Bulska, E., Godlewska-Zylkiewicz, B., Hedrich, M., Majcen, N., Magnusson, B., Marincic, S., Papadakis, I., Patriarca, M., Vassileva, E., Taylor, P., Analytical measurement: measurement uncertainty and statistics;ISBN 978-92-79-23070-7, 2012.
* [http://publications.npl.co.uk/npl_web/pdf/tqe2.pdf Arnaut, L. R. Measurement uncertainty in reverberation chambers - I. Sample statistics. Technical report TQE 2, 2nd. ed., National Physical Laboratory, 2008.]

[[Category:Applied mathematics]]
[[Category:Mathematical modeling]]
[[Category:Operations research]]
[[Category:Uncertainty of numbers]]