Inverse Problem

To predict the result of a measurement requires (1) a model of the system under investigation, and (2) a physical theory linking the parameters of the model to the parameters being measured. This prediction of observations, given the values of the parameters defining the model constitutes the "normal problem," or, in the jargon of inverse problem theory, the forward problem. The "inverse problem" consists in using the results of actual observations to infer the values of the parameters characterizing the system under investigation.

Inverse problems may be difficult to solve for at least two different reasons: (1) different values of the model parameters may be consistent with the data (knowing the height of the main-mast is not sufficient for calculating the age of the captain), and (2) discovering the values of the model parameters may require the exploration of a huge parameter space (finding a needle in a 100-dimensional haystack is difficult).

Although most of the formulations of inverse problems proceed directly to the setting of an optimization problem, it is actually best to start using a probabilistic formulation, the optimization formulation then appearing as a by-product.

Consider a manifold M with a notion of volume. Then for any A subset M,


A volumetric probability is a function f that to any A subset M associates its probability


If M is a metric manifold endowed with some coordinates {x^1,...,x^n}, then

 dV=sqrt(detg)dx^1 ^ ... ^ dx^n


P(A)=int_Adx^1 ^ ... ^ dx^nsqrt(detg)f_()
=int_Adx^1 ^ ... ^ dx^nf^_.

(Note that the volumetric probability f is an invariant, but the probability density f^_ is not; it is a density.)

A basic operation with volumetric probabilities is their product,


where nu=int_MdVf(P)g(P). This corresponds to a "combination of probabilities" well suited to many basic inference problems.

Consider an example in which two planes make two estimations of the geographical coordinates of a shipwrecked man. Let the probabilities be represented by the two volumetric probabilities f(phi,lambda) and g(phi,lambda). The volumetric probability that combines these two pieces of information is


This operation of product of volumetric probabilities extends to the following case:

1. There is a volumetric probability f(P) defined on a first manifold M.

2. There is another volumetric probability phi(Q) defined on a second manifold N.

3. There is an application P|->Q=Q(P) from M into N.

Then, the basic operation introduced above becomes


where nu=int_MdV(P)f(P)phi(Q(P)).

In a typical inverse problem, there is:

1. A set of model parameters {m^1,m^2,...,m^n}.

2. A set of observable parameters {o^1,o^2,...,o^n}.

3. A relation o^i=o^i(m^1,m^2,...,m^n) predicting the outcome of the possible observations.

The model parameters are coordinates on the model parameter manifold M while the observable parameters are coordinates over the observable parameter manifold O. When the points on M are denoted M, M^', ... and the points on O are denoted O, O^', ..., the relation between the model parameters an the observable parameters is written M|->O=O(M).

The three basic elements of a typical inverse problem are:

1. Some a priori information on the model parameters, represented by a volumetric probability rho_(prior)(M) defined over M.

2. Some experimental information obtained on the observable parameters, represented by a volumetric probability sigma_(obs)(O) defined over O.

3. The 'forward modeling' relation M|->O=O(M) that we have just seen.

The use of equation (8) leads to


where nu is a normalization constant. This volumetric probability represents the resulting information one has on the model parameters (obtained by combining the available information). Equation (9) provides the more general solution to the inverse problem. Common methods (Monte Carlo, optimization, etc.) can be seen as particular uses of this equation.

Considering an example from sampling, sample the a priori volumetric probability rho_(prior)(M) to obtain (many) random models M_1, M_2, .... For each model M_i, solve the forward modeling problem, O_i=O_i(M_i). Give to each model M_i a probability of 'survival' proportional to sigma_(obs)(O_i(M_i)). The surviving models M_(1^'), M_(2^'), ... are samples of the a posteriori volumetric probability


Considering an example from least-squares fitting, the model parameter manifold may be a linear space, with vectors denoted m, m^', ..., and the a priori information may have the Gaussian form


The observable parameter manifold may be a linear space, with vectors denoted o, o^', ... and the information brought by measurements may have the Gaussian form


The forward modeling relation becomes, with these notations,


Then, the posterior volumetric probability for the model parameters is


where the misfit function S(m) is the sum of squares


The maximum likelihood model is the model m maximizing rho_(post)(m). It is also the model minimizing S(m). It can be obtained using a quasi-Newton algorithm,


where the Hessian of S is


and the gradient of S is


Here, the tangent linear operator O_n is defined via


As we have seen, the model m_infty at which the algorithm converges maximizes the posterior volumetric probability rho_(post)(m).

To estimate the posterior uncertainties, one can demonstrate that the covariance operator of the Gaussian volumetric probability that is tangent to rho_(post)(m) at m_infty is H_infty^(-1)C_m.

This entry contributed by Albert Tarantola

Explore with Wolfram|Alpha


Groetsch, C. W. Inverse Problems: Activities for Undergraduates. Washington, DC: Math. Assoc. Amer., 1999.Kozhanov, A. I. Composite Type Equations and Inverse Problems. Utrecht, Netherlands: VSP, 1999.Mosegaard, K. and Tarantola, A. "Probabilistic Approach to Inverse Problems." In International Handbook of Earthquake & Engineering Seismology, Part A. New York: Academic Press, pp. 237-265, 2002.Prilepko, A. I.; Orlovsky, D. G.; and Vasin, I. A. Methods for Solving Inverse Problems in Mathematical Physics. New York: Dekker, 1999.Tarantola, A. Inverse Problem Theory and Model Parameter Estimation. Philadelphia, PA: SIAM, 2004.

Referenced on Wolfram|Alpha

Inverse Problem

Cite this as:

Tarantola, Albert. "Inverse Problem." From MathWorld--A Wolfram Web Resource, created by Eric W. Weisstein.

Subject classifications