TOPICS
Search

Bivariate Normal Distribution

DOWNLOAD Mathematica NotebookDownload Wolfram Notebook
BivariateNormalDistribution

A nonsingular bivariate normal distribution has probability density function

P(x_1,x_2)=1/(2pisigma_1sigma_2sqrt(1-rho^2))exp[-z/(2(1-rho^2))],
(1)

where

z=((x_1-mu_1)^2)/(sigma_1^2)-(2rho(x_1-mu_1)(x_2-mu_2))/(sigma_1sigma_2)+((x_2-mu_2)^2)/(sigma_2^2),
(2)

and

rho=cor(x_1,x_2)=(V_(12))/(sigma_1sigma_2)
(3)

is the correlation of x_1 and x_2 (Kenney and Keeping 1951, pp. 92 and 202-205; Whittaker and Robinson 1967, p. 329) and V_(12) is the covariance.

The bivariate normal distribution is implemented in the Wolfram Language as MultinormalDistribution[mu, v], where mu is the vector of means (mu_1,mu_2) and v is the covariance matrix V. The coefficients sigma_(ij) used in the derivation below are entries of the matrix A of a linear transformation, rather than entries of V. They satisfy V=AA^T, where T denotes the transpose.

The marginal probabilities are then

P(x_1)=int_(-infty)^inftyP(x_1,x_2)dx_2
(4)
=1/(sigma_1sqrt(2pi))e^(-(x_1-mu_1)^2/(2sigma_1^2))
(5)

and

P(x_2)=int_(-infty)^inftyP(x_1,x_2)dx_1
(6)
=1/(sigma_2sqrt(2pi))e^(-(x_2-mu_2)^2/(2sigma_2^2))
(7)

(Kenney and Keeping 1951, p. 202).

Let Z_1 and Z_2 be two independent normal variates with means mu_i=0 and sigma_i^2=1 for i=1, 2. Then the variables a_1 and a_2 defined below are normal bivariates with unit variance and correlation coefficient rho:

a_1=sqrt((1+rho)/2)z_1+sqrt((1-rho)/2)z_2
(8)
a_2=sqrt((1+rho)/2)z_1-sqrt((1-rho)/2)z_2.
(9)

To derive the bivariate normal probability function, let X_1 and X_2 be normally and independently distributed variates with mean 0 and variance 1, then define

Y_1=mu_1+sigma_(11)X_1+sigma_(12)X_2
(10)
Y_2=mu_2+sigma_(21)X_1+sigma_(22)X_2
(11)

(Kenney and Keeping 1951, p. 92). The variates Y_1 and Y_2 are then themselves normally distributed with means mu_1 and mu_2, variances

sigma_1^2=sigma_(11)^2+sigma_(12)^2
(12)
sigma_2^2=sigma_(21)^2+sigma_(22)^2,
(13)

and covariance

V_(12)=sigma_(11)sigma_(21)+sigma_(12)sigma_(22).
(14)

The covariance matrix is defined by

V=[sigma_1^2 rhosigma_1sigma_2; rhosigma_1sigma_2 sigma_2^2],
(15)

where

rho=(V_(12))/(sigma_1sigma_2)=(sigma_(11)sigma_(21)+sigma_(12)sigma_(22))/(sigma_1sigma_2).
(16)

Now, the joint probability density function for x_1 and x_2 is

f(x_1,x_2)dx_1dx_2=1/(2pi)e^(-(x_1^2+x_2^2)/2)dx_1dx_2,
(17)

but from (10) and (11), we have

[y_1-mu_1; y_2-mu_2]=[sigma_(11) sigma_(12); sigma_(21) sigma_(22)][x_1; x_2].
(18)

As long as

|sigma_(11) sigma_(12); sigma_(21) sigma_(22)|!=0,
(19)

this can be inverted to give

[x_1; x_2]=[sigma_(11) sigma_(12); sigma_(21) sigma_(22)]^(-1)[y_1-mu_1; y_2-mu_2]
=1/(sigma_(11)sigma_(22)-sigma_(12)sigma_(21))[sigma_(22) -sigma_(12); -sigma_(21) sigma_(11)][y_1-mu_1; y_2-mu_2].
(20)

Therefore,

x_1^2+x_2^2=([sigma_(22)(y_1-mu_1)-sigma_(12)(y_2-mu_2)]^2)/((sigma_(11)sigma_(22)-sigma_(12)sigma_(21))^2) +([-sigma_(21)(y_1-mu_1)+sigma_(11)(y_2-mu_2)]^2)/((sigma_(11)sigma_(22)-sigma_(12)sigma_(21))^2),
(21)

and expanding the numerator of (21) gives

sigma_(22)^2(y_1-mu_1)^2-2sigma_(12)sigma_(22)(y_1-mu_1)(y_2-mu_2)+sigma_(12)^2(y_2-mu_2)^2+sigma_(21)^2(y_1-mu_1)^2-2sigma_(11)sigma_(21)(y_1-mu_1)(y_2-mu_2)+sigma_(11)^2(y_2-mu_2)^2,
(22)

so

(x_1^2+x_2^2)(sigma_(11)sigma_(22)-sigma_(12)sigma_(21))^2 =(y_1-mu_1)^2(sigma_(21)^2+sigma_(22)^2)-2(y_1-mu_1)(y_2-mu_2)(sigma_(11)sigma_(21)+sigma_(12)sigma_(22))+(y_2-mu_2)^2(sigma_(11)^2+sigma_(12)^2) =sigma_2^2(y_1-mu_1)^2-2(y_1-mu_1)(y_2-mu_2)(rhosigma_1sigma_2)+sigma_1^2(y_2-mu_2)^2 =sigma_1^2sigma_2^2[((y_1-mu_1)^2)/(sigma_1^2)-(2rho(y_1-mu_1)(y_2-mu_2))/(sigma_1sigma_2)+((y_2-mu_2)^2)/(sigma_2^2)].
(23)

Now, the denominator of (21) is

sigma_(11)^2sigma_(21)^2+sigma_(11)^2sigma_(22)^2+sigma_(12)^2sigma_(21)^2+sigma_(12)^2sigma_(22)^2-sigma_(11)^2sigma_(21)^2 -2sigma_(11)sigma_(12)sigma_(21)sigma_(22)-sigma_(12)^2sigma_(22)^2=(sigma_(11)sigma_(22)-sigma_(12)sigma_(21))^2,
(24)

so

1/(1-rho^2)=1/(1-(V_(12)^2)/(sigma_1^2sigma_2^2))
(25)
=(sigma_1^2sigma_2^2)/(sigma_1^2sigma_2^2-V_(12)^2)
(26)
=(sigma_1^2sigma_2^2)/((sigma_(11)^2+sigma_(12)^2)(sigma_(21)^2+sigma_(22)^2)-(sigma_(11)sigma_(21)+sigma_(12)sigma_(22))^2).
(27)

can be written simply as

1/(1-rho^2)=(sigma_1^2sigma_2^2)/((sigma_(11)sigma_(22)-sigma_(12)sigma_(21))^2),
(28)

and

x_1^2+x_2^2=1/(1-rho^2)[((y_1-mu_1)^2)/(sigma_1^2)-(2rho(y_1-mu_1)(y_2-mu_2))/(sigma_1sigma_2)+((y_2-mu_2)^2)/(sigma_2^2)].
(29)

Writing the determinant as D=sigma_(11)sigma_(22)-sigma_(12)sigma_(21)!=0, the inverse transformation is

x_1=(sigma_(22)(y_1-mu_1)-sigma_(12)(y_2-mu_2))/D
(30)
x_2=(-sigma_(21)(y_1-mu_1)+sigma_(11)(y_2-mu_2))/D.
(31)

The absolute value of the Jacobian is

|J((x_1,x_2)/(y_1,y_2))|=1/(|D|)
(32)
=1/(sigma_1sigma_2sqrt(1-rho^2)),
(33)

so

dx_1dx_2=(dy_1dy_2)/(sigma_1sigma_2sqrt(1-rho^2))
(34)

and

1/(2pi)e^(-(x_1^2+x_2^2)/2)dx_1dx_2=1/(2pisigma_1sigma_2sqrt(1-rho^2))exp[-z/(2(1-rho^2))]dy_1dy_2,
(35)

where

z=((y_1-mu_1)^2)/(sigma_1^2)-(2rho(y_1-mu_1)(y_2-mu_2))/(sigma_1sigma_2)+((y_2-mu_2)^2)/(sigma_2^2).
(36)

Q.E.D.

The characteristic function of the bivariate normal distribution is given by

phi(t_1,t_2)=int_(-infty)^inftyint_(-infty)^inftye^(i(t_1x_1+t_2x_2))P(x_1,x_2)dx_1dx_2
(37)
=Nint_(-infty)^inftyint_(-infty)^inftye^(i(t_1x_1+t_2x_2))exp[-z/(2(1-rho^2))]dx_1dx_2,
(38)

where

z=[((x_1-mu_1)^2)/(sigma_1^2)-(2rho(x_1-mu_1)(x_2-mu_2))/(sigma_1sigma_2)+((x_2-mu_2)^2)/(sigma_2^2)]
(39)

and

N=1/(2pisigma_1sigma_2sqrt(1-rho^2)).
(40)

Now let

u=x_1-mu_1
(41)
w=x_2-mu_2.
(42)

Then

phi(t_1,t_2)=N^'int_(-infty)^infty(e^(it_2w)exp[-1/(2(1-rho^2))(w^2)/(sigma_2^2)])int_(-infty)^inftye^ve^(it_1u)dudw,
(43)

where

v=-1/(2(1-rho^2))1/(sigma_1^2)[u^2-(2rhosigma_1w)/(sigma_2)u]
(44)
N^'=(e^(i(t_1mu_1+t_2mu_2)))/(2pisigma_1sigma_2sqrt(1-rho^2)).
(45)

Complete the square in the inner integral

int_(-infty)^inftyexp{-1/(2(1-rho^2))1/(sigma_1^2)[u^2-(2rhosigma_1w)/(sigma_2)u]}e^(it_1u)du =int_(-infty)^inftyexp{-1/(2sigma_1^2(1-rho^2))[u-(rhosigma_1w)/(sigma_2)]^2}exp[1/(2sigma_1^2(1-rho^2))((rhosigma_1w)/(sigma_2))^2]e^(it_1u)du.
(46)

Rearranging to bring the exponential depending on w outside the inner integral, letting

v=u-rho(sigma_1w)/(sigma_2),
(47)

and writing

e^(it_1u)=cos(t_1u)+isin(t_1u)
(48)

gives

phi(t_1,t_2)=N^'int_(-infty)^inftye^(it_2w)exp[-1/(2sigma_2^2(1-rho^2))w^2]exp[(rho^2)/(2sigma_2^2(1-rho^2))w^2]int_(-infty)^inftyexp[-1/(2sigma_1^2(1-rho^2))v^2]{cos[t_1(v+(rhosigma_1w)/(sigma_2))]+isin[t_1(v+(rhosigma_1w)/(sigma_2))]}dvdw.
(49)

Expanding the term in braces gives

[cos(t_1v)cos((rhosigma_1wt_1)/(sigma_2))-sin(t_1v)sin((rhosigma_1wt_1)/(sigma_2))]+i[sin(t_1v)cos((rhosigma_1wt_1)/(sigma_2))+cos(t_1v)sin((rhosigma_1wt_1)/(sigma_2))] =[cos((rhosigma_1wt_1)/(sigma_2))+isin((rhosigma_1wt_1)/(sigma_2))][cos(t_1v)+isin(t_1v)]=exp((irhosigma_1w)/(sigma_2)t_1)[cos(t_1v)+isin(t_1v)].
(50)

But e^(-ax^2)sin(bx) is odd, so the integral over the sine term vanishes, and we are left with

phi(t_1,t_2)=N^'int_(-infty)^inftye^(it_2w)exp[-(w^2)/(2sigma_2^2(1-rho^2))]exp[(rho^2w^2)/(2sigma_2^2(1-rho^2))]exp[(irhosigma_1wt_1)/(sigma_2)]dwint_(-infty)^inftyexp[-(v^2)/(2sigma_1^2(1-rho^2))]cos(t_1v)dv =N^'int_(-infty)^inftyexp[iw(t_2+t_1(rho(sigma_1)/(sigma_2)))]exp[-(w^2)/(2sigma_2^2)]dwint_(-infty)^inftyexp[-(v^2)/(2sigma_1^2(1-rho^2))]cos(t_1v)dv.
(51)

Evaluating the Gaussian integral

int_(-infty)^inftye^(ikx)e^(-ax^2)dx=int_(-infty)^inftye^(-ax^2)cos(kx)dx
(52)
=sqrt(pi/a)e^(-k^2/(4a))
(53)

for a>0 and real k gives the explicit form of the characteristic function,

phi(t_1,t_2)=(e^(i(t_1mu_1+t_2mu_2)))/(2pisigma_1sigma_2sqrt(1-rho^2)){sigma_2sqrt(2pi)exp[-1/4(t_2+rho(sigma_1)/(sigma_2)t_1)^22sigma_2^2]}{sigma_1sqrt(2pi(1-rho^2))exp[-1/4t_1^22sigma_1^2(1-rho^2)]} =e^(i(t_1mu_1+t_2mu_2))exp{-1/2[t_2^2sigma_2^2+2rhosigma_1sigma_2t_1t_2+rho^2sigma_1^2t_1^2+(1-rho^2)sigma_1^2t_1^2]} =exp[i(t_1mu_1+t_2mu_2)-1/2(sigma_1^2t_1^2+2rhosigma_1sigma_2t_1t_2+sigma_2^2t_2^2)].
(54)

In the singular case that

|sigma_(11) sigma_(12); sigma_(21) sigma_(22)|=0
(55)

(Kenney and Keeping 1951, p. 94), it follows that

sigma_(11)sigma_(22)=sigma_(12)sigma_(21)
(56)
y_1=mu_1+sigma_(11)x_1+sigma_(12)x_2
(57)
y_2=mu_2+(sigma_(12)sigma_(21))/(sigma_(11))x_2
(58)
=mu_2+(sigma_(11)sigma_(21)x_1+sigma_(12)sigma_(21)x_2)/(sigma_(11))
(59)
=mu_2+(sigma_(21))/(sigma_(11))(sigma_(11)x_1+sigma_(12)x_2),
(60)

so

y_1=mu_1+x_3
(61)
y_2=mu_2+(sigma_(21))/(sigma_(11))x_3,
(62)

where

x_3=y_1-mu_1
(63)
=(sigma_(11))/(sigma_(21))(y_2-mu_2).
(64)

For a bivariate normal distribution with mu_1=mu_2=0, positive marginal standard deviations, and correlation coefficient -1<rho<1, the quadrant probability is given analytically by

P(x_1<=0,x_2<=0)=P(x_1>=0,x_2>=0)
(65)
=int_(-infty)^0int_(-infty)^0P(x_1,x_2)dx_1dx_2
(66)
=1/4+(sin^(-1)rho)/(2pi)
(67)

(Rose and Smith 1996; Stuart and Ord 1998; Rose and Smith 2002, p. 231). Similarly,

P(x_1<=0,x_2>=0)=P(x_1>=0,x_2<=0)
(68)
=int_(-infty)^0int_0^inftyP(x_1,x_2)dx_1dx_2
(69)
=(cos^(-1)rho)/(2pi).
(70)

See also

Box-Muller Transformation, Correlation Coefficient--Bivariate Normal Distribution, Multivariate Normal Distribution, Normal Distribution, Price's Theorem, Trivariate Normal Distribution

Explore with Wolfram|Alpha

References

Abramowitz, M. and Stegun, I. A. (Eds.). Handbook of Mathematical Functions with Formulas, Graphs, and Mathematical Tables, 9th printing. New York: Dover, pp. 936-937, 1972.Holst, E. "The Bivariate Normal Distribution." http://www.ami.dk/research/bivariate/.Kenney, J. F. and Keeping, E. S. Mathematics of Statistics, Pt. 2, 2nd ed. Princeton, NJ: Van Nostrand, 1951.Kotz, S.; Balakrishnan, N.; and Johnson, N. L. "Bivariate and Trivariate Normal Distributions." Ch. 46 in Continuous Multivariate Distributions, Vol. 1: Models and Applications, 2nd ed. New York: Wiley, pp. 251-348, 2000.Rose, C. and Smith, M. D. "The Multivariate Normal Distribution." Mathematica J. 6, 32-37, 1996.Rose, C. and Smith, M. D. "The Bivariate Normal." §6.4 A in Mathematical Statistics with Mathematica. New York: Springer-Verlag, pp. 216-226, 2002.Spiegel, M. R. Theory and Problems of Probability and Statistics. New York: McGraw-Hill, p. 118, 1992.Stuart, A.; and Ord, J. K. Kendall's Advanced Theory of Statistics, Vol. 1: Distribution Theory, 6th ed. New York: Oxford University Press, 1998.Whittaker, E. T. and Robinson, G. "Determination of the Constants in a Normal Frequency Distribution with Two Variables" and "The Frequencies of the Variables Taken Singly." §161-162 in The Calculus of Observations: A Treatise on Numerical Mathematics, 4th ed. New York: Dover, pp. 324-328, 1967.

Referenced on Wolfram|Alpha

Bivariate Normal Distribution

Cite this as:

Weisstein, Eric W. "Bivariate Normal Distribution." From MathWorld--A Wolfram Resource. https://mathworld.wolfram.com/BivariateNormalDistribution.html

Subject classifications