Conditional expectation of multivariate normal

273 Views Asked by At

Note: In trying to formalize my question, I think I found one answer to it. I have still posted the question, in part because I hope that someone else has a less algebraically intensive solution.

The question: I'm reading Lamperti's Probability, Second Edition. I'm trying to understand Example 3 of Section 4 of Chapter 1 (page 25 of my book), which has to do with conditional expectations applied to multivariate normal distributions. He makes a leap of logic that I don't follow.

Lamperti says that, given random variables $X_0, X_1, ..., X_n$ with positive and continuous joint density $f(t_0, t_1, ..., t_n)$, we can write $E[X_0 | X_1, ..., X_n]$ as the random variable $g(X_1, ..., X_n)$, with $g(t_1,...t_n)$ defined as

$$g(t_1, ..., t_n) := \frac{\int s f(s, t_1, ..., t_n) ds}{\int f(s, t_1, ..., t_n)ds}$$

So far, so good. He also says that $X_0, ..., X_n$ follow a multivariate normal distribution as long as they have a joint density of the form

$$ f(t_0, ..., t_n) = K \exp(-\frac{1}{2} \sum^n_{i,j=0}d_{ij}t_it_j)$$

with $K$ a normalizing constant and $[d_{ij}]$ a symmetric, positive-definite matrix. Also fine. Then, however, he says that, from the two facts above, we can deduce that $$E[X_0 | X_1, ..., X_n] = - \sum^n_{k=1} \frac{d_{k0}}{d_{00}}X_k$$

but I do not know how. Do you?

1

There are 1 best solutions below

1
On

This is an algebraically intensive answer that depends on completing the square inside the exponential, which is a common trick when working with normal distributions. However, @badatmath has suggested a shorter, more enlightening approach.

Since $f$ appears beneath the integral sign in both the numerator and the denominator of $g$, any terms that can be both factored out of $f$ and pulled out from beneath the integral sign will cancel in the fraction. Therefore, any such terms can be ignored. To that end, let us rewrite $f$ in a way that factors out any terms not depending on $t_0$.

$$f(t_0, ... , t_n)$$ $$\propto \exp(-\frac{1}{2} \sum^n_{i,j=0}d_{ij}t_it_j)$$ $$= \exp(-\frac{1}{2} (\sum^n_{i=0 \lor j=0}d_{ij}t_it_j + \sum^n_{i \neq 0 \land j \neq 0}d_{ij}t_it_j))$$ $$\propto \exp(-\frac{1}{2} (\sum^n_{i=0 \lor j=0}d_{ij}t_it_j)$$ $$= \exp(-\sum^n_{i=1} d_{i0}t_it_0 -\frac{1}{2}d_{00}t_0^2)$$

Now, for reasons that will soon be apparent, let us complete the square. Specifically, let $a^2 = \frac{1}{2} d_{00} t_0^2$ and $2ab = t_0 \sum^n_{i=1} d_{i0} t_i$. Then $a = \sqrt{\frac{1}{2} d_{00} t_0^2} = t_0 \sqrt{\frac{d_{00}}{2}}$, (where $d_{00} > 0$ because $[d_{ij}]$ is positive definite), and

$$ b $$ $$ = \frac{2ab}{2a}$$ $$ = \frac{t_0 \sum^n_{i=1} d_{i0} t_i}{t_0 \sqrt{2 d_{00}}}$$ $$ = \frac{\sum^n_{i=1} d_{i0} t_i}{\sqrt{2 d_{00}}}$$

where $b$ does not depend on $t_0$. Thus,

$$f(t_0,...t_n)$$ $$\propto \exp(-\sum^n_{i=1} d_{i0}t_it_0 -\frac{1}{2}d_{00}t_0^2)$$ $$=\exp(-2ab - a^2)$$ $$=\exp(b^2 - b^2 -2ab - a^2)$$ $$=\exp(b^2 - (a+b)^2)$$ $$\propto \exp((a+b)^2)$$ $$= \exp((\sqrt{\frac{d_{00}}{2}}t_0 + \frac{\sum^n_{i=1} d_{i0} t_i}{\sqrt{2 d_{00}}})^2)$$ $$= \exp(\frac{d_{00}}{2}(t_0 + \frac{\sum^n_{i=1} d_{i0} t_i}{ d_{00}})^2)$$

which is just the kernel of a univariate normal density with variable $t_0$ and mean $-\frac{\sum^n_{i=1} d_{i0} t_i}{ d_{00}}$. Plugging the kernel into our equation for $g$, we find

$$g(t_1,...t_n)$$ $$=\frac{\int t_0 * \mathrm{kernel} * dt_0}{\int \mathrm{kernel} * dt_0}$$ $$=\frac{-\frac{\sum^n_{i=1} d_{i0} t_i}{ d_{00}}}{1}$$ $$=-\sum^n_{i=1} \frac{d_{i0}}{d_{00}}t_i$$

which is what we wanted to show.