I have been reading the book Semiparametric Theory and Missing Data by Tsiatis. The question is about exercise 3(b) in chapter 2 Hilbert Space for Random Vectors. Here is the problem
Let $Z=\left(Z_1, Z_2\right)^T$ be a bivariate normally distributed vector with mean zero and covariance matrix $\left(\begin{array}{cc}\sigma_1^2 & \sigma_{12} \\ \sigma_{12} & \sigma_2^2\end{array}\right)$. Consider the Hilbert space of all one-dimensional measurable functions of $Z$ with mean zero, finite variance, and covariance inner product. Let $\mathcal{U}$ denote the linear subspace spanned by $Z_2$ and $\left(Z_1^2-\sigma_1^2\right)$; i.e., the space whose elements are $$ a_1\left(Z_1^2-\sigma_1^2\right)+a_2 Z_2 \text { for all } a_1, a_2 . $$ (a) Find the projection of $Z_1$ onto the space $\mathcal{U}$.
(b) Find the variance of the residual (i.e., $\operatorname{var}\left\{Z_1-\Pi\left(Z_1 \mid \mathcal{U}\right)\right\}$ ).
I have finished part (a) using projection theorem, and got
$$ \Pi\left(Z_1 \mid \mathcal{U}\right) = E\left[Z_1 u^{\top}\right]\left(E\left[u u^{\top}\right]\right)^{-1} u \text { where } u=\left(\begin{array}{l} z_1^2-\sigma_1^2 \\ z_2 \end{array}\right) $$
My question is regards to part (b). Since
$$ \begin{aligned} \operatorname{var}\left(Z_1-\pi\left(Z_1 \mid u\right)\right) & =\operatorname{var}\left(Z_1-a_0^{\top} u\right) \\ & =\operatorname{var}\left(Z_1\right)+\operatorname{var}\left(a_0^{\top} u\right) \\ & =\sigma_1^2+a_0^{\top} \operatorname{var}(u) a_0 \end{aligned} $$
Notice $ a_0 = E\left[Z_1 u^{\top}\right]\left(E\left[u u^{\top}\right]\right)^{-1}$.
To that end, I just need to figure out the variance-covariance matrix of $u$, which consists
$$ \begin{aligned} \operatorname{var}(u) & =\operatorname{var}\left(\left(\begin{array}{cc} Z_1^2-\sigma_1^2 \\ Z_2 \end{array}\right)\right) \\ & =\left(\begin{array}{ll} \operatorname{var}\left(Z_1^2-\sigma_1^2\right) & \operatorname{cov}\left(Z_1^2-\sigma_1^2 , Z_2\right) \\ \operatorname{cov}\left(Z_2, Z_1^2-\sigma_1^2\right) & \operatorname{var}\left(Z_2\right) \end{array}\right) \end{aligned} $$
All are straight forward except $\operatorname{cov}\left(Z_2, Z_1^2-\sigma_1^2\right)$. So my question is really about how to compute this quantity.
$$ \begin{aligned} \operatorname{cov}\left(Z_2, Z_1^2-\sigma_1^2\right) & =E\left[\left(Z_2-0\right)\left(\left(Z_1^2-\sigma_1^2\right)-0\right)\right] \\ & =E\left[Z_2\left(Z_1^2-\sigma_1^2\right)\right] \\ & =E\left[Z_1^2Z_2 \right]\\ \end{aligned} $$
The above used the fact that the mean of $Z_1 - \sigma^2 = 0$.
I did try to use transformation technique.
Define $g_1: \mathbb{R}^{-} \times \mathbb{R} \rightarrow \mathbb{R}^{\top} \times \mathbb{R}$ by $\quad g_1\left(\begin{array}{l}z_1 \\ z_2\end{array}\right)=\left(\begin{array}{l}z_1^2 \\ z_2\end{array}\right), \quad g_1^{-1}\left(\begin{array}{l}v \\ v\end{array}\right)=\left(\begin{array}{c}-\sqrt{u} \\ v\end{array}\right)$ $$ g_2: \mathbb{R}^{+} \times \mathbb{R} \rightarrow \mathbb{R}^{+} \times \mathbb{R} \text { by } \quad g_2\left(\begin{array}{l} z_1 \\ z_2 \end{array}\right)=\left(\begin{array}{l} z_1^2 \\ z_2 \end{array}\right), \quad g_2^{-1}\left(\begin{array}{l} u \\ v \end{array}\right)=\left(\begin{array}{l} \sqrt{u} \\ v \end{array}\right) $$
$$ f_{1, v}(u, v)=f_{z_1, z_2}\left(g_1^{-1}\left(\begin{array}{l} v \\ v \end{array}\right)\right)\left|J_{g-1}\left(\begin{array}{l} u \\ v \end{array}\right)\right|+f_{z_1, z_2}\left(g_2^{-1}\left(\begin{array}{l} u \\ v \end{array}\right)\right)\left|J_{g_2^{-1}}\left(\begin{array}{l} y \\ v \end{array}\right)\right| $$
$$ =f_{z_1 z_2}\left(\left(\begin{array}{c} -\sqrt{u} \\ v \end{array}\right)\right) \frac{1}{2 \sqrt{u}}+f_{z_1 z_2}\left(\left(\begin{array}{c} \sqrt{u} \\ v \end{array}\right)\right) \frac{1}{2 \sqrt{u}} $$
$$ =\frac{1}{2 \sqrt{u}}\left(f_{z_1 z_2}\left(\left(\begin{array}{c} -\sqrt{v} \\ v \end{array}\right)\right)+f_{z_1 z_2}\left(\left(\begin{array}{l} \sqrt{u} \\ v \end{array}\right)\right)\right) $$ $$ =\frac{1}{2 \sqrt{u}} \frac{1}{2 \pi \cdot \sqrt{\sigma_1^2 \sigma_2^2-\sigma_{12}^2}}\left(e^{-\frac{1}{2} \cdot \frac{u \sigma_2^2+v^2 \sigma_1^2}{\sigma_1^2 \sigma_2^2-\sigma_2^2}}+e^{-\frac{1}{2} \cdot \frac{u \sigma_2^2+v^2 \sigma_1^2}{\sigma_1^2 \sigma_2^2-\sigma_n^2}}\right) $$ $$ =\frac{1}{\sqrt{u}} \frac{1}{2 \pi \cdot \sqrt{\sigma_1^2 \sigma_2^2-\sigma_{12}^2}} e^{-\frac{1}{2} \cdot \frac{u \sigma_2^2+v^2 \sigma_1^2}{\sigma_1^2 \sigma_2^2-\sigma_{12}^2}} $$ $$ =\left(\frac{1}{\sqrt{u}} \frac{1}{2 \pi \cdot \sqrt{\sigma_1^2 \sigma_2^2-\sigma_{12}^2}} \quad e^{-\frac{1}{2} \frac{u \sigma_2^2}{\sigma_1^2 \sigma_2^2-\sigma_{12}^2}}\right)\left(e^{-\frac{1}{2} \frac{v^2 \sigma_2^2}{\sigma_1^2 \sigma_2^2-\sigma_{12}^2}}\right) $$
At the end I was able to obtain $f_{u, v}\left(u, v\right)$, however it is messy. But I'm thinking if I can use the result to show $u$ and $v$ are independent.
However, by inspecting the product of functions at the end, I'm not convinced that it is a product of a scaled Chi-square and a normal density.
May I get help on this? If I can show $u$ and $v$ are independent, then $E[uv] = E[u]E[v] = E[Z_1^2]E[Z_2]$, which will be easy.