Distribution of multivariate gaussian under linear constraints

416 Views Asked by At

Let $Z_1 \ldots Z_n$ be i.i.d Gaussians and introduce a set of linear constraints 1 through k, where the $j$'s constraint is the a linear combination of the $Z$'s take a specific value, i.e.

$ \sum_{i=1}^n w_{i,j}Z_i = c_j $

Where $w_{i,j}$ is a set of weights, and $c_j$ is the value for that constraint. Intuitively, this distribution should be also a gaussian, with now different first and second moments. I was able to find a derivation for what it would look like for a single constraint (see https://www.mdpi.com/2227-9091/6/3/64), but I'm unsure how to derive its form for more than one constraint.

1

There are 1 best solutions below

0
On

Partial answer, I may delete later:

Let $W \in \mathbb{R}^{n \times k}$ have entries $w_{i,j}$. Let $c = \begin{bmatrix} c_1 \\ \vdots \\ c_k\end{bmatrix}$ and $Z = \begin{bmatrix} Z_1 \\ \vdots \\ Z_n \end{bmatrix}$. The constraint is $W^\top Z = c$. I will assume the columns of $W$ are linearly independent.

Let $\tilde{W} \in \mathbb{R}^{n \times (n-k)}$ be such that $A:=\begin{bmatrix} W & \tilde{W}\end{bmatrix} \in \mathbb{R}^{n \times n}$. Then $$U := A^\top Z = \begin{bmatrix} W^\top Z \\ \tilde{W}^\top Z \end{bmatrix} \sim \mathcal{N}(0, \Sigma)$$ where $$\Sigma := A^\top A = \begin{bmatrix}W^\top W & W^\top \tilde{W} \\ \tilde{W}^\top W & \tilde{W}^\top \tilde{W}\end{bmatrix}.$$

The conditional distribution of $\tilde{W}^\top Z$ given $W^\top Z = c$, which is a Gaussian distribution whose parameters can be written down in terms of $\tilde{W}^\top Z$, $c$, and the blocks of $\Sigma$.