Variance of random matrix right multiplied by vector

Question

I'm working with random matrices and there is an identity I cannot find. Let A be a random matrix and b be a deterministic vector of appropriate dimensions. Is there an known identity for the following variance (aka variance-covariance) $$Var(Ab)$$ The identity I am finding always gives b as random and A as deterministic.

Thanks

Answer 1

There is no useful simplifying identity for right multiplication by a constant in general. Recall the covariance matrix of a random vector $X$ is given by

$$\text{Var}(X)=E[XX']-E[X]E[X]'.$$

If $A$ is constant and $b$ is random, by associativity and distributivity of matrix multiplication we have

$$\text{Var}(Ab)=E[(Ab)(Ab)']-E[Ab]E[Ab]'\\ =E[Abb'A']-E[Ab]E[b'A']\\ =A(E[bb']-E[b]E[b'])A'\\ =A\text{Var}(b)A'.$$

However, if $A$ is random and $b$ is constant, you can't proceed further than the second line in the above computation.

In any case, I'm not sure what kind of identity you were expecting, since the covariance matrix definition given above applies for random vectors, not random matrices to my knowledge (so we cannot have a "$\text{Var}(A)$"). If that definition was applied to a random matrix $A$, it wouldn't generally represent a matrix of all the different covariances between components of $A$ as it does in the case of random vectors.

Answer 2

So I had a go myself but I'm not sure if all the derivation is correct. Maybe someone can verify. Consider a random matrix B, random vector a and deterministic vector u where Bu reslts in a vector the size of a, than

$$ Cov(a,Bu)=E\left \{ a(Bu)^T\right \}-E\left \{a\right \}E\left \{Bu\right \}^T$$ $$ = E\left \{ au^TB^T\right \}-E\left \{ a\right \}E\left \{u^TB^T\right \}$$

Using matrix vectorization and considering that u is deterministic, the first term results in

$$ vec(E\left \{ au^TB^T\right \})= E\left \{vec\left [ au^TB^T \right ] \right \}$$

$$ = E\left \{ (B \otimes a)vec(u^T) \right \}$$ $$ = E\left \{ (B \otimes a) \right \}vec(u^T)$$

Using a similar procedure, the second term results in

$$ = [E\left \{B\right \} \otimes E\left \{a\right \}] vec(u^T)$$

and thus

$$ vec(Cov(a,Bu)) = [E\left \{ (B \otimes a) \right \}-E\left \{B\right \} \otimes E\left \{a\right \}]vec(u^T) $$

Taking the first kronocker product

$$E\left \{ (B \otimes a) \right \}=vec[E\left \{ aB^T \right \}] $$

and similarly

$$E\left \{B\right \} \otimes E\left \{a\right \}=vec[E\left \{a\right \}E\left \{B^T\right \}] $$

thus giving

$$ vec(Cov(a,Bu))=vec[E\left \{ aB^T \right \}-E\left \{a\right \}E\left \{B^T\right \}]vec(u^T) $$ $$ = vec(Cov(a,B))vec(u^T)$$

Does this result make sense?

Thanks