I have spent 2 days trying to search a formula that can help me obtain the covariance of (X+Y) especially when X and Y are independent with no luck. I could find for Variance(X+Y) however nothing for covariance. Nor could I find a way to obtain Covariance from Variance.
To the questions, say we have a model
$Z = aX + Y$
Where $X$ has a multivariate dist. with mean $\mu_X$ and $\sigma_X$
and $Y$ has a multivariate dist. with mean $\mu_Y$ and $\sigma_Y$
and $X$ and $Y$ are NOT independent.
And we were tasked with finding the mean and covariance for $Z$
For the mean, I just added the individual means
$\mu_Z = a*\mu_X + \mu_Y$
However, I am very much stuck for finding covariance for Z which I thought would be the same as $\operatorname{Cov}[aX + Y]$ . Is there a specific formula that I am missing? Or a property to calculate covariance of addition of 2 dependant variables?
Edit: Covariance of a single Variable as per the book : Probabilistic Machine Learning
First, you need a definition for covariance between two random vectors.
If $\boldsymbol X=(X_1,\ldots,X_m)^T$ and $\boldsymbol Y=(Y_1,\ldots,Y_n)^T$ are $m\times 1$ and $n\times 1$ random vectors respectively, then we define covariance between $\boldsymbol X$ and $\boldsymbol Y$ as the $m\times n$ matrix $\operatorname{Cov}(\boldsymbol X,\boldsymbol Y)$ whose $(i,j)$th entry is $\operatorname{Cov}(X_i,Y_j)$.
If $E(\boldsymbol X)=\boldsymbol{\alpha}$ and $E(\boldsymbol Y)=\boldsymbol{\beta}$ are the mean vectors, then it follows that
$$\operatorname{Cov}(\boldsymbol X,\boldsymbol Y)=E\left[(\boldsymbol X-\boldsymbol{\alpha})(\boldsymbol Y-\boldsymbol{\beta})^T\right]$$
The variance-covariance matrix or dispersion matrix of $\boldsymbol X$ is defined as
$$\operatorname{Var}(\boldsymbol X)=\operatorname{Cov}(\boldsymbol X,\boldsymbol X)=E\left[(\boldsymbol X-\boldsymbol{\alpha})(\boldsymbol X-\boldsymbol{\alpha})^T\right]$$
If $\boldsymbol X$ and $\boldsymbol Y$ are random vectors of same dimension and $a,b$ are real numbers, then using the above definition, one can show that
\begin{align} \operatorname{Var}(a\boldsymbol X+b\boldsymbol Y)&=\operatorname{Cov}(a\boldsymbol X+b\boldsymbol Y,a\boldsymbol X+b\boldsymbol Y) \\&=a^2\operatorname{Var}(\boldsymbol X)+b^2\operatorname{Var}(\boldsymbol Y)+ab(\operatorname{Cov}(\boldsymbol X,\boldsymbol Y)+\operatorname{Cov}(\boldsymbol Y,\boldsymbol X)) \end{align}