Referring to the above image, I wanted to know how to get the covariance matrix $\sum$.
My understanding is, $A$, is our transformation matrice, such that
$\begin{bmatrix} X_1 \\ X_2 \\ \end{bmatrix}$ = $\begin{bmatrix} Z_1 \\ Z_2 \\ \end{bmatrix} \cdot A$
How do we get the covariance matrices from here? Would it be like this:
$\sigma_{1,1} = COV(X_1,X_1) = VAR(X_1)$ Since $X_1 \sim N(0,\sigma_1)$, $VAR(X_1) = \sigma_1$ then $\sigma_{1,2} = COV(X_1,X_2) = COV(\sigma_1 Z_1, \rho \sigma_2 Z_1 + \sqrt{1 - \rho ^2} \sigma_2 Z_2) = COV(\sigma_1 Z_1, \rho \sigma_2 Z_1) + COV(\sigma_1 Z_1, \sqrt{1 - \rho ^2} \sigma_2 Z_2)$
Am I along the right track? What do I do after this, if I am?
You're on the right track; you can simplify using $\operatorname{Cov}(aZ_1,bZ_2)=ab\operatorname{Cov}(Z_1,Z_2)$, also the fact that $\operatorname{Cov}(Z_1,Z_2)= 0$ since $Z_1$ and $Z_2$ are independent, and the fact that $\operatorname{Cov}(Z_1,Z_1) = \operatorname{Var}(Z_1)=1$, and so on.
But there is a more direct method. If $X$ denotes the column vector $(X_1, X_2)^T$ and similarly we write $Z=(Z_1,Z_2)^T$, then you've got $X=AZ$. The general formula for covariance matrix is $$\operatorname{Cov}(X)=E[(X-E(X))(X-E(X))^T] \;.$$ Plug in $X=AZ$ and use matrix algebra to simplify this to $$ \operatorname{Cov}(X)=A\left[E(Z-E(Z))(Z-E(Z))^T\right]A^T=A[\operatorname{Cov}(Z)]A^T\;. $$ By assumption, the $Z_1, Z_2$ are standard normals, so that the covariance matrix for $Z$ is the identity matrix, and the covariance matrix for $X$ is the matrix product $AA^T$.