Suppose $Y_1, Y_2,$ and $X$ are random variables, then the iterated covariance formula is as follows:
$$Cov(Y_1, Y_2) = E(Cov(Y_1, Y_2|X)) + Cov(E(Y_1|X), E(Y_2|X))$$
Now let $Y = \begin{bmatrix}Y_1 \\ Y_2 \end{bmatrix}$. And let $X$ denote the set of random variables $X_1, \ldots, X_n$. And I'm interested in writing out the formula for $Cov(Y)$. Is the following correct?
$$Cov(Y) = E(Cov(Y|X)) + Cov(E(Y|X))$$
Or should I write it as
$$Cov(Y) = E(Cov(Y|X)) + Cov(E(Y_1|X), E(Y_2|X))$$
If $\text{Cov}(Y)$ denotes the covariance matrix of $Y$, then I think your first equation is fine. Since $E[Y \mid X]$ is a random vector, $\text{Cov}(E[Y\mid X])$ is another covariance matrix, which is fine.
You can check that it is true by checking each entry of the matrix. The equality of the off-diagonal entries is true by the law of total covariance. The equality of the diagonal entries is true by the law of total variance.