When can I transfer $(\cdot )^2$ to $(\cdot )(\cdot )^T$ in matrix form

Question

Following my own question When can I use the transformation:

$(Ux−U\mu_x)^2$ to $(Ux−U\mu_x)(Ux−U\mu_x)^T$

Is there a rule of thumb? Can I always substitute the two?

Answer 1

It boils down to the definition of $x^2$ where $x$ is a vector.

Suppose a vector, $x \in \mathbb{R}^3$, what do you mean by $x^2$?

Clearly it can't mean the regular matrix multiplication of $3 \times 1$ and $3 \times 1$ as the size is not compatible.

For a random vector $X$, we define the covariance matrix, $\Sigma$, to be the matrix where the $(i,j)$-entry satisfies $\Sigma_{i,j} = \mathbb{E}[(X_i-\mu_i)(X_j-\mu_j)]$. Of which, from this definition, we can prove that

$$\Sigma = \mathbb{E}\left((X-\mathbb{E}(X))(X-\mathbb{E}(X)^T \right).$$

When $X \in \mathbb{R}^\color{red}1$, this formula, collapse to $$\Sigma = \mathbb{E}((X-\mathbb{E}(X))(X-\mathbb{E}(X))^\color{red}T)\Sigma = \mathbb{E}((X-\mathbb{E}(X))(X-\mathbb{E}(X)))=\mathbb{E}((X-\mathbb{E}(X))^2)$$

since for a scalar, its transpose is equal to itself.

However, for higher dimension, as discussed earlier, the regular multiplication of two column vectors are not defined.

I would view it as an abuse of notation unless you explicitly define it such that

$$\color{red}{v^2=vv^T}$$

and if that is the case, the answer is "always" since you defined it to be so. I personally discourage such practice. If everyone casually define stuff unnecessarily, we will end up with more confusion rather than clarity.