Is it the case that $A\mathbf{z}\mathbf{z}^T A = A^2\mathbf{z}\mathbf{z}^T$, given $A$ is a positive definite matrix? Are there any circumstances this can be true?
I've been trying to work through a derivation that includes a term that appears to assume this, but I can't get my head around the steps to do this rearrangement.
[For background, its seen during the derivation of maximum likelihood of the covariance matrix in a normal distribution]
Let us first establish this:
For two non-zero vectors $\mathbf{u}$ and $\mathbf{v}$, $\mathbf{u}\mathbf{v}^T=\mathbf{v}\mathbf{u}^T$, if and only if $\mathbf{u}=\lambda\mathbf{v}$ for some $\lambda$.
This is obvious if you notice that the rows of $\mathbf{u}\mathbf{v}^T$ are vectors parallel to $\mathbf{v}$, while the rows of $\mathbf{v}\mathbf{u}^T$ are vectors parallel to $\mathbf{u}$, thus for them to be equal $\mathbf{u}$ must be parallel to $\mathbf{v}$ i.e. $\mathbf{u}=\lambda\mathbf{v}$ for some $\lambda$. The converse is obvious.
Coming back to the problem, firstly I'm assuming $A$ is symmetric (there are definitions of positive definiteness without this condition as well). Now if $A$ is positive definite, then it is certainly non-singular, and thus has an inverse and we can pre-multiply to equivalently obtain: $$A\mathbf{z}\mathbf{z}^TA=A^2\mathbf{z}\mathbf{z}^T\iff \mathbf{z}\mathbf{z}^TA=A\mathbf{z}\mathbf{z}^T$$ If we denote $\mathbf{y}=A\mathbf{z}$, then we need to consider when we can have, since $A$ is symmetric: $$\mathbf{z}\mathbf{y}^T=\mathbf{y}\mathbf{z}^T$$ But as we saw, this means either the trivial conditions where $\mathbf{z}=\mathbf{0}$ or $\mathbf{y}=\mathbf{0}$, of which the latter is not possible since that would imply that $\mathbf{z}$ is in the nullspace of $A$ but that is empty. Thus from what we saw this means that $\mathbf{z}$ and $A\mathbf{z}$ are parallel. Thus: $A\mathbf{z}=\lambda\mathbf{z}$ for some $\lambda$. Thus, $\mathbf{z}$ is one of the eigenvectors of $A$.
So concluding this holds if and only if $\mathbf{z}$ is either $\mathbf{0}$ or one of the eigenvectors of $A$