According to Wolfram Alpha, $\begin{bmatrix}3 & 1 \\ 5 & 2\end{bmatrix}^{-1}\begin{bmatrix}1 & 0 \\ 0 & 1\end{bmatrix}\neq \begin{bmatrix}3 & 1 \\ 5 & 2\end{bmatrix}^{-1}$.
Links to the two computations:
MatrixPower[{{3, 1}, {5, 2}}, -1] {{1, 0}, {0, 1}}
MatrixPower[{{3, 1}, {5, 2}}, -1]
To further confuse things, leaving out the inverse produces the expected result ($\begin{bmatrix}3 & 1 \\ 5 & 2\end{bmatrix}\begin{bmatrix}1 & 0 \\ 0 & 1\end{bmatrix}= \begin{bmatrix}3 & 1 \\ 5 & 2\end{bmatrix}$):
It turns out you need to explicitly add a period between them in the input, otherwise, it does elementwise multiplication. For some reason, it automatically adds this in the third link provided in the question.
MatrixPower[{{3, 1}, {5, 2}}, -1] . {{1, 0}, {0, 1}}