What algorithms or references can I look into in order to find a transformation $T$ that minimizes the norm of a given matrix $A$ with no particular properties?
$$\min_T \| T A T^{-1} \|$$
The norm I want to minimize is the $2$-norm, but the Frobenius norm also works because it bounds (tightly) the $2$-norm. Thus, it should yield similar $T$. Matrix $A$ is real in some applications and imaginary in others. Either solution would help.
The set of all invertible matrices is not compact. There is no guarantee that a minimum exists, but you may look at the infimum instead. We shall show that $\inf_T\|TAT^{-1}\|_2=\rho(A)$ and $\inf_T\|TAT^{-1}\|_F=\sqrt{\sum_i|\lambda_i(A)|^2}$.
It is clear that $\|TAT^{-1}\|_2\ge\rho(TAT^{-1})=\rho(A)$. Also, since Frobenius norm is unitarily invariant and every complex matrix is unitarily triangulable, $\|TAT^{-1}\|_F$ is always bounded below by $\sqrt{\sum_i|\lambda_i(A)|^2}$.
Now, suppose $T_1AT_1^{-1}$ is the Jordan normal form of $A$. Let $T_2=\operatorname{diag}(1,k,k^2,\ldots,k^{n-1})$ where $k>0$. When $k\to+\infty$, $T_2(T_1AT_1^{-1})T_2^{-1}\to\operatorname{diag}(\lambda_1,\ldots,\lambda_n)$. Therefore, its induced $2$-norm approaches $\rho(A)$ and its Frobenius norm approaches $\sqrt{\sum_i|\lambda_i(A)|^2}$. Hence the two lower bounds are infima.