If $G\in \mathscr M_n(\mathbf C)$ then it's well known that $\lim_{k\to \infty}\|G^k\|^{1/k}=\rho(G)$ where $\rho(G)$ is the spectral radius of $G$, the value of the limit does not depend on the choosen norm. Consequently if we take the Schur norm we get $$ \lim_{k\to \infty} \mathrm{Tr}(G^k{G^*}^k)^{1/2k}=\rho(G). $$ Now if $M$ is an hermitian positive definite matrix then $\|A\|_M=\sqrt{\mathrm{Tr}(AMA^*)}$ is still a norm on $\mathscr M_n(\mathbf C)$ and so we also get $$ \lim_{k\to \infty} \mathrm{Tr}(G^kM{G^*}^k)^{1/2k}=\rho(G). $$ I would like to know what happen when $M$ is only hermitian positive semi-definite. More precisely my question is the following.
Question : Let $G\in \mathscr M_n(\mathbf C)$ be an invertible matrix and let $M$ be an hermitian positive semi-definite matrix with $\mathrm{Tr}(M)=1$. Is it true that the limit $$\lim_{k\to \infty}\mathrm{Tr}(G^kM{G^*}^k)^{1/2k}$$ exists and is always equal to the modulus of an eigenvalue of $G$ ?
I know that the $\lim \inf$ is larger than the smallest singular value and that the $\lim\sup$ is smaller than the largest singular value.
Here is a slightly different answer based on several ideas of loup blanc. I think that the advantage of this answer is that we chose to decompose the vectors over a Jordan basis of $G$ which does not depend on $k$.
Let $(e_1,\ldots,e_n)$ be the canonical basis of $\mathbf C^n$ and $(\cdot, \cdot)$ be the standard Hermitian product. We then have $\mathrm{Tr}(A)=\sum_{i=1}^n(Ae_i,e_i)$ for every hermitian matrix $A$. If $P$ is a unitary matrix then $$\mathrm{Tr}(A)=\mathrm{Tr}(P^*AP)=\sum_{i=1}^n(P^*APe_i,e_i)=\sum_{i=1}^n(APe_i,Pe_i).$$ So for any orthonormal basis $(f_1,\ldots, f_n)$ and any hermitian matrix $A$ we have $$\mathrm{Tr}(A)=\sum_{i=1}^n(Af_i,f_i).$$ Now since the matrix $M$ is hermitian positive semi-definite there exists a matrix $N$ which is hermitian positive semi-definite such that $NN=M$. Consequently $$\mathrm{Tr}(G^kM{G^k}^*)=\mathrm{Tr}(G^kNN{G^k}^*)=\mathrm{Tr}((G^kN)^*G^kN)=\sum_{i=1}^n(G^kNf_i,G^kNf_i)$$ where $(f_1,\ldots,f_n)$ is any orthonormal basis. If we choose $(f_i)_i$ to be an orthonormal basis of eigenvectors of $N$ then we get $$ \mathrm{Tr}(G^kM{G^k}^*)=\sum_{i=1}^n\mu_i^2(G^kf_i,G^kf_i) = \sum_{i=1}^n\mu_i^2\|G^kf_i\|_2^2 $$ where $Nf_i = \mu f_i$ and $\|\cdot\|_2$ is the euclidean norm. We finally decomposes the $f_i$ over a Jordan basis for $G$ : $f_i=\sum \alpha_j^i v_j$ and compute the $G^kf_i$. If all the $v_j$ are eigenvectors then the computation is easy. If it's not the case then fix some integer $J$ and assume that $Gv_1 = \lambda_1 v_1 $ and $Gv_j = \lambda_1 v_j + v_{j-1}$ for every $j=2,\ldots,J$. We then have $$G^k v_J = \lambda_1^k v_J + \binom{k}{1}\lambda_1^{k-1}v_{J-1} + \ldots + \binom{k}{J-1}\lambda_1^{k-J+1}v_1=\lambda^{k-J+1}\binom{k}{J-1}(v_1+ o(1))$$ and for the same reason if $\beta_J \in \mathbf C \backslash \{0\} $ then $$ G^k(\beta_1 v_1 + \ldots + \beta_J v_J) = \beta_J\lambda_1^{k-J+1}\binom{k}{J-1}(v_1+ o(1)). $$ Consequently there exists some eigenvector $v_{i_1}$ of $G$, some non zero complexe number $\beta_{i_1}$ and some integer $N_{i_1}$ such that $$G^k f_1 = \beta_{i_1}\lambda_{i_1}^{k-N_{i_1}+1}\binom{k}{N_{i_1}}(v_{i_1}+o(1)).$$ This is enough to conclude using $\lim (|\lambda_1|^k+\ldots+|\lambda_n|^k)^{1/k}=\sup_{i=1,\ldots,n} |\lambda_i|$ and well known results on the comparative growth of functions :
$$ \lim_{k\to\infty} \mathrm{Tr}(G^kM{G^k}^*)^{1/2k}=|\lambda_i| $$ where $\lambda_i$ is some eigenvalue of $G$.