Let $A$ be a stable matrix (that is, $\rho(A) <1$) and $B$ be a symmetric positive semi-definite matrix. Now, if we define $$ f(N) = \sum_{i=1}^N \mbox{tr} \big((A^{i-1})' B A^{i-1} \big), $$ can we show that $f(N)$ grows with $\sqrt{N}$ rate?
My incomplete solution: \begin{align} f(N) &= \sum_{i=1}^N \mbox{tr} \big(B A^{i-1}(A^{i-1})' \big)\leq \sum_{i=1}^N \mbox{tr} \big(B \big) \mbox{tr} \big(A^{i-1}(A^{i-1})' \big) \\ & \leq trace \big(B \big) \sum_{i=1}^N \mbox{tr} \big(A^{i-1}(A^{i-1})' \big) \\ & \leq \mbox{tr} \big(B \big) \sum_{i=1}^N 0.5 \times [ \mbox{tr} \big(A^{i-1} A^{i-1}\big) + \mbox{tr} ((A^{i-1})' (A^{i-1})' ) ] \\ & \leq \mbox{tr} \big(B \big) \sum_{i=1}^N \mbox{tr} \big(A^{2i-2} \big) \end{align} The first inequality is correct because $B$ and $A^{i-1}(A^{i-1})'$ are both positive semi-definite and the third inequality is correct because $2\mbox{tr}(AB) \leq \mbox{tr}(A^2) + \mbox{tr}(B^2)$. Any idea for how should we continue with $\mbox{tr} \big(A^{2i-2}\big) $?
If we cannot say something when $A$ is a stable matrix, what if we know that $A$ is a strongly stable matrix (that is, $||A|| <1$)?
Because $B$ is positive semidefinite, we can write $B = \sum_{k=1}^r x_k x_k^T$ (where $r$ is the rank of $B$). Thus, we have $$ f(N) = \sum_{k=1}^r \sum_{i=1}^N \|(A^{i-1})'x_k\|^2 $$ Let $C = A'$, and let $\rho$ denote the spectral radius of $C$. Notably, we have $\rho < 1$. For all $x \in \Bbb R^n$, we have $$ \|C^{i-1}x\| \leq \|C^{i-1}\| \|x\| \leq \alpha \rho^{i-1} \|x\| $$ for some $\alpha > 0$. Thus, we can write $$ f(N) = \sum_k \sum_i \|C^i x_k\|^2 \leq \sum_k \sum_i \alpha^2 \rho^{2(i-1)} \|x_k\|^2 = \alpha^2 \frac{1 - \rho^{2N}}{1 - \rho} \sum_{k=1}^r \|x_k\|^2 \\ = \alpha^2 \frac{1 - \rho^{2N}}{1 - \rho} \operatorname{trace}(B) \leq \frac{\alpha^2 \operatorname{trace}(B)}{1 - \rho} $$ So, we have a stronger statement: there is an $M > 0$ such that $f(N) \leq M$ for all $N$. That is, $f(N)$ is $O(1)$.