Show that $\sum_{k=0}^\infty kA^k$ converges

Question

Let $\lVert\cdot\rVert$ be any matrix norm and let $A\in\mathbb{R}^{n\times n}$ be a matrix with $\lVert A\rVert < 1$. I'd like to show that $\sum_{k=0}^\infty kA^k$ converges, but I don't understand how to make use of the fact $\lVert A\rVert < 1$. Using the definition of a matrix norm, I get the following (useless) result. $$\left\lVert\sum_{k=0}^n kA^k\right\rVert \leq \sum_{k=0}^n k\left\lVert A\right\rVert^k = \frac{n\left\lVert A\right\rVert^{n+2} -\left(n+1\right)\left\lVert A\right\rVert^{n+1}+ \left\lVert A\right\rVert}{\left(1-\left\lVert A\right\rVert\right)^2}$$ I also looked into Cauchy's convergence test, but I had no success applying it. How can I prove convergence?

Answer 1

Let $V=\mathbb{R}^{n\times n}$ be the space of all $n\times n$ matrices with real entries, endowed with a matrix norm $\|\cdot\|$. We prove:

Theorem. Suppose $(X_k)_{k=1}^{\infty}$ is a sequence of matrices such that $\sum_{k=1}^{\infty} \|X_k\| < \infty$. Then $\sum_{k=1}^{\infty} X_k$ converges.

Proof. Let $S_k = \sum_{j=1}^{k} X_j$ be the partial sums. We need to prove that $(S_k)_{k=1}^{\infty}$ converges in $V$. Since $V$ is complete (in the sense that every Cauchy sequence converges in $V$), it suffices to show that $(S_k)_{k=1}^{\infty}$ is a Cauchy sequence. To make use of the assumption $\sum_{k=1}^{\infty} \|X_k\| < \infty$, we also introduce the partial sums $T_k = \sum_{j=1}^{k} \|X_j\|$. Then, for any $k < l$, we have

$$ \| S_l - S_k \| = \left\| \sum_{j=k+1}^{l} X_j \right\| \leq \sum_{j=k+1}^{l} \|X_j\| = T_l - T_k. $$

Since $T_l - T_k \to 0$ as $l, k \to \infty$, it follows that $(S_k)_{k=1}^{\infty}$ is indeed Cauchy and hence convergent. $\square$

Using this theorem, it is now straightforward to show that $\sum_{k=0}^{\infty} k A^k$ converges for $A$ with $\|A\| < 1$.

---

Addendum. In fact, the property of "absolutely convergent series being convergent series" is the characterizing property of Banach space:

Theorem. Let $V$ be a normed vector space endowed with a norm $\|\cdot\|$. Then the followings are equivalent:

1. $V$ is a Banach space, i.e., it is a complete normed space.
2. In $V$, every "absolutely convergent" series converges. More precisely, for any sequence $(x_k)$ in $V$ the following implication holds: $$ \text{if} \quad \sum_{k=1}^{\infty} \|x_k\| < \infty, \qquad \text{then} \quad \sum_{k=1}^{\infty} x_k \quad \text{converges.} $$

See the related article in Wikipedia, for instance.