Why does $(I-A)$ has inverse when $\|A \| < 1$

Question

I've seen a proof using the convergence of numerical series, but it is too large, could you please tell me if there is a shorter proof. Thank you!

Answer 1

When $\|A\|<1$, the series $\sum_{n=0}^\infty A^n$ converges to $(I-A)^{-1}$. Indeed, $$ (I-A)\sum_{n=0}^\infty A^n = \sum_{n=0}^\infty A^n - \sum_{n=0}^\infty A^{n+1} = A^0 = I, $$ and similarly for right-multiplication by $(I-A)$.

Answer 2

Spectral radius of a matrix $\rho(A)$ is a lower bound for matrix norms induced by vector norms. Hence, $1>||A||>\rho(A)$ for $||\cdot||$ a matrix norm induced by a vector norm. Note that, the eigenvalues of $I-A$ are the eigenvalues of $-A$ shifted by $1$. Since, we know that the largest (in magnitude) eigenvalue of $A$ is less than $1$, all eigenvalues of $I-A$ are nonzero. Therefore, $I-A$ is invertible.

Answer 3

We need the norm to be submultiplicative, i.e. $\|XY\|\le\|X\|\|Y\|$ for every pair of matrices $X$ and $Y$, otherwise the statement isn't true. For instance, let $\epsilon>0$ and define a matrix norm $\|X\|=\epsilon\sum_{i,j}|x_{ij}|$ for real matrices. Then $\|I\|<1$ when $\epsilon$ is sufficiently small, but $I-I=0$ is not invertible.

Suppose $\|A\|<1$ for some submultiplicative matrix norm $\|\cdot\|$. If $I-A$ is singular, then $(I-A)x=0$ for some nonzero vector $x$. Therefore $\|xx^T\|=\|Axx^T\|\le\|A\|\|xx^T\|<\|xx^T\|$, which is a contradiction.

Answer 4

Your matrix $A$ presumably comes from a ring $\mathscr{M}_n(\mathbb{C})$ of complex square matrices of order $n \in \mathbb{N}$, so the identity matrix in question should also be properly denoted as $\mathrm{I}_n$.

The result you ask is based on essential properties of the norm and of the metric space it induces. In a very general setting, you have the notion of a complex Banach algebra, which is a normed algebra $$(A, +, \cdot, \cdot, \lVert \cdot \rVert)$$ (the first multiplicative symbol denotes the internal multiplication of the algebra, the second one the external action of $\mathbb{C}$ on $A$) such that the metric induced by the norm is complete. In full detail, this means that the following axioms must be satisfied:

1. $(A, +, \cdot, \cdot)$ is a complex algebra.

2. The map $\lVert \cdot \rVert: A \to [0, \infty)$ is a norm on the complex vector space $(A, +, \cdot)$, in other words the relations:

   i) $(\forall x)(x \in A \wedge \lVert x \rVert=0 \Rightarrow x=0_A)$ (definiteness)

   ii) $(\forall x, y)(x, y \in A \Rightarrow \lVert x+y \rVert \leqslant \lVert x \rVert + \lVert y \rVert)$(subadditivity)

   iii) $(\forall \lambda, x)(\lambda \in \mathbb{C} \wedge x \in A \Rightarrow \lVert \lambda x \rVert =|\lambda| \lVert x \rVert)$ (absolute homogeneity)

must hold.

3. The norm must also be compatible with the internal multiplication in the following sense:

$$(\forall x, y)(x, y \in A \Rightarrow \lVert xy \rVert \leqslant \lVert x \rVert \lVert y \rVert)$$ (submultiplicativity)

rendering our structure into a normed algebra.

4. The norm is complete (i.e. the metric induced by the norm is complete).

Let us agree on the following notations:

• given metric space $(X, d)$, element $x \in X$ and $\rho \in (0, \infty)$ we write $\mathrm{B}_{\rho}(x)=\{y \in X|\ d(x,y)<\rho \}$ for the open ball of radius $\rho$ centred at $x$
• given arbitrary ring $A$, we write $0_A, 1_A$ for its zero and unity and respectively $\mathrm{U}(A)=\{x \in A|\ (\exists y)(y \in A \wedge xy=yx=1_A)\}$ for its set of invertible elements (also called units of the ring).

With this in place we can state the following general result (for simplicity, we will resort to an abuse of notation and refer to an algebra just by indicating its support set, omitting explicit reference to the structure which we implicitly understand):

Proposition: Let $A$ be an associative unital Banach algebra (unital in the sense of having a multiplicative unity). Then $\mathrm{B}_{1}(1_A) \subseteq \mathrm{U}(A)$.

Proof: First of all, $A$ being associative and endowed with a unity does indeed become a ring under the internal operations. By translation it is clear that $\mathrm{B}_{1}(1_A)=1_A+ \mathrm{B}_{1}(0_A)$ so let us consider an arbitrary $x \in A$ such that $\lVert x \rVert < 1$ and let us show that $1_A-x$ is invertible; to this end we will explicitly produce the inverse element by considering the sequence of partial sums $$u_n=\sum_{k=0}^{n} x^k$$ We argue that the series $$\sum_{n=0}^{\infty} x^n \tag{gs}$$ is convergent since it is absolutely convergent with respect to a complete norm; this is indeed so as we have the estimate $\lVert x^n \rVert \leqslant \lVert x \rVert^n$ for any $n \in \mathbb{N}^*$ (inequality obtained by applying induction to axiom 3 above) and hence we can bound from above $$\sum_{n=1}^{\infty} \lVert x^n \rVert \leqslant \sum_{n=1}^{\infty} \lVert x \rVert^n=\frac{\lVert x \rVert}{1- \lVert x \rVert}$$ with a geometric series of strictly subunitary ratio (the ratio is obviously $\lVert x \rVert <1$). Therefore, the series (gs) converges or equivalently we have $$\lim_{n \to \infty}u_n=y$$ for a certain $y \in A$.

By elementary algebra we have that $$(1_A-x)u_n=u_n(1_A-x)=1_A-x^{n+1} \tag{rel}$$

Since in a normed algebra all algebraic operations (the two internal ones as well as the external one) are continuous, we can pass to the limit in the previous relation (rel), bearing in mind that $$x^n \xrightarrow {n \to \infty} 0_A$$ in order to obtain the relation $(1_A-x)y=y(1_A-x)=1_A$. $\Box$

This will apply in particular to the norm you are considering on your unspecified matrix algebra, as long as it renders this latter structure into a Banach algebra (in other words as long as the axioms listed above are satisfied).