Need a second help in understanding a step in matrix representation of bounded linear operators.

Question

In completion to this question:

Need A help in understanding a step in matrix representation of bounded linear operators.

The book said:

"Now, $$A \phi_{j} = \sum_{k}<A \phi_{j},\phi_{k}> \phi_{k}......(2),$$ Combining (1) (where (1) is $Ax = \sum_{j}<x,\phi_{j}> A\phi_{j}$) and (2) gives, $$Ax = \sum_{j} \sum_{k}<x,\phi_{j}> <A \phi_{j},\phi_{k}> \phi_{k} = \sum_{k} \sum_{j}<x,\phi_{j}> <A \phi_{j},\phi_{k}> \phi_{k}.$$ "

My professor said that the interchange of the summation sign in the last step is by using the conditional convergence and absolute convergence, could anyone explain this in details for me please?

Thanks!

Answer 1

We do agree that:

$Ax=\sum_{k}\langle x,\phi_k\rangle A\phi_k$

now we can take again its prokections on the basis:

$\langle Ax,\phi_j\rangle=\sum_{k}\langle x,\phi_k\rangle \langle A\phi_k,\phi_j\rangle$

Again by definition:

$Ax=\sum_{j}\langle Ax,\phi_j\rangle\phi_j=\sum_{j}(\sum_{k}\langle x,\phi_k\rangle \langle A\phi_k,\phi_j\rangle)\phi_j$.

As you can see this is the correct way of proceed, and no need at all of interchanging sums of vectors.

In any case in order to interchange sums, for real numbers, you need to check the Tonelli condition (uniform convergence), see for istance http://www.math.ubc.ca/~feldman/m321/twosum.pdf.

Answer 2

Personally, I don't think the absolute convergence plays the role of exchanging the double sum, @Diesirae92 has explained part of the double sum. Let us look if the exchanging is okay:

First of all, @Diesirae92 wrote that

\begin{align*} Ax=\sum_{j}\sum_{k}\left<x,\phi_{k}\right>\left<A\phi_{k},\phi_{j}\right>\phi_{j}, \end{align*} now we proceed that

\begin{align*} \sum_{k}\sum_{j}\left<x,\phi_{k}\right>\left<A\phi_{k},\phi_{j}\right>\phi_{j}&=\sum_{k}\left<x,\phi_{k}\right>\sum_{j}\left<A\phi_{k},\phi_{j}\right>\phi_{j}\\ &=\sum_{k}\left<x,\phi_{k}\right>A\phi_{k}\\ &=Ax \end{align*} because

\begin{align*} Ax&=A\left(\sum_{k}\left<x,\phi_{k}\right>\phi_{k}\right)\\ &=A\left(\lim_{N\rightarrow\infty}\sum_{k=1}^{N}\left<x,\phi_{k}\right>\phi_{k}\right)\\ &=\lim_{N\rightarrow\infty}A\left(\sum_{k=1}^{N}\left<x,\phi_{k}\right>\phi_{k}\right)\\ &=\lim_{N\rightarrow\infty}\sum_{k=1}^{N}A\left(\left<x,\phi_{k}\right>\phi_{k}\right)\\ &=\lim_{N\rightarrow\infty}\sum_{k=1}^{N}\left<x,\phi_{k}\right>A\phi_{k}\\ &=\sum_{k}\left<x,\phi_{k}\right>A\phi_{k}, \end{align*} in other words, the exchange of the double sum goes through. Anyway, the above reasoning has nothing to do with absolute convergence.

Rather, if $A=I$ the identity operator, then

\begin{align*} \sum_{k,j}|\left<x,\phi_{j}\right>|\cdot|\left<A\phi_{j},\phi_{k}\right>|\cdot\|\phi_{k}\|&=\sum_{k,j}|\left<x,\phi_{j}\right>|\cdot|\left<\phi_{j},\phi_{k}\right>|\cdot\|\phi_{k}\|\\ &=\sum_{k,j}|\left<x,\phi_{j}\right>|\cdot|\left<\phi_{j},\phi_{k}\right>|\\ &=\sum_{j}|\left<x,\phi_{j}\right>|\cdot\|\phi_{j}\|^{2}\\ &=\sum_{j}|\left<x,\phi_{j}\right>|, \end{align*} if it were absolutely convergent, then \begin{align*} \sum_{j}|\left<x,\phi_{j}\right>|<\infty, \end{align*} this is not true in general, take for example in $L^{2}[-2,2]$ with basis $\{1,\cos(n\pi x)/2,\sin(n\pi x)/2\}_{n}$, the function $x$ has expansion $\dfrac{4}{\pi}\displaystyle\sum_{n=1}^{\infty}\dfrac{(-1)^{n+1}}{n}\sin\left(\dfrac{n\pi x}{2}\right)$, so the coefficients are $\dfrac{4}{\pi}\dfrac{(-1)^{n+1}}{n}$ which their absolute sum is not convergent.

Answer 3

As already stated in other answers the interchange of sums do not need a closer study of the type of convergence. But there is some sense in what your professor said about absolute convergence (see below), and the statement is in fact quite non-trivial (see the final remarks).

Let us write $x_j=\langle x, \phi_j \rangle$ and $P_n x = \sum_{j=1}^n \langle x, \phi_j \rangle \phi_j = \sum_{j=1}^n x_j \phi_j$ for the projection onto the span of the first $n$ basis vectors. Also let $a_{jk} = \langle A \phi_j , \phi_k \rangle$ denote the matrix elements of $A$. For any $x\in H$, we have $P_n x \stackrel{n}{\rightarrow} x$ in $H$ (i.e. $P_n$ converges strongly to the identity operator). For finite $n,m$: $$ P_m A P_n x = \sum_{k= 1}^m \langle A P_n x, \phi_k \rangle \phi_k = \sum_{k= 1}^m \sum_{j=1}^n x_j a_{jk} \phi_k. $$ Since $A$ is continuous we get by taking the $m\rightarrow \infty$ limit first: $$ P_m A P_n x \stackrel{m}{\rightarrow} A P_n x \stackrel{n}{\rightarrow} A x = \lim_{n\rightarrow \infty} \sum_{j=1}^n x_j \left( \lim_{m\rightarrow \infty} \sum_{k= 1}^m a_{jk} \phi_k \right) $$ On the other hand, $\|P_m A\|\leq \|A\|$ so by continuity of $P_m A$: $$ P_m A P_n x \stackrel{n}{\rightarrow} P_m A x \stackrel{m}{\rightarrow} A x = \lim_{m\rightarrow \infty} \sum_{k=1}^n \left( \lim_{n\rightarrow \infty} \sum_{j=1}^n x_j a_{jk} \right) \phi_k $$ So you have your stated identity. On the other hand the last expression has the advantage that the convergence in the paranthesis for fixed $k$ is absolute. To see this note that: $$ \left| \sum_{j=1}^n x_j a_{jk} \right| = \left| \langle A P_n x, \phi_k\rangle \right| \leq \|A\| \; \|P_n x\| \leq \|A\|\; \|x\|. $$ But $\|x\|$ is invariant if we replace $x_j$ by $x_j e^{i\theta_j}$ for any real $\theta_j$'s, so in particular, when chosing $\theta_j$ so that each $x_j a_{jk}$ becomes real and non-negative. Therefore, we get the bound on the sum of absolute values: $$ \sum_{j=1}^n |x_j|\; |a_{jk}| \leq \|A\|\; \|x\|. $$ This bound then also holds when $n\rightarrow \infty$, so, as stated, $ y_k = \sum_{j=1}^\infty x_j a_{jk} $ is absolutely convergent and we have: $$ Ax = \sum_{k\geq 1} y_k \phi_k = \sum_{k\geq 1}\left( \sum_{j\geq1} x_j a_{jk} \right) \phi_k $$ with absolute convergence in the inner sum.

Note, however, that the convergence of the double sum in general is conditional, i.e. depends upon the order in which you sum over couples $(k,j)$. More precisely, with $H=\ell^2({\Bbb N})$, you may construct (a bit too involved, so I omit details) an operator $A\in L(H)$ of operator norm one, and a vector $x\in H$ of norm one with the following property: For any $R<+\infty$ you may find a finite subset $\Omega\in {\Bbb N} \times {\Bbb N}$ such that: $$ \| \sum_{(k,j)\in \Omega} x_j a_{jk} \phi_k \| > R .$$ This may sound rather weird, but shows that your identity can be very sensitive to the way you perform the double sum. Nevertheless, summing first over one of the indices, then over the other, you do have convergence and the limit is the same.