When I read the book Iterative Methods for Sparse Linear Systems, Second Edition, I get stuck with the following proof. The yellow highlight parts are the positions I have trouble to understand. Something maybe really obvious. But for me, it is not :(.
On
If $A$ is normal, left and right eigenvectors conincide.
The matrix $A$ is normal if $$\tag{1}A^*A=AA^*.$$ Let $(\lambda,v)$ be an eigenpair of $A$: $Av=\lambda v$. We show that $v^*A=\lambda v^*$ (that is, $v^*$ is the left eigenvector of $A$), or, equivalently, $A^*v=\bar{\lambda} v$ (that is, $v$ is the right eigenvector of $A^*$). Since $Av-\lambda v=0$, we have $$\tag{2} \begin{split} 0=\|Av-\lambda v\|_2^2 &= v^*A^*Av-\lambda v^*A^*v-\bar{\lambda}v^*Av+|\lambda|^2v^*v. \end{split} $$ Using (1) and (2), we have $$ \begin{split} \|A^*v-\bar{\lambda}v\|_2^2&=v^*AA^*v-\lambda v^*A^*v-\bar{\lambda}v^*Av+|\lambda|^2 v^*v\\ &=v^*A^*Av-\lambda v^*A^*v-\bar{\lambda}v^*Av+|\lambda|^2 v^*v =\|Av-\lambda v\|_2^2 = 0. \end{split} $$ Hence, $A^*v=\bar{\lambda}v$.
A simpler (but less elementary; probably referred to as "trivial" in the text) approach to show this is using the fact that a normal $A$ can be unitarily diagonalized: there is a unitary $V=[v_1,\ldots,v_n]$ and diagonal $\Lambda=\mathrm{diag}(\lambda_1,\ldots,\lambda_n)$ such that $A=V\Lambda V^*$. By examining the equivalent relation $AU=U\Lambda$ it follows that $\lambda_i$ are the eigenvalues of $A$ with the associated eigenvectors $v_i$. Since $A=V\Lambda V^*$, we have $A^*=(V\Lambda V^*)^*=V\bar{\Lambda}V^*$, that is, $A^*V=V\bar{\Lambda}$. This confirms what we shown above.
For the other direction, the key is to show that if the left and right eigenvectors coincide, then the eigenvectors form the basis of $\mathbb{C}^n$. You know that if $V$ is nonsingular such that $AV=VC$ and $A^*V=VD$ for diagonal $C$ and $D$ (that is, both $A$ and $A^*$ have the same full set of eigenvectors), then $$A^*AV=A^*VC=V(DC)$$ and $$AA^*V=AVD=V(CD)=V(DC)$$ and hence $$(A^*A-AA^*)V=0,$$ which (since $V$ is nonsingular) implies $A^*A=AA^*$ ($A$ is normal).
You already know that if the left and right eigenvectors coincide, they correspond to the same eigenvalues (or, in other words, the left eigenvectors are right eigenvectors of $A^*$ with the complex conjugate eigenvalues). Now the assumption (to arive to the contradiction) is that some of the eigenvalues $\lambda_i$ is defective (its algebraic multiplicity is larger than its geometric multiplicity) and has a non-trivial Jordan block. It follows that if $v_i$ is the associated eigenvector, there is a nonzero vector $w_i$ such that $$ A[v_i,w_i]=[v_i,w_i]\pmatrix{\lambda_i&1\\0&\lambda_i}, $$ that is, $$ (A-\lambda_i I)w_i=v_i. $$ (Of course, the dimension of the Jordan block can be larger than 2 but we are interested here only in the existence of the first generalized eigenvector.) Now by using (1.31) and (1.32) you arrive to the conclusion that $v_i$ is zero which contradicts the fact that it is an eigenvector.
Hence, there is no nontrivial Jordan block and the matrix is diagonalizable. Therefore, the eigenvectors for the basis of $\mathbb{C}^n$.
Technically, the first highlighted remark is not correct because it equates row vectors and columns vectors. However, the statement which is correct--and this is the one the author uses--is that $Ax=\lambda x$ iff $A^{H}x=\overline{\lambda}x$. That statement is correct.
Next, the use of the term elementary zero divisions is not defined. What he is referring to here is the Jordan canonical form. For a root $\lambda$ of the characteristic polynomial $p$ of $A$, the restriction of $A$ to a Jordan block associated with $\lambda$ has the property that there is a least $k$ for which $(A-\lambda I)^{k}x=0$ for the vectors $x$ associated with such a block, and $k$ is, of course, equal to the size of the block. The block has size 1 if $(A-\lambda I)x=0$ for the vectors $x$ associated with the block. By "no elementary zero divisor" the author means that $(A-\lambda I)^{k}x=0$ for some $k > 0$ iff $(A-\lambda I)x=0$, which is an interesting property of normal operators that forces all Jordan blocks to be diagonal blocks of size 1, and ultimately forces the Jordan form to be diagonal, which means a full basis of eigenvectors.
What the author does to prove to prove "no zero divisors" is to assume $(A-\lambda I)^{2}x=0$ for some $x$, and then proceeds to show that $(A-\lambda I)x=0$. And that's enough to do the job because $(A-\lambda I)^{k}x=0$ for $k > 2$ can be written as $(A-\lambda I)^{2}(A-\lambda I)^{k-2}x=0$ which then implies $(A-\lambda I)(A-\lambda I)^{k-2}x=0$; so you can repeat that argument as many times as you need to reduce to $(A-\lambda I)x=0$. And that's what eliminates all Jordan blocks except blocks of size 1, and gives you a diagonal Jordan form.
To prove the case for $k=2$, the author assumes $(A-\lambda_{i} I)^{2}u_{i}=0$ and sets $v_{i}=(A-\lambda_{i} I)u_{i}$, which is then an eigenvector with eigenvalue $\lambda_{i}$. The author shows that $v_{i}=0$ in order to obtain the desired result. First note that $Au_{i}=\lambda_{i} u_{i}+v_{i}$ where $Av_{i}=\lambda v_{i}$ and, hence, also $A^{H}v_{i}=\overline{\lambda_{i}}v_{i}$. Putting these pieces together yields $$ \begin{align} & (Au_{i},v_{i})=\lambda_{i}(u_{i},v_{i})+(v_{i},v_{i}) \\ & \;\;\;|| \\ & (u_{i},A^{H}v_{i}) = \lambda_{i}(u_{i},v_{i}). \end{align} $$ The inevitable conclusion is $(v_{i},v_{i})=0$ or $0=v_{i}=(A-\lambda_{i}I)u_{i}$. In other words, $$ (A-\lambda_{i})^{2}v_{i} = 0 \implies (A-\lambda_{i}I)v_{i}=0. $$