I have a hard time understand the following statements below about PCA (normed or not normed).
a) the matrix to diagonalize is the matrix of linear correlations of original variables.
b) An illustrative variable is well represented on a factorial axis if its contribution is high on this axis.
It would be really helpful if someone could explain.
I suggest you read the chapter for the Singular Value Decomposition (SVD) in the book 'Linear Algebra' of Gilbert Strang. There's no other book that explains it more intuitively and clearly. The SVD and PCA have the same purpose in practical applications: Identifying the directions in which the data varies the most. In the end one tries to describe (approximate) the data with fewer dimensions by leaving out the dimensions that don't vary a lot.
I can't explain the whole SVD in detail. But here's the big picture: We'd like to decompose $A$ into $A=U\Sigma V^T$, where $U$ and $V$ have orthogonal columns and $\Sigma$ is just a diagonal matrix.
We know that $A^\top A$ is symmetric and thus diagonalizable. Since $$ A^\top A = (U\Sigma V^\top)^\top U\Sigma V^\top =V\Sigma U^\top U\Sigma V^\top = V\Sigma^2 V^T, $$ by diagonalising $A^\top A$ and orthonormalising the eigenvectors of the decomposition we can get $V$ and $\Sigma$ from this process. The eigenvalues of $\Sigma$ are usually sorted in decreasing order. We then just use $A=U\Sigma V^\top \Longleftrightarrow AV=U\Sigma$ in order to determine $U$.
See also:
https://en.wikipedia.org/wiki/Singular_value_decomposition (Has some good pictures) https://en.wikipedia.org/wiki/Principal_component_analysis