Motivation of eigenvalues and eigenvectors

Question

I have been reading a book (Linear Algebra, Michel Queysanne) in which there is an explanation about the motivation of defining eigenvalues and eigenvectors.

I was wondering if anyone could explain this in a clearer way.

Basically, the author argues that we first need to find a matrix for an endomorphism f which is associated to a basis $(a_1,a_2, \dots, a_n)$. Then, from this point on, I get completely lost. The author mentions a matrix $A$ which is made of $4$ blocks where one is $$A’= M(g, a_i) \qquad(1\le i\le n),$$ but no explanation about the other three blocks is provided. Then he says $g$ is the linear map induced by $f$ on $F$ (which is an eigensubspace) Why do we have to talk about a function g now, and what is with induced linear maps?

Later, he talks about a family of stable subspaces $f$ through $f$ such that $E$ (vector space) equals the direct sum of $F_1, \dots, F_m.$

Finally, he gathers the bases of $F_1, \dots, F_m.$ in a diagonal matrix $A =M(f,a_i)$. Why is it diagonal? He makes the point that the matrices $A_j$ of order strictly lower than that of $M(f)$ have to be simpified. He then connects this fact to homotecic transformations, i.e. $x\mapsto cx$ for some scalar $c$.

Could some shed some light on this topic please?

Answer 1

Unfortunately, I can't really understand your actual questions about your book, they are pretty heavily stripped of context. For example, I have no idea what the definition of $A'$ means. So I will just give you a general discussion. I will choose to use some algebraic terminology like "endomorphism" instead of "square matrix" because you did so in your question. If you're more comfortable with more elementary terminology then I can change it.

As a rule, in math we want to take a general thing and do the best we can to reduce it to a simple thing. In the case of eigenvalue/eigenvectors, the general thing is "linear transformations" and the simple thing is "scalar multiplication". Thus we want to reduce linear transformations to scalar multiplication. This is kind of a non-starter in the rectangular case (scalar multiplication is an endomorphism), so we look at endomorphisms.

Even in this case, we can't just perform the reduction, linear transformations are more complicated than that. The trick is to find restrictions of our linear transformation to subsets of their domain on which they act like scalar multiplication. That is we look for subsets of the domain where $Ax=\lambda x$. These subsets are called eigenspaces; their nonzero elements are called eigenvectors; the corresponding scalars are called eigenvalues. We call them eigenspaces because they are vector subspaces, as is easily checked from the definition.

Our original endomorphism is now obtained on the direct sum of all its eigenspaces by the requirement of linearity: for $x_i$ eigenvectors and $c_i$ scalars, you have $A(\sum_{i=1}^n c_i x_i)=\sum_{i=1}^n c_i \lambda_i x_i$. Hopefully, the direct sum of all the eigenspaces is the whole domain. In this case we say the endomorphism is diagonalizable, because it can be written as a diagonal matrix if we choose an eigenvector basis (a union of bases of each eigenspace). If the endomorphism is not diagonalizable, then we are not done, there are still some parts of the domain where we don't understand how $A$ works. We sometimes choose to get out of this by passing to the Jordan form.

Answer 2

When you hear for the first time the words eigenvectors and eigenvalues it might sound like something really abstract and from another world. But when you discover when and where they are used in practice, everything’s gets clearer.

Well, I will tell you where they are used in practice: first of all in linear isometries. Linear isometries are any kind of transformations, movements and motions. Given a matrix, you can’t know which movement it is without knowing the eigenvects and eigenvals (you can read about the Spectral Theorem for Isometries).

Eigenvectors are just specific subspaces that have some characteristics (for example, a subspace of fixed points of eigenval 1, the direction of the projection of eigenval 0, the direction of the symmetry eigenval -1).

Any movement. It is really used for example in 3D video games, though quaternions are also really used in game industry for rotations, for example. I hope this motivates to understand eigen....

Answer 3

There’s a very simple geometric question that can motivate introducing eigenvectors and eigenvalues:

Are there any lines through the origin that are mapped to themselves by the linear transformation $T$?

Every vector on such a line gets mapped to some scalar multiple of itself and since the transformation is linear, this scalar must be the same for every vector on the line. So, finding invariant lines becomes a problem of finding vectors $\mathbf v$ and scalars $\lambda\ne0$ such that $T(\mathbf v)=\lambda\mathbf v$. This equation of course has the trivial solution $\mathbf v=0$, but that doesn’t generate a line, so we sharpen this condition up a bit and require that $\mathbf v\ne0$.

From this simple beginning, we can generalize and expand the question. Allowing $\lambda=0$, for instance, finds lines that are collapsed to the origin by $T$. We can also ask if there are higher-dimensional subspaces that are invariant under $T$, and so on.

Answer 4

The way I understand eigen values and eigen vectors are via the problem of diagonalization.

We say a square matrix $A$ of dimension $n \times n$ is said to be diagonizable if there exists an invertible matrix $P$ such that $P^{-1}AP= \Lambda$, where $\Lambda$ is a diagonal matrix of same dimension that of $A$.

Now suppose we assume that $A$ is diagonizable, then by definition there exists an invertible matrix $P$ such that $AP= \lambda P$.

Let the columns vectors of $P$ be represented as $(p_1 \space p_2 \space p_3 \dots p_n)$ and the $n$ scalars associated with the diagonal matrix be $\lambda_1,\lambda_2 \dots \lambda_n$. then $AP=(Ap_1 \space Ap_2 \dots Ap_n)= (\lambda_1 p_2 \space \lambda_2 p_2 \dots \lambda_n p_n)$.

This then forces that $Ap_j= \lambda_j p_j$. Thus for a matrix to be diagonizable we need to find $n$ pairs of scalars $\{\lambda_j \}$ and $n$ linearly independent vectors $p_1,p_2,\dots p_n$ such that $Ap_j=\lambda_jp_j$. Those set of scalars are then the $\textit{eigen values}$ of the matrix $A$ and the $n$ linearly independent vectors are the $\textit{eigen vectors } $associated with each of the eigen values of $A$. If we could find them, then the matrix is diagonizable.

Question as how to find them?

We know that each $\textit{eigen pair}$, $(\lambda,p)$, if they exists, satisfies the condition $Ap =\lambda p$. This then would imply that $(A-\lambda I)p=0$. Since $p$'s are linearly independent, $p \neq 0$ and hence, $A_{\lambda}:=A-\lambda I$ is not invertible. Thus $\det A_\lambda=0$ which sets up the characteristic equation.

$\det A_\lambda= \prod \limits_{i=1}^k(\lambda-\lambda_i)^{a_i}$ such that $\sum a_i=n$. Setting $\det A_\lambda=0$ fetches the $n$ eigen values. Now note that associated with each eigen value is a matrix $A_\lambda=A-\lambda I$. To get the eigen vector we simply find the null space of $A_\lambda$. From the theory we know that null space forms a subspace, hence for an eigen value, the eigen vectors are not unique.

But what's interesting is that, associated with eigen value is a subspace $W_j= Ker(A_{\lambda_j})$. Since there are $k$ distinct eigen values we have $k$ distinct subspaces. Suppose we form a set which is union of the basis vectors for each of this subspaces then it can be shown that if they are $n$ linearly independent vectors then they indeed diagonalize the matrix $A$.

The number of times a root repeats itself is called the algebraic multiplicity of the eigen value and the dimension of the subspace $W_j$ is called the geometric multiplicity.

Now suppose that algebraic multiplicity equals the geometric multiplicity then $\sum dim W_j =n$ and hence we are assured with $n$ linearly independent vectors $p_1,p_2,\dots p_n$ which then satisfies the condition $P^{-1}AP = \Lambda$.

As an example, the class of matrices for which algebraic multiplicity = geometric multiplicity are the class of Hermitian matrices in the complex case and the symmetric matrices in the real case. However, there do exist "many" matrices which aren't Hermitian(or symmetric) yet tend to be diagonizable.