If I have two eigenvalue $\lambda_1$ and $\lambda_2$ and two associated normalized eigenvector $\mathbf e_1$ and $\mathbf e_2$ respectively, and I want to draw ellipse, How can I know which eigenvalue and eigenvector will construct the major axis and which one will be associated with minor axis ?
The ellipse looks like the following :
On
An intuition of why the large eigenvalue corresponds to the short axis.
Think of the graph of an ellipse as a contour curve of that quadratic form. If there are more contour curve plotted, they would be something like this:
(ref of the figure: https://users.oden.utexas.edu/~omar/inverse_probs/numopt.pdf)
Along the short axis, the contour curves are "denser", meaning that the quadratic function grows faster in this direction (compare to the direction of the other axis). So this must be the direction of larger eigenvalue.
The major axis corresponding to the small eigenvalue, while the minor axis corresponding to the big eigenvalue. Here is a simple explanation,
An eclipse can be thought of a section of quadratic form $x^T Ax$, i.e. $x^T Ax=1$. ($A$ must be a postive definite matrix) In 2-dimentional case, $A$ is a 2 by 2 matrix. Now factorize A to eigenvalue and eigonvector.
$$A=\begin{pmatrix}e_1&e_2\end{pmatrix}\begin{pmatrix}\lambda_1&\\&\lambda_2\end{pmatrix}\begin{pmatrix}e_1^T\\e_2^T\end{pmatrix}$$
Now the equation for the ellipse become,
$$x^T\begin{pmatrix}e_1&e_2\end{pmatrix}\begin{pmatrix}\lambda_1&\\&\lambda_2\end{pmatrix}\begin{pmatrix}e_1^T\\e_2^T\end{pmatrix}x=1$$
$$\lambda_1 x^T e_1 e_1^T x+\lambda_2 x^T e_2 e_2^T x=1$$
There comes the familar ellipse equation,
$$\frac{(e_1^T x)^2}{(\frac{1}{\sqrt{\lambda_1}})^2}+\frac{(e_2^T x)^2}{(\frac{1}{\sqrt{\lambda_2}})^2}=1$$
Assuming $\lambda_1$ is smaller, from the equation, we can see that eigonvector $e_1$ and $e_2$ are corresponding to the major and minor axis direction, eigenvalue $\frac{1}{\sqrt{\lambda_1}}$ and $\frac{1}{\sqrt{\lambda_2}}$ are corresponding to the length of major and minor axis.