I have been learning about singular value decomposition from http://www.ams.org/publicoutreach/feature-column/fcarc-svd and they say that orthongoal vectors in the domain are mapped to orthogonal vectors in the co-domain. For the 2D case we can image a sphere being deformed by a matrix transformation to an ellipse. Then these domain vectors paticularly map to the major and minor axis of the ellipse.
These vectors in the domain turn out to be the eigenvectors of $A^T A$. Should'nt the eigenvector of A with the highest eigenvalue be the vector that is turned to the major axis of the ellipse? As it is stretched the most of any vector it would be the major axis of the ellipse I suppose. In this case that vector must be an eigenvector of $A$ and $A^T A$. The other eigenvector may not be transformed to the minor axis due to the angle it makes with this one. Can someone verify or expand upon my intution?
Huh, why? The matrix $A$ may not possess any eigenvector in the first place. Even if it has a real and complete eigenbasis, the eigenvector corresponding to the largest-sized eigenvalue is stretched the most by $A$ only among all eigenvectors, but it can happen that some other vector that is not an eigenvector is stretched even further.
For instance, suppose $A=\pmatrix{\epsilon_1&1\\ 0&\epsilon_2}$ for some very small $\epsilon_1>\epsilon_2>0$. Let $e_1=(1,0)^T$ and $e_2=(0,1)^T$. Then the eigenvector corresponding to the largest-sized eigenvalue is $e_1$ but $\|Ae_1\|=\epsilon_1\ll1\approx\|Ae_2\|$.
In general, the vector that is stretched the most by $A$ is a singular vector corresponding to the largest singular value, not an eigenvector corresponding to the largest-sized eigenvalue. The largest singular value of $A$ is always bounded below by (and is often strictly greater than) the spectral radius of $A$. In fact, $\|A\|\ge\rho(A)$ for every submultiplicative matrix norm $\|\cdot\|$, including the induced $2$-norm $\|A\|_2=\sigma_1(A)$.
From another perspective, if $A$ and $A^TA$ share an eigenvector for some nonzero eigenvalue, $A$ and $A^T$ must share the same eigenvector as well. In fact, if $Av=\lambda v\ne0$ and $A^TAv=\mu v$, then $$ A^Tv=A^T\left(\frac1\lambda Av\right)=\frac1\lambda A^TAv=\frac{\mu}{\lambda}v. $$ Yet, there is no reason to believe that $A^T$ and $A$ in general will share any eigenvector at all. E.g. if $$ A=\pmatrix{1&1\\ &\ddots&\ddots\\ &&\ddots&1\\ &&&1}, $$ then the only eigenspace of $A$ is the span of $(1,0,\ldots,0)^T$ but the only eigenspace of $A^T$ is the span of $(0,\ldots,0,1)^T$. (This counterexample actually works over any field, not just $\mathbb R$.)