Linear transformations for which the matrix is diagonalizable have been described as anisotropic scalings, presumably along orthogonal directions. So based on lots of web browsing to get the intuition behind this, I now picture such a transformation as rotating/flipping a vector until the original coordinate axes are aligned with the axes for anisotropic scaling, doing the scaling, then reversing the rotations/flippings to orient the vector back in the n-space of its original axes.
Is rotating/flipping the only transformations that can be used to diagonalize, or can the transformation be nonorthogonal? (I'm not talking about the anisotropic scaling itself, just aligning the input vector for scaling, and reverse transformation after the scaling).
Also, does the anisotropic scaling always occur along orthogonal axes? I realize that after the diagonlizing, the scaling is along orthogonal axes. But if we picture a cloud of points in n-space being subjected to an anisotropic scaling that corresponds to a diagonalizable matrix, will the directions of the anisotropic scalings also be orthogonal?
In general, the directions along which the action of a diagonalizable linear transformation is a simple scaling are not orthogonal. When they are, you can diagonalize the mapping with an orthogonal matrix, which represents a combination of rotations and reflections. In this situation your mental model of flipping the vector around, scaling and flipping it back is reasonable. In the general case, though, this mental model breaks down. A better way to imagine the transformation is as breaking down the vector into its components in these special directions, applying the appropriate scale factor to each component, and then reassembling them.
First, a brief digression to establish some vocabulary that you may already be familiar with. A question we might ask about a linear transformation $L$ is: are there any lines (through the origin) that are mapped to themselves by $L$? Linearity requires that, for all vectors $\mathbf v$ on such a line, $L(\mathbf v)=\lambda\mathbf v$ for some fixed scalar $\lambda$. A non-zero vector $\mathbf v$ that satisfies such an equation is called an eigenvector (characteristic vector) of $L$ and the corresponding scalar an eigenvalue (characteristic value). It should be fairly obvious that if $\mathbf v$ is an eigenvector with eigenvalue $\lambda$, then so is any non-zero scalar multiple of it. Thus, another way of characterizing an eigenvector of $L$ is that it specifies a direction in which the action of $L$ is scaling by some constant factor $\lambda$. It’s a simple matter to show that the set of eigenvectors corresponding to a particular eigenvalue $\lambda$ form a subspace, called the eigenspace of $\lambda$. Eigenspaces aren’t always one-dimensional. When an eigenvalue is repeated, the corresponding eigenspace might be a plane or higher-dimensional subspace.
When a linear transformation $L:V\to V$ is diagonalizable, there exists a basis of $V$ that consists entirely of eigenvectors of $L$. That is, one can find $\dim V$ different directions in which the action of $L$ is simple scaling by some constant factor $\lambda_k$. In the finite-dimensional case, the matrix of $L$ relative to this basis is the diagonal matrix $\Lambda$, which has the eigenvalues $\lambda_k$ as its main diagonal elements. Relative to some other basis, the matrix will be $B\Lambda B^{-1}$, where $B$ is the change-of-basis matrix that has the corresponding eigenvectors of $L$, expressed relative to this other basis, as its columns. In general, $B$ is not orthogonal. It’s always possible to normalize the basis vectors without affecting the above product, of course, but there’s no guarantee that they’ll be orthogonal (except in some important special cases).
You can view the action of $B^{-1}$ as breaking down a vector into its components in the directions of the eigenvectors. The matrix $\Lambda$ then scales each of these components by the appropriate $\lambda_k$ factor, and then $B$ reassembles the scaled vector. In fact, it’s possible to decompose a diagonalizable linear transformation into the sum of a set of scaled projections onto the distinct eigenspaces of $L$.
To illustrate, consider the linear transformation $L:\mathbb R^2\to\mathbb R^2$ given by the matrix $M=\tiny{\pmatrix{3&-2\\2&-2}}$ relative to the standard basis. This transformation has eigenvalues $2$ and $-1$ with eigenvectors $(2,1)^T$ and $(1,2)^T$, respectively. Their dot product is equal to $4$, so these vectors are not orthogonal. Thus, $L$ can be described as scaling by $2$ in the $(2,1)^T$ direction and reversing the $(1,2)^T$ direction. We compute $L(2,3)^T$ in two ways. Directly multiplying this vector by $M$ produces $(0,-2)^T$. Alternatively, apply $B^{-1}$ to the vector producing $\left(\frac13,\frac43\right)^T$, that is, decompose $(2,3)^T$ into $\frac13(2,1)^T+\frac43(1,2)^T$. The result of applying $L$ should then be $2\cdot\frac13(2,1)^T-\frac43(1,2)^T$, which is indeed also $(0,-2)^T$.
We can also decompose $L$ into a sum of scaled projections. Let $\lambda_1=2$ and $\lambda_2=-1$, the eigenvalues of $L$. Then define $$P_1={L-\lambda_2\operatorname{id}\over\lambda_1-\lambda_2} \\ P_2={L-\lambda_1\operatorname{id}\over\lambda_2-\lambda_1}.$$ The matrices of these transformations are $\frac13\tiny{\pmatrix{4&-2\\2&-1}}$ and $\frac13\tiny{\pmatrix{-1&2\\-2&4}}$, respectively. You can verify that $P_1^2=P_1$ and $P_2^2=P_2$ so that these are in fact projections, and that their images are the spans of $(2,1)^T$ and $(1,2)^T$, respectively. Moreover, $P_1(2,3)^T=\left(\frac23,\frac13\right)^T=\frac13(2,1)^T$ and $P_2(2,3)^T=\left(\frac43,\frac83\right)^T=\frac43(1,2)^T$, just as we had with $B^{-1}$. So, $L$ can be decomposed into the sum of scaled projections $\lambda_1P_1+\lambda_2P_2=2P_1-P_2$. The diagram below illustrates this decomposition with $v=(2,3)^T$.