Let $f:{\mathcal M}_2(\mathbb{R})\to{\mathcal M}_2(\mathbb{R}), f(X)= X- X^t.$ Find if $f$ is diagonalisable.
We take the canonic base in ${\mathcal M}_2(\mathbb{R})$, $B=\{{\begin{pmatrix}1&0\\0&0\end{pmatrix},\begin{pmatrix}0&1\\0&0\end{pmatrix},\begin{pmatrix}0&0\\1&0\end{pmatrix},\begin{pmatrix}0&0\\0&1\end{pmatrix}}\}$. Let's call them in order $E_1,E_2,E_3,E_4.$
Now to find $M_f$ in $B$. We have to do the following:
$$f(E_1)=\lambda_{1,1}E_1+...+\lambda_{1,4}E_4$$ $$...$$ $$f(E_4)=\lambda_{4,1}E_1+...+\lambda_{4,4}E_4$$
And $M_f$ is all those $\lambda$'s on columns. Well then $M_f$ will be:
$$M_f=\begin{pmatrix}0&0&0&0\\0&1&-1&0\\0&-1&1&0\\0&0&0&0\end{pmatrix}$$
So then to find it's eigenvalues we have to find it's characteristic polynomial, to do that, we have that:
$$P_{M_f}(x)=det(M_f-xI_4)=x^3(x-2).$$
So we have eigenvalues: $\lambda_1=0$ with the algebraic multiplicity $a(\lambda_1)=3$ and $\lambda_2=2$ with $a(\lambda_2)=1.$ We now want to find $V_{\lambda_{1,2}}.$ To do that, don't we have to $(M_f-\lambda_1I_4)\begin{pmatrix}x\\y\\z\\t\end{pmatrix}=\begin{pmatrix}0\\0\\0\\0\end{pmatrix}$?
Because that gives me $V_{\lambda_1}=\{\alpha\begin{pmatrix}0&1\\1&0\end{pmatrix}|\alpha\in\mathbb{R}\}.$
And that's not correct, what am I doing wrong?
You are not doing anything wrong yet. Next, you need to find the other eigenspace and if its dimension is $3$, then the matrix is readily diagonalizable. But if not, then the best you can do is you can get a Jordan normal form.
I think you are being confused about the eigenvectors. Note that, once you represent a linear transformation as a matrix with a fixed basis, then the eigenvectors you are finding are of that matrix. As such, they are elements of $\mathbb{R^n}$, if the original linear transformation is acting on a vector space with dimension $n.$