Consider a endomorphism $u$ from $\mathbb R^n$ to $\mathbb R^n$ and a matrix $A\in\mathbb R^{k,k}$ such that there is an isomorphism $\varphi$ between the invariant subspace of $u$ and the kernel of $A$. This proves that the invariant subspace and the kernel have same dimensions. But at the same time $k\neq n$, so I am a bit confused: let's say $x\in\mathbb R$ is such that $u(x)=x$ and $y\in\mathbb R^k$ is such that $Ay=0$. Is the isomorphism $\varphi$ fully described by a relation of the type $$ x=Cy,\quad C\in\mathbb R^{n,k}$$ The confusion comes from the fact that $C$ is not invertible (it's not a square matrix). Is it necessary to also have the reciprocal isomorphism $$y=Lx,\quad L\in\mathbb R^{k,n}$$ to fully described $\varphi$?
In other words, does it make sense to say that there exist an isomorphism defined from $\mathbb R^n$ over $\mathbb R^k$? It seems impossible to me if $k\neq n$...
There exists such matrix $L$ (from $\Bbb R^n$ to $\Bbb R^k$), and it is not invertible in general. You can choose it so that it maps the vectors of the invariant subspace isomorphically to the kernel and maps some chosen complement of the invariant subspace to $0$ (or anything). There is no reason why $L$ should be invertible.
Similarly, you could define a matrix $C$ from $\Bbb R^k$ to $\Bbb R^n$ that maps the kernel isomorphically to the invariant subspace and maps some complement of the kernel arbitrarily. Also this doesn't need to be invertible.
However, if you choose a basis of the invariant subspace and a basis of the kernel, then you can write the isomorphism $\varphi$ with respect to these bases and get a regular (invertible) matrix $\in \Bbb R^{u,u}$ for $u=\dim\mathrm{Ker}$.
$\Bbb R^n$ is isomorphic to $\Bbb R^k$ only for $k=n$, which easily follows from the kernel-rank theorem.