My lecture notes include the following result with no proof:
Theorem: Let $f:\mathbb{R}^3\rightarrow\mathbb{R}^3$ be an affine map with corresponding linear function $\phi:\mathbb{R}^3\rightarrow\mathbb{R}^3$, that is, for any vector $w\in \mathbb{R}^3$ and point $p\in \mathbb{R}^3$ (the affine space we are dealing with has $\mathbb{R}^3$ as both its set of point and as its vector space) we have that $$f(p+v)=f(p)+\phi(v).$$ Then the plane $P$ described by the equation $ax+by+cz+d=0$ is invariant under $f$ if and only if the vector $$u=(a,b,c,d)^T$$ is an eigenvector of $M^T$, where $M$ is the augmented matrix of $f$.
I've tried playing with the identities $$v\in P \Leftrightarrow u\cdot \begin{pmatrix} v \\ 1 \end{pmatrix} = u\cdot \begin{pmatrix} x\\ y \\ z \\ 1 \end{pmatrix} =0$$
as well as
$$v'\in P \Leftrightarrow u\cdot \begin{pmatrix} v' \\ 1 \end{pmatrix} = u\cdot \begin{pmatrix} x'\\ y' \\ z' \\ 1 \end{pmatrix} =u\cdot (Mv)=0$$
where $v'=f(v)=(x',y',z')$ and $M$ is the augmented matrix of $f$. Although I have not been able to prove either direction.
How could one prove the result?
Let's use $v$ to stand for $(x,y,z,1)^T$. Then use $u\cdot v=u^T v$, and your derivations are almost there.
$u^T M v =0=v^T M^T u$ and $u^T v=0 = v^T u$ for all $v$ of the form $(x,y,z,1)^T$ implies $$M^T u = \lambda u.$$