consider $\frac{dx}{dt} = Ax$ where $A$ is the matrix $$ \begin{bmatrix} 1 & 0 & 1 \\ 0 & 0 & -2 \\ 0 & 1 & 0 \\ \end{bmatrix} $$
construct a real matrix $P$ such that the change of coordinates $x=Py$ transforms our real equation to
$\frac{dy}{dt}=By$ where $B$ is the matrix
$$ \begin{bmatrix} 1 & 0 & 1 \\ 0 & 0 & -\sqrt{2} \\ 0 & \sqrt{2} & 0 \\ \end{bmatrix} $$
then solve explicity for $y$ and evaluate the solution in terms of our original $x=Py$
My Solution so far...
I have diagonalized both $A$ and $B$ and got the following matrices respectively $$ \begin{bmatrix} 1 & 0 & 0 \\ 0 &\sqrt{2} & 0) \\ 0 & 0 & -\sqrt{2} \\ \end{bmatrix} $$ and for $B$ $$ \begin{bmatrix} 1 & 0 & 1 \\ 0 & \sqrt{2} & 0 \\ 0 & 0 & -\sqrt{2} \\ \end{bmatrix} $$ the eigenvectors for $A$ make the matrix $$ \begin{bmatrix} 1 & 1+\sqrt{2} & -1-\sqrt{2} \\ 0 & 1/\sqrt{2} & -1/\sqrt{2} \\ 0 & 1 & 1 \\ \end{bmatrix} $$
and the eigenvectors for $B$ make the matrix $$ \begin{bmatrix} 1 & 1/\sqrt{2} & -1/\sqrt{2} \\ 0 & 0 & 0 \\ 0 & 0 &0 \\ \end{bmatrix} $$
Thus the determinant of the eigenvector matrix for $B$ is $0$...I am now stuck. Please help!
First of all, your eigenvalues (for both $A$ and $B$) should be $1$ and $\pm i\sqrt{2}$. Since you have three distinct eigenvalues, the eigenvectors of either matrix are guaranteed to be linearly independent, and thus constitute a basis. (By necessity, the eigenvectors corresponding to the complex eigenvalues must be complex.)
If $A=S_1\Lambda S_1^{-1}$ and $B=S_2\Lambda S_2^{-1}$, then $B=S_2S_1^{-1}AS_1S_2^{-1}$. This means that you get the desired transformation $P=S_1S_2^{-1}$, and with some luck this will be a real matrix (both $S_1$ and $S_2$ are complex).