Find an orthogonal matrix that transfers one plane to another

Question

I need to find an orthogonal matrix that transfers the plane $\pi_1=\Bigg\{\begin{pmatrix} x\\ y\\z\end{pmatrix}\Bigg|3x+4y+5z=0\Bigg\}$ to the plane $\pi=\Bigg\{\begin{pmatrix} x\\ y\\z\end{pmatrix}\Bigg|5x+5y=0\Bigg\}$. At first I thought that I maybe need to find two lines from $\pi_1$, then I'll complete a basis by adding one more independent vector, then I'll do $gram-shmidt$, and then Ill do the same with the other plane. And the answer will be the second matrix times the transpose of the first matrix. The problem is that I don't know if it is true, or how to find two lines from $\pi_1$ and $\pi_2$.

Answer 1

As hinted in my first comment above, your method is correct but a simpler one is to find some orthogonal matrix $R$ sending $u$ to $v$, where $$x:=\begin{pmatrix} 3\\ 4\\5\end{pmatrix}\quad\text{and}\quad y:=\begin{pmatrix} 5\\ 5\\0\end{pmatrix}.$$ A simple $R$ for this is the reflection in the plane orthogonal to $$z:=y-x=\begin{pmatrix}2\\1\\-5\end{pmatrix}:$$ $$R:=I_3-\frac2{z^T\,z}z\,z^T=\frac1{15}\begin{pmatrix}11&-2&10\\-2&14&5\\10&5&-10\end{pmatrix}.$$

Answer 2

Here's a guideline without doing the calculations.

Let $x = (3,4,5)^t$ and $y=(5,5,0)^t$. You want an orthogonal matrix that maps $x$ to $y$. Note that both have Euclidean norm $5\sqrt{2}$. Their inner products is $35$, so if $\theta\in [0,\pi]$ is the angle between them then $\cos(\theta) = 35/50 = 0.7$ so you know they're not co-linear. You also know $\sin(\theta) = \sqrt{1-(7/10)^2} = \sqrt{51}/10$.

Now let $u = x/(5\sqrt{2})$, $w = (x\times y)/(50\sin(\theta))$ and $v=w\times u$ where $\times$ is the cross product. Then $(u, v, w)$ is a right-handed orthonormal basis. In this basis, $y/(5\sqrt{2}) = \cos(\theta) u + \sin(\theta) v$. Let $A$ be the orthogonal matrix with columns $(u, v, w)$. You can get an orthogonal matrix rotating $x$ to $y$ by $$A\pmatrix{ \cos(\theta)&-\sin(\theta)&0\\ \sin(\theta)&\cos(\theta)&0\\ 0&0&1 }A^t$$