Transformation matrix: rotation in $\mathbb R^3$

Question

Operator $\phi: \mathbb R^3 \to \mathbb R^3$ is rotation around line $p: x_1 - x_2 = 0,$ $ x_3=0$, $\phi (0,0,2) = (\sqrt2,-\sqrt2,0)$. I need to find transforamtion matrix $A$: $\phi(x)=Ax$ in standart base of $\mathbb R^3$. How do I do that?

Answer 1

A is a rotation (i. e. $A$ is orthogonal and $\det A = 1$), with $A(0,0,2) = (\sqrt{2},-\sqrt{2},0)$ and a fixed line $x_1-x_2 =0$, so e. g. $(1,1,0)$ is fixed. This should be sufficient information to determine the images of the orthogonal basis vectors $(0,0,2), (1,1,0), (1,-1,0)$ under $\phi$. Then you can use a basis transformation to get $A$.

Answer 2

we know that the third column of the matrix is $(\sqrt 2/2, -\sqrt 2/2, 0).$ by requiring that the axis $(1,1,0)$ is fixed we have $$\pmatrix{a&1-a&\sqrt 2/2\\b & 1-b&-\sqrt 2/2\\c&-c&0} $$ requiring that the columns one and three are orthogonal gives you $a = b.$ requiring the length of columns one and two is one gets you $$1=a^2+a^2 + c^2 = (1-a)^2 +(1-a)^2 + c^2 \implies a = 1/2,c = \pm1/\sqrt 2. $$ so far we have i don't know how to get rid of the $\pm$ sign.

using the user zardo's suggestion that the determinant of the rotation matrix is one we have $$\pmatrix{1/2&1/2&\sqrt 2/2\\1/2&1/2&-\sqrt 2/2\\-\sqrt2/2&\sqrt2/2&0}. $$