Let's begin with the definition of the inner product. It's as follows:
$$ A . B = ||A|| \ ||B||\cos(\theta) . $$
My understanding is that the $\theta$ is the angle between the two vectors. As such this can be used to find the axis of rotation about a line, given that you have the line about which the rotation occurs, the transformation matrix and a vector orthogonal to the line.
I have been trying to solve an example question and have come as far as figuring out that the transformation is isometric ($A^tA = I) $ and that the $detA = 1$. Thus I know it's a rotation and now I just need to find the line and then I can proceed with finding the angle of rotation.
In short the transformation matrix $A$ = $1\over7$$\begin{bmatrix}3&6&2\\-6&2&3\\2&-3&6\end{bmatrix}$ and the line about which the rotation occurs is $t(1, 0 ,2)$. I picked a vector orthogonal to this line, $\vec v = (0,1,0)$, and now I want to use the definition of the inner product (the one defined earlier) to find the angle.
I had a look at the answer to this example and it turns out that the way they did it - which I do not fully understand - is by using this formula, if you will; $$(A\vec v)^t \vec v = cos \theta $$ Now the thing I don't get is , how does this formula work? I don't really understand the $(A\vec V)^t$ part and why it works.
This seems like a fundamental concept that I am unfortunately lacking an understanding of. I ask for your help, how does the forumla work?
Thanks
The meaning of the notation $(A\vec v)^t\vec v$ is the inner product between the vector $Av$ and the vector $v$. Since $v$ is normal to the line about which the rotation takes place, the vector $Av$ will be another vector normal to the rotation-line, and the angle between $v$ and $Av$ will precisely yield the angle of rotation. It does not matter which normal vector you begin with.