In Chapter 4 of the book Classical Mechanics by Goldstein, it was written that "By means of some similarity transformation, it is always possible to transform the matrix A to a system of coordinates where the z axis lies along the axis of rotation".
I don't understand how this is done. Can anyone please explain this to me with appropriate physical example with mathematical calculation.
Thank you in advance.

Let $R$ be the given rotation matrix expressed in the basis of the world frame.
Then the image of a point $p$ is $p'$ given by
$p' = R p$
Assume that you know or can determine the axis $a$ and angle of rotation $\theta$ of matrix $R$, and using this information, you define a coordinate system through the rotation matrix $R_1$ where R1 has its third column equal to the axis $a$, then it follows that
$p = R1 q$
where $q$ is the coordinate of the point $p$ expressed in the $R_1$ frame.
Matrix $R_1$ takes the form
$R1 = [u_1, u_2, a ]$
where $a \cdot u_1 = a \cdot u_2 = u_1 \cdot u_2 = 0 $, and
$ u_1 \times u_2 = a , \ a \times u_1 = u_2 , \ a \times u_2 = - u_1 $
Therefore, now, the image $p'$ can expressed as
$ p' = R_1 q' $
and also,
$ p' = R p = R R_1 q $
Therefore,
$q' = R_1^{-1} R R_1 q = R_1^T R R_1 = R' q $
where $R' = R_1^T R R_1 $
This defines the similarity relation between $R$ and $R'$.
From Rodrigues' rotation matrix formula, the rotation matrix R is given by
$R = a a^T + (I - a a^T) \ \cos \phi + S_a \ \sin \phi $
where $S_a v = a \times v $ for any vector $v$.
It follows that,
$R R_1 = [ a a^T + (I - a a^T) \ \cos \phi + S_a \ \sin \phi ][ u_1, u_2, a] $
This evaluates to
$ R R_1 = [ u_1 \ \cos \phi + u_2 \ \sin \phi , u_2 \ \cos \phi - u_1 \ \sin \phi , a ] $
Hence,
$R_1^T R R_1 = \begin{bmatrix} u_1^T \\ u_2^T \\ a^T \end{bmatrix} [ u_1 \ \cos \phi + u_2 \ \sin \phi , u_2 \ \cos \phi - u_1 \ \sin \phi , a ] $
and this evaluates to
$R' = R_1^T R R_1 = \begin{bmatrix} \cos \phi && - \sin \phi && 0 \\ \sin \phi && \cos \phi && 0 \\ 0 && 0 && 1 \end{bmatrix} $
which is what we wanted to show. Note that matrix $R_1$ is not unique.