Given $\gamma: \mathbb{R}^2 \rightarrow \mathbb{R}^2$ (rotation around $o$) and $\sigma: \mathbb{R}^2 \rightarrow \mathbb{R}^2$ (reflection in one of the lines through the origin), I have to show that $\gamma \circ \sigma$ is also a reflection.
I have two questions.
Is there any more elegant way of calculating $\gamma \circ \sigma$ than, the way I chose: $=A_\gamma^\mathcal{B}\cdot A_\sigma^\mathcal{B}$ and then simplifying this to the form $\pmatrix{ \cos(\varphi) & \sin(\varphi) \\ \sin(\varphi) & -\cos(\varphi)}$ using trigonometric identities? $\mathcal{B}$ is orthonormalbase of $\mathbb{R}^2$.
What is the axis of this reflection? (why is important that we have $\sigma $ defined as reflection in one of the lines through the origin)
One useful thing to know is that the composition of two reflections in intersecting lines in the plane is a rotation around the point of intersection by twice the angle between the lines (in the direction from the first axis of reflection towards the second axis); this can be shown purely geometrically. Now you can decompose the rotation $\gamma$ in such a way, and since there is freedom in choosing the exact axes of reflection, one can arrange that $\gamma=\tau\circ\sigma$ (the axis of $\tau$ is obtained by rotating the axis of $\sigma$ by half the angle of rotation for$~\gamma$). Now $\gamma\circ\sigma=\tau\circ\sigma^2=\tau$ which is a relflection.
If $\sigma$ were reflection in a line not through the origin, this argument would not work. Indeed the best one could get (by decomposing $\gamma$ with one axis of reflection parallel to the axis of $\sigma$) is writing $\gamma$ as the composition of a translation followed by a reflection. Such compositions only rarely have any fixed points at all (it only happens if the translation is perpendicular to the axis of reflection), and if not, the composition cannot possibly be a reflection (instead it will be a glide-reflection).