Distance between $3\times3$ matrices (isometries)

Question

The problem is pretty long, so I'm going to describe it while writting what I have so far.
First I have to asociate the $3\times3$ matrices with $\Bbb R^9$, so we defined $\phi:M_{3\times3}(\Bbb R) \to \Bbb R^9$ as the following mapping: $(a_{ij}) \mapsto (a_{1,1},...,a_{9,9})$. Then define the distance between matrices (using the distance in $\Bbb R^9$), so we get $d: M_{3\times3}(\Bbb R) \times M_{3\times3}(\Bbb R) \to \Bbb R$ where $(A,B) \mapsto d_{\Bbb R^9}(\phi(A),\phi (B))$, where $d_{\Bbb R^9}$ is the usual metric of $\Bbb R^9$, clearly $d$ is a metric.
Now, I have to prove that if we restrict the distance between the matrices to the elements of $O (3,\Bbb R)$,wich is the orthogonal group, for every $M \in SO(3,\Bbb R)$ exists $\varepsilon>0$ such that if $d(M,M')<\varepsilon$ then $M'$ is not a reflection.
I don't know how to do that, we know that the distance $d$ is a continuous function, and the determinant of the elements in $SO(3,\Bbb R)$ is allways $1$, but the determinant of a reflection is $-1$,I belive we have to use that, maybe compose the distance with the determinant, since the determinant is also continuous (isn't it?).

Answer 1

In dimension 3, if $$ A A^t = I, \; \; \det A = 1, $$ and $$ B B^t = I, \; \; \det B = -1, $$ then $$ 2 \leq \operatorname{dist} (A,B) \leq 2 \sqrt 3. $$ In addition, if $A=I$ and $B$ is a genuine reflection, meaning no rotation in the (setwise) fixed plane and eigenvalues $-1,1,1,$ then the distance between $I(A)$ and $B$ is exactly $2.$ For this example, you could just have $B$ the diagonal matrix with diagonal entries $-1,1,1.$

Let me explain that your distance is simply this: the distance of any matrix $M$ from the matrix with all entries $0$ is just $\sqrt {\operatorname{trace} M M^t}.$ See FROBENIUS. Very important that you know $$ \operatorname{trace} M N^t = \operatorname{trace} M^t N = \operatorname{trace} N M^t = \operatorname{trace} N^t M. $$

Then for any pair of matrices, $$ \operatorname{dist} (M,N) = \sqrt {\operatorname{trace} (M - N) (M^t - N^t)}. $$

Oh, if we have some $R R^T = I,$ then $$ \operatorname{dist} (RM,RN) = \operatorname{dist} (MR,NR) = \operatorname{dist} (M,N). $$ For this one, we do not care about the determinant of $R,$ it is allowed $\pm 1.$

As a result, given $A A^t = 1, \det A = 1,$ while $B$ is a genuine reflection, then $$ \operatorname{dist} (A,AB) = \operatorname{dist} (A,BA) $$ is exactly $2.$

On the theory that this does little harm if nobody reads it; the dot product on $\mathbb R^{n^2}$ is exactly this thing I wrote before, $$ \operatorname{trace} M N^t = \operatorname{trace} M^t N = \operatorname{trace} N M^t = \operatorname{trace} N^t M. $$ Write it out for 2 by 2 matrices $M,N,$ you'll see.

EDIT, Wednesday, why not. As in comment, we are finding the minimum of $ \operatorname{dist} (I,C)$ where $\det C = -1, \; \; C C^T = I.$ Writing $$ d = \operatorname{dist} (I,C) $$ we get $$ d^2 = \operatorname{trace} (I - C) (I - C^t) = \operatorname{trace} (2I - C - C^t) = \operatorname{trace} (2I - 2C), $$ or $$ d^2 = 2 \operatorname{trace} (I - C). $$ We are in dimension 3, which is odd. The eigenvalues of $C$ are $-1, \cos \theta + i \sin \theta, \cos \theta - i \sin \theta.$ As a result, the trace of $C$ is $2 \cos \theta - 1,$ so $-3 \leq \operatorname{trace} C \leq 1, $ then $-1 \leq \operatorname{trace} (-C) \leq 3, $ next $2 \leq \operatorname{trace} (I-C) \leq 6, $ finally $$4 \leq 2 \operatorname{trace} (I-C) \leq 12. $$ With $ d = \operatorname{dist} (I,C) $ and $ d^2 = 2 \operatorname{trace} (I - C), $ we arrive at $$4 \leq d^2 \leq 12, $$ so $2 \leq d \leq 2 \sqrt 3, $ that is $$2 \leq \operatorname{dist} (I,C) \leq 2 \sqrt 3, $$ for dimension 3, $\det C = -1, \; C C^T = I.$

NOTES, Thursday, 12 September 2013: $$\begin{array}{l}\text{What is reflection, saith my sufferings, then?}\cr \text{If all the pens that ever poets held}\cr \text{Had fed the feeling of their masters' thoughts,}\cr \text{And every sweetness that inspir'd their hearts,}\cr \text{Their minds, and muses on admired themes;}\cr \text{If all the heavenly quintessence they still}\cr \text{From their immortal flowers of poesy,}\cr \text{Wherein, as in a mirror, we perceive}\cr \text{The highest reaches of a human wit;}\cr \text{If these had made one poem's period,}\cr \text{And all combin'd in beauty's worthiness,}\cr \text{Yet should there hover in their restless heads}\cr \text{One thought, one grace, one wonder, at the least,}\cr \text{Which into words no virtue can digest.}\end{array}$$

A reflection in $\mathbb R^3$ negates one vector, call it $r,$ (and demand that $r$ be unit length) but fixes every vector in the plane orthogonal to $r.$ We can take an orthonormal basis of the plane, call that $s,t.$ Each of these is fixed. So $r,s,t$ are an orthonormal basis of eigenvectors (of the reflection) with eigenvalues $-1,1,1.$ That is to say, in this basis, the matrix of the reflection is the diagonal matrix $$ D \; = \; \left( \begin{array}{rrr} -1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{array} \right). $$
Thus, if $P$ is the matrix with comlumns given by the entries $r,s,t,$ then $P P^T = I.$ If the matrix of the reflection in the original basis is $W,$ then $$ D = P^T W P $$ and $$ W = P D P^T. $$ So, what do we know about $W?$ That it is evidently symmetric, orthogonally similar to $D,$ that $W^2 = I,$ and finally $\operatorname{trace} W = 1. $ For dimension $n$ instead of dimension $3,$ the definition of reflection would be $W$ symmetric, $W^2 = I,$ and $\operatorname{trace} W = n-2.$

Finally, as I proved above in dimension $3,$ the reflections are the closest orientation-reversing isometries to the identity matrix, distance $2.$

Answer 2

Yes, your approach is correct. Let $\epsilon=1$ and you're about done.