distance SO(3) rotation matrix

Question

According to M. Moakher's

Means and averaging in the group of rotations

and I. Sharf's

Arithmetic and geometric solutions for average rigid-body rotation

the distance between two rotation matrices is $$\| R_1 - R_2 \|_{\text{F}}$$ where $\| \cdot \|_{\text{F}}$ denotes the Frobenious norm. Does it mean $\left\| R_1 - R_2 \right\|_{\text{F}}$ or $\left\| R_1^T R_2 \right\|_{\text{F}}$?

• $\left\| R_1 - R_2 \right\|_{\text{F}}$ does not make sense as $(R_1-R_2) \notin$ SO(3).

• $\left\| R_1^T R_2 \right\|_{\text{F}}$ is also strange: suppose $R_1=R_2$, the distance is $3$. I find out that the distance between two rotation matrices is less than $3$. Such metric is against my intuition (the distance between two identical elements is largest!).

Answer 1

It means $\|(R_1-R_2)\|_F.$

It does not matter that $R_1-R_2$ is not special-orthogonal as we can still take the norm. Consider that the distance between two points on a circle is related to the length of the vector between them, even though that vector is not itself on the circle. This is much like the $SO(2)$ version of your question.

Answer 2

The slightly counterintuitive bit is that the metric comes from the (normed) vector space structure of the matrix algebra, but when you're dealing with $SO_3$ you're focusing on the complementary structure (the bilinear operation of multiplication). Still, the Frobenius norm gives us an extrinsic metric on $SO_3$ when we consider it embedded in matrix land: $d(R_1, R_2) = ||R_1 - R_2||_F$. $SO_3$ is not a metric vector space, but a subset of a metric is still a metric space.

Let's look at another related example.
Consider the complex numbers, $\mathbb{C}$, which form an algebra over the reals under the operations of multiplication and addition. We have a standard norm in the complex numbers, $$|a + bi| = \sqrt{a^2 + b^2}$$ which induces a metric in which the distance between $u = a+bi$ and $v = c + di$ is $d(u,v) = |u-v|$. Now this metric gives a perfectly good way to measure the distance between any two complex numbers. This metric still provides a consistent way to conceive of distance between elements if we restrict our attention to the unit circle (those complex numbers $z$ such that $|z| = 1$, which form a group under multiplication). However, you might have the objection that, considering the unit circle as embedded in $\mathbb{C}$, this measure of distance is extrinsic. If that's an issue for you, you can establish an intrinsic metric by taking the distance of two points on the unit circle to be the minimum length among all paths between the points restricted to the unit circle.

Your matrix case is much the same. True you are considering $SO_3$ which is a special subset of matrices that forms a group under multiplication, but that does not invalidate the Frobenius norm on all matrices as a measure of distance.