I'm using a quaternion to represent the orientation in a kalman filter. My algorithm works fine until I rotate "upside down". I think this is because there seems to be multiple ways to represent the same orientation.
Suppose I have a quaternion that represents no rotation q = (1,0,0,0).
A 180 degree rotation about the x axis should result in the same end orientation as if I were to rotate 180 degree about the y axis. However, the resulting quaternion doesn't reflect this.
My quaternion for rotation 180 degrees about the x axis is: $q_{rx} = (0,1,0,0)$ and rotation 180 degrees about the y axis is $q_{ry} = (0,0,1,0)$
$$ qrx * q = (0,1,0,0) $$ $$ qry * q = (0,0,1,0) $$ But, like I said, these are the same orientation.
Am I thinking about this the wrong way?
EDIT: I think I have found my problem. I was trying to use a single quaternion to represent the orientation in the Kalman filter. Since there are multiple ways to represent a single orientation with a quaternion, the Kalman filter would go unstable at these orientations. What I did to fix the problem was to represent orientation as $\Delta q * q$ in the filter. The Kalman filter then only acts on $\Delta q$ and it only represents small rotations. At the end of each Kalman "predict" and "update" stage, I update $q$. Therefore, we never get the ambiguity of orientation I was experiencing before.
As an aside, it makes the calculations of the Jacobians for the Kalman filter considerably more complicated.
I'm interpreting this all as the standard representation of $3$-d rotations using the conjugation action of unit quaternions on the "pure quaternions" (those with real part $0$).
The transformation represented by the quaternion $q_{rx}=(0,1,0,0)=i$ indeed rotates the pure quaternions around the $x$ axis. It leaves $i$ fixed (the $x$ axis), and maps $j\mapsto -j$, $k\mapsto -k$, so that the $y,z$ plane has experienced a $180$ degree rotation.
Similarly, the transformation represented by the quaternion $q_{ry}=(0,0,1,0)=j$ fixes the $y$ axis and turns the $x,z$ plane $180$ degrees.
Both of these are nonreflective rotations, and therefore preserve orientation. Therefore they have the same orientation, but they do not, of course, result in the same transformation.
Going back to what you said before your example, it is true that rotations have more than one representation in this scheme, but you will probably not be impressed by the difference. It's easy to see that a unit quaternion $q$ represents the same rotation as $-q$ does. A quick computation confirms that $(-q)^{-1}=-q^{-1}$, and then $(-q)x(-q)^{-1}=(-1)^2 qxq^{-1}=qxq^{-1}$ follows immediately.