Optimization of three right-angled vectors

Question

In my case, I have three given vectors $\vec{a}, \vec{b}, \vec{c}$ with $$\vec{a}= \begin{pmatrix} x \\ y \\ z \\ \end{pmatrix} $$ and these vectors span a coordinate system. But so far they are not totally perpendicular to each other. So they wouldn't span a totally right-angled coordinate system. Is there a way to change these three vectors that the axes are perpendicular and the change of the vectors direction is quite low?

Answer 1

Here is an answer taking into account your desire of "low change" (of course, it depends of what one takes as a measure of "low change"...). The spirit of my answer is that I assume that these vectors, call them ${a_1, a_2, a_3}$, are already unit vectors (if this has not been done previously, it is a necessary step), not too far from constituting an orthonormal basis.

We will work on the unit sphere $S^2$ of $\mathbb{R}^3$.

1) compute $u=a_1+a_2+a_3$. Divide by its norm in order that this "mid-vector", call it $a_4$ be on $S^2$.

2) then "unfold/fold" as you could do it with an umbrella with axis directed by $a_4$ and ribs ${a_1, a_2, a_3}$, in order to position these ribs as mutually orthonormal. This can be done independently for each $k=1,2,3$ in the following way: rotate $a_k$ around $b_k$ := cross product $a_4 \times a_k$ by angle $\alpha_k = \pi/2-arccos(a_4.a_k)$ (dot-product). I can provide supplementary computational details if this kind of interpretation agrees with your objective.

Answer 2

You can do a regularized optimization:

$$\min_{{\vec b_v},{\vec c_v}}\left\{\|{\vec a}^T[{{\vec b} + \vec b_v},{\vec c + \vec c_v}]\|_2^2 + \lambda_b\|{\vec b_v}\|_2^2 + \lambda_c\|{\vec c_v}\|_2^2\right\}$$

with $\lambda_b, \lambda_c$ you can steer the importance of the changes (variations) ${\vec b_v} \& {\vec c_v}$ being small.

Teh first term will make $\vec b+\vec b_v$ and $\vec c+ \vec c_v$ more orthogonal to $\vec a$ as minimize the two-norm of their scalar products, then you need to make them orthogonal to each other and you can do that with a subsequent optimization of the same kind.