Make two vectors mutually orthogonal while minimizing change to both vectors

Question

Given two vectors $u$ and $v$, by what procedure can I make them orthogonal to each other (such that $u'\cdot v'=0$) while minimizing "change to both of them" (e.g. minimizing $(v'-v)^2+(u'-u)^2$)?

EDIT

Following @SiddharthJoshi's advice, I formulate the Lagrangian:

$$L(u',v',\lambda)=(u'-u)^2+(v'-v)^2+\lambda(u'\cdot v')$$

yielding:

$$\frac{\partial L}{\partial u'}=2(u'-u)+\lambda v'=0$$

$$\frac{\partial L}{\partial v'}=2(v'-v)+\lambda u'=0$$

$$\frac{\partial L}{\partial \lambda}=u'\cdot v'=0$$

This implies that the change to $u$ should be parallel to $v'$ and the change to $v$ should be parallel to $u'$, but I'm not sure whether it can be taken any further.

Answer 1

We know that we can represent, for example, vector $v$ as $v = v^{\perp u} + v^{\parallel u}$. Now, to make $v$ orthogonal $u$, we write $v^{\perp u} = v - v^{\parallel u}$.

We can calculate $v^{\parallel u}$ as $v^{\parallel u} = \frac{\langle v, u \rangle}{\langle u, u \rangle}u$.

Answer 2

Taking $X = u$, $Y = v$, $x = u'$ and $y = v'$, we have the following optimization problem $$\min_{x, y} f(x, y)=\ (x-X)^2 + (y-Y)^2$$ subject to the constraint $c(x, y) = x^{T}y = 0$. By Lagrange Multiplier Method, we have that $$\nabla_x(f(x, y)-\lambda * g(x, y)) = \nabla_y (f(x, y) - \lambda * g(x, y)) = 0$$ Now $$\nabla_x(f(x, y)-\lambda * g(x, y))=2(x-X)-\lambda * y = 0 \implies x=X+\frac{\lambda}{2} * y$$ and also $$\nabla_y(f(x, y)-\lambda * g(x, y))=2(y-Y) - \lambda * x \implies y = Y + \frac{\lambda}{2}*x$$For simplicity take $\mu = \frac{\lambda}{2}$. Solving for $x, y$, you will get that $x = \frac{X + \mu Y}{1-\mu^2}$ and $y = \frac{Y + \mu X}{1-\mu^2}$. Now to solve for $\mu$, use the constraint that $x^Ty=0$. You will get that $\mu^2(X^TY)+(X^TX + Y^TY)\mu+Y^TX=0$. This quadratic equation would always have real solutions and there will be two possible values of $\mu$.

Now since $\nabla_{(x, y)}^2 f= \begin{bmatrix} 2I & 0 \\ 0 & 2I \end{bmatrix}$ is a positive definite matrix, the solution we have is a local minimum for $f$ and since there are only two critical points, one of them must be the global minimum.

You can try this out numerically for some values of $x, y$ and see if you get a satisfactory answer.

Answer 3

The following two corner cases need special treatments and they are ignored here:

1. the vector space in question is at most one-dimensional, or
2. $u$ and $v$ are linearly dependent.

Suppose that the vector space in question is at least two-dimensional and $u,v$ are linearly independent. Let $u'=ax$ and $v'=by$, where $\{x,y\}$ is orthonormal and $a,b\in\mathbb R$. Then \begin{align} &\|u-u'\|^2+\|v-v'\|^2\\ &=\langle u-ax,u-ax\rangle^2+\langle v-by,v-by\rangle^2\\ &=(a-\langle u,x\rangle)^2 +(b-\langle v,y\rangle)^2-\langle u,x\rangle^2-\langle v,y\rangle^2+\|u\|^2+\|v\|^2. \end{align} It is minimised when $a=\langle u,x\rangle$ and $b=\langle v,y\rangle$. The problem thus reduces to maximising $S=\langle u,x\rangle^2+\langle v,y\rangle^2$ subject to the constraint that $\{x,y\}$ is orthonormal.

Let $x$ be fixed. Then $v=v_x+v_{x^\perp}$ where $v_x:=\langle v,x\rangle x$ is parallel to $x$ and $v_{x^\perp}:=(v-\langle v,x\rangle x)$ is orthogonal to $x$. Since $\langle v,y\rangle=\langle v_{x^\perp},y\rangle$, the maximum value of $S$ is always attained when $v_{x^\perp}$ is parallel to $y$. In that case, we have $$ S=\langle u,x\rangle^2+\|v_{x^\perp}\|^2 =\langle u,x\rangle^2+\|v\|^2-\langle v,x\rangle^2. $$ If the ambient space is some $\mathbb R^n$, the expression above can be rewritten as $S=x^T(uu^T-vv^T)x+\|v\|^2$. Hence one optimal $x$ is given by a unit eigenvector corresponding to the maximum eigenvalue of $uu^T-vv^T$. With this $x$, an optimal $y$ is given by a unit vector in $\operatorname{span}\{u,v\}$ that is orthogonal to $x$. From these two vectors, we obtain an optimal solution $u'=ax=\langle u,x\rangle x$ and $v'=by=\langle v,y\rangle y$.