Projection of a Point onto a Hyperplane

Question

Given the coordinates of a point x and the parameters describing a hyperplane in N dimensions as the N-1 by 1 matrix theta and shift theta_0, evaluate the coordinates of the projection of point x onto the plane represented by theta and theta_0.

Answer 1

You want to project a point $v$ onto the hyperplane whose equation is $\theta^T x + \theta_0 = 0 $

By definition, you want to find $x$ that minimizes

$ f(x, v) = (x - v)^T (x - v) $ subject to $ \theta^T x + \theta_0 = 0$

Using Lagrange multiplier method, set up the following function, using the Lagrange multiplier $\lambda$

$ g(x, v, \lambda) = (x - v)^T (x - v) + \lambda ( \theta^T x + \theta_0 ) $

The gradient vector of $g$ with respect to $x$ is given by

$ \nabla_x g = 2 (x - v) + \lambda \theta = 0 \hspace{20pt}(1) $

And the derivative of $g$ with respect to $\lambda $ is

$ \theta^T x + \theta_0 = 0 \hspace{20pt}(2) $

Solving the first equation for $x$,

$ x = v - \frac{1}{2} \lambda \theta \hspace{20pt}(3) $

Substitute this into the second equation,

$ \theta^T v - \frac{1}{2} \lambda \theta^T \theta + \theta_0 = 0 $

Hence,

$ \lambda = \dfrac{2 (\theta^T v + \theta_0) }{\theta^T \theta }\hspace{20pt}(4) $

Substitute this back into $(3)$,

$ x = v - \dfrac{(\theta^T v + \theta_0) \theta }{\theta^T \theta }\hspace{20pt}(5) $

After a simple manipulation this becomes

$ x = \bigg( I - \dfrac{ \theta \theta^T }{\theta^T \theta } \bigg) v - \dfrac{\theta_0 \theta} {\theta^T \theta} $

And this is the expression for the projection of $v$ onto the hyperplane.