Quadratic matrix equation: ellipse of all solutions

Question

Consider the following equation in $Z$:

$$-2 (\pmb X^T Y)^T Z+Z^T(\pmb X^T \pmb X)Z = 0$$

where:

$\pmb X\in\mathbb{R}^{n\times p}$ and $Y\in\mathbb{R}^n$ with $n>p$ are known and $Z\in\mathbb{R}^p$ is the vector of unknown.

Geometrically, I can see that all solutions $Z$ to this equation lie on an ellipse ($\pmb X^T \pmb X\succeq0$) passing through the origin. My question is how to characterize this ellipse in terms of the known quantity of this problem ($\pmb X$ and $Y$).

Answer 1

When you see a quadratic form always think of $x^TAx$ structure. Moreover, if you see a matrix term has scalar factor, always start from the fact that a scalar's transpose is itself.

Here we can slowly reconstruct the form as follows (to make sure that I'm not covering my footsteps):

$$ (Z^TX^T)(XZ) - Y^T(XZ) - (XZ)^TY = 0 $$

now call $XZ=W$ or whatever you Wish, then we have

$$ W^TW - Y^TW - W^TY=0 $$ write it as a quadratic form

$$ \begin{pmatrix}W\\Y\end{pmatrix}^T \begin{pmatrix}I&-I\\-I&0\end{pmatrix} \begin{pmatrix}W\\Y\end{pmatrix} = 0 $$ then take $X$ from $W$ and put it inside the multiplier in the middle

$$ \begin{pmatrix}Z\\Y\end{pmatrix}^T \begin{pmatrix}X^TX&-X^T\\-X&0\end{pmatrix} \begin{pmatrix}Z\\Y\end{pmatrix} = 0 $$ Then I think, you can see why ellipsoid comes into play. And also a tall matrix $A$ cannot have a positive definite $A^TA$ but only and necessarily positive semi-definite, if $n>p$.

Answer 2

First notice that:

$$Z=2(\pmb X^\top Y)(\pmb X^\top\pmb X)^{-1}$$

is (with the origin) a second solution to this homogeneous quadratic vector equation (this follows from generalising to vectors the usual roots of quadratic homogeneous equation formula).

From this it follows that the ellipse of all solutions to the vector quadratic equation is centred at $(\pmb X^\top Y)(\pmb X^\top\pmb X)^{-1}$.

With a bit of trial and errors one sees that this ellipse has shape matrix $(\pmb X^\top\pmb X)^{-1}$.

Matching all of the above, one sees that this ellipse has radius:

$$\sqrt{((\pmb X^\top Y)(\pmb X^\top\pmb X)^{-1})^\top(\pmb X^\top\pmb X)((\pmb X^\top Y)(\pmb X^\top\pmb X)^{-1})}$$