How do I minimize the following?
$ min_{z>0} - zt + 1/2\ z\ ||\ Y + X_k\ /\ z\ ||_2^2 $
Also, $X_k^TX_k = 1 \ \ \forall k $
I am given that the answer should be : $ \sqrt{Y^T - 2t} + Y^TX$
But I don't understand how to get this.
Here are my steps so far:
Taking the derivative and setting to $ 0$
$ X_k/z - Y^TX_k /z - 1/(z^2) = Y^TY - 2t $
But if I were to take the roots of this equation the solution does not come out to be : $ \sqrt{Y^T - 2t} + Y^TX$
Suggestions?
You are on the right track; you've made some mistake in your derivative calculation, and notice also that the question is asking you for the minimum and not the minimizer.
Might as well expand out that inner product:
$$\min_{z>0}\ -zt + \frac{1}{2} z\left(Y^T Y + Y^TX/z + X^TY/z + X^TX/z^2\right)$$ $$\min_{z>0}\ -zt + \frac{1}{2} \left(Y^T Yz + 2 X^TY + 1/z\right).$$ Differentiating I get $$0 = -t + \frac{1}{2}Y^TY -\frac{1}{2}z^{-2}$$ so $$z = (Y^T Y -2t)^{-1/2}.$$
Plugging this minimizer into the objective gives me $$\frac{1}{2}(Y^TY-2t)(Y^TY-2t)^{-1/2} + X^TY + \frac{1}{2}(Y^TY-2t)^{1/2}$$ $$\frac{1}{2}(Y^TY-2t)^{1/2} + X^TY + \frac{1}{2}(Y^TY-2t)^{1/2}$$ $$\sqrt{(Y^TY-2t)} + X^TY$$
I leave it to you to check that the above critical point is indeed a local minimum, and that this local minimum is global.