I am learning Lagrangian duality methods in an optimization course.
Below is from my lecture note.
Let $\ell(\textbf{x}; \lambda) =-x_1x_2+\lambda[(x_1-3)^2+x^2_2-5]$.
Then, the Hessian of $\ell(\textbf{x}; \lambda)$ is
$ \nabla^2\ell(\textbf{x}; \lambda) = \left[ {\begin{array}{cc} 2\lambda & -1 \\ -1 & 2\lambda \\ \end{array} } \right] $
To give you more context of the problem, $\space\phi(\lambda) =$ Min$\ell(\textbf{x};\lambda)$ is the dual function.
Since $\textbf{x}$ is a column vector with 2 variables $x_1$ and $x_2$, I think the hessian should be a 3 by 3 matrix.
Thanks in advance.
Since $ \ell $ is a function of $ 3 $ variables, it's true that $ \nabla ^ 2 \ell ( \mathbf x ; \lambda ) $ is really a $ 3 $-by-$ 3 $ matrix, specifically $$ \left [ \matrix { 2 \lambda & - 1 & 2 ( x _ 1 - 3 ) \\ - 1 & 2 \lambda & 2 x _ 2 \\ 2 ( x _ 1 - 3 ) & 2 x _ 2 & 0 } \right ] \text . $$ However, in finding $ \phi ( \lambda ) : = \min _ { \mathbf x } \ell ( \mathbf x ; \lambda ) $, you're not really minimizing the function $ \ell $, but rather minimizing a function of $ \mathbf x $ alone. If we write $ \ell _ \lambda ( \mathbf x ) : = \ell ( \mathbf x ; \lambda ) $, then $ \ell _ \lambda $ is a function of only $ 2 $ variables, and $ \nabla ^ 2 \ell _ \lambda ( \mathbf x ) $ is the the $ 2 $-by-$ 2 $ matrix $$ \left [ \matrix { 2 \lambda & - 1 \\ - 1 & 2 \lambda } \right ] $$ that you have in your notes. Since $ \ell _ \lambda $ is the relevant function to analyse to find $ \phi $, this is the relevant Hessian matrix. (And the relevant gradient to set to $ \mathbf 0 $ is $ \nabla \ell _ \lambda ( \mathbf x ) = [ - x _ 2 + 2 \lambda ( x _ 1 - 3 ) , - x _ 1 + 2 \lambda x _ 2 ] $. Also notice that this only actually gives you a minimum when $ \lambda > 0 $.)