I have to show that the function $f(x, y) = x^2 + y^2$ is convex so I am thinking to demonstrate that the graph of the function lies above any of its tangent planes.But I want to prove it with linear algebra perspective.
To do this, i used the definition of a convex function in terms of its Hessian matrix. The Hessian matrix of a function with two variables is a $2 \times 2$ matrix that contains its second partial derivatives.
If the Hessian matrix is positive semidefinite then the function is convex.Am I correct ?
To find the Hessian matrix of $f(x, y) = x^2 + y^2$, i need to compute its second partial derivatives:
\begin{align*} \frac{\partial^2 f}{\partial x^2} &= \frac{\partial}{\partial x}\left(\frac{\partial f}{\partial x}\right) \\ &= \frac{\partial}{\partial x}\left(2x\right) \\ &= 2 \end{align*}
\begin{align*} \frac{\partial^2 f}{\partial y^2} &= \frac{\partial}{\partial y}\left(\frac{\partial f}{\partial y}\right) \\ &= \frac{\partial}{\partial y}\left(2y\right) \\ &= 2 \end{align*}
\begin{align*} \frac{\partial^2 f}{\partial x\partial y} &= \frac{\partial}{\partial x}\left(\frac{\partial f}{\partial y}\right) \\ &= \frac{\partial}{\partial x}\left(0\right) \\ &= 0 \end{align*}
\begin{align*} \frac{\partial^2 f}{\partial y\partial x} &= \frac{\partial}{\partial y}\left(\frac{\partial f}{\partial x}\right) \\ &= \frac{\partial}{\partial y}\left(0\right) \\ &= 0 \end{align*}
Therefore, the Hessian matrix of $$f(x, y) = x^2 + y^2$$ is:
$$H = \begin{pmatrix} 2 & 0 \\ 0 & 2\\ \end{pmatrix}$$
To show that the Hessian matrix of $f(x, y) = x^2 + y^2$ is positive semidefinite, i need to demonstrate that for any vector $x = [u, v]^T$, the quadratic form $x^T H x$ is non-negative:
\begin{align*} x^T H x &= \begin{bmatrix} u & v \end{bmatrix}^T \begin{pmatrix} 2 & 0 \\ 0 & 2 \end{pmatrix} \begin{bmatrix} u \ v \end{bmatrix} \\ &= \begin{bmatrix} 2u & 2v \end{bmatrix} \begin{bmatrix} u \ v \end{bmatrix} \\ &= 2u^2 + 2v^2 \\ &= 2(x_1^2 + x_2^2) \end{align*}
Since $x^T H x$ is equal to 2 times the sum of squares of $x$'s components, it is non-negative for any choice of $x$.
Therefore, the Hessian matrix of $f(x, y)$ is positive semidefinite and therefore the function is convex.Am I correct ?
Great job! You are correct.
Another way to show positive semidefiniteness of the matrix is note that all the eigenvalues are nonnegative.
For a diagonal matrix, the eigenvalues are the diagonal entries. In your case, the eigenvalues are $2 \ge 0$.