How does positive (semi)definiteness help with showing convexity of quadratic forms?

260 Views Asked by At

I wish to prove the following two claims from definition:

For all $x\in \mathbb{R}^n$,

  • Let $A$ be PD, then we claim that $f(x) = x^TAx$ is strictly convex.

  • Let $A$ be PSD, then we claim that $f(x) = x^TAx$ is convex.

However, I am not seeing how convexity helps. Consider the following.

Suppose that $A$ is PSD. Take two vectors $x,y \in \mathbb{R}^n$ and $\forall \alpha \in (0,1)$,

$f(\alpha x + (1-\alpha)y) = (\alpha x + (1-\alpha)y)^TA(\alpha x + (1-\alpha)y) = \alpha^2 f(x) + (1-\alpha)^2 f(y)$.

It is not obvious from here how the property on $A$ helps. Also, since $\alpha \in (0,1)$, wouldn't the square on $\alpha$ and $1-\alpha$ make both terms smaller? Hence it doesn't seem to be true that $f(\alpha x + (1-\alpha)y) = \alpha^2 f(x) + (1-\alpha)^2 f(y) \leq \alpha f(x) + (1-\alpha) f(y)$.

1

There are 1 best solutions below

0
On BEST ANSWER

$$\eqalign{f(t x + (1-t) y) - t f(x) - (1-t) f(y) &= (t x^T + (1-t) y^T) A (t x + (1-t) y) - t x^T A x - (1-t) y^T A y \cr &= t^2 (x-y)^T A (x-y) + t ((x-y)^T A y + y^T A (x-y) - x^T A x + y^T A y)\cr &=(t^2-t) (x-y)^T A (x-y) \le 0}$$ for all $x,y$ with $0 \le t \le 1$ if $A$ is positive semidefinite, and $< 0$ if $0 < t < 1$ with $x \ne y$ and $A$ positive definite.