Proving that the solution of a norm constrained optimization is on the boundary of the set

Question

I am trying to solve the following maximization problem

$$\max_{||x|| \leq c} x^H A x,$$ where matrix $A$ is hermitian symmetric.

I have been told that the argument of the maximum is on the ellipsoid {$||x|| = c$}, but am finding it difficult to prove it. Any help will be greatly appreciated.

Answer 1

The following discussion will assume the quadratic form $x^H A x$ has only real values (so that the "max" operator can be directly applied to these values), although it is not necessarily positive definite.

The maximum will be attained on the boundary $||x|| = c$ (assuming $c \ge 0$) provided $A$ is not negative definite. That is, if $x^H A x \ge 0$ for some $x$ with norm less than $c$, then:

$$ \frac{c x^H}{||x||} A \frac{c x}{||x||} \ge x^H A x $$

since $c^2 \gt ||x||^2$.

However if $x^H A x \lt 0$ for all nonzero $x$ (negative definite), then the maximum occurs only at $x = 0$. For a discussion of negative definite (resp. positive definite) implying Hermitian (self-adjoint) in the complex vector space setting, see the Wikipedia discussion.