I have a problem with finding:
$$\max_{a \in \mathbb{R}^p} \frac{a’Ca}{a’Ba},$$
where $C$ and $B$ are $p \times p$ symmetric matrices.
After differentiation I get the following result:
$$\frac{2a’C(a’Ba) - 2a’B(a’Ca)}{(a’Ba)(a’B’a)} =0$$
I can’t solve the above equation for $a$. How to find it?
On
Denote the scalar which you wish to optimize as $$\lambda = \frac{a^TCa}{a^TBa}$$ Take your gradient result, transpose it, multiply it by $\,\frac{a^TBa}{2},\,$ and write it as $$Ca = \lambda Ba$$ This is a generalized eigenvalue problem and you want the eigenvector corresponding to the largest such eigenvalue.
If $\,B^{-1}$ exists, then this can be reduced to an ordinary eigenvalue problem for $L=B^{-1}C$. $$La = \lambda a$$
If $\,C^{-1}$ exists, then it becomes an eigenvalue problem for the reciprocal eigenvalues $\big(\mu=\frac{1}{\lambda}\big)$ of the matrix $M=C^{-1}B$. $$Ma = \mu a$$
The maximum won't generally exist. It can be guaranteed to exist, however, if $B$ is positive definite.
In this case, suppose that $B = M'M$ for some invertible matrix $M$ (we can compute $M$ via a Cholesky decomposition for instance). We then have $$ \max_a \frac{a'Ca}{a'Ba} = \max_a \frac{a'Ca}{a'M'Ma} = \max_a \frac{a'Ca}{(Ma)'(Ma)} \\ = \max_b \frac{(M^{-1}b)'C(M^{-1}b)}{b'b} = \max_b \frac{b'([M']^{-1}CM^{-1})b}{b'b}. $$ By the Rayleigh-Ritz theorem, this maximum is the largest eigenvalue of $[M']^{-1}CM^{-1}$.