I have a cost function $f= a^TK_P^TK_Pa + b^TK_Pa$, where $a$ and $b$ are known user defined 3*1 real valued vectors and $K_P$ is an unknown real valued $3*3$ positive definite matrix. Could anyone give me an analytical method to find the optimal matrix $K_P$ to minimize the above cost function using Lagrange multipliers method?
The fact that $K_P$ has to be positive definite, is troubling me as that inequality constraint is real hard to incorporate. I'd really appreciate if I find an easier way.
Assuming $a\ne 0$.
There is a minimizer if and only if $a^T b < 0$!
Let $f(K) = a^T K^T K a + b^T K a$. Then, we have $$ Df(K)[H] = 2 a^T K^T H a + b^T H a = \operatorname{tr}((2Kaa^T + ba^T)^T H ),$$ that is $$ \nabla f(K) = 2Kaa^T + ba^T. $$
Now, if the gradient vanishes at a positive definite $K$ it follows $$ a^T b = -2a^T K a < 0. $$
On other hand one solution is given by $$ Ka = -\frac12 b. $$ Define $$ K = \left(I - \frac{ab^T}{a^T b}\right)^T \left(I - \frac{ab^T}{a^T b}\right) - \frac12\frac{bb^T}{a^T b}. $$ Then, $K$ is positive definite and $$ Ka = \left(I - \frac{ab^T}{a^T b}\right)^T (a - a) - \frac12 b = -\frac12 b. $$