Let $b \in \mathbb{C}^n$, i'm trying to find the min for the following function:
$H(b) = b^\star P b - q^\star b - b^\star q + s$
I know it's gonna be $b = P^{-1} q$, as P is invertible (also positive semi-definite), but i've seen how to reach it without the term $b^\star q$ where $H'= Pb - q$ and it follows from solving the linear system.
But i'm not used to working with this kind of objective functions (or optimization at all), how can i work with this?
On
Writing $b = x + yi$, you get $$H(b) = (x-yi)^T P (x+yi) - q^T(x+yi) - q^T(x-yi) + s=x^TPx + y^TPy-q^Tx+s. $$ If $P$ is positive definite, you can take the derivative to get the minimum: $2Px-q=0$ and $2Py=0$, so $x=0.5P^{-1}q$ and $y=0$.
On
Hint.
Assuming $P\in \mathbb{R}^{n\times n}$ and calling
$$ \cases{ b = x + i y\\ b^*= x - i y\\ q = u + i v\\ q^*= u - i v } $$
we have
$$ H(b) = (x-i y) P (x+ i y) - (u-i v) (x+i y) -(x-i y) (u+i v)+ s $$
or
$$ H(x,y) = x'\cdot P\cdot x + y'\cdot P\cdot y-2(u'\cdot x+v'\cdot y)+s $$
or
$$ H(x,y) = \left(\begin{array}{c}x \\ y\end{array}\right)'\left(\begin{array}{cc}P & 0 \\ 0 & P\end{array}\right)\left(\begin{array}{c}x \\ y\end{array}\right)-2\left(\begin{array}{c}u \\ v\end{array}\right)'\left(\begin{array}{c}x \\ y\end{array}\right)+s $$
etc.
You can do it by completing the square.
Try introducing an $a$ and a $c$ like this, and then solve for them to match your expression
$H(b) = (b^\star -a^\star)P(b-a) + c = b^\star P b - a^\star P b - b^\star P a - a^\star P a + c$
Comparing with your expression,
$H(b) = b^\star P b - q^\star b - b^\star q + s$
we see that to match up we need $b^\star P a = b^\star q$ and $a^\star P b = a^\star b$. Because your matrix is positive definite, then $q^\star P^{-1} = (P^{-1}q)^\star$, so we can solve both conditions simultaneously to get $a = P^{-1} q$. We also need $c = s - q^\star P^{-1} q$.
Now we have
$H(b) = (b^\star - q^\star P^{-1})P(b-P^{-1}q) + s - q^\star P^{-1} q$,
and your function is written explicitly as a square plus a constant. The minimum of this function will be when $b-P^{-1}q = 0$, with minimum value $H_{min} = s - q^\star P^{-1} q$ at that point.
It may already be clear to you why $b=P^{-1}q$ minimises the function, but if it's not yet then you can see this by diagonalising the matrix. $P$ is positive definite and hence $P = P^ \star$, and any such matrix can be diagonalised by a unitary matrix. This means that there exists an $S$ with $S^\star = S^{-1}$ such that $P = S^\star D S$, where $D$ is diagonal and every diagonal element is positive. Then
$H(b) = (b^\star - q^\star P^{-1})S^\star D S(b-P^{-1}q) + s - q^\star P^{-1} q$
Defining a new variable $y = S(b-P^{-1}q)$, then
$H(y) = y^\star D y + s - q^\star P^{-1} q = \sum_i \lambda_i |y_i|^2 + s - q^\star P^{-1} q$,
where $\lambda_i > 0$ are the eigenvalues of $P$, and $y_i \in \mathbb{C}$ are the coordinates of $y$. Hopefully it's now clear to inspect that this function is minimised by $y = 0$, which implies that $b=P^{-1}q$.