What is the second-order Taylor expansion of a function $f : \mathbb C^n \to \mathbb R$?

171 Views Asked by At

Consider for example $f(x)=\|Ax-b\|_2^2$, where $A \in \mathbb C^{m \times n}$, $x \in \mathbb C^n$, $b \in \mathbb C^m$.

$y := Ax - b \implies dy = A dx \implies dy^* = A^* dx^*$.

Taking the Wirtinger derivative of $f = y^* : y$ and using all the tricks that I learned from greg gives

\begin{align} df & = dy^* : y + y^* :dy \\ & = A^* dx^* : y + y^* : A dx \\ & = A^H y : dx^* + A^T y^* : dx. \end{align}

Thus $g := \frac{\partial f}{\partial x} = A^T y^*.$ Taking the derivative gives

$$ dg = A^T dy^* = A^T A^* dx^* \implies \frac{\partial g}{\partial x^*} = A^TA^*. $$

In the end we have $h := \frac{\partial g}{\partial x} = A^H A.$

Is it true that the second-order Taylor expansion at a point $x_0 \in \mathbb C^n$ is given by the following equation?

\begin{align} \hat f(x) & = f(x_0) + \mathrm{Re}( g^H (x - x_0) ) + \frac 1 2 (x - x_0)^H h (x - x_0) \\ & = f(x_0) + \mathrm{Re}((A^H (A x_0 - b))^T (x - x_0)) + \frac 1 2 \|A(x - x_0)\|_2^2. \end{align}