I need to find a minimum for
$f(x,y)=Y^T(x,y)A(x,y)Y(x,y)$
where $A(x,y)$ is a symmetric matrix and $Y(x,y)$ is a vector whose components are functions of $x$ and $y$.
Eg in $\mathbb{R}^2$:
$$Y(x,y) = \left(\begin{array}{c}{g(x,y) \\ h(x,y)}\end{array}\right)$$
where $g,h:\mathbb{R}^2\rightarrow\mathbb{R}$. Similarly for $A(x,y)$
This is in $\mathbb{R}$ (but It would also be interesting to consider the complex case)
How can I find a matricial expression for the gradient of $f(x,y)$?
So you want to find the derivative of the quadratic form $f(x,y)=(x,y)^{T}A(x,y)$ $$f(x,y) = (x,y)^{T}\cdot\left(x\left(\begin{array}{c} a_{1,1}\\ a_{2,1} \end{array}\right)+y\left(\begin{array}{c} a_{2,1}\\ a_{2,2} \end{array}\right)\right)$$ $$= (x,y)^{T}\cdot\left(\begin{array}{c} xa_{1,1}+ya_{1,2}\\ xa_{2,1}+ya_{2,2} \end{array}\right)$$ $$= x\left(xa_{1,1}+ya_{1,2}\right)+y\left(xa_{2,1}+ya_{2,2}\right)$$ $$=x^{2}a_{1,1}+2xy\cdot a_{1,2}+y^{2}a_{2,2}$$
where we used the fact that $A$ is symmetric for the last equality.
so, you can take the partial derivative of $f(x,y)=x^{2}a_{1,1}+2xy\cdot a_{1,2}+y^{2}a_{2,2}$ with respect to $x$ and $y$ to get $\nabla f =2\cdot A(x,y)$. I'll leave those details for you to fill in.