I have the following energy function and I need to calculate its derivative with respect to $\alpha$ and I don't know how to do it. I would really appreciate if anyone can help.
$$ E=\alpha ^{T}L\alpha ^{T}+\lambda (\alpha-\beta )^{T}D(\alpha -\beta )$$
I need to find $$\frac{dE}{d\alpha }$$
where $L$ and $D$ are $n\times n$ matrices.
Here is a sketch of the solution.
First, notice that the derivative of $E$ with respect to $\alpha$ is really the gradient $\nabla _\alpha E$.
We can also expand the expression $$ E=\alpha^TL\alpha+\lambda(\alpha−\beta)^TD(\alpha−\beta) $$ into $$ E=\alpha ^T L \alpha + \lambda (\alpha ^T D \alpha-\alpha ^T D \beta-\beta^T D\alpha+\beta ^TD\beta ) $$
The terms without $\alpha$ in them will fall away, so we can ignore the $\beta^T D \beta$ term. Now we just have some inner products and quadratic forms. For the derivative of a quadratic form, suppose we take $a^T X a$ and take the gradient with respect to $a$. That would mean we take $$ \begin{split} \lim _{h \to 0}& \frac{(a+h\vec v)^TX(a+h\vec v)-a^TXa}h\\ &=\lim _{h \to 0} \frac{(a+h\vec v)^TX(a+h\vec v)-a^TXa}h\\ &=\lim _{h \to 0} \frac{a^T X a+h\vec v^TXa+a^TXh\vec v-a^TXa}h\\ &=\lim _{h \to 0} \frac{h\vec v^TXa+a^TXh\vec v+h^2\vec v^TX\vec v}h\\ &=\lim _{h \to 0} \vec v^TXa+a^TX\vec v+h\vec v^TX\vec v\\ &=\vec v^TXa+a^TX\vec v \end{split} $$
Now in order to obtain the gradient, we would simply take $\vec v$ to be each of the standard basis vectors, as that would be taking the partial with respect to each component of $\alpha$. This gives us that $$ \frac{\partial}{\partial a_i} a^TXa = [X^T a + X a]_i $$ as we have reduced the derivative of the quadratic form to taking the derivative of a dot product, which will simply be the ith component of vector we are taking the dot product with. You can apply this formula, and the derivative of dot product on the expanded expression to obtain the derivative of your energy function.