Let $$ \alpha = x^TAx \enspace x\in \mathbf{R}^{nx1}, A \in \mathbf{R}^{nxn} $$
How do I compute the derivative $\Large \frac{\partial\alpha}{\partial x}$ without using the coordinate way i.e, writing A in terms of $\large A = (a_{ij})$
My attempt is as follows (using the product rule):
$$ \begin{align} \frac{\partial\alpha}{\partial x} = &\frac{\partial(x^TAx)}{\partial x}\\ =&\frac{\partial(x^TA)x}{\partial x} + x^TA\frac{\partial x}{\partial x}\\ =&\frac{\partial(A^Tx)^Tx}{\partial x} + x^TAI\\ =&\left(\frac{\partial A^Tx}{\partial x}\right)^Tx+ x^TA\\ =&Ax+x^TA \end{align} $$
This is different from the answer that I find in wikipedia which says is to be:
$$ \frac{\partial\alpha}{\partial x} = x^TA+x^TA^T $$
Where am I going wrong, please present your answer in the product rule form.
The product rule is not valid for the gradients of matrix/vector expressions.
For example $$\frac{\partial(a^Tb)}{\partial x} \ne \Bigg(\frac{\partial a^T}{\partial x}\Bigg)b + a^T\Bigg(\frac{\partial b}{\partial x}\Bigg)$$ However, the product rule is valid for the differentials of such expressions $$d(a^Tb) = (da^T)b + a^T(db)$$
Note that the gradient of a vector is a matrix, which behaves differently than a vector.
But the differential of a vector is just another vector, and behaves accordingly.
So let's approach your problem using differentials. $$\eqalign{ \alpha &= x^TAx \cr d\alpha &= dx^T\,Ax + x^TA\,dx \cr &= x^TA^T\,dx + x^TA\,dx \cr \frac{\partial \alpha}{\partial x} &= x^TA^T + x^TA \cr\cr }$$