Critical Points of Quadratic Forms

Question

just a question about finding critical points (points where the differential is not surjective). I have the equation $$ f(x) = x^tAx $$ where $A$ is a symmetric $n$ by $n$ matrix and $x$ is an element of $\mathbb{R}^n$. Now, I've found the derivative of the function to be $$ df(x) = 2x^tA = 2Ax $$ by symmetry of A. I'm having trouble determining values in $\mathbb{R}^n$ where the derivative is not surjective. It's clear that for $x = (0,...,0)$, the derivative is not surjective, but my intuition is that for any other value of $x$, and for any value in $\mathbb{R}^n$, we can choose a corresponding $A$. Am I thinking about this correctly? Is my derivative correct for the quadratic form $ f(x) = x^tAx $?

Answer 1

As I said in my comment, because $f$ is a map from $\mathbb R^n$ to $\mathbb R$, its derivative at a point $x\in\mathbb R^n$ will be a linear map $df_x\colon \mathbb R^n\to \mathbb R$. You can compute the value of this linear map applied to a vector $y\in\mathbb R^n$ by taking the ordinary derivative (with respect to $s$) of the smooth function $f(x+sy)$, and setting $s=0$. Thus \begin{align*} df_x(y) &= \left.\frac{d}{ds}\right|_{s=0} \big((x+sy)^t A (x+sy)\big)\\ &= \left.\frac{d}{ds}\right|_{s=0} \big(x^tAx + s y^tAx + sx^tAy + s^2 y^tAy\big)\\ &= y^tAx + x^t Ay. \end{align*} Because each term above is a scalar quantity (i.e., a $1\times 1$ matrix, which is equal to its own transpose) and $A$ is symmetric, this can be written as \begin{align*} df_x(y) &= (y^tAx)^t + x^t Ay = x^t A^ty + x^t Ay= 2 x^t A y. \end{align*} Therefore, $df_x$ is the linear map represented by the row matrix $2x^tA$. (But this is not equal to $2Ax$, because the latter is a column matrix, and therefore doesn't represent a linear map from $\mathbb R^n$ to $\mathbb R$.)