I know that the gradient of the determinant function $\mathrm{det}:\mathbb{R}^{n\times n}\to\mathbb R$ at point $A\in\mathbb{R}^{n\times n}$ is the matrix $C$ of cofactors of $A$ (easy to show using Laplace expansion). Using explicit expansion of determinants in terms of matrix entries, I can show that:
- ($n=1$) The "cofactor matrix" of $x\in\mathbb R=\mathbb R^{1\times 1}$ is always $1\in\mathbb R^{1\times 1}$ (even when $x=0$). This is consistent with $\det x=x\therefore \det'x=1$.
- ($n=2$) The cofactor matrix of $A\in\mathbb R^{2\times 2}$ is zero if and only if $A=0$. (Trivial)
- ($n=3$) The cofactor matrix of $A\in\mathbb R^{3\times 3}$ is zero if and only if $A$ has rank $1$. (Tricky, but can be done in one page of manual computations by considering separately $a_{11}=0$ and $a_{11}\neq 0$.)
This looks like it generalizes as "the cofactor matrix of $A$ is zero if and only if $A$ has rank $\leqslant n-2$", which looks intuitive to me: the determinant, being a polynomial in the matrix entries, will have a zero derivative wherever there's a zero of order two or more. I can't formalize this argument, however. How should I do it? Is it actually true?
[Context: Reading the section on the implicit function theorem, my textbook mentions the level surfaces with level $\neq 0$ of the determinant function as examples of $C^\infty$ manifolds embedded in $\mathbb R^{n\times n}$. I wanted to explore what the (non-differentiable) level surface of level 0 looked like---in particular, at which points it is guaranteed to be differentiable.]
The rank of $A$ is equal to the dimension of the span of its columns. And the determinant is non-zero iff this is $n$. Now consider what happens if the rank of $A$ is $n-2$ or lower. Then there is no single column you can change to make the rank of $A$ increase to $n$. In particular, there is no single entry you can change. Thus the gradient of the determinant at $A$ is $0$.
On the other hand, if the rank of $A$ is $n-1$, then take a column $v$ which is a linear combination of the other columns. It has $n$ different entries you can change. If changing the $i$th entry makes $v$ still a linear combination of the other columns, then the change itself, $[0,0,\ldots,0,1,0,\ldots,0]^T$ must also be a linear combination of those other columns. But we can't have all basis column vectors be linear combinations of the $n-1$ other column vectors.
Thus there must be at least one component of $v$ which, when changed, makes $v$ not a linear combination of the other columns. Changing this component makes the rank of $A$ equal to $n$, and the determinant non-zero.