nth derivative of determinant wrt matrix

Question

I'm working on an expression for the nth derivative of a (symmetric) matrix, i.e. \begin{equation}\frac{\partial^{n} \det(A)}{\partial A^{n}}\end{equation} Starting with \begin{equation}\frac{\partial \det(A)}{\partial A}=\det(A) A^{-1}\end{equation} Then naturally the next derivative is \begin{equation}\frac{\partial^{2}\det(A)}{\partial A^{2}}=\frac{\partial}{\partial A}\left(\det(A)A^{-1}\right)=\det(A)A^{-2}-\det(A)A^{-2}=0\end{equation} I doubt this is right, can someone point out my mistake? I'm actually working on an expression for the nth derivative of $\det(A)^{-1/2}$ but a general formula for the simple case would be fine.

Answer 1

I have to admit this example is pretty interesting. There are a few mistakes here.

At first, the derivative of the determinant of a symmetric matrix w.r.t itself is $$ \frac{\partial}{\partial \mathbf{X}} \det(\mathbf{X}) = \det(\mathbf{X}) \, (2 \mathbf{X}^{-1} - (\mathbf{X}^{-1} \circ \mathbf{I})) $$ (where $\circ$ denotes Hadamard product) is no long the formula you wrote for an invertible matrix with no special structure. The reason can be found in this post.

Second, two derivatives of $\det A$ and $A^{-1}$ with respect to $A$ has totally different interpretations. $$\left(\frac{\partial}{\partial A}\det A\right)_{ij}=\frac{\partial \det A}{\partial a_{ij}}$$ is a matrix composing of different derivatives w.r.t scalar. While $$\frac{\partial}{\partial A} A^{-1}(B)=\lim_{t\to 0}\frac{(A+tB)^{-1}-A^{-1}}{t}=-A^{-1}BA^{-1}\neq-A^{-2}B$$ is Fretchet derivative, or directional derivative along $B$. To use chain rule, you have to unify the definition of derivatives.

But I think the most important thing is that you have to make sure why do you want to take second derivative? Notations serve for mathematics, but mathematics doesn't explain notations.

Answer 2

Tee-Jay, that you call a derivative is in fact a gradient. Let $f\colon U\mapsto \det(U)$. We assume that $A$ is invertible. The derivative is a linear application:

$Df_A\colon H\mapsto \det(A)\operatorname*{trace}(HA^{-1})$.

The second derivative is a symmetric bilinear form:

$$D^2f_A\colon(H,K)\mapsto \det(A)\operatorname*{trace}(KA^{-1})\operatorname*{trace}(HA^{-1})+\det(A)\operatorname*{trace}(H(-A^{-1}KA^{-1})) \qquad\qquad\qquad\qquad\qquad\quad=\det(A)(\operatorname*{trace}(HA^{-1})\operatorname*{trace}(KA^{-1})-\operatorname*{trace}(HA^{-1}KA^{-1})).$$

In particular, the associated quadratic form is:

$$D^2f_A(H,H)=\det(A)((\operatorname*{trace}(HA^{-1}))^2-\operatorname*{trace}((HA^{-1})^2).$$

EDIT 1: let $(\lambda_i)$ be the spectrum of $HA^{-1}$, $(\sigma_i)$ be the elementary symmetric polynomials associated to the $(\lambda_i)$ and $S_k=\sum_i {\lambda_i}^k=\operatorname*{trace}((HA^{-1})^k)$. Then $$Df_A(H)=\det(A)\sigma_1,D^2f_A(H,H)=\det(A)2\sigma_2.$$ In fact we can generalize this result.

$\det(A+H)=\det(A)\det(I+HA^{-1})$ and we may assume that $A=I$ and $(\lambda_i)$ is the spectrum of $H$. Then, according to the Taylor formula, $$\det(I+H)=\Pi_i(1+\lambda_i)=1+\sum_{k=1}^n\sigma_k=1+\sum_{k=1}^n 1/k!D^kf_I(H,\cdots,H).$$ By identifying the terms of the same degree, we obtain:

$$D^kf_I(H,\cdots,H)=k!\sigma_k=k!P_k(S_1,\cdots,S_k)=k!P_k(\operatorname*{trace}(H),\cdots, \operatorname*{trace}(H^k)$$ where $P_k$ is the polynomial given by the Newton's identities, cf., http://en.wikipedia.org/wiki/Newton_identities

For instance, $$D^1f_I=S_1,D^2f_I={S_1}^2-S_2,D^3f_I={S_1}^3-3S_1S_2+2S_3,\\ D^4f_I={S_1}^4-6{S_1}^2S_2+3{S_2}^2+8S_1S_3-6S_4.$$

EDIT 2: Example. Let $A,H\in GL_n(\mathbb{C})\times M_n(\mathbb{C})$. $$\det(A+H)=\det(A)(1+S_1+1/2({S_1}^2-S_2)+1/6({S_1}^3-3S_1S_2+2S_3)+O(\lVert{H}\rVert^4))$$ where $S_k=\operatorname*{trace}((HA^{-1})^k)$.