Second Hadamard Variation Formula

Question

For a symmetric matrix $H$ that depends smoothly on a variable $t$, let $u_i$ be the eigenvector corresponding to eigenvalue $\lambda_i$. From the definition $$ H(t) u(t) = \lambda_i(t) u(t) $$ and $u(t)^T u(t) = 1$, we can easily take the derivative to find that $$ H'(t) u(t) + H(t) u'(t) = \lambda_i'(t) u(t) + \lambda_i(t) u'(t) $$ and $$ u_i'^T u_i + u_i^T u_i' = 0. $$ Taking the inner product of the first equation with $u_i$ and incorporating the second immediately yields $$ \lambda'(t) = u_i(t)^T A'(t) u_i(t) $$ which is known as Hadamard's first variation formula. I would like to find a simple proof for Hadamard's second variation formula: $$ \lambda_i''(t) = u_i(t)^T A''(t) u_i(t) + 2 \sum_{j \neq i} \frac{|u_j^T A'(t) u_i|^2}{\lambda_i -\lambda_j}. $$ I tried taking two derivatives of the definitional equations but couldn't find a way to incorporate the other eigenvalues. Also, we can assume that all the eigenvalues are distinct.

Answer 1

Consider the equation you got $$ H'(t) u_i(t) + H(t)u_i'(t) = \lambda_i'(t)u_i(t) + \lambda_i(t)u_i'(t)$$ Dot product the whole equation with $u_j(t)$ for $j \neq i$, to get $$ u_j(t)^\top H'(t) u_i(t) + \lambda_j(t)u_j(t)^\top u_i'(t) = \lambda_i(t)u_j(t)^\top u_i'(t)$$ which simplifies to

$$u_j(t)^\top H'(t)u_i(t) = (\lambda_i(t) - \lambda_j(t))u_j(t)^\top u_i'(t) $$ Call this equation $\alpha$.

Now differentiate the equation you got for the first derivative of $\lambda_i(t)$, to get $$ \lambda''_i(t) = u_i(t)^\top H''(t) u_i(t) + 2 u_i'(t)^\top H'(t) u_i(t)$$ Call this equation $\beta$.

Now, we know that $u_i'(t)$ is orthogonal to $u_i(t)$ and that the $u_i(t)$ form a orthonormal basis. So, we can write $$u_i'(t) =\sum\limits_{j \neq i} u_i'(t)^\top u_j(t) u_j(t)$$. Using equation $\alpha$, we get, $$u_i'(t) =\sum\limits_{j \neq i} \frac{u_j(t)^\top H'(t) u_i(t)}{\lambda_i-\lambda_j} u_j(t)$$ Plugging this into the equation $\beta$, we get the desired equality. Let me know if you have any questions. Cheers