The question is if $A$ is an invertible matrix with real entries of size $n$. Is $f(A)=det(A^{-1}-A)$ differentiable? and what is the differential.
I think I managed to show it's differentiable. the determinant is a polynomial. If we derive it by $A_{ij}$ we will get either a polynomial or a constant, at any rate, it will be continuous, and so all the partial derivatives are continuous, and that implies that $f$ is differentiable.
But what is the differential?
Note that $$ \frac{d}{dt} \det(H(t))=\det(H(t))\cdot\operatorname{trace}\!\left(H(t)^{-1}\frac{d}{dt}H(t)\right) = \operatorname{trace}\!\left(H(t)^{\#}\frac{d}{dt}H(t)\right) $$ where $H^\#$ is the adjoint/adjugate matrix of $H$.
Now set $H(t)=A+tX$ to compute the directional derivatives of the function and finally the gradient or full derivative.
One could of course first simplify the expression by noting that $$ \det(A^{-1}-A)=\frac{\det(I-A)\det(I+A)}{\det(A)} $$