I want to take the derivative with respect to $o$ and $V$ of: $$-\log |I + \frac {1} {o^2} V^T V|$$
The formulate for the log-determinant derivative is define on page 15 of these notes.
$$\frac d {d \theta_k} \log (\det (V(\theta))) = \mathrm{tr} (V^{-1} \frac d {d \theta_k} V^t(\theta))$$
I am guessing that $V^t = V^T$ here. Right now I am thinking of how to implement this and am finding myself confused because of that $V^T V$. I know that derivative of $AB$ with respect to $A$ is $\delta B^T$, but I've never given it thought as to what I would do when the errors are not given directly. I am not that sure about the trace either.
How could I use the formula as part of the backpropagation process?
I noticed that $\partial \mathrm{tr} (X) = \mathrm{tr} (\partial X)$. Since trace can be defined as $\sum diag(X)$, in order to backprop through that I'd have to propagate through the sum and then through the $diag$. Propagating through the sum is straightforward and all that propagating through the diag does is add adjoints to the diagonal elements.
If the cost function was $$ tr (I + \frac {1} {o^2} V^T V)$$ instead of what I wrote out above, I'd just set the error matrix $\delta$ to an identity matrix and propagate it from there. In the case of $$-\log |I + \frac {1} {o^2} V^T V|$$ all I'd have to do is make sure to propagate it through that extra inverse term which is $\partial \log \det (X) = \mathrm{tr} (X^{-1} \partial X)$. This means that at the top level the actual error would be $-IX^{-1}\partial X$ which makes things simple.