Gradient of $x \mapsto \log \left( \det \left( \dfrac {3}{10}I+xx^{T}\right) \right) $

Question

Let $f : \mathbb{R}^{n} \to \mathbb{R}$ be defined by

$$f (x) := \log \left( \det \left( \dfrac {3}{10}I+xx^{T}\right) \right)$$

Calculate the gradient $\nabla f$.

---

I have been struggling to calculate $\nabla f$. My attempt is as follows.

$$\dfrac {df}{dx} = \left( \det \left( \dfrac {3}{10}I+xx^{T}\right) \right)^{-1} \cdot\dfrac {dg}{dx}$$

where $m : = \left(\dfrac {3}{10}I+xx^{T}\right)$

$$g := \det(m) = \det \left( \dfrac {3}{10}I+xx^{T}\right)$$

$$ \dfrac {dg}{dx} = \det \left( \dfrac {3}{10}I+xx^{T}\right).Tr( \left( \dfrac {3}{10}I+xx^{T}\right)^{-1}.\dfrac {dm}{dx}) $$

$$\dfrac {dm}{dx} = \dfrac {d(xx^{T})}{dx}$$

is an $n \times n \times n$ tensor, and $Tr(\dfrac {dm}{dx})= 2x$. So, I get

$$\dfrac {df}{dx} = Tr(\left( \dfrac {3}{10}I+xx^{T}\right)^{-1}.\dfrac {d(xx^{T})}{dx})$$

A second question is about checking this or other complex multivariate differentials, I know my attempts are wrong by using autograd in Python. It would be really helpful to know if this is possible with SymPy or something similar, or it is rarely done.

Answer 1

Let function $f : \mathbb{R}^n \to \mathbb{R}$ be defined by

$$f (\mathrm x) := \log \left( \det \left( \gamma \, \mathrm I_n + \mathrm x \mathrm x^\top \right) \right)$$

where $\gamma = \frac{3}{10}$. Using the matrix determinant lemma,

$$\det \left( \gamma \, \mathrm I_n + \mathrm x \mathrm x^\top \right) = \cdots = \gamma^n \left( 1 + \frac{1}{\gamma} \mathrm x^\top \mathrm x \right)$$

and, thus,

$$f (\mathrm x) = n \log (\gamma) + \log \left( 1 + \frac{1}{\gamma} \mathrm x^\top \mathrm x \right)$$

Taking the partial derivative with respect to $x_i$,

$$\partial_{x_i} f (\mathrm x) = \frac{\frac{2}{\gamma} x_i}{1 + \frac{1}{\gamma} \mathrm x^\top \mathrm x} = \left(\frac{2}{\gamma + \mathrm x^\top \mathrm x}\right) x_i$$

and, hence, the gradient of $f$ is

$$\nabla f (\mathrm x) = \color{blue}{\left(\frac{2}{\gamma + \mathrm x^\top \mathrm x}\right) \mathrm x}$$

Answer 2

Let $M = \left( xx^T + 0.3I \right)$ and $f = \log \det \left( M \right)$.

We will utilize the following the identities

• Trace and Frobenius product relation $$A:B={\rm tr}(A^TB)$$ or $$A^T:B={\rm tr}(AB)$$
• Cyclic property of Trace/Frobenius product $$\eqalign{ A:BC &= AC^T:B \cr &= B^TA:C \cr &= {\text etc.} \cr }$$
• Jacobi's formula (for nonsingular matrix $M$) in terms of differential $$d\log \det \left( M \right) = d{\rm tr}\log\left( M \right) .$$

Now, we obtain the differential first and thereafter we obtain the gradient.

So, \begin{align} df &= d \log \det \left( M \right) \\ &= d \ {\rm tr}\left( \log\left( M \right) \right) \hspace{8mm} \text{note: utilized Jacobi's formula} \\ &= {\rm tr} \left( M^{-1} dM \right) \\ &= M^{-T} \ : \ dM \hspace{8mm} \text{note: utilized trace and Frobenius relation} \\ &=M^{-1}\ : \ \left( dxx^T + xdx^T\right)\\ &= 2M^{-1}x \ : \ dx \\ \end{align}

So, the derivative of $f = \log \det \left( xx^T + 0.3I \right)$ with respect to $x$ is \begin{align} \frac{\partial}{\partial x} f = \frac{\partial}{\partial x} \log \det \left( xx^T + 0.3I \right) = 2 \left(xx^T + 0.3I\right)^{-1} x .\\ \end{align}