Show that $H_f$ is positive definite

55 Views Asked by At

Let $V$, $D$ be $n\times n$ positive definite matrices, with $D$ diagonal. Let $k<n$ and let $f:\mathbb R^{n\times k}\to \mathbb R$ be the function defined by

$$f(X)=\log \det |\Sigma|+ \text{Tr}(\Sigma^{-1}V)$$

where $\Sigma=XX'+D$. Consider the problem if minimizing $f$. Using the rules of matrix differentiation we find that

$$\frac {\partial f}{\partial X}=2\Sigma^{-1}X-2\Sigma^{-1}V\Sigma^{-1}X=2[I_n-\Sigma^{-1}V]\Sigma^{-1}X$$

Hence a necessary condition for a local minimum is $$\frac {\partial f}{\partial X}=0 \iff \Sigma^{-1}X=\Sigma^{-1}V\Sigma^{-1}X \quad (1)$$

I want to show that $(1)$ is also sufficient for a local minimum. I computed the Hessian:

$$H_f:=\frac {\partial f}{\partial vec(X)\partial vec(X)^\top}=$$

$$2[X'\Sigma^{-1}V\Sigma^{-1}X\otimes \Sigma^{-1}]+2[X'\Sigma^{-1}V\Sigma^{-1}\otimes \Sigma^{-1}X]K^{n,k}$$

$$-2[X'\Sigma^{-1}X\otimes(\Sigma^{-1}-\Sigma^{-1}V\Sigma^{-1})]-2[X'\Sigma^{-1}\otimes(\Sigma^{-1}X-\Sigma^{-1}V\Sigma^{-1}X)]K^{n,k}$$

$$+2[I_k\otimes(\Sigma^{-1}-\Sigma^{-1}V\Sigma^{-1})]$$

where $K^{n,k}$ denotes the commutation matrix of order $(n,k)$. One can check that $H_f$ is symmetric using the commutation property of $K^{n,k}$. If $(1)$ is satisfied then we get the simplification

$$H_f^{(1)}=2[I_k\otimes (\Sigma^{-1}-\Sigma^{-1}V\Sigma^{-1})]+ 2[X^\top\Sigma^{-1}X\otimes \Sigma^{-1}V\Sigma^{-1}]$$ $$+2[X^\top \Sigma^{-1}\otimes \Sigma^{-1}V\Sigma^{-1}X]K^{n,k}$$

where the third term is symmetric.

Question: Suppose $X$ has full column rank and $\Sigma^{-1}-\Sigma^{-1}V\Sigma^{-1}=0$. Then is $H_f^{(1)}$ positive definite?