Let the matrix be:
$$A = \begin{bmatrix}a_1^T X a_1 & a_2^TXa_2\\ a_3^TXa_3 & a_4^TXa_4 \end{bmatrix}$$
Where $a_k \in \mathbb{R}^n$. I want to minimize the trace of inverse of this matrix so:
$$\min_{X} \qquad \frac{a_1^TXa_1+a_4^TXa^4}{a_1^TXa_1 a^T_4Xa_4-a_2^TXa_2 a_3^T X a_3}$$ $$\text{subject to:} \qquad \text{trace}(X) = 1$$ $$\text{and} \qquad X \succeq 0$$
If $n>4$ and the $a$s are linearly indepedent, I don't see why a minimum exists.
Pick an orthogonal basis $\{b_1, b_2, b_3, b_4, \ldots\}$ for $\mathbb{R}^n$ where the first four basis vectors span the same space as the $a$s, and the rest span the orthogonal complement. Pick $b_3$ and $b_4$ so that they are orthogonal to $a_1$ and $a_4$.
Now set $X$ to be the matrix whose eigenvectors are the $b_i$s, where the corresponding eigenvalues are $$\lambda_i = \begin{cases} \epsilon^3, &i=1,2\\ \epsilon^2, &i=3,4\\ \frac{1-2\epsilon^3-2\epsilon^2}{n-4}, &i>4.\end{cases} $$
For sufficiently small $\epsilon$, your objective function is negative with arbitrarily large magnitude.