Which Operation, Similar to a Log(), Keeps All Eigenvalues Positive?

Question

I have a NxN symmetric matrix C which has all positive eigenvalues.

When I take the log of C, I get negative eigenvalues in the result. If I do log(C+1) I still get negative eigenvalues...

I am looking for an operation similar to a log, that will keep the matrix with all positive eigenvalues.

Answer 1

If matrix $A = (a_{ij})$ is positive definite, so does every matrix of the form $(p(a_{ij}))^{\color{blue}{[1]}}$ where $p(x)$ is any non-constant polynomial with non-negative coefficients.

When $p(x)$ is a power of $x$, i.e. $p(x) = x^k$ for some $k > 0$, this is a corollary of Schur product theorem. Since positive definite matrices are closed under positive linear combinations, the matrix $(p(a_{ij}))$ will be positive definite when $p(x)$ is a positive linear combination of powers of $x$. i.e. when $p(x)$ has the form $\sum_{k=1}^m b_k x^k$ where $b_k \ge 0$ for all $k$ and $\ne 0$ for some $k$. Notice the matrix whose entries are all one are positive semi-definite and the sum of a positive semi-definite matrix and a positive definite matrix is positive definite. The matrix $(p(a_{ij}))$ is also positive definite when $p(x)$ is a non-constant polynomial with non-negative coefficients.

It is easy to generalize this result to power series.

If matrix $A = (a_{ij})$ is positive definite, then for any non-constant function $g(z)$ whose power series expansion has non-negative coefficients and radius of convergence $> \max |a_{ij}|$, the matrix $(g(a_{ij}))$ will be positive definite.

For example, for any positive definite matrix $A = (a_{ij})$,

• matrix $(-\log(1 - a_{ij}))$ will be positive definite if all $|a_{ij}| < 1$.
• matrix $( e^{a_{ij}} - 1)$ will always be positive definite.

Notes

• $\color{blue}{[1]}$ - we use the notation $(f(a_{ij}))$ to denote the matrix whose entries at row $i$, column $j$ equals to $f(a_{ij})$. i.e. the one obtained from $A = (a_{ij})$ by entry-wise application of function $f$.

Answer 2

Entry-wise operations don't play nicely with matrices but it turns out that analytic functions like $\exp$ and $\log$ have a natural extension to matrices.

For example,

$$ \exp(x) = \sum_{n \ge 0} \frac{x^n}{n!} $$

is the power series expansion of $\exp$. If $M$ is a matrix, we can just go ahead and plop it right in there:

$$ \exp(M) = \sum_{n \ge 0} \frac{M^n}{n!}. $$

Similarly we have a series expansion of $\log(1 + x)$:

$$ \log(1 + x) = \sum_{n \ge 1} (-1)^{n - 1}\frac{x^n}{n}. $$

Hence, for matrices,

$$ \log(I + M) = \sum_{n \ge 1} (-1)^{n - 1}\frac{M^n}{n}. $$

There is a small issue here which is that the series expansion for $\log(1 + x)$ only converges for $-1 < x \le 1$. Likewise, the matrix summation won't converge if $||M|| > 1$ where $||\cdot||$ is some kind of norm on the space of matrices.

We can get around this by noticing the following. Let $f(x) = \sum_{n\ge 0} a_n x^n$. If $M$ is diagonalizable and $M = PDP^{-1}$ with $D$ a diagonal matrix, then we have

\begin{align} f(M) &= f(PDP^{-1}) \\ &= \sum_{n \ge 0} a_n (PDP^{-1})^n \\ &= \sum_{n \ge 0} a_n \underbrace{(PDP^{-1})(PDP^{-1})\cdots(PDP^{-1})}_n \\ &= \sum_{n \ge 0} a_n PD^nP^{-1} \\ &= P \left( \sum_{n \ge 0} a_nD^n \right) P^{-1} \\ &= Pf(D)P^{-1}. \end{align}

For diagonal matrices, you should check that

$$ f \left( \begin{pmatrix} d_1 & 0 & \cdots & 0 \\ 0 & d_2 & \cdots & 0 \\ \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & \cdots & d_n \end{pmatrix} \right) = \begin{pmatrix} f(d_1) & 0 & \cdots & 0 \\ 0 & f(d_2) & \cdots & 0 \\ \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & \cdots & f(d_n) \end{pmatrix}. $$

We can use this to define $\log(I + M)$ when $M$ is diagonalizable by just applying $\log(1 + x)$ to all of the entries of $D$ when $M = PDP^{-1}$. That is, $\log(I + M) = P\log(I + D)P^{-1}$ and $\log(I + D)$ is computed entry-wise.

By the Spectral Theorem, every symmetric matrix is diagonalizable which handles the case you are interested in. Also notice that the entries of $D$ are the eigenvalues of the matrix and the entries of $\log(I + D)$ are positive when the entries of $D$ are.