Computational complexity of the Cholesky factorization

1k Views Asked by At

According to the Cholesky factorization on Wikipedia, the computational complexity of it is $\frac{n^3}{3}$ FLOPs where $n$ is the size of the considered matrix $\mathbf{A}$.

There are various methods for calculating the Cholesky decomposition. The computational complexity of commonly used algorithms is $\mathcal{O}(n^3)$ in general.The algorithms described below all involve about $n^3/3$ FLOPs ($n^3/6$ multiplications and the same number of additions), where $n$ is the size of the matrix A. Hence, they have half the cost of the LU decomposition, which uses $2n^3/3$ FLOPs (see Trefethen and Bau 1997).

To more understand how this number comes out, I was using a $3\times3$ real symmetric matrix, which can be found from this lecture note, pp.12.17--12.18.

For convenience, let me assume the computational complexity (or the floating point operation, FLOP) for multiplication, addition/subtraction, division, and square-root is $1$.

The matrix $\mathbf{A}$ is defined as

$$ \mathbf{A} = \begin{bmatrix} 25 & 15 & -5\\15 & 18 & 0\\-5 & 0 & 11\end{bmatrix} = \begin{bmatrix} R_{11} & 0 & 0\\R_{12} & R_{22} & 0\\R_{13} & R_{23} & R_{33} \end{bmatrix} \begin{bmatrix} R_{11} & R_{12} & R_{13}\\ 0 & R_{22} & R_{23} \\ 0 & 0 & R_{33} \end{bmatrix}\ . $$

We need to compute the following to find $R_{11}$:

$$R_{11}^2 = 25 \rightarrow R_{11} = 5,$$

and the plus 5 is chosen for convenience.

From the equation above, we easily can see that one squareroot is required to compute $R_{11}$.

For $R_{12}$ and $R_{13}$, we might compute the following:

$$ R_{12} = 15R_{11}^{-1} = 3, $$

$$ R_{13} = -5R_{11}^{-1} = -1, $$

which requires one division, respectively.

In total, so far, a total of $3$ FLOPs is required for $R_{11}$, $R_{12}$, and $R_{13}$.

Let me write the rest elements below.

$$R_{22} = \sqrt{18-R_{12}^2} = 3,$$

$$R_{23} = \frac{0-R_{13}R_{12}}{R_{22}} = 1,$$

$$R_{33} = \sqrt{11 - R_{13}^2 - R_{23}^2} = 3,$$

Therefore, it is a total of 14 FLOPS for the Cholesky factorization, which can be found in the following table:

Element $\times$ $+, -$ $\div$ $\sqrt{\cdot}$
$R_{11}$ $0$ $0$ $0$ $1$
$R_{12}$ $0$ $0$ $1$ $0$
$R_{13}$ $0$ $0$ $1$ $0$
$R_{22}$ $1$ $1$ $0$ $1$
$R_{23}$ $1$ $1$ $1$ $0$
$R_{33}$ $2$ $2$ $0$ $1$
Total $4$ $4$ $3$ $3$

It should have been $9$ FLOPs according to $n^3/3$, but the FLOPs that were obtained were $14$.

Is there any miscomputation? Or there should be something that I am missing or misunderstanding?