Matrix Factorization Difference

Question

I've just learned about $LDL^T$ decomposition. And i found that there are many other decomposition such as QR decomposition and cholesky decomposition. I don't understand what's the difference between them, i mean the basic idea of them. Do they have relation with spectral decomposition? Thanks..

Answer 1

The different decomposition have totally different properties. The Cholesky decomposition for example decomposes a symmetric (or hermitian) positive definitee matrix $A$ into a lower triangle matrix such that $A = LL^t$. This can on one hand be used for solving $Ax=b$ if $A$ has the desired properties, but the algorithm of the decomposition can also be used to check whether a matrix $A$ is positive definite.

The $QR$ decomposition decomposes an arbitrary matrix (does not even have to be square) into a square matrix where $Q$ is an orthognonal matrix (sqare) and $R$ an upper triangle matrix (does not have to be square). This one can e.g. be used for linear regression, where you have to solve an over determined system of linear equation $QRx = Ax \simeq b$ which then can be solved as $x = Q^t R^{-1} b$ (where $R^{-1}$ is not a conventioinal inverse but can be solved via substitution as it is in the $LU$ decomposition. The solution will then provide a least square solution for the regression function.

The $LU$ decomposition decomposes an invertible matrix $A$ into a lower and upper triangle matrix $L$ and $U$. This is also known as the gaussian elimination for solving general systems of linear equations.

These are just 3 of IMHO the most important matrix decompositions. They all have different properties and are used in different situations. Many numerical algorithms rely on solving systems of linear equations. But when solving huge systems you most often do not want to solve them exaclty (as done with those decompositions) but rather approximate the solutions. Those algorithms for approximating solutions again rely on properties of those decompositions or use modified decompositions.

The spectral decomposition $D = T^{-1} A T$ where $D$ is diagonal is ofcourse one of the most important decompositions, but it is mainly (but not only) a tool for theoretical stuff as proofs, and of course the study of eigenvalues and eigenvectors, but again is a different concept and used in different situations to the ones above.

I hope I could answer your question by giving a small overview over those decompositions with some of their properties. You can find a bigger list of matrix decompositions on this wikipedia site.