Why is my $D'D$ matrix not positive definite?

284 Views Asked by At

I am trying to write a quadratic program to minimize a sum of squares. My goal is to obtain a "smooth" vector $x$, so I'm trying to minimize $(x_1-x_2)^2 + (x_2-x_3)^2 + (x_3-x_4)^2 + (x_4 + x_5)^2$ (with some restrictions).

If I understood correctly, the quadratic program representation is then:

$\min x^\top A x$, where $A=D^\top D$.

The first order difference matrix is $D = \begin{pmatrix} -1 & 1 & 0 & 0 & 0 \\ 0 & -1 & 1 & 0 & 0 \\ 0 & 0 & -1 & 1 & 0 \\ 0 & 0 & 0 & -1 & 1 \end{pmatrix}$

Then, $A$ will be $A = D^\top D = \begin{pmatrix} 1 & -1 & 0 & 0 & 0 \\ -1 & 2 & -1 & 0 & 0 \\ 0 & -1 & 2 & -1 & 0 \\ 0 & 0 & -1 & 2 & -1 \\ 0 & 0 & 0 & -1 & 1 \end{pmatrix}$

This matrix $A$ is not positive definite (but semi-definite) according to my quadratic program solver, which throws an error. On Wikipedia, though, it says that all gram matrices of the form $D^\top D$ are positive definite.

I seem to have a mistake somewhere. My hunch is that my matrix $A$ does not represent what I want, which is to minimize $(x_1-x_2)^2 + (x_2-x_3)^2 + (x_3-x_4)^2 + (x_4 + x_5)^2$.

So, my question: Where is my mistake? My sum of squares is clearly positive for all $x$, but my matrix $A$ is not positive definite.

Wikipedia also gives an example that a matrix $\begin{pmatrix} 2 & -1 & 0 \\ -1 & 2 & -1 \\ 0 & -1 & 2 \end{pmatrix}$ is positive definite. This one differs in my version only in the top left and lower right elements. Does that have anything to do with what I'm trying to solve?

1

There are 1 best solutions below

3
On BEST ANSWER

$D$ is $4\times 5$ matrix, rank of $D$ $\le 4$. $D^t$ is $5\times 4$ matrix, thus $A$ is a $5 \times 5$. Rank $A$ $\le $ Rank $D$ $= 4 $. Thus $A$ is a $5 \times 5$ matrix of rank $\le 4$. Thus $\exists \ v \in \mathbb{R}^{5}$ such that $Av =0$. So $A$ is not positive definite and only semi-definite.