In order to use a minimum squares estimator over some discrete dynamic system parameters, it is necessary to prove that the product $X^T X$ is invertible. Consider the following $N$ by $n+1$ matrix $X$: $$X=\underbrace{\begin{bmatrix} u(0) & 0 & \ldots & 0 \\ u(1) & u(0) & 0 &\ldots \\ & \vdots & & \\ u(n) & u(n-1) & \ldots & u(0) \\ & \vdots & & \\ u(N) & u(N-1) & \ldots & u(N-n) \end{bmatrix}}_{n+1} $$ Where $u(k)=0,\forall k<0$ is the input of the system (which is deterministic and known, but could be an i.i.d sequence). In the case $n=N$, $X$ will be a Toeplitz matrix, if not, a rectangular one "Toeplitz-like". I did the algebra for the case $n<N$, and the product $X^TX$ is effectively a "symmetric" square matrix of size $(n+1)^2$, but I don't know how to start to prove that $X^TX$ is positive definite, or have non zero singular values (or eigenvalues) in order to be invertible...
Thanks a lot in advance for any help or opinion.
Just look at the column rank of $X$. The matrix has full column rank if and only if it has a nonzero diagonal (not necessarily the main diagonal, but any diagonal of length $n+1$). Therefore $X^\top X$ is positive definite if and only if at least one of $u(0),u(1),\ldots,u(N-n)$ is nonzero, otherwise the matrix product is merely positive semidefinite.